Human Head-Shoulder Segmentation
|
|
- Griselda Blake
- 6 years ago
- Views:
Transcription
1 Human Head-Shoulder Segmentation Hai Xin, Haizhou Ai Computer Science and Technology Tsinghua University Beijing, China Hui Chao, Daniel Tretter Hewlett-Packard Labs 1501 Page Mill Rd. Palo Alto CA, USA {Hui.chao, Abstract In this paper, an automatic head-shoulder segmentation method for human photos based on graph cut with shape sketch constraint and border detection through learning is presented. We propose a new shape constraint method based upon graph cut for head-shoulder photos. First, a watershed algorithm is used to over segment the photo into superpixels; next, an iterative shape mask guided graph cut algorithm with sketch constraint is applied to the superpixel level graph to get a border that segments the head-shoulder from its background; finally, a border detector, which is trained by AdaBoost, is used to refine the border. Experiments on consumer photo images demonstrate its effectiveness. Keywords-component; Human Segmentation, Shape Sketch, Graph Cut, AdaBoost, Border Detection I. INTRODUCTION Automatic and efficient human head-shoulder segmentation has great practical importance in human photo analysis and editing. It is an important part of face contextual region analysis for the purpose of human recognition and tracking. General contextual information, such as clothing, hair style, etc., is very useful for people identification, especially when the facial features alone do not provide sufficient information. Gallagher et al [1] has demonstrated that clothing information with time stamp can be effectively used in recognizing people for family photo collection. For editing, segmented headshoulder images are often used in new image and video composition by replacing or modifying the foreground or background. In this paper, we present a non-interactive human face contextual region segmentation method based on graph cut with shape sketch constraint and boundary detection through learning. Object segmentation is one of the basic issues in image processing and computer vision. Extensive studies on segmentation algorithms have been presented in the literature. Among them, GrabCut [2] built on Graph cut [3] proves to be a powerful interactive method with much reduced human interaction. After user roughly draw the object location, it initializes graph cut algorithm iteratively for more precise segmentation. Graph cut in its original form builds a graph on pixel level in which each pixel is a vertex and between any two neighboring pixels is an edge. Each pixel has a probability of foreground called data cost, and each edge has an edge cost based on the brightness and distance of the two pixels. Graph cut translates the segmentation problem into a min-cut problem, which can be solved by network flow algorithms. Learning based border and edge detection based on low level features has shown to be highly adaptive and effective [4, 5, 6, 7]. Shahrokni et al [4] trained a detector by AdaBoost for texture boundary detection. Their results showed accurate object borders identified as texture transitions in complex scenes. Our basic idea in developing a robust algorithm for automatic non-interactive head-shoulder region segmentation is to incorporate the domain knowledge that is specific to consumer face photos rather than general objects. Start with an extended face area as the region of interest, to improve the speed of graph cut, watershed segmentation is used first to convert this region into a superpixel image; graph cut with head-shoulder constraint is then applied to the superpixel level graph and followed by a special hair region processing to get a border that segments the head-shoulder region from its background. For boundary refinement, we explore the method of border detection through machine learning as [4]. A border detector is trained using AdaBoost where ribbon patches along the segmented border in the normal direction is used as positive samples and patches away from border as negative samples. We use this detector to check ribbon patches in the normal direction of previous segmented border resulted from graph cut to refine the boundary. The rest of the paper is organized as follows: Section 2 reviews related work. Section 3 introduces over segmentation and superpixel graph. Section 4 describes Shape sketch constraint. Section 5 presents segmentation with iterative mask. Section 6 provides border patches and border detector training. Section 7 shows the experiment results. And finally a conclusion is given in section 8. II. RELATED WORK In principle, graph cut is developed for interactive image segmentation by Y. Boykov [3]. The approach is to segment image by labeling each pixel to 0 (background) or 1 (foreground). For a given image, it builds a graph, G=(V,E), where V is a set of vertices and E is a set of edges. Each vertex v i represents a pixel in original image and edges are connection relationship between adjacent pixels, which can be brightness, color, texture, space distance or hybrid of them. Moreover,
2 Figure 2. Over segementation into superpixels. Figure 1. Head-shoulder part graph cut has two special terminate vertices called source and sink, which delegate foreground and background. Algorithm tries to minimize global cost energy E: where (1) E R ( A ) B B p p p, q pp ( p, q) N, Ap Aq feature dist( p, q) exp( ) 2 pq, 2 where P is the set of pixels and N is the set of neighboring pixels. A is the label of segmentation, A(p)=0 if p is background, and A(p)=1 for foreground. R p (A p ) is the prior data likelihood to assign p to foreground, which comes by interactive marking in [3]. B p,q is image connection cost, which is the possibility p and q will be assigned to the same class. This energy will be high when p and q have similar feature but different labels. Then, the problem can be solved in polynomial time through graph min-cut algorithm. Shape prior is a useful tool for segmentation. Borenstein and Malik [15] use top-down shape template to guide low level pixels merge. Some researchers use level-set active contour algorithm along with principal component analysis (PCA) [11, 12] to segment with shape priors. Freedman and Zhang [8, 9] introduced level-set active contour template into traditional graph cut to incorporate with shape priors. It merges shape priors information along with original image into new edge weight. In their approach, they add [ mid ( p q)] into B p,q in (1). φ is the shape prior function, it works on the midpoint of p and q, which are adjacent pixels with different labels. The energy will be minimal when midpoint of p and q is on the shape prior, and it increases while the midpoint is away from shape prior. Moreover, they apply Procrustes Method [9, 10] for template rotation and Gaussian pyramid for scale. Chang, Yang and Parvin [14] improved Freedman and Zhang s method. They design a Bayesian approach to manage shape template scaling, translation and rotation. (2) Border detection is another way for segmentation. Some researchers use various features to train border patches [5, 6, 13]. Dollar, Tu and Belongie [5] use a large number of generic features across different scales to train a boosting tree for detection. Shahrokni et al [4] use some simple feature on ribbon patch. III. OVER SEGMENTATION AND SUPERPIXEL GRAPH Our approach starts from face locating. For a given frontal human photo, we use a face detector to find the position and size of the human face. Centered on the face, we extend the face area to three times the size of the face and normalize it to a 400*400 image. We use this region as head-shoulder part of a human image. An example is shown in Fig. 1. The normalized photo is 400*400 with pixels, which will cost too much time for using graph cut algorithm directly on a pixel level graph. Thus, a watershed segmentation algorithm is applied first to split the photo into superpixels. Examples are shown in Fig. 2, where about 2700 superpixels are obtained. The speed is greatly improved with much reduced pixel size at superpixel level. And the border information still keeps in the superpixel graph. An image graph is then constructed based on superpixels. Each superpixel is represented by the mean RGB color and the histogram of its pixels. The edge weight of two connected or close superpixels is calculated by the histogram distance and the boundary length. The formula of edge weight of two superpixels A and B is as follows: D p( A) p( B) AB ( A B) 2 2 ( Ai Bi) tab ( A) tab ( B) (3) A B 2 i i i i i i where Ai Bi is the Chi-Square distance of the histogram, p(a) is the most frequent histogram color of A. t AB (A) is the ratio of boundary length of AB over whole border of A, and similarly, t AB (B) is the ratio of the boundary length of AB over the whole border of B.
3 Figure 3. Head shape sketch IV. SHAPE SKETCH CONSTRAINT In this section, we introduce an energy item to represent the difference between the segmentation result and a prior shape model. The function is not related to the model s scale, but the shape of model. A. Head-Shoulder Shape Human head-shoulder parts all have a similar Omega like shape, as Fig. 3. Since the head part is guaranteed at the center of the normalized image by face detector. We compute the head shape sketch through statistical average position. However, the shoulder part has various shapes and changes a lot in different photos. Thus, we only restrict the resulted shape sketch of head border. The target is to limit resulted shape to a common sense head shape, but with various scale. B. Sketch Energy Function Shape sketch constraint is a high level knowledge for object segmentation, while lower-level superpixels don t carry this information. Thus we provide the following method to add the shape information into the graph cut framework. Keep data energy R p (A p ) the same as (1). Add a sketch energy term S p,q into the global energy equation: (4) E R ( A ) B S p p p, q p, q pp ( p, q) N, Ap Aq ( p, q) S, Ap Aq Same as traditional graph cut, B p,q is pixel connection energy, where feature can be brightness, color, texture and etc. Since we work on superpixel level graph, B p,q is the histogram distance in (3). S p,q is the sketch energy and S is the sketch neighborhood set, shown as follows. In Fig. 4(a) and 4(b), the green line l 0 is the average position of head shape sketch synthesized from ground truth. For each pair of contiguous superpixels on the green line, add it to neighborhood set S and assign a shape energy E l to them. As in Fig. 4(c), superpixel A and B are neighbors on shape sketch line l, we put pair (A, B) into set S and add energy E l (A, B) to S p,q. S p, q S p, q El ( p, q) (5) Figure 4. Sketch energy: (a) Average sketch position. (b, c) superpixels on sketch line. Assuming a segment result crosses the shape sketch, it will definitely intersect with some pair (P, Q) in set S. So the energy E L (P, Q) will be added to E, which increase the global energy. Therefore, global optimal result will avoid crossing this green shape sketch. To force segmentation result along with shape sketch, we add more sketch lines besides the green one. As Fig. 5(a), we resize the sketch and add all this class of shape lines to the model, repeating the above method. For each resized sketch line l, add energy E l (A, B) to each adjacent neighbor pair. Therefore, sketch energy S will be: S p, q El ( p, q) (6) ll E ( p, q) dist( l, l ) l where L is the set of resized sketch lines. E l gets larger while l is far away from l 0. This energy penalizes segment result crossing shape sketch but less for the sketch line near average position. The further the result is away from original shape, the more shape lines segment border will cross, which will produce a lot of energy. Energy graph is shown in Fig. 5(b), where the darker, the larger cost is. Figure 5. Sketch model. (a) Resized sketch lines. (b) sketch line energy. 0 (7)
4 Figure 6. Shape Masks V. ITERATIVE MASK SEGMENTATION For a human photo, his or her head can be accurately located by a face detector, although the size of the head remains uncertain. But shoulder and body parts of humans can have various shape forms that change differently. We use K- means algorithm to cluster all ground truth. Experiments show it works best when dividing into three classes, as Fig. 6. Then we provide the following algorithm to find the segmentation result through iterative improvement shape mask. A. Shape mask guided graph cut Given a shape mask synthesized with ground truth, as Fig. 6(b), each superpixel has a probability, p, to belong to foreground or background. By deciding a probability threshold, we can divide all superpixels into 3 classes: foreground (where p> ), background (p<1- ), or unknown (else). In our implementation, we use = 1%. Based on the mask, we assign each superpixel to foreground, background or unknown and build Gaussian Mixture Models (GMMs)for foreground and background respectively. We then compute the probabilities of each unknown superpixel X in foreground and background class. In (4), data energy is expressed by R p (A p ), which is the probability of p in foreground. We merge shape mask along with GMM, getting the following formula: GMM fore( X ) P( X fore) d( X fore) GMM ( X ) P( X fore) GMM ( X ) P( X back) fore GMM back ( X ) P( X back) d( X back) GMM ( X ) P( X fore) GMM ( X ) P( X back) fore where GMM fore (X) is the probability that X is in foreground GMM and GMM back (X) is the probability X in background GMM. P(X fore) is the probability that X is in foreground of the shape mask and P(X back) is the probability that X in shape mask s background. B. Iterative mask update Human shoulder and body parts differ a lot from each other, even the clustered average shape model is not accurate due to large data space. To match various shoulder shape photos, we update shape mask through iterative graph cut. back back (8) (9) Figure 7. Iterative shape mask. (a) Result from first time segmentation. (b) Generated shape mask. (c) Final result. First, we merge similar superpixels and choose the closest shape mask in Fig. 6. Based on the mask, we apply graph cut on (4), and get a segmentation result. Then we blur the border of the result to create a new shape mask and segment again. The iteration will continue until the result is stable. As Fig. 7, the iterative shape improvement can lead segmentation result close to the ground truth step by step. The whole segmentation algorithm is as follows: 1) Detect head shoulder part and normalize photo size. 2) Apply watershed algorithm to segment photo into superpixels and find closest mask. 3) Graph cut image based on sketch model and shape mask, according to (4). 4) Regenerate shape mask and sketch, redo step 3, until result stable. The algorithm converges to final result quickly, which will iterate 2 to 4 times in practice. VI. BORDER DETECTION To refine the segmentation border further, we explore the border detection method proposed in [4]. We extract image patches on the normal direction of the borders from ground truth. We use AdaBoost to train a head-shoulder border detector and use it to improve the boundary from segmentation result. A. Patch Training We collect image patches on the border of ground truth. Each patch is a 64*16 rectangle along the normal direction of the border. Since the direction is not precise, we add several small degree angle distortions on each direction and get several patches. We scale the patches down to 32*8 and use them as positive samples. The white rectangles in Fig. 9(a) show the positive samples. The negative samples come from ribbon patches sliding away from each positive sample in the normal direction of the border, as the yellow or green rectangle in Fig. 9(b). Similarly, we extract 64*16 rectangle images, scale them down to 32*8, and use these as negative samples. The samples are shown in Fig. 9(c) and (d). Figure 8. Patch feature
5 TABLE I. EXPERIMENT RESULTS Overlap Ratio % 90-95% 80-90% <80% Our approach 45% 29% 24% 2% Without border refine 42% 30% 26% 2% Without sketch constraint 39% 32% 27% 2% GrabCut [2] 15% 24% 26% 35% Figure 9. Patch samples For a 32*8 ribbon patch, the left and right average RGB color differences at different window sizes and locations are used as weak features. As shown in Fig. 8, the patch is divided into 32 vertical bars. Assuming average RGB values of the left are l 1, l 2,, l 16, the right are r 1, r 2,, r 16, the following features are computed: b d li rk ia k c fa, b, c, d, 1 a, b, c, d 16 b a d c (10) We collect about 3,000 positive samples and 74,000 negative ones, and use AdaBoost algorithm to train a detector. The detector on the training set has 99% positive pass rate and 3.7% negative pass rate. B. Refine Border by Detection For the segmentation result, we check the border along the normal direction by the border detector. As in the training method, we get 64*16 rectangle patches by several distortions and down scale to 32*8 for detector. The detector finds several possible border points, as illustrated as the red points in Fig. 10(b), and then the confidence weighted average position is chosen as the new border. The refined border result as the white outline is shown in Fig. 10(b) which is better than previous segmentation result in Fig. 10(a). VII. EXPERIMENTS Experiments are carried on 1000 frontal human photos, with various hair styles and different skin colors. The photos are from our own photo collections. The segmentation result is compared with ground truth and the overlap ratio is used as the evaluation criterion. Figure 10. Refine border: (a) previous; (b) refined. Ground Segment overlap Ground Segment (11) where Ground is the image ground truth, and Segment is the result of our algorithm. Experiments shows 74% photos have an overlap more than 90%. We compare our results with GrabCut [2] algorithm, for which we manually interactive segment on arbitrary 100 photos in the same dataset. Details are shown in Table I. Some examples are shown in Fig. 11. Due to the shape priors, our algorithm gives a rough headshoulder Omega-like border correctly on complex or confusable background, while general GrabCut fails sometime. With the help of sketch constraint, our algorithm fixes head part in most cases. Moreover, the border refine process further improves result locally while shape mask disturbs local superpixel feature information in global energy. However, the algorithm fails when the ground truth is far away from trained shape masks. VIII. CONCLUSION In this paper, we propose an iterative segmentation algorithm with shape priors constraint and apply it on human head-shoulder photo. A graph cut algorithm with shape sketch constraint and mask guide is applied on superpixels generated by watershed algorithm. Then an AdaBoosted border detector is used to refine the segmentation. Experiment results on different style human photos demonstrate its effectiveness. The new method improves precision of graph cut. It can extend to other object segmentation with shape priors. ACKNOWLEDGMENT This work is supported by a grant from Hewlett-Packard Company, and it is also supported in part by National Science Foundation of China under grant No REFERENCES [1] Andrew C. Gallagher, Tsuhan Chen, Clothing Segmentation for Recognizing People, IEEE Conference, Computer Vision and Pattern Recognition, [2] Carsten Rother, Vladimir Kolmogorov and Andrew Blake, GrabCut - Interactive Foreground Extraction using Iterated Graph Cuts, ACM SIGGRAPH [3] Y. Boykov, M. Jolly. Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images. International Conference on Computer Vision, [4] A. Shahrokni, T. Drummond, F. Fleuret, and P. Fua, Classification- Based Probabilistic Modeling of Texture Transition for Fast Line Search Tracking and Delineation, IEEE Trans.Pattern Analysis and Machine Intelligence, Vol.31, No.3, March 2009
6 [5] P. Dollar, Z. Tu, and S. Belongie, Supervised Learning of Edges and Object Boundaries, Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June [6] D. Martin, C. Fowlkes, and J. Malik, Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues, IEEE Trans.Pattern Analysis and Machine Intelligence, vol. 26, no. 5, May [7] E. Borenstein and S. Ullman, Class-Specific, Top-Down Segmentation, European Conference on Computer Vision, 2002 [8] D. Freedman and T. Zhang, Interactive graph cut based segmentation with shape priors, IEEE Conference on Computer Vision and Pattern Recognition, 2005 [9] T. Zhang and D. Freeman. Tracking objects using density matching and shape priors. IEEE International Conference on Computer Vision, [10] F. L. Bookstein, landmark methods for forms without landmarks localizing group differences in outline shape. Mediacal Image Analysis, [11] S. Pizer, G. Gerig, S. Joshi, and S. Aylward. Multiscale medial shapebased analysis of image objects. Proceedings of the IEEE, 91(10): , [12] A. Tsai, W. Wells, C. Tempany, E. Grimson, and A. Willsky. Mutual information in coupled multi-shape model for medical image segmentation. Medical Image Analysis 8 (2004) [13] A. Shahrokni, T. Drummond, P. Fua. Texture Boundary Detection for Real-Time Tracking. European Conference on Computer Vision, [14] H. Chang, Q. Yang and B. Parvin. A Bayesian Approach for Image Segmentation with Shape Priors. IEEE Conference on Computer Vision and Pattern Recognition, [15] E.Borenstein and J. Malik. Shape Guided Object Segmentation, Computer Vision and Pattern Recognition, Figure 11. Head-shoulder segmentation results. (a, d) Results from GrabCut [2] algorithm. (b, e) Shape mask guided graph cut algorithm without sketch constraint. (c, f) Our approach results, with both shape mask and sketch constraint.
Image Segmentation Using Iterated Graph Cuts Based on Multi-scale Smoothing
Image Segmentation Using Iterated Graph Cuts Based on Multi-scale Smoothing Tomoyuki Nagahashi 1, Hironobu Fujiyoshi 1, and Takeo Kanade 2 1 Dept. of Computer Science, Chubu University. Matsumoto 1200,
More informationSupervised texture detection in images
Supervised texture detection in images Branislav Mičušík and Allan Hanbury Pattern Recognition and Image Processing Group, Institute of Computer Aided Automation, Vienna University of Technology Favoritenstraße
More informationImage Segmentation Using Iterated Graph Cuts BasedonMulti-scaleSmoothing
Image Segmentation Using Iterated Graph Cuts BasedonMulti-scaleSmoothing Tomoyuki Nagahashi 1, Hironobu Fujiyoshi 1, and Takeo Kanade 2 1 Dept. of Computer Science, Chubu University. Matsumoto 1200, Kasugai,
More informationA Feature Point Matching Based Approach for Video Objects Segmentation
A Feature Point Matching Based Approach for Video Objects Segmentation Yan Zhang, Zhong Zhou, Wei Wu State Key Laboratory of Virtual Reality Technology and Systems, Beijing, P.R. China School of Computer
More informationSegmentation with non-linear constraints on appearance, complexity, and geometry
IPAM February 2013 Western Univesity Segmentation with non-linear constraints on appearance, complexity, and geometry Yuri Boykov Andrew Delong Lena Gorelick Hossam Isack Anton Osokin Frank Schmidt Olga
More informationTri-modal Human Body Segmentation
Tri-modal Human Body Segmentation Master of Science Thesis Cristina Palmero Cantariño Advisor: Sergio Escalera Guerrero February 6, 2014 Outline 1 Introduction 2 Tri-modal dataset 3 Proposed baseline 4
More informationA Benchmark for Interactive Image Segmentation Algorithms
A Benchmark for Interactive Image Segmentation Algorithms Yibiao Zhao 1,3, Xiaohan Nie 2,3, Yanbiao Duan 2,3, Yaping Huang 1, Siwei Luo 1 1 Beijing Jiaotong University, 2 Beijing Institute of Technology,
More informationEnergy Minimization for Segmentation in Computer Vision
S * = arg S min E(S) Energy Minimization for Segmentation in Computer Vision Meng Tang, Dmitrii Marin, Ismail Ben Ayed, Yuri Boykov Outline Clustering/segmentation methods K-means, GrabCut, Normalized
More informationAnnouncements. Image Segmentation. From images to objects. Extracting objects. Status reports next Thursday ~5min presentations in class
Image Segmentation Announcements Status reports next Thursday ~5min presentations in class Project voting From Sandlot Science Today s Readings Forsyth & Ponce, Chapter 1 (plus lots of optional references
More informationDot Text Detection Based on FAST Points
Dot Text Detection Based on FAST Points Yuning Du, Haizhou Ai Computer Science & Technology Department Tsinghua University Beijing, China dyn10@mails.tsinghua.edu.cn, ahz@mail.tsinghua.edu.cn Shihong Lao
More informationEstimating Human Pose in Images. Navraj Singh December 11, 2009
Estimating Human Pose in Images Navraj Singh December 11, 2009 Introduction This project attempts to improve the performance of an existing method of estimating the pose of humans in still images. Tasks
More informationFace Recognition Using Vector Quantization Histogram and Support Vector Machine Classifier Rong-sheng LI, Fei-fei LEE *, Yan YAN and Qiu CHEN
2016 International Conference on Artificial Intelligence: Techniques and Applications (AITA 2016) ISBN: 978-1-60595-389-2 Face Recognition Using Vector Quantization Histogram and Support Vector Machine
More informationMRFs and Segmentation with Graph Cuts
02/24/10 MRFs and Segmentation with Graph Cuts Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Today s class Finish up EM MRFs w ij i Segmentation with Graph Cuts j EM Algorithm: Recap
More informationintro, applications MRF, labeling... how it can be computed at all? Applications in segmentation: GraphCut, GrabCut, demos
Image as Markov Random Field and Applications 1 Tomáš Svoboda, svoboda@cmp.felk.cvut.cz Czech Technical University in Prague, Center for Machine Perception http://cmp.felk.cvut.cz Talk Outline Last update:
More informationAUTONOMOUS IMAGE EXTRACTION AND SEGMENTATION OF IMAGE USING UAV S
AUTONOMOUS IMAGE EXTRACTION AND SEGMENTATION OF IMAGE USING UAV S Radha Krishna Rambola, Associate Professor, NMIMS University, India Akash Agrawal, Student at NMIMS University, India ABSTRACT Due to the
More informationObject Detection Using Segmented Images
Object Detection Using Segmented Images Naran Bayanbat Stanford University Palo Alto, CA naranb@stanford.edu Jason Chen Stanford University Palo Alto, CA jasonch@stanford.edu Abstract Object detection
More informationFace Alignment Under Various Poses and Expressions
Face Alignment Under Various Poses and Expressions Shengjun Xin and Haizhou Ai Computer Science and Technology Department, Tsinghua University, Beijing 100084, China ahz@mail.tsinghua.edu.cn Abstract.
More informationCombining Top-down and Bottom-up Segmentation
Combining Top-down and Bottom-up Segmentation Authors: Eran Borenstein, Eitan Sharon, Shimon Ullman Presenter: Collin McCarthy Introduction Goal Separate object from background Problems Inaccuracies Top-down
More informationClassifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao
Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao Motivation Image search Building large sets of classified images Robotics Background Object recognition is unsolved Deformable shaped
More informationCS 534: Computer Vision Segmentation and Perceptual Grouping
CS 534: Computer Vision Segmentation and Perceptual Grouping Ahmed Elgammal Dept of Computer Science CS 534 Segmentation - 1 Outlines Mid-level vision What is segmentation Perceptual Grouping Segmentation
More informationBus Detection and recognition for visually impaired people
Bus Detection and recognition for visually impaired people Hangrong Pan, Chucai Yi, and Yingli Tian The City College of New York The Graduate Center The City University of New York MAP4VIP Outline Motivation
More informationMarkov Random Fields and Segmentation with Graph Cuts
Markov Random Fields and Segmentation with Graph Cuts Computer Vision Jia-Bin Huang, Virginia Tech Many slides from D. Hoiem Administrative stuffs Final project Proposal due Oct 27 (Thursday) HW 4 is out
More informationGraph-Based Superpixel Labeling for Enhancement of Online Video Segmentation
Graph-Based Superpixel Labeling for Enhancement of Online Video Segmentation Alaa E. Abdel-Hakim Electrical Engineering Department Assiut University Assiut, Egypt alaa.aly@eng.au.edu.eg Mostafa Izz Cairo
More informationLearning and Inferring Depth from Monocular Images. Jiyan Pan April 1, 2009
Learning and Inferring Depth from Monocular Images Jiyan Pan April 1, 2009 Traditional ways of inferring depth Binocular disparity Structure from motion Defocus Given a single monocular image, how to infer
More informationMarkov/Conditional Random Fields, Graph Cut, and applications in Computer Vision
Markov/Conditional Random Fields, Graph Cut, and applications in Computer Vision Fuxin Li Slides and materials from Le Song, Tucker Hermans, Pawan Kumar, Carsten Rother, Peter Orchard, and others Recap:
More informationDigital Makeup Face Generation
Digital Makeup Face Generation Wut Yee Oo Mechanical Engineering Stanford University wutyee@stanford.edu Abstract Make up applications offer photoshop tools to get users inputs in generating a make up
More informationidentified and grouped together.
Segmentation ti of Images SEGMENTATION If an image has been preprocessed appropriately to remove noise and artifacts, segmentation is often the key step in interpreting the image. Image segmentation is
More informationApplications. Foreground / background segmentation Finding skin-colored regions. Finding the moving objects. Intelligent scissors
Segmentation I Goal Separate image into coherent regions Berkeley segmentation database: http://www.eecs.berkeley.edu/research/projects/cs/vision/grouping/segbench/ Slide by L. Lazebnik Applications Intelligent
More informationSegmentation and Grouping
CS 1699: Intro to Computer Vision Segmentation and Grouping Prof. Adriana Kovashka University of Pittsburgh September 24, 2015 Goals: Grouping in vision Gather features that belong together Obtain an intermediate
More informationProf. Feng Liu. Spring /17/2017. With slides by F. Durand, Y.Y. Chuang, R. Raskar, and C.
Prof. Feng Liu Spring 2017 http://www.cs.pdx.edu/~fliu/courses/cs510/ 05/17/2017 With slides by F. Durand, Y.Y. Chuang, R. Raskar, and C. Rother Last Time Image segmentation Normalized cut and segmentation
More informationLinear combinations of simple classifiers for the PASCAL challenge
Linear combinations of simple classifiers for the PASCAL challenge Nik A. Melchior and David Lee 16 721 Advanced Perception The Robotics Institute Carnegie Mellon University Email: melchior@cmu.edu, dlee1@andrew.cmu.edu
More informationLecture 11: E-M and MeanShift. CAP 5415 Fall 2007
Lecture 11: E-M and MeanShift CAP 5415 Fall 2007 Review on Segmentation by Clustering Each Pixel Data Vector Example (From Comanciu and Meer) Review of k-means Let's find three clusters in this data These
More informationSegmentation. Separate image into coherent regions
Segmentation II Segmentation Separate image into coherent regions Berkeley segmentation database: http://www.eecs.berkeley.edu/research/projects/cs/vision/grouping/segbench/ Slide by L. Lazebnik Interactive
More informationQUT Digital Repository: This is the author version published as:
QUT Digital Repository: http://eprints.qut.edu.au/ This is the author version published as: This is the accepted version of this article. To be published as : This is the author version published as: Chen,
More information3 October, 2013 MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos
Machine Learning for Computer Vision 1 3 October, 2013 MVA ENS Cachan Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Department of Applied Mathematics Ecole Centrale
More informationIterative MAP and ML Estimations for Image Segmentation
Iterative MAP and ML Estimations for Image Segmentation Shifeng Chen 1, Liangliang Cao 2, Jianzhuang Liu 1, and Xiaoou Tang 1,3 1 Dept. of IE, The Chinese University of Hong Kong {sfchen5, jzliu}@ie.cuhk.edu.hk
More informationIntroduction to Medical Imaging (5XSA0) Module 5
Introduction to Medical Imaging (5XSA0) Module 5 Segmentation Jungong Han, Dirk Farin, Sveta Zinger ( s.zinger@tue.nl ) 1 Outline Introduction Color Segmentation region-growing region-merging watershed
More informationA New Algorithm for Shape Detection
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 19, Issue 3, Ver. I (May.-June. 2017), PP 71-76 www.iosrjournals.org A New Algorithm for Shape Detection Hewa
More informationImproved head-shoulder human contour estimation through clusters of learned shape models
Improved head-shoulder human contour estimation through clusters of learned shape models Julio Cezar Silveira Jacques Junior and Soraia Raupp Musse PUCRS - Faculdade de Informática Porto Alegre, Brazil
More informationSegmentation of Images
Segmentation of Images SEGMENTATION If an image has been preprocessed appropriately to remove noise and artifacts, segmentation is often the key step in interpreting the image. Image segmentation is a
More informationPrincipal-channels for One-sided Object Cutout
Principal-channels for One-sided Object Cutout Lior Gavish, Lior Wolf, Lior Shapira, and Daniel Cohen-Or Tel-Aviv University, Tel-Aviv, Israel Abstract We introduce principal-channels for cutting out objects
More informationGlobal Probability of Boundary
Global Probability of Boundary Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues Martin, Fowlkes, Malik Using Contours to Detect and Localize Junctions in Natural
More informationPart III: Affinity Functions for Image Segmentation
Part III: Affinity Functions for Image Segmentation Charless Fowlkes joint work with David Martin and Jitendra Malik at University of California at Berkeley 1 Q: What measurements should we use for constructing
More informationGeneric Face Alignment Using an Improved Active Shape Model
Generic Face Alignment Using an Improved Active Shape Model Liting Wang, Xiaoqing Ding, Chi Fang Electronic Engineering Department, Tsinghua University, Beijing, China {wanglt, dxq, fangchi} @ocrserv.ee.tsinghua.edu.cn
More informationFigure-Ground Segmentation Techniques
Figure-Ground Segmentation Techniques Snehal P. Ambulkar 1, Nikhil S. Sakhare 2 1 2 nd Year Student, Master of Technology, Computer Science & Engineering, Rajiv Gandhi College of Engineering & Research,
More informationECG782: Multidimensional Digital Signal Processing
Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu ECG782: Multidimensional Digital Signal Processing Spring 2014 TTh 14:30-15:45 CBC C313 Lecture 10 Segmentation 14/02/27 http://www.ee.unlv.edu/~b1morris/ecg782/
More informationBoundaries and Sketches
Boundaries and Sketches Szeliski 4.2 Computer Vision James Hays Many slides from Michael Maire, Jitendra Malek Today s lecture Segmentation vs Boundary Detection Why boundaries / Grouping? Recap: Canny
More informationSegmentation. (template) matching
Segmentation. (template) matching 1 Announcements Midterm March 3 2010, in class Duration: 45 min Weight: 20% of final course mark Closed book, closed notes Calculator allowed Practice midterm posted.
More informationObject Recognition Using Pictorial Structures. Daniel Huttenlocher Computer Science Department. In This Talk. Object recognition in computer vision
Object Recognition Using Pictorial Structures Daniel Huttenlocher Computer Science Department Joint work with Pedro Felzenszwalb, MIT AI Lab In This Talk Object recognition in computer vision Brief definition
More informationScene Text Detection Using Machine Learning Classifiers
601 Scene Text Detection Using Machine Learning Classifiers Nafla C.N. 1, Sneha K. 2, Divya K.P. 3 1 (Department of CSE, RCET, Akkikkvu, Thrissur) 2 (Department of CSE, RCET, Akkikkvu, Thrissur) 3 (Department
More informationIterated Graph Cuts for Image Segmentation
Iterated Graph Cuts for Image Segmentation Bo Peng 1, Lei Zhang 1, and Jian Yang 2 1 Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China. 2 School of Computer Science
More informationImage Segmentation continued Graph Based Methods. Some slides: courtesy of O. Capms, Penn State, J.Ponce and D. Fortsyth, Computer Vision Book
Image Segmentation continued Graph Based Methods Some slides: courtesy of O. Capms, Penn State, J.Ponce and D. Fortsyth, Computer Vision Book Previously Binary segmentation Segmentation by thresholding
More informationCellular Learning Automata-Based Color Image Segmentation using Adaptive Chains
Cellular Learning Automata-Based Color Image Segmentation using Adaptive Chains Ahmad Ali Abin, Mehran Fotouhi, Shohreh Kasaei, Senior Member, IEEE Sharif University of Technology, Tehran, Iran abin@ce.sharif.edu,
More informationLecture 16 Segmentation and Scene understanding
Lecture 16 Segmentation and Scene understanding Introduction! Mean-shift! Graph-based segmentation! Top-down segmentation! Silvio Savarese Lecture 15 -! 3-Mar-14 Segmentation Silvio Savarese Lecture 15
More informationConsistent Line Clusters for Building Recognition in CBIR
Consistent Line Clusters for Building Recognition in CBIR Yi Li and Linda G. Shapiro Department of Computer Science and Engineering University of Washington Seattle, WA 98195-250 shapiro,yi @cs.washington.edu
More informationDiscovering Visual Hierarchy through Unsupervised Learning Haider Razvi
Discovering Visual Hierarchy through Unsupervised Learning Haider Razvi hrazvi@stanford.edu 1 Introduction: We present a method for discovering visual hierarchy in a set of images. Automatically grouping
More informationData-driven Depth Inference from a Single Still Image
Data-driven Depth Inference from a Single Still Image Kyunghee Kim Computer Science Department Stanford University kyunghee.kim@stanford.edu Abstract Given an indoor image, how to recover its depth information
More informationTime Stamp Detection and Recognition in Video Frames
Time Stamp Detection and Recognition in Video Frames Nongluk Covavisaruch and Chetsada Saengpanit Department of Computer Engineering, Chulalongkorn University, Bangkok 10330, Thailand E-mail: nongluk.c@chula.ac.th
More informationFinding people in repeated shots of the same scene
1 Finding people in repeated shots of the same scene Josef Sivic 1 C. Lawrence Zitnick Richard Szeliski 1 University of Oxford Microsoft Research Abstract The goal of this work is to find all occurrences
More informationAn Adaptive Threshold LBP Algorithm for Face Recognition
An Adaptive Threshold LBP Algorithm for Face Recognition Xiaoping Jiang 1, Chuyu Guo 1,*, Hua Zhang 1, and Chenghua Li 1 1 College of Electronics and Information Engineering, Hubei Key Laboratory of Intelligent
More informationAn embedded system of Face Recognition based on ARM and HMM
An embedded system of Face Recognition based on ARM and HMM Yanbin Sun 1,2, Lun Xie 1, Zhiliang Wang 1,Yi An 2 1 Department of Electronic Information Engineering, School of Information Engineering, University
More informationInteractive Graph Cut Based Segmentation With Shape Priors
Interactive Graph Cut Based Segmentation With Shape Priors Daniel Freedman and Tao Zhang Computer Science Department, Rensselaer Polytechnic Institute, Troy, NY 1180 Abstract Interactive or semi-automatic
More informationImproving Latent Fingerprint Matching Performance by Orientation Field Estimation using Localized Dictionaries
Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 11, November 2014,
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More informationApplied Bayesian Nonparametrics 5. Spatial Models via Gaussian Processes, not MRFs Tutorial at CVPR 2012 Erik Sudderth Brown University
Applied Bayesian Nonparametrics 5. Spatial Models via Gaussian Processes, not MRFs Tutorial at CVPR 2012 Erik Sudderth Brown University NIPS 2008: E. Sudderth & M. Jordan, Shared Segmentation of Natural
More informationDIGITAL IMAGE ANALYSIS. Image Classification: Object-based Classification
DIGITAL IMAGE ANALYSIS Image Classification: Object-based Classification Image classification Quantitative analysis used to automate the identification of features Spectral pattern recognition Unsupervised
More informationThe goals of segmentation
Image segmentation The goals of segmentation Group together similar-looking pixels for efficiency of further processing Bottom-up process Unsupervised superpixels X. Ren and J. Malik. Learning a classification
More informationInteractive Image Segmentation with GrabCut
Interactive Image Segmentation with GrabCut Bryan Anenberg Stanford University anenberg@stanford.edu Michela Meister Stanford University mmeister@stanford.edu Abstract We implement GrabCut and experiment
More informationADAPTIVE GRAPH CUTS WITH TISSUE PRIORS FOR BRAIN MRI SEGMENTATION
ADAPTIVE GRAPH CUTS WITH TISSUE PRIORS FOR BRAIN MRI SEGMENTATION Abstract: MIP Project Report Spring 2013 Gaurav Mittal 201232644 This is a detailed report about the course project, which was to implement
More informationAn Introduction to Pattern Recognition
An Introduction to Pattern Recognition Speaker : Wei lun Chao Advisor : Prof. Jian-jiun Ding DISP Lab Graduate Institute of Communication Engineering 1 Abstract Not a new research field Wide range included
More informationIMA Preprint Series # 2153
DISTANCECUT: INTERACTIVE REAL-TIME SEGMENTATION AND MATTING OF IMAGES AND VIDEOS By Xue Bai and Guillermo Sapiro IMA Preprint Series # 2153 ( January 2007 ) INSTITUTE FOR MATHEMATICS AND ITS APPLICATIONS
More informationCS 664 Segmentation. Daniel Huttenlocher
CS 664 Segmentation Daniel Huttenlocher Grouping Perceptual Organization Structural relationships between tokens Parallelism, symmetry, alignment Similarity of token properties Often strong psychophysical
More information6.801/866. Segmentation and Line Fitting. T. Darrell
6.801/866 Segmentation and Line Fitting T. Darrell Segmentation and Line Fitting Gestalt grouping Background subtraction K-Means Graph cuts Hough transform Iterative fitting (Next time: Probabilistic segmentation)
More informationSTUDYING THE FEASIBILITY AND IMPORTANCE OF GRAPH-BASED IMAGE SEGMENTATION TECHNIQUES
25-29 JATIT. All rights reserved. STUDYING THE FEASIBILITY AND IMPORTANCE OF GRAPH-BASED IMAGE SEGMENTATION TECHNIQUES DR.S.V.KASMIR RAJA, 2 A.SHAIK ABDUL KHADIR, 3 DR.S.S.RIAZ AHAMED. Dean (Research),
More informationRegion-based Segmentation
Region-based Segmentation Image Segmentation Group similar components (such as, pixels in an image, image frames in a video) to obtain a compact representation. Applications: Finding tumors, veins, etc.
More informationImage Segmentation Techniques
A Study On Image Segmentation Techniques Palwinder Singh 1, Amarbir Singh 2 1,2 Department of Computer Science, GNDU Amritsar Abstract Image segmentation is very important step of image analysis which
More informationCSE/EE-576, Final Project
1 CSE/EE-576, Final Project Torso tracking Ke-Yu Chen Introduction Human 3D modeling and reconstruction from 2D sequences has been researcher s interests for years. Torso is the main part of the human
More informationTexture Segmentation by Windowed Projection
Texture Segmentation by Windowed Projection 1, 2 Fan-Chen Tseng, 2 Ching-Chi Hsu, 2 Chiou-Shann Fuh 1 Department of Electronic Engineering National I-Lan Institute of Technology e-mail : fctseng@ccmail.ilantech.edu.tw
More informationRobust Face Alignment Based on Hierarchical Classifier Network
Robust Face Alignment Based on Hierarchical Classifier Network Li Zhang 1, Haizhou Ai 1, and Shihong Lao 2 1 Department of Computer Science, Tsinghua University, Beiing 0084, China 2 Sensing and Control
More informationA Hierarchical Compositional System for Rapid Object Detection
A Hierarchical Compositional System for Rapid Object Detection Long Zhu and Alan Yuille Department of Statistics University of California at Los Angeles Los Angeles, CA 90095 {lzhu,yuille}@stat.ucla.edu
More informationAn Object Detection System using Image Reconstruction with PCA
An Object Detection System using Image Reconstruction with PCA Luis Malagón-Borja and Olac Fuentes Instituto Nacional de Astrofísica Óptica y Electrónica, Puebla, 72840 Mexico jmb@ccc.inaoep.mx, fuentes@inaoep.mx
More informationExtracting Layers and Recognizing Features for Automatic Map Understanding. Yao-Yi Chiang
Extracting Layers and Recognizing Features for Automatic Map Understanding Yao-Yi Chiang 0 Outline Introduction/ Problem Motivation Map Processing Overview Map Decomposition Feature Recognition Discussion
More informationTexture. Texture. 2) Synthesis. Objectives: 1) Discrimination/Analysis
Texture Texture D. Forsythe and J. Ponce Computer Vision modern approach Chapter 9 (Slides D. Lowe, UBC) Key issue: How do we represent texture? Topics: Texture segmentation Texture-based matching Texture
More informationTexture. Texture is a description of the spatial arrangement of color or intensities in an image or a selected region of an image.
Texture Texture is a description of the spatial arrangement of color or intensities in an image or a selected region of an image. Structural approach: a set of texels in some regular or repeated pattern
More informationMarkov Random Fields and Segmentation with Graph Cuts
Markov Random Fields and Segmentation with Graph Cuts Computer Vision Jia-Bin Huang, Virginia Tech Many slides from D. Hoiem Administrative stuffs Final project Proposal due Oct 30 (Monday) HW 4 is out
More informationPredicting Types of Clothing Using SURF and LDP based on Bag of Features
Predicting Types of Clothing Using SURF and LDP based on Bag of Features Wisarut Surakarin Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok, Thailand ws.sarut@gmail.com
More informationComputer Vision 5 Segmentation by Clustering
Computer Vision 5 Segmentation by Clustering MAP-I Doctoral Programme Miguel Tavares Coimbra Outline Introduction Applications Simple clustering K-means clustering Graph-theoretic clustering Acknowledgements:
More informationAutomatic Shadow Removal by Illuminance in HSV Color Space
Computer Science and Information Technology 3(3): 70-75, 2015 DOI: 10.13189/csit.2015.030303 http://www.hrpub.org Automatic Shadow Removal by Illuminance in HSV Color Space Wenbo Huang 1, KyoungYeon Kim
More informationShweta Gandhi, Dr.D.M.Yadav JSPM S Bhivarabai sawant Institute of technology & research Electronics and telecom.dept, Wagholi, Pune
Face sketch photo synthesis Shweta Gandhi, Dr.D.M.Yadav JSPM S Bhivarabai sawant Institute of technology & research Electronics and telecom.dept, Wagholi, Pune Abstract Face sketch to photo synthesis has
More informationSupplemental Material: Detailed, accurate, human shape estimation from clothed 3D scan sequences
Supplemental Material: Detailed, accurate, human shape estimation from clothed 3D scan sequences Chao Zhang 1,2, Sergi Pujades 1, Michael Black 1, and Gerard Pons-Moll 1 1 MPI for Intelligent Systems,
More informationEnhanced Active Shape Models with Global Texture Constraints for Image Analysis
Enhanced Active Shape Models with Global Texture Constraints for Image Analysis Shiguang Shan, Wen Gao, Wei Wang, Debin Zhao, Baocai Yin Institute of Computing Technology, Chinese Academy of Sciences,
More informationNearest Clustering Algorithm for Satellite Image Classification in Remote Sensing Applications
Nearest Clustering Algorithm for Satellite Image Classification in Remote Sensing Applications Anil K Goswami 1, Swati Sharma 2, Praveen Kumar 3 1 DRDO, New Delhi, India 2 PDM College of Engineering for
More informationTransductive Object Cutout
Transductive Object Cutout Jingyu Cui 1, Qiong Yang 3, Fang Wen 2, Qiying Wu 4, Changshui Zhang 1, Luc Van Gool 3, and Xiaoou Tang 2 1 Tsinghua University, Beijing, China cui-jy@mails.thu.edu.cn, zcs@mail.thu.edu.cn
More informationSegmentation Computer Vision Spring 2018, Lecture 27
Segmentation http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 218, Lecture 27 Course announcements Homework 7 is due on Sunday 6 th. - Any questions about homework 7? - How many of you have
More informationUsing the Kolmogorov-Smirnov Test for Image Segmentation
Using the Kolmogorov-Smirnov Test for Image Segmentation Yong Jae Lee CS395T Computational Statistics Final Project Report May 6th, 2009 I. INTRODUCTION Image segmentation is a fundamental task in computer
More informationAnalysis: TextonBoost and Semantic Texton Forests. Daniel Munoz Februrary 9, 2009
Analysis: TextonBoost and Semantic Texton Forests Daniel Munoz 16-721 Februrary 9, 2009 Papers [shotton-eccv-06] J. Shotton, J. Winn, C. Rother, A. Criminisi, TextonBoost: Joint Appearance, Shape and Context
More informationLOOSECUT: INTERACTIVE IMAGE SEGMENTATION WITH LOOSELY BOUNDED BOXES
Loose Input Box LOOSECUT: INTERACTIVE IMAGE SEGMENTATION WITH LOOSELY BOUNDED BOXES Hongkai Yu 1, Youjie Zhou 1, Hui Qian 2, Min Xian 3, and Song Wang 1 1 University of South Carolina, SC 2 Zhejiang University,
More informationTRANSPARENT OBJECT DETECTION USING REGIONS WITH CONVOLUTIONAL NEURAL NETWORK
TRANSPARENT OBJECT DETECTION USING REGIONS WITH CONVOLUTIONAL NEURAL NETWORK 1 Po-Jen Lai ( 賴柏任 ), 2 Chiou-Shann Fuh ( 傅楸善 ) 1 Dept. of Electrical Engineering, National Taiwan University, Taiwan 2 Dept.
More informationI. INTRODUCTION. Figure-1 Basic block of text analysis
ISSN: 2349-7637 (Online) (RHIMRJ) Research Paper Available online at: www.rhimrj.com Detection and Localization of Texts from Natural Scene Images: A Hybrid Approach Priyanka Muchhadiya Post Graduate Fellow,
More informationColor Image Segmentation
Color Image Segmentation Yining Deng, B. S. Manjunath and Hyundoo Shin* Department of Electrical and Computer Engineering University of California, Santa Barbara, CA 93106-9560 *Samsung Electronics Inc.
More information