Computer Vision 10. Segmentation Computer Engineering, Sejong University Dongil Han Image Segmentation Image segmentation Subdivides an image into its constituent regions or objects - After an image has been segmented, the aggregate of segmented pixels is represented and described and recognized. One of the most difficult tasks in image processing - Industrial inspection application : control the environment for easy segmentation - If control is impossible : focus on selecting the type of sensor - Infrared imaging to detect heat signatures in military applications Segmentation accuracy determines the eventual success or failure of analysis procedures 2/59
Image Segmentation Example 3/59 Image Segmentation Example 4/59
Image Segmentation Segmentation Cues Similar Color Similar Texture Combination of previous segmentation results Segmentation methods Use discontinuity ( between different region ) - Edge based segmentation Use similarity ( within the region ) - Based on thresholding, region growing, region splitting, region merging 5/59 Image Segmentation Discontinuity based segmentation Point detection Line detection Edge detection - Gradient operator - Laplacian operator => Stage for connecting edge segments(edge linking) is required 6/59
Image Segmentation Edge linking and Boundary detection Edge detection methods generate spurious intensity discontinuities - Because of noise, nonuniform illumination, and other effects edge detection algorithms - Gradient operators : Roberts, Prewitt, Sobel, etc. - Laplacian, LoG Edge detection algorithms are followed by edge linking procedures Edge linking methods Local processing Hough transform Graph-theoretic technique 7/59 Edge linking Local processing Analyze the characteristics of pixels in a small neighborhood All points that are similar according to a set of predefined criteria are linked Usually 3x3, 5x5 window is used Two principal properties foe establishing similarity of edge pixel - Strength of the response of the gradient operator - Direction of the gradient vector 8/59
9/59 Edge linking 10/59 Edge linking Gradient operator The gradient of an image f(x,y) at location (x,y) is defined as the vector The magnitude of this vector Let (x,y) represent the direction angle of the vector at (x,y) y f x f y G x G f ) ( 1 tan ), ( x y G G y x 2 2 ) ( y G x G or y G x G mag f f
Edge linking Edge detection and linking A point in the predefined neighborhood of (x, y) is linked to the pixel at (x, y) if the following two criteria are satisfied f ( x, y) f ( x 0, y 0 ) E ( x, y) ( x 0, y 0 ) A E, A : nonnegative thresholds This process is repeated at every location in the image 11/59 Edge linking 12/59
Edge linking Local processing vs. Global processing Local processing - Simple and fast - Edge linking results depend on the size of window - Edge linking results depend on the selection of threshold value - Sensitive to noise Global processing - Complex and slow - Exact and robust to noise - Popular techniques 1. Hough transform 2. Graph-theoretic technique 13/59 Hough Transform Concept Global edge detection in the parameter space - Consider a point (x i, y i ) - General equation of a straight line in slope-intercept form ax b - Infinitely many lines pass through (x i, y i ) for varying values of a and b - If we considering the ab-plane y i b x a i i y i - Here, ab-plane is called parameter space 14/59
Hough Transform Concept All points contained on one line have lines in parameter space that intersect at the same point (a, b ) 15/59 Hough Transform Implementation Subdivide the parameter space into accumulator cells - Expected ranges of slope and intercept values (a max, a min ), (b max, b min ) - Determine the resolution of accumulator cells A(i, j) - Initially accumulator cells are set to zero - For every point (x k, y k ) in the image plane, increase the accumulator cells which satisfy the following equations b x a k y k - The value of each accumulator cell A(i, j) presents the number of points lying on the line y = a i x + b j in xy-plane 16/59
Hough Transform Accumulator cell example 17/59 Hough Transform Selection of Parameter space ab-plane - The slope approaches infinity as the line approaches the vertical - The size of accumulator cell is unequal -plane : the smallest distance between the line and origin : the angle of the locus vector from the origin to this closest point x i cos y sin - Loci are sinusoidal curves in the -plane - Widely used i 18/59
Hough Transform accumulator cell example with -plane 19/59 Hough Transform Hough Transform Example 20/59
Hough Transform Edge linking with Hough Transform Compute the gradient of an image Threshold it to obtain a binary image Specify subdivisions in the -plane Hough transform for edge pixels Examine the counts of the accumulator cells for high pixel concentrations Examine the relationship between pixels in a chosen cell 21/59 Hough Transform Line detection with Hough Transform 22/59
Hough Transform Generalization Similar transform can be used for finding any shape which can be represented by a set of parameters Possible form of Generalized Hough transform g( v, c) 0 Here, v: coordinate vector, c: parameter vector Example : detection of circle ( x c c 2 2 1 ) ( y c2) 3-dimensional parameter space is used 2 3 23/59 Graph-Theoretic Technique Global processing via graph-theoretic technique Represents edge segments in the form of graph and searching the graph for edge linking Low cost path => significant edge Robust to noise Procedure is considerably complicated Requires more processing time 24/59
Graph-Theoretic Technique Edge element edge element : the boundary between two pixels p and q If we let xy-coordinate of point p and q be (x p, y p ), (x q, y q ), the edge element is defined by ( x p, y p )( xq, yq ) Edge : a sequence of connected elements => requires the decision of edge element and edge linking process 25/59 Graph-Theoretic Technique Detection of edge element The cost function defined by pixels p and q c( p, q) H [ f ( p) f ( q)] H : Highest gray-level value in the image f(p), f(q) : the gray-level values of p and q 26/59
Graph-Theoretic Technique Linking of Edge element Edge linking : Finding a minimum cost path 27/59 Graph-Theoretic Technique Noisy image and edge boundary 28/59
Graph-Theoretic Technique Noisy image and edge boundary 29/59 Image Segmentation Similarity based segmentation Thresholding : simple and widely used - Global thresholding - Adaptive thresholding - Local thresholding Region based segmentation : finding the regions directly - Region Growing : group pixels or subregions into larger regions based on predefined criteria - Region splitting and merging : subdivide an image initially into a set of arbitrary, disjointed regions, and then merge/split the regions to satisfy the predefined conditions 30/59
Thresholding Concept Single threshold : a threshold T separates and extract the objects from the background if f(x,y) > T then object point else background point Multiple threshold : region growing is more proper algorithm if T1 < f(x,y) <= T2 then object1 else if f(x,y) > T2 then object2 else background 31/59 General form of threshold value T Thresholding T T[ x, y, p( x, y), f ( x, y)] f(x, y) : gray level of point (x, y) p(x, y) : some local property of point (x, y) ex) average gray level of a neighborhood centered on (x, y) A thresholded image g(x,y) is defined as g( x, y) 1 0 if if f ( x, f ( x, y) T y) T 32/59
Thresholding Global thresholding T T[ f ( x, y)] Local thresholding T T[ p( x, y), f ( x, y)] Dynamic thresholding, adaptive thresholding T T[ x, y, p( x, y), f ( x, y)] 33/59 Role of illumination Thresholding 34/59
Thresholding Global thresholding Highly controlled environment is required : industrial inspection applications Simplest Approach 35/59 Selection of global thresholding value T heuristic approach automatic approach automatic approach Thresholding 1. Select an initial estimate for T 2. Segment the image using T. This will produce two groups of pixels : pixels in G 1 > T, pixels in G 2 T 3. Compute the average gray level values, 1, 2 for the pixels in regions G 1, G 2 4. Compute the new threshold value : 5. Repeat steps 2 through 4 until the difference in T in successive T 1 2 ( 1 2 ) iterations is smaller than a predefined parameter T 0 36/59
Thresholding automatic thresholding result 37/59 Thresholding Adaptive thresholding Uneven illumination : global threshold can not segment perfectly Divide original image into subimages and utilize different threshold to segment each subimage 38/59
Adaptive thresholding Thresholding Failed subimage further subdivided into much smaller subimages - adaptive 39/59 Thresholding Optimal thresholding a method for estimating thresholds that produce the minimum average segmentation error. 40/59