(10) Image Segmentation

Transcription

1 (0) Image Segmentation - Image analysis Low-level image processing: inputs and outputs are all images Mid-/High-level image processing: inputs are images; outputs are information or attributes of the images Image analysis: etract useful information from the result of low-level image processing Image segmentation: divides an image into meaningful parts or objects - Image segmentation Two graylevel characteristics are generally considered: * Discontinuities: segment the image according to the abrupt change in graylevels # E.g.: point, line, edge detection * Similarity: segment the image according to some predefined criteria of similarity 0-

2 # E.g.: thresholding, region growing, region splitting and merging (a) Detection of discontinuities - Point detection By a mask operation Mask: Image: w w w 3 z z z 9 3 w 4 w 5 w 6 z 4 z 5 z 6 R = w i z i i= w 7 w 8 w 9 z 7 z 8 z 9 If R T, a point is detected isolated point (significant graylevel difference with neighboring piels) E.g., using the mask: 8 * MATLAB: w = [- - -; - 8 -; - - -]; 0-

3 g = abs(imfilter(double(f), w) >= T; - Line detection E.g., using the following masks: Horizontal (R ) 45 (R ) 45 (R 3 ) Vertical (R 4 ) R i T : one-piel wide line * MATLAB: w = [- - -; ; - - -]; g = abs(imfilter(double(f), w) >= T; % Horizontal line - Edge detection Common method to detect discontinuities because isolated points or one-piel wide lines are rare Edge: the borders of regions with different graylevels Digital edge model: 0-3

4 Perfect edge Ramp edge (more realistic in images) Ramp edge * More clear edge: width of the ramp is narrow; fuzzy edge: width of the ramp is wider * All piels in the ramp are edge points * First derivatives G f # Vector: f = = Gy f y f = mag( f ) = G + G G + # Magnitude: [ ] y y / Determines if a piel is in the ramp edge G 0-4

5 Gy # Direction of gradient: tan G f f * Second derivative: f + y Profile of graylevel edge First derivative Second derivative Zero-crossing 0-5

6 # Sign of second derivative determines if an edge point is on the bright or dark side - There are positive and negative values on a ramp edge - Zero-crossing: intersection of the 0-value line and the line connecting the positive and negative points position of edge Definition: * Edge point: nd order derivative is greater than a threshold * Edge: set of connected edge points (need to predefine connection ) * Edge segment: short edges Implementation: * st order derivative: gradient operator # Sobel operators # Prewitt operators (including slanted line)

7 # Roberts operators * nd order derivative: Laplacian or LoG operator Sobel LoG Thresholded LoG Zero crossing MATLAB bw = edge(im, method, parameters) * method: 'sobel', 'prewitt', 'roberts', 'log', 'zerocross', 'canny' 0-7

8 (b) Thresholding - Thresholding: setting threshold T in histogram to segment the image T Two-modal T T Multimodal Threshold: T[, y, p(, y), f(, y)] (, y): piel coordinate, p(, y): region property, and f(, y): graylevel * Two-modal single thresholding 0 if f (, y) > T g(, y) = g(, y) = : obj 0 ; g(, y) = : obj if f (, y) T * Multimodal multilevel thresholding 0-8

9 0 if T < f (, y) g(, y) = if T f (, y) < T M * T related to f(, y) only: global thresholding * T related to f(, y) and p(, y) only: local thresholding * T related to f(, y), p(, y), and (, y): dynamic (adaptive) thresholding E.g., Tiffany: T, T, T 3, T 4 = 50, 70, 90, 0 E.g., Tool: T = 7 0-9

10 - Global thresholding algorithm. Select an initial threshold T: usually, T = (gray ma + gray min )/. Segment the image using T: two groups of piels G (< T) and G ( T) 3. Compute average intensities of G and G : µ and µ 4. Compute a new threshold T: T = (µ + µ )/ 5. Repeat Steps to 4 until the difference in T in successive iterations is smaller than a predefined parameter T 0 MATLAB: T = 0.5*(double(min(f(:))) + double(ma(f(:)))); % f: input image done = false; while ~done 0-0

11 g = f >= T; Tnet = 0.5*(mean(f(g)) + mean(f(~g))); done = abs(t - Tnet) < 0.5; T = Tnet; end MATLAB graythresh( ): Otsu [979] * Histogram represented by a discrete probability density function: nq pr ( rq ) =, q = 0,,,..., L n where n: total number of piels, n q : number of piels with graylevel r q, and L: number of intensity levels * Suppose threshold k is chosen, two set of piels: C 0 (< k) and C ( k) * Define between-class variance, σ B : σ B = ω0( µ 0 µ T ) + ω( µ µ T ) = ω0ω ( µ µ 0), where ω k = 0 p q ( r q ) q= 0 L p q ( r q ) k ω =, q=, 0-

12 k µ = 0 qpq( r q ) / ω0 µ = qpq( r ) /, q ω, and µ T = q= 0 q= k q= * Objective: find k such that σ B is maimized If σ B is maimized, within-class variance, levels, σ T, are also maimized: W ω σ, σ = T = ( i µ T ) σ = ω0σ 0 + (c) Region-based segmentation - Region-based segmentation Partition an image into regions L L i p i σ W L 0 qp q ( r, and total variance of - Basic formula Let R represent the entire image Segmentation: partition R into n subregions, R, R,, R n, such that q ) 0-

13 n U * R R i= i = Segmentation must be complete * R i is a connected region, i =,, n Points in a region must be connected * R i R j =, for all i and j, i j Regions must be disjoint * P(R i ) = TRUE, i =,, n Piels in a region must satisfy some properties (e.g., similar color) # P(R i ): a logical predicate defined on every point in R i * P(R i R j ) = FALSE for adjacent regions R i and R j R i and R j are different in the sense of predicate P - Region growing Group piels or subregions into larger regions based on predefined criteria for growth Method:. Start with a set of seed points 0-3

14 . Grow regions by appending to each seed those neighboring piels that have predefined properties similar to the seed (e.g., graylevel, teture ) * E.g, points (3, ) and (3, 4) are seeds; property: graylevel difference < 3: a a a b b 8 7 a a a b b a a b b b a a b b b a a b b b Problems: how to select seeds and define properties - Region splitting and merging Partion image into disjoint subregions and then perform splitting and merging Method: subdivide R until subregion P(R i ) = TRUE. Splitting: if P(R) = FALSE, divide R into quadrants (R i, i =,,, 4); if P(R i ) = FALSE, divide R i into subquadrants, and so on Quadtree of quadregions # MATLAB quadtree decomposition: qtdecomp( ) 0-4

15 . Merging: two adjacent regions R j and R k are merged if P(R j R k ) = TRUE stop if no further merging is possible Splitting: R R R R R 3 R 4 R 3 R 4 R R R 3 R 4 R R R 3 R 4 Merging: R R R R 3 R 3 R 4 R R R R 3 R R R R 3 R 4 Problem: translation of the object results in a different quadtree 0-5

16 R R R R R R 4? R 3 R R 3 R 4 R R R 3 R 4 (d) Canny edge detector - Criteria for edge detection Low error rate of detection: all edges should be found and nothing but edges Localization of edges: distance between actual edges in the image and edges found should be minimized Single response: multiple edge piels should not be returned when only a single edge eists 0-6

17 - Canny edge detection algorithm [986]. Gaussian smoothing followed by derivative of Gaussian Smooth image,, and then find possible candidate piels * To reduce computational cost, the filter can be applied to columns first and then to rows (Gaussian filter is separable): # Create a D Gaussian filter g # Create a D filter d g according to the following equation ep( ) σ σ # Convolve g with d g to obtain gd g # Apply gd g to input image producing # Apply (gd g )' to producing # Edge image (magnitude): e = +. Nonmaimum suppression * Thresholding e to remove piels with low edge magnitude will produce multiple edge responses 0-7

18 * Compute the edge direction of piel p (i.e., edge gradient) Edge gradient: g = tan ( / ) * For p to be considered as a true edge piel, p must have a greater magnitude than its neighbors in the edge direction (both ends) # Approach (taking the following figure as an eample) - Compute interpolated magnitudes (m ) of a and b - Compute interpolated magnitudes (m ) of c and d - If e (p) < m or e (p) < m, delete p a c p b 0 d g (p) = 30 Approach Approach 0-8

19 # Approach : - Quantize g (p) to 0, 45, 90, or 35 - If e (p) < e (a) or e (p) < e (d), delete p 3. Hysteresis thresholding * Thresholding with a single value is not appropriate because edges tend to be broken * Two threshold values: a low value t L and a high value t H * If e (p) < t L, p is deleted (nonedge piel) * If e (p) t H, p is marked as an edge piel (edge piel) * If t L e (p) < t H (possible edge piel) and if starting from p and following other possible edge piels can lead to an edge piel, p is also marked as an edge piel 0-9

20 Edge Edge Possible edge Edge Possible edge Nonedge Edge (e) Harris edge/corner detector - Moravec interest point detector Measure directional variance over a small window (auto-correlation): E(, y) = Σ u,v {I +u,y+v I u,v }, = Σ i=~9 {P i Q i } where I is the image, (, y) is the shift of the window, and the center of the 0-0

21 window is regarded as the origin Q v u (0,0) 6 P (, y) Window of (u, v), u, v =, 0, Define 8 shift directions:, y =, 0,, ecluding (0, 0), total response: E = = (,,0, y, y) (0,0) E(, y) 0-

22 If the center is in a smooth local area, E will be small If E T, an interesting point is found - Harris edge/corner detector Moravec s measure is noisy due to the binary window ( s within the window and 0 s elsewhere) * Improvement: use a smooth circular window, e.g, a Gaussian window w u,v = ep (u +v )/σ E(, y) = Σ u,v w u,v {I +u,y+v I u,v } * Typically, a 3 3 Gaussian with σ = 0.5 is w w w w = = w4 w5 w w 7 w8 w Moravec s measure is anisotropic (only 8 discrete directions) * Improvement: use the Taylor series to represent I +u,y+v to achieve 0-

23 isotropy; Taylor epansion of a D function f about a point (, y): f ( +, * Hence, y + y) = f (,! y) + { f (, y) + f (, y) y} { f (, y) + f (, y) y + f (, y) y } + L E(, y) = Σ u,v w u,v {I u,v + X + yy + O(, y ) + I u,v } = Σ u,v w u,v { X + yy + O(, y ) + }, where the derivatives are approimated by X = I [, 0, ] = I/ Y = I [, 0, ] T = I/ y ( : convolution) * Denoting and y by and y, respectively, and ignoring high order terms: E(, y) = Σ u,v w u,v { X + yy} = y y + yy 0-3

24 0-4 Σ u,v w u,v {X + XYy + Y y } = A + Cy + By, where A = X w, B = Y w, and C = (XY) w. * Rewritten: E(, y) = [ y] M [ y] T, where = B C C A M * A linear algebra problem: eigan values and eigan vectors M = λ where M is a matri, is a vector, and λ is a scalar A vector transformed by a matri is equal to the vector multiplied by a scalar # E.g., = = = 4, 4 M λ 0 4) ( 0 ) ( or 4 = + = = + = + λ λ λ λ λ = 3, λ = : eigan values with corresponding eigan vectors

25 * Geometric meaning: Direction of the fastest change Direction of the slowest change (λ ma ) -/ (λ min ) -/ Define the corner response by matri determinant and trace (avoid computing eigenvalues): R = Det(M) k Tr(M) = (AB C ) k(a+b) 0-5

26 where k 0.04 (Note: R = λ λ k(λ + λ )) R depends only on eigenvalues of M * R is large for a corner * R is negative with large magnitude for an edge * R is small for a flat region Iso-response contours of R on the λ λ -plane: β R = R 3 R = R R = R α 0-6

27 Classification of image points using eigenvalues of M: λ Edge λ >> λ Corner λ and λ are large, λ ~ λ ; E increases in all directions λ and λ are small; E is almost constant in all directions Flat region Edge λ >> λ λ 0-7