Chapter - 3 : IMAGE SEGMENTATION Segmentation subdivides an image into its constituent s parts or objects. The level to which this subdivision is carried depends on the problem being solved. That means segmentation should stop, when the objects of interest in an application have been isolated. Segmentation algorithm for monochrome images generally are based on one of two basic properties of gray level values: discontinuity and similarity. In the first category, the approach is to partition an image based on abrupt changes in gray level. The principal areas of interest within this category are detection of isolated points and detection of lines and edges in an image. In the second category, the principal approach is threshold, region growing and region splitting and merging. The concept of segmenting an image based on discontinuity or similarity of the gray level values of its pixels is applicable to both static & dynamic (time varying) images. --------------------------------------------------------------------------------------------------- Q() Explain basic principles of detecting following in the images () Points () Lines (3) Edges (0M May 07 IT) Three basic types of discontinuities in a digital image are i) point, ii) lines, iii) edges. i) Point Detection : - The detection of isolated point different from constant background image can be done using the following mask. - - - - 8 - - - - A point has been detected at the location on which the mask is centered if R > T where T is a non negative threshold and R = wz + wz + +w9z9 IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 39 39
The idea is that the gray level of an isolated points will be quite different from the gray levels of its neighbours. The mask operation measure the weighted differences between the center point and its neighbours. The differences that are large enough as determined by T considered isolated points in image of interest. ii) Line Detection : - Line masks - - - - - - - - - - - - - - - - - - - - - - - - (a) Horizontal (b) 4 degrees (c) vertical (d) -4 degrees If the mask (a) is moved around an image, it would respond more strongly to lines oriented horizontally. With constant background, the maximum respond would result when the line is passing through the middle row of the mask. Now, Let R, R, R3, R4 denotes the responses of the above 4 masks. Suppose that all masks are run through image. If at a certain point in the image Ri > Rj for all j i that point is said to be more likely associated with a line in the direction of mask(i). For ex. if at a point (x,y) in the image if R > Rj for j =, 3, 4. i.e. R > R R > R3 R > R4 That particular point is said to be more likely associated with a horizontal line. iii) Edge Detection :- Edges characterize object boundaries are therefore useful for segmentation and identification of objects in scene. Edge point can be thought of as pixel location of abrupt gray level changes. It is the boundary between two regions with relatively distinct gray level properties. Edge Detection by derivative operators :- * * * * * IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 40 40
() Gradient Operators :- The gradient of an image f(x,y) at any pt. (x,y) is the vector The Magnitude of gradient is, This quantity equals the max. rate of increase of f(x,y) per unit distance in the direction of Common practice is to approximate the gradient with absolute values. The Direction of gradient vector :- Let (x,y) represent the direction angle of the vector at (x,y). Then from vector analysis, where angle is measured w.r.t the x-axis. The computation of the gradient of an image is based on obtaining the partial derivative and at every pixel location. Derivative based on Sobel operators masks are, Gx = ( z7+ z8 + z9 ) ( z + z + z3 ) Gy = ( z3 + z + z9 ) ( z + z4 + z7 ) where z to z9 are the gray levels of the pixels overlaps by the mask at any location in an image. z Z z3 - - - - 0 z4 Z z 0 0 0-0 z7 Z8 z9-0 3*3 image region mask used to compute Gx mask used to compute Gy Some common Gradient operators :- IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 4 4
Gx Gy Robert 0 - - 0 0 0 - - - - 0 Prewit 0 0 0-0 - 0 - - - 0 Isotropic 0 0 0 0 Fri-Chen - 0 () Compass Operators :- Compass operators measure gradients in a selected number of directions as given below- 0 0-0 - - 0 0 0 0-0 - 0 - - - - 0 - - 0-0 - - - - - 0-0 0 0 0 0-0 - 0-0 0-0 - - 0 Four different compass gradient for NORTH going edges. () () - -3 0-3 - - - -3-3 -3 (3) (4) 0 0 0 0 0 0 - - - - - - Each clockwise circular shift of elements about the center, rotates the gradient direction by 4 degrees. IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 4 4
(3) Laplacian Operator :- The foregoing methods of estimating the gradients work best when the gray level transition is quite abrupt like a step function. As the transition region gets widens it is more advantageous to apply the second order derivatives. One frequently encountered operator is the Laplacian operator defined as, () 0-0 () - - - (3) - - 4 - - 8 - - 4-0 - 0 - - - - Q() What is an edge? What is a directional edge detector? How can you implement it? Where will you use a directional edge detector? Give edge detector masks for detecting edges. (0M May0 Comp) Q(3) Explain operation and application of each of the following: Give 3x3 mask wherever applicable ] Laplacian (M May0 Etrx) ] Horizontal line detection (M May0 Etrx) Q(4) Show that subtracting the Laplacian from an image is proportional to unsharp masking. (M May0 I.T) Q() Short note: Edge linking and Boundary detection via graph theoretic Technique. (M Dec 04 Comp) (M May0 I.T) Q() Can Graph theoretic technique be used in Image Segmentation? If so, how? (M Dec0 Comp) As the gradient is a derivative, the operation is seldom suitable as a preprocessing step in situation characterized by high noise contents. Graph theoretic method provides a rugged approach that performs well in the presence of noise. The procedure is considerably more complicated and requires more processing time. Each edge element defined by pixels p & q has an associated cost, defined as, c(p,q) = H f(p) f(q) where H is the highest intensity value in the image; f(p) and f(q) are the intensity of pixel p & q respectively. IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 43 43
Q(7) What is the most typical problem of edge based segmentation? (4M May03 Etrx) Q(8) Justify /contradict the following statements: Mini. cost algorithm is meant for detection of edges. (M May0 Comp) Q(9) The 3x3 matrix shown below is frequently used to compute the derivative in x direction at each point in an image. Give an ALU procedure to implement this operation. (0M Dec0 Etrx) - - - 0 0 0 Q(0) TRUE or FALSE and Justify a) Poorly illuminated images can be easily segmented.(3m May0 Etrx) b) Image resulting from poor illumination cannot be segmented easily. (8M May0 Comp) (4M May07 IT) Q() Explain following methods of image segmentation. a) Region growing. (8M May0 Etrx) b) Splitting and merging. (0M May 07 I.T.) Q() Explain region oriented segmentation. [0 M, MAY 07, ETRX] The objective of segmentation is to partition an image into regions. In this section, segmentation is done by finding the regions directly. Let R represents the entire image region segmentation as a process that partitions R into n subregions R, R,--------Rn such that ) ) Ri is connected region I =,.n 3) (4) P(Ri) = TRUE for all I =,.n and There are two different approaches for region oriented segmentation. () Region Growing by Pixel Aggregation :- Region growing is a procedure that groups pixels or subregions into larger regions. Pixel aggregation procedure starts with a set of seed point and from these grows region by appending for each seed point those neighbouring pixels that have similar proportion. IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 44 44
() Region Splitting & Merging :- In this method an image is first subdivided into a set of arbitrary disjointed region and then merge and/or split the regions. Let R represent the entire image region and then select a predicate P. For image one approach for segmenting R is to subdivide it successively into smaller and smaller quadrant region so that for any region Ri, P(Ri)=TRUE. i.e. if P ( R ) = FALSE, divide the image into quadrants. If P is False for any quadrant, subdivide that quadrant into subquadrants and so on. Algorithm : ) Split into four disjointed quadrants any region Ri when P(Ri) = False. ) Merge any adjacent regions Rj & Rk for which P(Rj U Rk) = TRUE 3) Stop when no further merging or splitting is possible. Refer... Q(3) For image given below perform segmentation using region growing by pixel aggregation. Choose appropriate threshold and seed points. F 7 = 4 4 7 7 4 4 3 4 7 0 4 0 3 3 0 0 4 3 3 3 Q(4) Explain region splitting and merging technique for image segmentation. (M Dec0 Etrx) (0M Dec0 IT) (0M May0 Comp) Q() Explain segmentation based on discontinuity and segmentation based on similarities. [8 M, MAY 07, ETRX] Q() Write short notes on : Split and Merge (M Dec0 IT) IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 4 4
Q(7) Assume that the edge in the gray level image starts in the first column and ends in the last column. Find the cost of all possible edges and sketch them. Find the edge corresponding to minimum cost path: (M May0 Etrx) 7 7 0 Q(8) Assume that the edge starts in the first row and ends in the last row for the following Gray image--- (0M Dec0 IT) Sketch all possible paths and determine the edge corresponding to minimum cost Path. 7 7 0 3 Q(9) Assuming that edge starts in the first row and ends in the last row. For the following gray level image, sketch all possible paths and determine edge corresponding to minimum cost path. (0M May 07 I.T.) 7 7 0 Q(0) Design compass gradient operator of the size 3x3 to measure gradients of edges oriented in eight directions. E, NE, N, NW, W, SW, S and SE. Give the form of these eight operators using coefficients valued 0,, or - also specify the gradient direction of each mark. [8M, MAY 07, ETRX] Q() Explain the main conceptual differences in edge based and region based approaches to Image segmentation. (4M May03 Etrx) Q() Explain why prior information about edge detections may increase the speed of Hough transform-based Image segmentation. (4M May03 Etrx) Q(3) Hough Transform [M MAY 07, Etrx] (M Dec0 Comp)(M Dec0 IT) Q(4) What is threshold? Explain how to obtain the threshold for image segmentation. (M Dec0 Etrx) () Simple Global Thresholding :- The simplest of all thresholding technique is to partition the image histogram by using a single threshold T. Segmentation is accomplished by scanning the image pixel by pixel and labeling each pixel as object or background, depending on whether the gray level of that pixel is greater or less than the value of T. IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 4 4
ALGORITHM to calculate Threshold: I. Select an initial estimate for T II. Segment the image using T. This will produce two groups of pixels : G consisting of all pixels with gray level values > T and G consisting of all pixels with gray level values <= T. III. Compute the average gray level values µ and µ for the pixels in the regions G and G. µ + µ IV. Compute a new threshold value, T = V. Repeat steps II through IV until the difference in T is successive iterations is smaller than a predefined parameter T0 () Basic Adaptive Thresholding An uneven transformation can transform a perfectly segmentable histogram into a histogram that can not be partitioned effectively by a Single global threshold. Basic adaptive thresholding is an attempt to solve such problem. Subdivide the orginal image into small areas. Utilize a different threshold to each sub-images. Since the threshold used for each pixel depends on the location of the pixel in terms of the subimages, this type of thresholding is adaptive. (3) Threshold Selection based on Boundary Characteristics :- The gradient f at any point (x,y) in an image is given by, f = G + G G + x y x G y and f = + G x G y Those two quantities are used to form a three level image. s( x, y) = 0 + if if if f f f < T T T AND AND f f 0 < 0 IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 47 47
The symbols 0, +, - represent any three distinct gray levels, T is a threshold and the gradient & laplacian are computed at every point (x,y). For a dark object on light background the above technique produces an image S(x,y) in which all pixels that are not on an edge are labeled 0, all pixels on the dark side of an edge are labeled + and all pixels on the light sides of an edge are labeled -. The information obtained by using this procedure can be used to generate a segmented, binary image in which objects of interest and 0 background. The transition from a light background to a dark object must be characterised by the occurances of a followed by + in s(x,y). The interior of the object is composed of pixels that are labeled either or. Finally, the transition from the object to background is characterised by the occurances of + followed by. All the pixels that are not on the boundary are labeled o, all the pixels that are on dark side of an edge are labeled + and all the pixels that are on light side of an edge are labeled Q() Segment the following given image using Split and Merge technique. Predicate : All the pixels in the region must have same intensity value. Step I : Image Splitting () For region R Entire region is neither Black nor White. Predicate ( R ) = FALSE Split R into 4 parts. IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 48 48
R R R R 3 R 4 () For subregion R Entire Region R is white. Predicate ( R ) = TRUE. (3) For subregion R Entire region is neither Black nor White. Predicate ( R ) = FALSE Split R into 4 parts. R R R R 3 R 4 Sub regions R, R, R 3 and R 4 are single pixel regions. And Predicate( Single Pixel Region ) is TRUE. (4) For subregion R 3 Entire Region R 3 is Black. Predicate ( R 3 ) = TRUE. () For subregion R 4 Entire region is neither Black nor White. Predicate ( R 4 ) = FALSE Split R into 4 parts. R 4 R 4 R 4 R 43 R 44 Sub regions R 4, R 4, R 43 and R 44 are single pixel regions. Result of Image Splitting : R R R R R 3 R 4 R 4 R 4 R 43 R 44 IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 49 49
Step I I : Image Merging () For sub regions R, R, R 3 and R 4 Adjacent Regions : { R, R }, { R, R 3 },{ R, R 4 }, (a) Region { R U R 4 } is completely Black. Predicate { R U R 4 } = TRUE Merge { R U R 3 } Let M = { R U R 3 } (b) Region { R 3 U M } is completely Black. Predicate { R 3 U M } = TRUE Merge { R U M } Let M = { R U M } R M () For sub regions R 4, R 4, R 43 and R 44 Adjacent Regions : { R 4, R 43 }, { R 4, R 44 }, (a) Region { R 4 U R 43 } is completely Black. Predicate { R 4 U R 43 } = TRUE Merge { R 4 U R 43 } Let M 3 = { R 4 U R 43 } (b) Region { R 4 U R 44 } is completely White Predicate { R 4 U R 44 } = TRUE Merge { R 4 U R 44 } Let M 4 = { R 4 U R 44 } M 3 M4 R R M R 3 M 3 M 4 IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 0 0
(3) For regions R, R, R 3, M, M 3 and M 4 Adjacent Regions : { R, R }, { M, M 3 }, { R 3, M 3 } (a) Region { R U R } is completely White. Predicate { R U R } = TRUE Merge { R U R } Let M = { R U R } (b) Region { M U M 3 } is completely Black Predicate { M U M 3 } = TRUE Merge { M U M 3 } Let M = { M U M 3 } (c) Region { R 3 U M } is completely Black Predicate { R 3 U M } = TRUE Merge { R 3 U M } Let M 7 = { R 3 U M } M M 7 M 4 ANS IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com
Q() For the following given image, apply split and merge technique and segment the image. Solution : Predicate : All the pixels in the region must have same intensity value. Step I : Image Splitting () For region R Entire region is neither Black nor White. Predicate ( R ) = FALSE Split R into 4 parts. R R R R 3 R 4 (7) For subregion R Entire Region R is white. Predicate ( R ) = TRUE. (8) For subregion R Entire region is neither Black nor White. Predicate ( R ) = FALSE Split R into 4 parts. R R R R 3 R 4 Sub regions R, R, R 3 and R 4 are single pixel regions. And Predicate( Single Pixel Region ) is TRUE. (9) For subregion R 3 Entire region is neither Black nor White. Predicate ( R 3 ) = FALSE Split R into 4 parts. R 3 R 3 R 3 R 33 R 34 Sub regions R 3, R 3, R 33 and R 34 are single pixel regions IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com
(0) For subregion R 4 Entire Region R is Black. Predicate ( R 4 ) = TRUE. Result of Image Splitting : R R R R R 3 R 4 R 3 R 3 R 4 R 33 R 34 Step I I : Image Merging () For sub regions R, R, R 3 and R 4 Adjacent Regions : { R, R }, { R, R 3 },{ R, R 4 }, (a) Region { R U R 4 } is completely Black. Predicate { R U R 4 } = TRUE Merge { R U R 3 } Let M = { R U R 3 } (b) Region { R 3 U M } is completely Black. Predicate { R 3 U M } = TRUE Merge { R U M } Let M = { R U M } R M () For sub regions R 3, R 3, R 33 and R 34 Adjacent Regions : { R 3, R }, { R 3, R 33 },{ R 3, R 34 }, (a) Region { R 3 U R 34 } is completely Black. Predicate { R 3 U R 34 } = TRUE Merge { R 3 U R 33 } Let M 3 = { R 3 U R 33 } (b) Region { R 3 U M } is completely Black. Predicate { R 33 U M 3 } = TRUE Merge { R 3 U M 3 } Let M 4 = { R 3 U M 3 } IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 3 3
R 3 M 4 R R M R 3 M 4 R4 (3) For regions R, R, R 3, M and M 4 Adjacent Regions : { R, R }, { R, R 3 },{ R, M 4 }, { R, M }, { R, M }, {M, R 4 },{ R 3, M 4 }, { M 4, R 4 }, (a) Region { R U R } is completely White. Predicate { R U R } = TRUE Merge { R U R } Let M = { R U R } (b) Region { M U R 4 } is completely Black Predicate { M U R 4 } = TRUE Merge { M U R 4 } Let M = { M U R 4 } (c) Region { M 4 U M } is completely Black Predicate { M 4 U M } = TRUE Merge { M 4 U M } Let M 7 = { M 4 U M } (d) Region { R 3 U M } is completely White Predicate { R 3 U M } = TRUE Merge { R 3 U M } Let M 8 = { R 3 U M } M 8 M7 ANS IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com 4 4
Q(7) What is an edge? What is a directional edge detector? How can you implement it? Where will you use a directional edge detector? Give edge detector masks for detecting edges. (0M May0 Comp) Directional Edge Detectors are Derivative Operators [ i.e. First order and second order derivative operators] Directional Derivative Operators are used to detect Step and Ramp edges. Explain merits and demerits of each operators. Write mask of each of them. Q(8) Show that Laplacian is not a good edge detector. Because the derivative operator acts as a highpass filter, second order derivative edge detectors are more sensitive to noise. It is easy for noise inherent in an image to corrupt the real edges by shifting their apparent locations and by adding many false edge pixels. Very small local peaks in the first derivative will result in zero crossings in the second derivative, Q() Explain Unsharp Masking. Unsharp masking is one of the techniques typically used for edge enhancement. In this approach, a smoothened version of the image is subtracted from the original image; hence, tipping the image balance towards the sharper content of the image. The procedure to perform unsharp masking is given below: Blur filter the image. Subtract the result obtained from step from the original image. 3 Multiply the result obtained in step by some weighting fraction. Add the result obtained in step 3 to the original image. Q(9) Show that subtracting the laplacian from an image is proportional to unsharp masking.. z z z3 Let f(x,y) be the original Image. f ( x, y) = z4 z z z 7 z8 z9 0 0 The 4 directional Laplacian mask is given by, 4 0 0 The response of the mask at (x,y) position is given by, R = Z + Z 4 + Z +Z 8 4 Z Subtracting the laplacian from an image we get, IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com
Let R = f(x,y) R R = Z ( Z + Z 4 + Z + Z 8 4 Z ) R = Z ( Z + Z 4 + Z + Z 8 +Z )+ Z + 4 Z ) R = Z ( Z + Z 4 + Z + Z 8 +Z ) R = { / Z / ( Z + Z 4 + Z + Z 8 +Z ) } R = {. Z Mean } RHS gives the equation of unsharp masking. Thus subtracting the laplacian from an image is equivalent to unsharp masking. Q(30) What is the most typical problem of edge based segmentation? (4M May03 Etrx) Edge-based segmentations rely on edges found in an image by edge detecting operators -- these edges mark image locations of discontinuities in gray level, color, texture, etc. The most common problems of edge-based segmentation are an edge presence in locations where there is no border, and no edge presence where a real border exists. (false alarms and missed detections) Q(3) Explain why prior information about edge detections may increase the speed of Hough transform-based Image segmentation. (4M May03 Etrx) In the most general case, nothing is known about lines in the image, and therefore lines of any direction may go through any of the edge pixels. In reality, the number of these lines is infinite, however, for practical purposes, only a limited number of line directions may be considered. With prior information about edge detections it is possible to consider limited number of line directions that reduces the complexity and aid the Hough transform-based Image segmentation Q(3) Explain the main conceptual differences in edge based and region based approaches to Image segmentation. (4M May03 Etrx) In edge based segmentation, the region is partitioned based on abrupt discontinuities in the intensity values. Abrupt discontinuities characterizes the edges in the image. In region oriented segmentation, region is partitioned based on predefined similarity of intensity values. Here, pixel values similar in amplitude are grouped in the same region. IP Help Line : 9 9 8 7 0 3 0 8 8 www.guideforengineers.com