Digital Image Processing Lecture 7 p. Segmentation and labeling of objects p. Segmentation and labeling Region growing Region splitting and merging Labeling Watersheds MSER (extra, optional) More morphological algorithms from G&W Gonzales & Woods: Chapter 0 pp. 76-778 Chapter 9 pp. 64-664 or Paperbac 0: Add to the page numbers above. Gray scale image Segmented and labeled image? Maria Magnusson, Computer Vision Lab., Dept. of Electrical Engineering, Linöping University Segmentation subdivides an image into its constituent regions or objects. p. p. 4 Methods for segmentation Point, line and edge detection Thresholding Region-based segmentation Region growing Region splitting and merging Watershed segmentation Labeling The RT-algorithm (Run-trac) Lab The RB-algorithm (Raster-scan Border-follow) Labeling, different algorithms The RT-algorithm (Run-trac) ast The RB-algorithm (Raster-scan Border-follow) Not as fast as the RT-algorithm Gives more information There are also other methods, for example: lood fill from computer graphics Gonzalez & Woods: Extraction of connected components, see figure 9.7 and 9.8. now already today later
The RB-algorithm (Raster-scan Border-follow) (0,0) Raster-scan the image from left to right and from top to bottom. When the scanning reach a border (0-> or ->0), stop the scanning and start border-follow instead. During the border-following process, labels are set to the right of the border. inally all labels are expanded to the right, see next slide. After border-following of the first object After border-following of both objects and one hole p. 5 The RB-algorithm, final result 4 4 5 4 5 5 5 Chain code: 0 (0,0) (,8) 0 0 0 0 0 0 0 (,6) 0 (,6) 0 0 0 more about chain code later 5 Topological graph: p. 6 5 Region growing Start with a set of seed points and expand them to neighboring pixels if they satisfy a certain criterion. Examples of criteria are: Right intensity Right color Right texture or a simple example see next slide. It is from G&W :nd edition, in the :rd edition they have made it a bit too complicated, I thin. a) Original -ray image of a defective weld with cracs. The challenge is to determine the size and location of the cracs. b) The seed points are obtained using a high threshold, T. c) Result of region growing to intensity pixels >T. (T<T.) d) Boundaries of segmented defective weld (blac) overlaid on original image. There is a similarity with thresholding with hysteres! Lab 6 Variants The criteria can also be altered in every step. Ex) if abs(pixelvalue mean(grown region)) < value: Then grow!. The growing can stop at a high magnitude of the gradient. p. 7 Region growing, cont. p. 8 ig. 0.5 a) b) c) d)
Region splitting and merging ) Split into four disjoint quadrants any region R i for which Q(R i )=ALSE. ) When no further splitting is possible, merge any adjacent region R j and R i for which Q(R j U R )=TRUE. (Alternative simplification: Q(R j )=TRUE and Q(R )=TRUE) )Stop when no further merging is possible p. 9 ig. 0.5 Region splitting and merging ig. 0.5 Original image of the Cygnus Loop supernova Smallest Quad- Region: x p. 0 Smallest Quad- Region: 6x6 Smallest Quad- Region: 8x8 TRUE, Q ALSE, if ( std ) a AND otherwise 0 m(mean) b Watershed segmentation, introduction p. Watershed segmentation, introduction p. ig. 0.54 a) The watershed algorithm regards a gray scale image as a landscape, where the laes are preferably dar pixels and the ridges and mountains are preferably bright pixels. A watershed = a ridge between individual laes and rivers. A catchments basin = a geographic area whose water flows to a certain lae. ig. 0.54 h) The watershed algorithm can find the watersheds in the image. They form closed curves and the area inside each closed curve, the catchments basin, can be regarded as a segmented, individual object. a) h) ig. 0.54
Watershed segmentation, description a) b) e) f) c) d) g) h) p. ig. 0.54 Watershed segmentation, description ig. 0.54 b) is a topographic view of 0.54 a). It is intended to give a D-feeling. In the beginning, the landscape in a) and b) is totally dry. Local minima are detected and holes are punched in each of them. Then the entire landscape is flooded from below by letting water rise through the holes at a uniform rate. In c), there is water (light gray) in the bacground. In d), there is also water in the left lae. In e), there is water in both laes. In f), there is the first sign on overflow between two catchments basins. Therefore, a short dam has been built between them. In g), there are rather long dams. inally, the whole landscape is over flooded with water, separated by high dams, watersheds. In h), these watersheds are shown overlaid on the original image. p. 4 Watershed segmentation, dam construction a) b) d (8) = c) :rd dilation :st dilation :nd dilation Dam points p. 5 ig. 0.55 Watershed segmentation, dam construction Let C n- (M ) denote the pixels in lae M at flooding step n-. These are shown to the left in a). Let C n- (M) denote the pixels in lae M at flooding step n-. These are shown to the right in a). looding step n is shown in b). Let C(n-) denote the union of C n- (M ) and C n- (M ). There are objects in C(n-) in a) and object in C(n) in b). This indicates that something have happened in step n and that some action must be performed. Dilate the pixels in a) with the d (8) -structuring element considering the following constraints: Dilation is not allowed outside b). Dilation is not allowed so that the pixels in M and M are united. Instead, if the pixels in M and M are going to be united, denote: dam pixel. The first dilation in c) gives the light gray pixels. The second dilation in c) gives the dar gray pixels and the -mared dam pixels. p. 6
Watershed segmentation, Ex ) p. 7 Watershed segmentation, Ex ) p. 8 a) c) b) d) a) Original image with dar objects that are going to be segmented. b) A usual pre-processing step is to calculate the magnitude of the gradient. c) The result of the watershed algorithm overlaid on the gradient image. d) The result of the watershed algorithm overlaid on the original image. ig. 0.56 Watershed segmentation, Ex ) oversegmentation Extra, optional p. 9 Watershed segmentation, Ex ) oversegmentation a) Original image with dar object to be segmented. Extra, optional Pre-processing by computation of the magnitude of the gradient. Result from the watershed algorithm overlaid on the original image. Oversegmentation! The oversegmentation was probably caused by too many local minima. p. 0 a) b) ig. 0.57
Watershed segmentation, Ex ) oversegmentation cured a) b) Extra, optional p. Watershed segmentation,ex ) oversegmentation cured Extra, optional ig. 0.58 a) The light gray internal marers are regions received by applying a low-pass filter on the original image followed by consideration of the following criteria: The region must be surrounded by pixels of higher values. The region must consist of a number of connected pixels. The points in the region must have approximately the same gray scale value. The external marers are the watersheds received when the watershed-algorithm was applied to the low-pass filtered image. The internal marers where then used as local minima. The external marers in a) divides the original image into regions. Each region is processed individually. The magnitude-of-the-gradient image was calculated, the internal marer was used as local minimum and the watershed-algorithm was applied. The result is shown in ig. 0.58 b)! p. ig. 0.58 Watershed segmentation, Ex ) for overlapping binary objects p. Ex) Generate a distance map of the objects and invert it p. 4 The watershed algorithm can also be used to segment overlapping binary objects. Distance mapping is used to construct a landscape with laes. Computation: See this and the following slides. Multiply with - Bacground=>-00 Lab 5! Binary input image Distance map Inverted distance map
Ex) Detect local min & apply the watershed algorithm on the inverted distance map p. 5 MSER yet another segmentation Method Extra, optional p. 6 Labeled local min Result of the watershed algorithm MSER = Maximally Stable Extremal Region. Somewhat similar to the watershed algorithm. The MSER detector finds regions that are stable over a wide range of thresholds of a grayscale image. Teached in the course TSBB5 Computer Vision. Popular at our CVL division, e.g. used as a part of a method for traffic sign recognition, see figure (thesis by redri Larsson). J. Matas, O. Chum, M. Urban, and T. Pajdla: "Robust wide baseline stereo from maximally stable extremal regions." In Proc. of British Machine Vision Conference, pages 84-96, July 00. Some basic morphological algorithms: ) Boundary extraction p. 7 ) inding connected components p. 8 Extra object does not matter The image 0 contains the initial seed, which is iteratively dilated by B and multiplied by A. ig. 9. Binary multiplication A A AB The object A eroded by B (9.5-) ig. 9.4 The object - dilated by B B A,,,,... (9.5-) ig. 9.7
) inding connected components Original image Thresholded image p. 9 The eroded image can be rasterscanned until a pixel is found. This pixel is labeled by i and the connected component is reconstructed. Its area (number of pixels) is noted. Then the image is scanned again until the next unlabeled pixel is found and labeled by i+, etc. ) Convex hull An set A is said to be convex if the straight line segment joining any two point in A lies entirely within A. The convex hull of S is the smallest convex set containing S. The element B i found a match union by A i 0 A p. 0 Eroded image ig. 9.8 in the image - i *, Small error in G&W here C A i i i B i,,,...,,,... 4 i D, where D i i i for converging (9.5-4) (9.5-5) ) Convex hull Original image A Convex hull, preliminary result p. Improved result by considering vertical & horizontal directions ig. 9.9 ig. 9.0 4) G&W:s thinning, connectivity preserving shrining (to 8-connective seleton) A B B Thinning of A by B B, B A A, B B... A B A (9.5-6) * B B found a match in A,..., Repeat (9.5-8)!... (9.5-7) B n p. (9.5-8) n B ig. 9.
5) Pruning p. Something similar in Lab? 6a) Geodesic dilation of size p. 4 Original image A H A (9.5-7) B Repeat (9.5-7) times Repeat (9.5-9) times (9.5-9) H A 8 * (9.5-8) B (9.5-0) ig. 9.5 The marer image dilated by B D G multiplied by G B G (9.5-) The marer image contains the initial seed, which is dilated by B and multiplied by the mas G. The mas G limit the growth of marer ig. 9.6 6b) Morphological reconstruction by geodesic dilation iterated until stability D D D : R D when G G p. 5 Similar to (9.5-) G G (9.5-5) 7) Opening by reconstruction Ex) Extracting characters with long vertical stroes O n D R nb (9.5-7) n erosions by B R p. 6 ig. 9.8 Original image erosion by a 5 element ig. 9.9 Resulted image
8) illing holes I x, y, if x,y x, y 0, otherwise is on the border of I H D R I C C p. 7 (9.5-8) (9.5-9) 8) illing holes Original image I p. 8 Complement image I C used as a mas ig. 9.0 Marer image, white frame Resulted image ig. 9. p. 9 9) Border clearing I x, y, if x,y x, y 0, is on the otherwise I R border D I of I (9.5-0) (9.5-) Image with objects touching the the border R D I Resulted image, error in G&W ig. 9.