Biomedical Image Analysis. Mathematical Morphology

Transcription

1 Biomedical Image Analysis Mathematical Morphology Contents: Foundation of Mathematical Morphology Structuring Elements Applications BMIA 15 V. Roth & P. Cattin 265

2 Foundations of Mathematical Morphology The foundation of Mathematical Morphology lies in set theory It aims at analyzing and manipulating the shape of objects It is very useful and often used in Computer Vision. BMIA 15 V. Roth & P. Cattin 266

3 What is it good for...? Image enhancement, segmentation, restoration Image compression Edge detection Shape analysis Post-processing of regions: thinning, hole filling, boundary extraction, extraction of connected components,... BMIA 15 V. Roth & P. Cattin 267

4 Some Basic Concepts from Set Theory Let A be a set in Z 2. If a = (a 1, a 2 ) is an element of A, then we write a A, else a / A. The set with no elements is called the null or empty set. A set is specified by { }. In our context: elements of sets are coordinates of pixels representing objects in an image. Example: C = {w w = d, for d D} means the set of elements, w, such that w is formed by multiplying each of the two coordinates of all the elements of set D by 1. If every element of A is also an element B, then A B. BMIA 15 V. Roth & P. Cattin 268

5 Some Basic Concepts from Set Theory The union C = A B is the set of all elements belonging to either A, B, or both. The intersection D = A B is the set of all elements belonging to both A and B. Two sets A and B are disjoint or mutually exclusive if they have no common elements A B =. The complement A c = {w w / A} is the set of elements not contained in A. The difference A B = {w A, w / B} = A B c is the set of elements that belong to A, but not to B. BMIA 15 V. Roth & P. Cattin 269

6 Some Basic Concepts from Set Theory BMIA 15 V. Roth & P. Cattin 270

7 Sets: Reflections and translations The reflection of set B is ˆB = {w w = b, for b B}. The translation of set A by point z = (z 1, z 2 ) is (A) z, = {c c = a + z, for a A}. BMIA 15 V. Roth & P. Cattin 271

8 Morphological Operators The primary morphological operators are Dilation expansion Erosion shrinkage More sophisticated morphological operators, such as Opening and Closing can be designed by combining erosions and dilations. BMIA 15 V. Roth & P. Cattin 272

9 Structuring Element Structuring elements (SEs) are small set of images used to probe the image under study. BMIA 15 V. Roth & P. Cattin 273

10 Morphological Operation Example Create a new image by running B over A so that the origin of B visits all image locations of A If B is completely contained in A mark that location as black, else mark it white If B is on a border element, some part of B is not contained in A elimination of border element. Result: boundary of the object is eroded. BMIA 15 V. Roth & P. Cattin 274

11 Erosion The Erosion of A by the structuring element B is the set of all points z such that B, translated by z, is contained in A: A B = {z (B) z A} BMIA 15 V. Roth & P. Cattin 275

12 Erosion Example (a) With Otsu segmented noisy image (b) Results after eroding image (a) Erosion successfully removed all the small noisy spots. But the gaps in the central object also got bigger! BMIA 15 V. Roth & P. Cattin 276

13 Erosion Example 2 BMIA 15 V. Roth & P. Cattin 277

14 Dilation The Dilation of A by B is the set of all displacements, z, such that ˆB and A overlap by at least one element: A B = {z ( ˆB) z A } BMIA 15 V. Roth & P. Cattin 278

15 Dilation Example (a) With Otsu segmented noisy image (b) Result of dilating image (a) Dilation successfully filled all the small gaps in the central object. But it also amplified the noise! BMIA 15 V. Roth & P. Cattin 279

16 Dilation Example 2 BMIA 15 V. Roth & P. Cattin 280

17 Duality of Erosion and Dilation Erosion and dilation are dual to each other with respect to set complementation and reflection, that is (A B) c = A c ˆB and (A B) c = A c ˆB The duality property is particularly useful when the SE is symmetric, so that ˆB = B. We can then obtain the erosion of an image by simply dilating its background, i.e. dilating A c, and complementing the result. BMIA 15 V. Roth & P. Cattin 281

18 Opening and Closing Erosion and dilation form the basis for two more sophisticated morphological operations: Opening: smoothens the contour, breaks narrow isthmuses, and eliminates thin protrusions Closing: also smoothens the contour, fuses narrow breaks, eliminates small holes, fills gaps in contours Opening of A by SE B: A B = (A B) B opening = erosion + dilation. Closing: A B = (A B) B closing = dilation + erosion. BMIA 15 V. Roth & P. Cattin 282

19 Opening Morphological opening has a simple geometric interpretation. The boundary of A B is established by rolling the SE B inside the contour A. The result is obtained by all the points that could be reached by B. BMIA 15 V. Roth & P. Cattin 283

20 Closing Morphological closing has a similar geometric interpretation as opening, except that we roll B on the outside of the boundary. BMIA 15 V. Roth & P. Cattin 284

21 Duality As with dilation and erosion, opening and closing are dual to each other, that is (A B) c = A c ˆB and (A B) c = A c ˆB The duality property is particularly useful when the SE is symmetric, so that ˆB = B. We can then obtain the opening by simply closing its background, i.e. closing A c, and complementing the result. BMIA 15 V. Roth & P. Cattin 285

22 Opening Example In this example we apply opening in the hope to separate more cells by cutting small isthmuses Indeed two more cells could be separated into disconnected components. (a) Binary image of yeast cells (b) Opening result BMIA 15 V. Roth & P. Cattin 286

23 Opening and Closing Example Erosion successfully removes the spots, but it also increases the wholes in the main body. Opening removes the spots while retaining the gap sizes. A further closing step then finally closes these gaps as well. (a) Erosion (b) Opening (c) Opening then closing BMIA 15 V. Roth & P. Cattin 287

24 Example 2 BMIA 15 V. Roth & P. Cattin 288

25 The Hit-or-Miss transform The Hit-or-Miss transform is the basis for shape detection. Assume set A = C D E (Fig. 9.12). Objective: find location of shape D. Let D be enclosed by a small window W. The local background of D w.r.t. W is the set difference (W D). Erosion of A by D (Fig. 9.12(d)) gives the set of all locations of the origin of D at which D found a hit in A. BMIA 15 V. Roth & P. Cattin 289

26 The Hit-or-Miss transform (2) Erosion of the complement A c by background (W D) gives the set of all locations of the origin of the background at which (W D) found a hit in A c. Intersection of A D and A c (W D) gives set of locations where D (and background) exactly fits inside A. Generalization: Set B = B 1 (object) B 2 (background) set of matches = (A B 1 ) (A c B 2 ). Underlying idea: two objects are distinct only if they form disjoint sets. This is guaranteed by by requiring that each object have at least a one-pixel background around it. BMIA 15 V. Roth & P. Cattin 290

27 The Hit-or-Miss transform BMIA 15 V. Roth & P. Cattin 291

28 Boundary Extraction The boundary of set A is denoted as β(a) and can be obtained by first eroding A by B and then performing the set difference, thus β(a) = A (A B) where A is the binary image and B the structuring element. BMIA 15 V. Roth & P. Cattin 292

29 Boundary Extraction Example (a) Segmented yeast cells (b) Their morphological boundary BMIA 15 V. Roth & P. Cattin 293

30 Hole Filling A hole is a background region surrounded by a connected border of foreground pixels. Let A denote a set whose elements are 8-connected boundaries, each of which encloses a hole. Goal: Given one initial point in each hole, fill all the holes. Define array X 0 of 0s (same size as array containing A). Set locations in X 0 corresponding to the initial points inside holes to 1. BMIA 15 V. Roth & P. Cattin 294

31 Hole Filling Iterate X k = (X k 1 B) A c, k = 1, 2,..., where B is the symmetric SE in Fig. 9.15(c). Without the intersection operation, the dilation would fill the entire area. Intersection with A c limits result to inside the region of interest (the holes). Example of conditional dilation. BMIA 15 V. Roth & P. Cattin 295

32 Hole Filling BMIA 15 V. Roth & P. Cattin 296

33 Hole Filling Example BMIA 15 V. Roth & P. Cattin 297

34 Connected Components Extraction of connected components from binary images is central to many applications. Let A denote a set containing one or more connected components. Goal: Extract all connected components. Assume we have one initial point in each component. Define array X 0 of 0s (same size as array containing A). Set locations in X 0 corresponding to the initial points inside connected components to 1. BMIA 15 V. Roth & P. Cattin 298

35 Connected Components Iterate X k = (X k 1 B) A, k = 1, 2,..., where B is the symmetric SE in Fig. 9.17(a). Terminates when X k = X k 1, with X k containing all the connected components. Again: without intersection operation, the dilation would fill the entire area. Intersection with A limits result to inside the region of interest (the foreground points in the components). BMIA 15 V. Roth & P. Cattin 299

36 Connected Components BMIA 15 V. Roth & P. Cattin 300

37 Connected Components Example BMIA 15 V. Roth & P. Cattin 301

38 Other Mathematical Morphology Operators Convex Hull Thinning & Thickening Skeletons Pruning etc. etc. BMIA 15 V. Roth & P. Cattin 302

39 Morphological Reconstruction Powerful transformation that involves two images and a SE. One image, the marker, contains the starting points for the transformation. Other image, the mask, contains the transformation itself. The SE defines connectivity. BMIA 15 V. Roth & P. Cattin 303

40 Geodesic dilation and erosion Let F be the marker and G be the mask. Both are binary images and F G. The geodesic dilation of size 1 of F w.r.t. G is defined as D (1) G (F ) = (F B) G. The geodesic dilation of size n is defined as D (n) G (F ) = D (1) G [D(n1) G ](F ) with D(0) G (F ) = F. Geodesic erosion is defined analogously with the operation. BMIA 15 V. Roth & P. Cattin 304

41 Geodesic dilation BMIA 15 V. Roth & P. Cattin 305

42 Geodesic erosion BMIA 15 V. Roth & P. Cattin 306

43 Morphological Reconstruction by dilation Idea: Iterate until stability is achieved: RG D (F ) = D(k) G (F ) with k such that D(k) G (F ) = D(k 1) G (F ) BMIA 15 V. Roth & P. Cattin 307

44 Morphological Opening by Reconstruction In a morphological opening, erosion removes small objects and the subsequent dilation attempts to restore the shape of objects that remain. Problem: accuracy of this reconstruction depends on similarity of the shapes of the objects and the SE. Morphological opening by reconstruction restores exactly the shapes of the objects that remain after erosion. Definition: reconstruction by dilation of F from n erosions of F by SE B: O (n) R (F ) = RD F [(F nb)]. Note that F is used as the mask! BMIA 15 V. Roth & P. Cattin 308

45 Morphological Opening by Reconstruction BMIA 15 V. Roth & P. Cattin 309

46 Morphological Reconstruction: hole filling Form a marker image F that is zero, except at the image border (where it contains the flipped pixels): { 1 I(x, y), if (x, y) is on the border of I F (x, y) = 0, else Then, H = [R D I c (F )] c is a equal to I with all holes filled. BMIA 15 V. Roth & P. Cattin 310

47 Morphological Reconstruction: hole filling BMIA 15 V. Roth & P. Cattin 311

48 Morphology of Grayscale Images BMIA 15 V. Roth & P. Cattin 312

49 Morphology of Grayscale Images The erosion of f by a flat structuring element b at location (x, y) is defined as [f b] (x, y) = min (s,t) b similar the definition for dilation is [f b] (x, y) = max (s,t) b {f(x + s, y + t)} {f(x s, y t)} Grayscale erosion shrinks positive peaks. Peaks thinner than the structuring element disappear. It also expands valleys and sinks. Grey-scale dilation has the dual effect. BMIA 15 V. Roth & P. Cattin 313

50 Morphology of Grayscale Images BMIA 15 V. Roth & P. Cattin 314

51 Morphology of Grayscale Images BMIA 15 V. Roth & P. Cattin 315

52 Morphology of Grayscale Images Opening of f by SE b: f b = (f b) b opening = erosion + dilation. Closing: f b = (f b) b closing = dilation + erosion. Duality: (f b) c = f c ˆb (f b) c = f c ˆb, with f c = f(x, y) and ˆb = b( x, y). BMIA 15 V. Roth & P. Cattin 316

53 Morphology of Grayscale Images BMIA 15 V. Roth & P. Cattin 317

54 Morphology of Grayscale Images BMIA 15 V. Roth & P. Cattin 318

55 Morphological Gradient Example This example show the application of morphology to calculate the gradient of a gray-scale image. Idea: Dilation thickens regions, erosion shrinks them. Their difference emphasizes boundaries between regions. (a) Original CT scan A (b) Dilated scan A B (c) Eroded scan A B (d) Morphological gradient (A B) (A B) BMIA 15 V. Roth & P. Cattin 319

56 Morphological Smoothing Opening suppresses bright details Closing suppresses dark details use combinations for image smoothing and noise removal BMIA 15 V. Roth & P. Cattin 320

57 Top-hat and bottom-hat transforms Top-hat of f: f minus its opening: T hat (f) = f (f b) Bottom-hat of f: closing of f -f: B hat (f) = (f b) f Main idea: use structure element in opening /closing that does not fit the objects to be removed. Difference operation then yields image with only the removed objects. Top-hat used for light objects on dark background, bottom hat for the converse. BMIA 15 V. Roth & P. Cattin 321

58 Top-hat Example: illumination correction BMIA 15 V. Roth & P. Cattin 322