Image Segmentation Using Topological Persistence

Transcription

1 Image Segmentation Using Toological Persistence David Letscher and Jason Fritts Saint Louis University Deartment of Mathematics and Comuter Science {letscher, Abstract. This aer resents a new hybrid slit-and-merge image segmentation method based on comutational geometry and toology using ersistent homology. The algorithm uses edge-directed toology to initially slit the image into a set of regions based on the Delaunay triangulations of the oints in the edge ma. Persistent homology is used to generate three tyes of regions: -ersistent regions, -transient regions, and d-triangles.the-ersistent regions corresond to core objects in the image, while -transient regions and d-triangles are smaller regions that may be combined in the merge hase, either with -ersistent regions to refine the core or with other -transient and d-triangles regions to otentially form new core objects. Performing image segmentation based on toology and ersistent homology guarantees several nice roerties, and initial results demonstrate high quality image segmentation. 1 Introduction Image segmentation algorithms classically fall into one of two classes: region-based methods or edge-based methods. Region-based methods tyically emloy clustering, region growing, or slit-and-merge methods to construct regions based on feature characteristics of the region(s). Conversely, edge-based methods use edge detection to find edges within the image, and then use some method to link the edges to form boundaries that define regions. Both classes of segmentation methods have their advantages and disadvantages, so research has also delved into hybrid methods that combine region- and edge-based segmentation [6]. This aer resents a new hybrid aroach to image segmentation based on comutational toology. The method is fundamentally based on the slit-andmerge aradigm, using edge-directed toology to initially slit the image into a set of regions, and then using region-based merging to combine select regions into the final segmentation. The algorithm uses a three-ste aroach: 1) Perform edge detection. 2) Slit the image into regions using edge-directed comutational toology, based on ersistent homology. 3) Merge select regions with similar features in orderoftoological ersistence. W.G. Kroatsch, M. Kamel, and A. Hanbury (Eds.): CAIP 2007, LNCS 4673, , c Sringer-Verlag Berlin Heidelberg 2007

2 588 D. Letscher and J. Fritts More secifically, the algorithm first uses edge-detection to roduce an edge ma, which serves as the guide for toological region slitting. Given this edge ma, edges are linked to form regions based on the theory of ersistent homology. The result of this toological slitting is an initial region segmentation that contains three tyes of regions: -ersistent regions, -transient regions, and d- triangle regions. The -ersistent regions are larger regions corresonding to core objects in the image. The -transient and d-triangle regions are smaller regions of the image, which corresond to either additional sections of the existing core objects or sub-sections of new core objects in the image. The -transient and d-triangle regions are merged, in order of increasing ersistence, either with - ersistent regions to further refine the existing core objects or with each other to otentially define new core objects in the final segmentation. The rimary contribution of this aer is the novel use of ersistent homologybased toology in the second ste, and the corresonding order it imoses on region merging in the third ste. Any edge detection method could be used in ste one, and any feature characteristics could be used for region merging in ste three. The second ste is similar to the algorithm in [11]. However, the third ste adds toological guarantees for the comleted segmentation and incororates region information to imrove the quality of the final result. In the remainder of the aer, we discuss the roosed method in greater detail, with emhasis on the details of the toological region slitting and merge ordering in stes two and three, resectively. Section 2 resents the ertinent background in comutational toology and ersistent homology. Section 3 resents the roosed edge-directed toological segmentation algorithm. Section 4 resents reliminary results for this new algorithm. Section 5 resents related work, and Section 6 concludes the aer and discusses future directions. 2 Background in Toology Since the key asect of the roosed image segmentation algorithm is our aroach to region slitting and merging based on comutational toology, we first resent the necessary background on toology and ersistent homology. We resent these toological concets as they ertain to images and image segmentation, so we assume the existence of an image I with oints in R Alha Comlexes and Delaunay Triangulations Suose we have a set of oints X = {x 1,...,x n } in I that have been identified by some edge detection algorithm. For any ositive radius α, we can consider the set of all disks of radius α with centers in X (see Fig. 1). As α grows, larger regions of the image are covered and disks begin overlaing. The centers of the overlaing disks can be connected to form edges and triangles. The attern in which these disks intersect defines a triangulation, with the vertices of the triangulation corresonding to the oints in X. An edge in the triangulation is formed when two disks meet in a oint that is not in the interior of another disk. Similarly, a triangle is formed when three disks meet in a single

3 Image Segmentation Using Toological Persistence 589 Fig. 1. α-comlexes for α =1.2, 1.6, and 2.0, and the full Delaunay triangulation that is formed at α 4.6 oint that is not in the interior of another disk. The triangulation constructed in this way is called the Delaunay triangulation and has many nice roerties [2]. This triangulations can be found in O(n log n) time in two dimensions [2,5,7]. Figure 1 illustrates the alha comlexes for a set of oints with α =1.2, 1.6, 2.0, and 4.6. As visible in the figure, at α =1.2, the disks are of radius 1.2 and fourteen airs of disks intersect to form 14 edges. Some of these edges are connected with other edges, forming four connected comonents. When the radius α grows to 1.6, more edges are formed, and all of the edges are connected into a single connected comonent. Additionally, three trilets of disks intersect to form three Delaunay triangles. As the radius further grows to α =2.0, 5 additional edges and 2 additional triangles are formed. Finally, when α grows to α =4.6, the full Delaunay triangulation over all the vertices is formed. For a given value of α, the set of vertices, edges, and triangles defines the subcomlex, K α. This comlex accurately reresents the toology generated by the union of disks with radius α centered at each vertex [3]. An imortant characteristic of comlexes is that, for two radii, α and α, such that α<α, the subcomlex of α is a subset of the subcomlex of α, i.e. K α K α.any sequence of comlexes with one a subset of the next is referred to as a filtration. At sufficiently large α, the comlex becomes a full triangulation, in which all regions in the grah over X are covered by Delaunay triangles, as demonstrated in Fig. 1 for α 4.6. This comlex is a suerset of all other subcomlexes in X. The edges and triangles in K α iece the oints in X into a cohesive union. Recalling that we are defining X as the set of edge oints generated from erforming edge detection on I, one roblem that occurs is that small regions may be left in the comlement of the subcomlex. These regions would disaear for slightly larger α. However, increasing α to remove these regions may create new small regions. To counter this roblem and effectively deal with small regions in the comlement of the comlex, we turn to ersistent homology. 2.2 Persistent Homology In algebraic toology, homology rovides a way to measure the comlexity of a sace. In the study of comlexes inside the lane we are concerned with the first homology grou, H 1 (K), and its corresonding Betti number, β 1. Fortunately, in this alication, β 1 can be readily defined without in-deth knowledge of algebraic toology. For a connected grah where V and E are the number of vertices and edges in a grah, resectively, the Betti number is defined as β 1 = E V +1.If

4 590 D. Letscher and J. Fritts we have a triangulated subcomlex in the lane, β 1 = C V + E F,whereC is the number of connected comonents and F the number of faces (or triangles) in the comlex. The full definitions are more comlex (see [9] for a full treatment), but in our situation the theory reduces to the formulae above. As mentioned in the revious section, as α increases, the resulting sequence of subcomlexes is known as a filtration. As the triangulations change, the Betti number, β 1 (K α ), for each subcomlex K α in the filtration will change with α. As α increases, β 1 will decrease when a hole is filled in (i.e. a triangle is created), and may increase when a new edge is created (see Fig. 2). In other words, when a hole is filled, F increases by one. Conversely, when an edge forms, the number of edges E increases by one, and the number of connected comonents C may decrease by one (if the edge connects two reviously searate comonents). Related to this, ersistence refers to how long regions in the comlement of K α last until they are contained in K α+ for some > 0. The -ersistent Betti number of K α, (K α), is β 1 (K α ) minus the number of regions in the comlement of K α that are contained in K α+. In sirit, it is the Betti number of the comlex obtained from K α with small holes filled in. For formal definitions in a more general setting, and algorithms for their calculation, see [4]. Consider two subcomlexes in a filtration, K α and K α+,where>0. Recall that this means that K α K α+,sok α+ contains the same vertices, edges, and triangles as K α, but may also contain new edges and triangles. Holes in K α that are still holes in K α+ are regions that are ersistent in. Weshallrefer to these regions as -ersistent regions. Holes in K α that are filled in between K α and K α+ are regions that are not ersistent in. We shall refer to these as -transient regions. And finally, the individual triangles that were already filled in K α will be referred to as d-triangles. For subcomlexes of the lane, loos that do not ersist bound regions that disaear comletely in K α+. An equivalent definition to the more general one, is (K α)=c V + E F R, wherer is the number of regions in the comlement of K α that are contained in K α+. Note that this is not necessarily true in higher dimensions. 3 Segmentation Algorithm The roosed segmentation algorithm is a hybrid slit-and-merge segmentation algorithm that emloys comutational toology and ersistent homology for image slitting and region feature characteristics for region merging. The algorithm first emloys some edge detection algorithm, such as the Canny algorithm [1] or the wavelet-based detector that we used [8], to determine the set of edge oints, X, in the image. Toological slitting is the erformed over X to generate an initial segmentation with three tyes of regions: -ersistent regions, -transient regions, and d-triangles. The toological slitting is controlled by two arameters, α and, whereα defines the radius of the disks and indicates the ersistence. In the final ste, the algorithm merges the -transient and

5 Image Segmentation Using Toological Persistence 591 β α C =4 E =14 F =0 β 1 =0 E =22 F =0 β 1 =5 E =27 F =5 β 1 =5 E =35 F =10 β 1 =8 E =37 F =13 β 1 =7 E =38 F =18 β 1 =3 Fig. 2. This figure demonstrates various comlexes in a filtration, including their Betti numbers and ersistent Betti numbers for each K α and K α+ d-triangle regions with either the -ersistent regions or each other to generate the final segmentation. As discussed in the revious section, the -ersistent regions reresent larger regions that remain (are not filled in) in K α+. Conversely, -transient are smaller regions that are filled in between K α and K α+, while d-triangles are the smallest regions, which already existed as triangles in K α.thed-triangles are akin to thickened edges searating regions in the initial segmentation. The -ersistent regions are crucial in that they define the core objects in the initial segmentation. There are two reasons to believe these regions are imortant to the segmentation: they are comletely surrounded by edges, and they all contain a disk of radius α + in their interiors. These initial regions can then be exanded uon to create a full segmentation. We erform this rocess in such a way that the ersistent homology is resected. One aroach would be to use the simlification algorithms resented in [4] to exand these regions and reserve this homological information. However, this method uses only the edge information, essentially exanding the regions so that the region boundaries consist of the shortest ossible edges. This aroach has the significant drawback that it does not incororate any region information. An alternative strategy for exanding the ersistent regions to a full segmentation is to merge neighboring regions based on their feature characteristics. In our case, we simly used the average color value for each region, but any desired feature vector could be used. Homological ersistence dictates the merge order of the regions, with regions being ordered according to the disk radius α for which the triangle was formed, starting from the smallest. In order to reserve the toology of the segmentation, regions will only be merged if the ersistent Betti number remains unchanged.

6 592 D. Letscher and J. Fritts Algorithm 1. Algorithm for edge-directed toological image segmentation 1: 2: 3: 4: 5: 6: 7: 8: 4 Identify set of edges, X, in the image. Find the Delaunay triangulation of X. Calculate β1 (Kα ) and identify regions in 2 Kα as -ersistent, -transient, or d-triangle regions. Let F be the set of regions ordered by increasing α values. while (F is not emty) do Let σ be the first face in F. And let F = F {σ}. Merge σ with the most similar adjacent region rovided that β1 (merged regions) = β1 (Kα ) end while Results Figures 3 and 4 demonstrate the results of the algorithm for a samle image, both over multile stages of the rocess, and over multile searate images, resectively. As evident, each of the resulting segmentations effectively distinguishes the critical objects in the image. Figure 5 demonstrates how the choice of α and affects the resulting segmentation. For small values of α large regions are not searated from each other in the initial stages of the algorithm. For fixed α larger values of remove small regions from consideration. Fig. 3. Major stes in the algorithm: (a) original image (b) comlement of the α comlex for α = 4, = 0. (c) the ersistent regions at α = 4, = 3 (d) final segmentation. Fig. 4. Examle segmentations a sto sign, a rose and a baseball layer

7 Image Segmentation Using Toological Persistence =0 23 Regions =2 22 Regions =4 13 Regions 47 Regions 34 Regions 21 Regions 43 Regions 28 Regions 22 Regions 593 α=2 α=4 α=6 Fig. 5. Affect on the choice of arameters of the resulting segmentation The segmentation algorithm has several nice guarantees. In articular, large regions in the comlement of the edges will not be broken into searate regions. Secifically, any ball of radius in the comlement of the edges will not be divided between multile segments. The choice of α will affect how well the oints identified by the edge detection fill out to comlete the boundary of regions. In articular, if the oints identified as edges have neighbors with a distance of α/2 in each direction then boundaries of regions will be correctly reconstructed. 5 Related Work An early image segmentation method using Delaunay triangulations was resented in [6]. Their method is distinct in that they do not use ersistent homology, but instead start with a full Delaunay triangulation and generate the core objects in the final segmentation solely through region merging. A more recent image segmentation method based on Delaunay triangulation is resented in [11]. They use similar criteria for doing their initial segmentation. However, whereas we use Delaunay triangulation for defining different tyes of regions and then erform region merging to generate the final segmentation, they use Delaunay triangulation for defining boundaries, and use thinning to eliminate undesired edges in the final segmentation.

8 594 D. Letscher and J. Fritts Another recent image segmentation method based on Delaunay triangulation is resented in [10]. They generate the full Delaunay triangulation first, followed by region merging. They merge regions by extracting a skeleton of the segmented image from the connected comonents grah of the triangulations. Each searate connected comonent in the skeleton defines one region in the final segmentation. 6 Conclusion and Future Directions We have resented an image segmentation algorithm that uses both edge and region information and uses techniques from comutational toology. The segmentations obtained clearly delineate key objects in the images and are good initial indicators of the effectiveness of the method. There are several directions of followu for this algorithm. These include studying different orderings for merging the d-triangles and -transient regions, and using alternate feature characteristics for region merging (as oosed to just average region color value). Significant imrovement could also otentially be achieved in using machine learning methods to hel identify aroriate arameterizations of α and according to the characteristics of the image and edge features. Another imrovement is to utilize more information from the edge detection algorithms. The current algorithm only uses the magnitude of the estimated gradient. The direction of the gradient can also be incororated in finding an anisotroic Delaunay triangulation. These triangulations can better reresent the regions divided by the edge information. References 1. Canny, J.: A Comutational Aroach To Edge Detection. IEEE Trans. Pattern Analysis and Machine Intelligence 8, (1986) 2. Edelsbrunner, H.: Algorithms in Combinatorial Geometry. Sringer, New York (1987) 3. Edelsbrunner, H.: The Union of Balls and its Dual Shae. In: Proceedings of the Ninth Annual Symosium on Comutational Geometry, (1993) 4. Edelsbrunner, H., Letscher, D., Zomorodian, A.: Toological Persistence and Simlification. Discrete and Comutational Geometry 28, (2002) 5. Fortune, S.: A Sweeline Algorithm for Voronoi Diagrams. Algorithmica 2, (1987) 6. Gevers, T., Smeulders, A.W.M.: Combining Region Slitting and Edge Detection through Guided Delaunay Image Subdivision. In: Proc. of the 1997 International Conference on Comuter Vision and Pattern Recognition, (1997) 7. Guibas, L., Knuth, D., Sharir, M.: Randomized Incremental Construction of Delaunay and Voronoi Diagrams. Algorithmica 7, (1992) 8. Mallat, S., Zhong, S.: Characterization of signals from Multiscale Edges. IEEE Trans. Patt. Anal. and Mach. Intell. 14, (1992) 9. Massey, W.: A Basic Course in Algebraic Toology. Sringer, Heidelberg (1991)

9 Image Segmentation Using Toological Persistence Prasad, L., Skourikhine, A.N.: Vectorized Image Segmentation via Trixel Agglomeration. In: Brun, L., Vento, M. (eds.) GbRPR LNCS, vol. 3434, Sringer, Heidelberg (2005) 11. Stelldinger, P., Ullrich, K., Meine, H.: Toologically Correct Image Segmentation Using Alha Shaes. In: Kuba, A., Nyúl, L.G., Palágyi, K. (eds.) DGCI LNCS, vol. 4245, Sringer, Heidelberg (2006)