Image Types int, double, char,... Morphological Algorithms. Data Structures

Transcription

1 GENERIC ALGORITHMS FOR MORPHOLOGICAL IMAGE OPERATORS A Case Study Using Watersheds D'ORNELLAS, M. C. and VAN DEN BOOMGAARD, R. Intelligent Sensory Information Systems University of Amsterdam, Faculty WINS Kruislaan, SJ Amsterdam fornellas,reing@wins.uva.nl FAX: (31)(20) Abstract. The aim of the present work is to combine both the advantages of generic programming approach and the wave front propagation interpretation to develop generic algorithms for morphological image operators. The watershed operator is implemented based on this framework and a performance evaluation, using processing time, is provided to compare the generic watershed implementation in contrast to the classical implementation using hierarchical queues. Key words: generic algorithms, mathematical morphology, priority queues, wave front propagation, and watersheds. 1. Introduction Mathematical morphology is a powerful and unied approach for geometrical shape analysis and description based on a complete lattice framework. A complete lattice is a partially ordered set L such that every subset K has an inmum, denoted by ^K, and a supremum, denoted by _K. From a theoretic point of view, a dilation, erosion, adjunction, opening, and closing are the most important algebraic notions in mathematical morphology to characterize operators on lattices. The present study addresses the generic programming approach to morphological algorithm development. Generic programming reduces the task of creating morphological algorithms available for all combinations of images (i.e., lattices) and data structures. Since pixel values in an image form a partially ordered set and several morphological operators can be most eciently implemented when this ordering is explicitly used, it seems natural to provide data structures to deal with the ordering. A priority queue structure is used to organize these pixels according to specied priorities while the wave front propagation interpretation is employed to handle the region growing process. The organization of the rest of this paper is as follows. In section 2, we describe Formerly at Universidade Federal de Santa Maria, Departamento de Eletr^onica e Computac~ao, Santa Maria-RS, Brazil. Supported by CAPES Foundation under grant BEX 2780/95-0.

2 2 D'ORNELLAS, M. C. AND VAN DEN BOOMGAARD, R. the generic programming approach. In section 3 we give a brief introduction to the wave front propagation interpretation of morphological image operators. We also discuss the data structure and the way it can be used to work with every image type. Then in section 4, a watershed segmentation algorithm is implemented based on wave front propagation and following a generic approach. Some code optimizations and enhancements are also provided. The evaluation performance and results of our generic watershed algorithm are shown in section 5. Conclusion and further research are given in section Morphological Algorithm Development 2.1. Generic Programming Approach The object oriented programming paradigm is so strongly devoted on specifying interfaces for encapsulation that programmers sometimes keep out of sight from data structures and algorithms. Once a class interface has been built, the details of its implementation are said to be of little importance. However, some care must be taken whether or not it has been implemented correctly and eciently and whether or not the algorithms and data structures have been chosen properly. Generic programming gives and alternative to this approach. While object oriented programming takes abstract objects and gives them real world representations, generic programming does the same with algorithms. Generic programming also has a number of important advantages:? it represents ecient algorithms independently of any particular data structure;? providing an interface to a diverse set of data structures not only gives us the exibility to choose appropriate ones, but also facilitates the development of more abstract algorithms. The generic programming paradigm within the C++ context was rst proposed by Stepanov and Musser [5] and became popular with the inclusion of their Standard Template Library (STL) into the C++ standard. In C++, the template mechanism is used to build generic algorithms by means of compile-time polymorphism so that the performance overhead is minimal and sometimes, non existent. Generic algorithms naturally lead to the iterator concepts. Iterators are used in the STL to access data structures, rather than interacting with data structures directly. Moreover, all generic data structures provide the most advanced type of iterator they can implement eciently. In this way, iterators serve as interfaces to both generic algorithms and generic data structures Working with Generic Images, Algorithms, and Data Structures Consider software components as a three-dimensional space as shown in gure 1. One dimension represents the image data types (int, oat, complex,... ), the second dimension represents the data structures (queue, linked-list, heap, splay, priority queue,... ) and the third dimension represents the morphological algorithms (erosion, dilation, reconstruction, watersheds,... ). Based on this scenario, x y z dierent versions of code have to be designed - an watershed algorithm for a priority queue of int, a reconstruction algorithm for a priority queue of double and so on. By using template functions that are parametrized

3 GENERIC ALGORITHMS FOR MORPHOLOGICAL IMAGE OPERATORS 3 X Image Types int, double, char,... Morphological Algorithms Z dilation, erosion, watersheds,... Data Structures Y queue, linked-list, heap, splay,... Fig. 1. Software Component Space by data types, the x-axis can be dropped and only y z versions of code have to be designed, because there has to be only one priority queue implementation which them can hold objects of any data type. The next step is to make the algorithms work on dierent data structures - that means that a watershed algorithm should work on hierarchical queues as well as on priority queues, etc. Then, only y + z versions of code have to be created. With these notions in mind we can dene a unique interface between morphological algorithms and the data structures so that the same algorithm implementation can be applied to any number of dierent image types. Our intention is to assure that one can have algorithms dened as generically as possible without loosing eciency. 3. The Wave Front Interpretation of Algorithms 3.1. Wave Front Propagation Methodology Some of the most powerful and often used algorithms in morphological image processing are based on wave front propagation. A wave front propagation interpretation implies that the action of the morphological operator is closely associated with the notion of connectivity. It also implies that the action is based on a kind of growing process, where the information is propagated through the image. The wave front propagation interpretation of morphological operations was pionered by Van Vliet and Verwer [8]. In wave front propagation algorithms, the following aspects can be distinguished:? selection of the initial wave front (i.e., which pixels are selected for processing);? selection of the points to which a point on the wave front is propagated. (i.e., the elementary wave front); It should be noted that the main idea of a wave front propagation implies that all points on the wave front act like point sources. The envelope of all innitesimal wave fronts emanating from all those individual points forms the macroscopic wave front. Also, observe that the morphological selection of minima (or maxima) corresponds with this natural selection of the envelope wave front Wave Front Propagation Algorithms Using Priority Queues Vincent and Soille [7] were the rst to use a priority-like ordered queue in mathematical morphology. Another approach to deal with morphological algorithms using

4 4 D'ORNELLAS, M. C. AND VAN DEN BOOMGAARD, R. priority queues was introduced by Beucher and Meyer [1] and is known as the hierarchical queue. The inner problem of these two approaches is that they are limited to a nite range of priorities and to discrete values. To overcome this problem a heap structure was used by Vincent [6]. However, a heap is inherently unstable, in the sense that items with the same priority are not handled in FIFO order. Breen and Monro stated that the stability can be achieved at a cost of one integer per item [2]. Recently, Noguet et al. [4] have shown that a priority queue structure can be eciently used to handle the sorting step if a morphological operator can be treated as a wave front propagation from initial points inside a set of points sharing some features based on the connectivity rules. A priority queue is an abstract data structure that can deal with an arbitrary number of items with priorities. In our implementation, a priority queue of objects is constructed which every object has its pixel's address and a set of keys (k1; k2; : : : ; kn) that is sorted using the lexicographical ordering. An ordered pair (k1; k2) is lexicographically less than (k1 0 ; k2 0 ) if either k1 < k1 0 or k1 = k1 0 and k2 k2 0 [9]. Lexicographical ordering can be extended to many keys in a straightforward manner. A priority queue is then specied in table I: Method PQ q; q.enq(key1, key2,..., keyn ); q.deq(); q.empty(); TABLE I Priority Queue Fundamental Methods Description priority queue object declaration. put the tuple (key1,key2,...,keyn) on the queue. get a tuple (key1,key2,...,keyn) from the queue. checks whether there are items on the queue. In the next section, we will concentrate in the watershed operator to demonstrate the generic programming approach. However, we should have in mind that the generic algorithm concept is planned to serve as a framework to the development of other morphological operators like reconstruction, skeleton, SKIZ, etc. 4. A Generic Watershed Algorithm Implementation Based on Wave Front Propagation Watershed analysis has proven to be a powerful tool for many image segmentation problems. In the ooding scheme, water slowly rises within the topographic surface represented by an image, so that all point below water level are immersed. Holes are punched in the regional minima and the topography is ooded from below. As the water rises, more surface minima are pierced, which in turn starts more catchment basins. The catchment basins expand as the water rises and oods more points. When two oods from dierent catchment basins meet, a dam is built at these points to prevent the catchment basins from merging. After the surface is completely ooded, only the tops of the dams are visible and are treated as dividing lines. These watershed lines separate the surface into catchment basins.

5 GENERIC ALGORITHMS FOR MORPHOLOGICAL IMAGE OPERATORS 5 In this paper, we consider images as mappings from D I Z 2 into Z. The grid G Z 2 Z 2 denes the neighborhood operations (4 or 8 connectivity) and N G (p) = fp 0 2 Z 2 j(p; p 0 ) 2 Gg denotes the neighbors of a pixel p, according to a grid G. Also, M is the labeled markers image and f(p) corresponds to the grey-value of pixel p. All the algorithms presented in this paper extend the original image outside the window to cope with borders. The bare-bones watershed implementation can be composed by two steps: a initialization step, and a data driven propagation step. The pseudocode is given as follows: Initialization Step: 01: PQ q; 02: For (every p in domain D) f 03: If ( M(p)!= 0 and (exists (p 0 ) in NG(p) : M(p 0 ) == 0)) 04: q.enq( p, f(p)); 05: g Data Driven Propagation Step: 06: While (!(q.empty())) f 07: q.deq(); 08: For (every (p 0 ) in (NG(p) \ D)) 09: If (M(p 0 ) == M(p)) 10: q.enq( p 0, f(p 0 )); 11: g Fig. 2. Wathershed Algorithm - WS1 The algorithm layout in gure 2 can be found in Noguet et al. [4], Beucher and Meyer [1], and Najman and Schmitt [3]. The data structure used is an ordered queue in the sense that the method q.deq(); returns the element from the queue that has minimal grey-value. Apart from its well dened structure, there are several drawbacks in this algorithm as stated below:? the catchment basins are calculated. Although the watersheds themselves are readily calculated from the catchment basins, it is preferred that the watershed algorithm oers at least the possibility to calculate the watershed lines directly;? in case there are several object tuples on the queue with the same grey-value and associated positions (i.e.,there is a plateau in the image), the above algorithm does not specify in what order the pixels on the plateau are processed. It would be preferred in case the pixels on a plateau are processed in accordance to their distance from the regional extremum they belong to, as stated by Beucher and Meyer [1], and Najman and Schmitt [3]. In order to be able to mark the contours of the catchment basins explicitly, the basic algorithm must be extended. This is shown in gure 3: The extension of this algorithm is potentially rather time consuming because for each pixel p each neighbor p 0 is checked and also for each p 0 the neighbors p 00 are checked. This increases the number of pixels to be checked from 9 to 81. It should be noted though that the number of pixels to be checked can be reduced quite signicantly. Coming from pixel p that is labeled with M = M(p) we are about

6 6 D'ORNELLAS, M. C. AND VAN DEN BOOMGAARD, R. Initialization Step: 01: PQ q; 02: For (every p in domain D) f 03: If ( M(p)!= 0 and (exists (p 0 ) in NG(p) : M(p 0 ) == 0)) 04: q.enq( p, f(p)); 05: g Data Driven Propagation Step: 06: While (!(q.empty())) f 07: q.deq(); 08: For (every (p 0 ) in (NG(p) \ D)) 09: If (M(p 0 ) == 0) 10: If (exists (p 00 ) in NG(p 0 ) : M(p 00 )!= M(p)) 11: M(p 0 ) = WSHED; 12: else f 13: M(p 0 ) = M(p); 14: q.enq( p 0, f(p 0 )); 15: g 16: g Fig. 3. Wathershed Algorithm - WS2 to label p 0 with M as well. But we can only do that in case this new pixel p 0 is not connected to a pixel with a label unequal to M. Since p, p 0 and p 00 are connected within a 3 3 neighborhood and because all pixels in the 3 3 neighborhood of p are known to be unlabeled or with label M, we don't have to check all pixels. So, the pixels in N G (p) do not have to be taken into account when looking for the pixels p 00 in the neighborhood of N G (p 0 ). With this optimization, we get a small change reducing the complexity from 81 to at most 45 or 36 on average. This algorithm optimization can be achieved by changing the line 10: in gure 3 by: 10: If (exists (p 00 ) in (NG(p 0 ) n NG(p)) : M(p 00 )!= M(p)) Generic Algorithms can be extended by means of whether or not the wave front propagation is included in the initialization step. By changing the data driven propagation position and some conditional statements one can implement distinct operations like reconstruction, skeleton, and extrema detection easily. 5. Experimental Results The performance of the watershed algorithms presented in this paper, are compared with the original algorithm by Vincent and Soille [7]. This is illustrated by applying these algorithms to a sample data set in gure 4(a) using a marker image as a reference. Describing the technique used to generate the marker image used would go beyond the scope of this paper. The result of the rst watershed algorithm can be seen in gure 4(c) where just the catchment basins are shown. Figure 4(d) shows both the catchment basins and the watershed lines according to the second implementation. Figure 4(e) shows the results by Vincent and Soille. The original image in gure 4(a) is then superimposed

7 GENERIC ALGORITHMS FOR MORPHOLOGICAL IMAGE OPERATORS 7 by the watershed lines from gure 4(d), to emphasize the segmentation of the cells in gure 4(f). Fig. 4. Input image(a), Morphological gradient(b), Catchment basins(c), Watershed lines(d), Vincent and Soille implementation(e), and Input image with watershed lines(f). The processing time for the implementations are given in table II for a 256x256 grey-level image based on int type, using 8-connected neighborhood. The watershed implementations are labeled by WS1,WS2, and the optimization by WS3. The last one,ws4, is the implementation by Vincent and Soille previously cited. All the algorithms were tested on a Sun Sparcstation Ultra-1 running Solaris 2.5 (UNIX), using g++ compiler with optimizations enabled. Note that WS4 also includes the detection of the regional minima of the image whereas WS1,WS2 and WS3 start from this minima. TABLE II Performance Evaluation (time in (s)) Algorithms WS1 WS2 WS3 WS4 Connectivity Regional Minima Detection Watershed Algorithm Total Processing time

8 8 D'ORNELLAS, M. C. AND VAN DEN BOOMGAARD, R. 6. Conclusions and Further Research In this paper we have shown that generic programming approach can be applied to morphological image operators based on wave front propagation like watersheds. Generic programming tools like STL let us apply a more evolutionary and experimental approach to morphological algorithm development. Using the proposed technique, reuse of morphological algorithms should become more easier. A performance evaluation based on processing time were used to compare our generic watershed algorithms in contrast to the classical implementation. The watershed optimized version WS3 proved to be faster than the classical one even when the regional minima detection time is considered. Real images often possess plateaus. If we want thin watershed lines, the unpredictable propagation in the plateaus should be considered. This will be the subject of study for future implementations. One solution is to extend the basic algorithm so that when propagating the labels also the chamfer distance from the regional minimum is propagated. Acknowledgements Thanks are due to Niels Nes and Dennis Koelma for making constructive comments on the implementation aspects of our work. References 1. S. Beucher and F. Meyer. The morphological approach to segmentation: the watershed transformation. In E. R. Dougherty, editor, Mathematical Morphology in Image Processing, chapter 12, pages 433{481. Marcel Dekker, New York, E. J. Breen and D. H. Monro. An evaluation of priority queues for mathematical morphology. In J. Serra and P. Soille, editors, Mathematical Morphology and Its Applications to Image Processing, pages 249{256. Kluwer Academic Publishers, The Netherlands, L. Najman and M. Schmitt. Geodesic saliency of watershed contours and hierarchical segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(12):1163{1173, D. Noguet, A. Merle, and D. Lattard. A data dependent architecture based on seeded region growing strategy for advanced morphological operators. In P. Maragos, R. W. Schafer, and M. A. Butt, editors, Mathematical Morphology and Its Applications to Image and Signal Processing, pages 235{243, The Netherlands, Kluwer Academic Publisher. 5. A. Stepanov and D. Musser. Algorithm-oriented generic libraries. Software - Practice and Experience, 24(7):623{642, L. Vincent. Morphological grayscale reconstruction in image analysis: Applications and ecient algorithms. IEEE Transactions on Image Processing, 2:176{201, L. Vincent and P. Soille. Watersheds in digital spaces: An ecient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6):583{ 598, L. J. V. Vliet and B. J. H. Verwer. A contour processing method for fast binary neighbourhood operations. Pattern Recognition Letters, 7:27{36, D. Wood. Data Structures, Algorithms, and Performance. Addison Wesley, New York, 1993.