Line Segment Based Watershed Segmentation Johan De Bock 1 and Wilfried Philips Dep. TELIN/TW07, Ghent University Sint-Pietersnieuwstraat 41, B-9000 Ghent, Belgium jdebock@telin.ugent.be Abstract. In this paper we present an overview of our novel line segment based watershed segmentation algorithm. Most of the existing watershed algorithms use the region label image as the main data structure for its ease of use. These type of watershed algorithms have a relatively large memory footprint and need unbounded memory access. For our new watershed algorithm we replaced the traditional region label image with a data structure that stores the regions in linked line segments. Consequently, the new algorithm has a much smaller memory footprint. Using the new data structure also makes it possible to create an efficient algorithm that only needs one scan over the input image and that only needs the last 3 lines and a small part of the data structure in memory. 1 Introduction Image segmentation is the process of partitioning a digital image in meaningful segments, i.e. segments that show a certain degree of homogeneity. This can be homogeneity of any type, such as intensity, color, texture or higher level objects. It is crucial in a number of applications, such as image coding, tracking, content-based image retrieval and object recognition. The image segmentation concept can be translated to many techniques, the technique we will discuss is the watershed segmentation. It is a well-known low-level image segmentation technique. It can be attributed properties of both edge directed and region directed segmentation classes. First we generate an edge indication image, i.e. an image with for each pixel a value that indicates the probability of an edge going through the pixel in question. In case of intensity segmentation, the edge indication image can be created by calculating the norm of the gradient vector at each pixel position. Then the watershed algorithm will try to find the homogeneous closed regions by using the edge indication image as input. The watershed algorithm achieves this by regarding the edge indication image as a topographic landscape in which valleys correspond to the interior of segments, whereas the mountains correspond to the boundaries of segments. The watershed algorithm derives the mountain rims from the landscape and those mountain rims then delineate the segments in the image. Watershed algorithms can be divided in two classes depending on the method that is used to extract the mountain rims from the topographic landscape. The The final publication is available at Springer via DOI 10.1007/978-3-540-71457-6 53
Fig. 1. Chronological stages in the flooding process first class contains the flooding watershed algorithms. These type of watershed algorithms extract the mountain rims by gradually flooding the landscape. The points where the waterfronts meet each other constitute the mountain rims. This process is displayed chronologically in Fig. 1. The classic example of this class is the discrete Vincent-Soille flooding watershed algorithm [1]. The second class contains the rainfalling watershed algorithms. The algorithm discussed in this paper belongs to this class. Other examples of this class are the algorithm described in [2] and our previous algorithms [3,4]. In Sect. 2 we will describe the general layout of a rainfalling watershed algorithm. In Sect. 3 we will explain our line segment based rainfalling watershed algorithm. We will give a detailed report on the execution times of the algorithm for different parameters and for both natural and artificial images in Sect. 4. Finally we will draw some conclusions in Sect. 5. 2 General Layout of a Rainfalling Watershed Algorithm As the name partly reveals, a rainfalling watershed algorithm exploits a different concept (compared to the flooding watershed) to extract the mountain rims. For each point on the topographic landscape an algorithm tracks the path that a virtual droplet of water would follow if it would fall on the landscape at that point. All droplets or points that flow to the same local minimum constitute a segment. This concept is depicted in Fig. 2 for the two-dimensional case. The lowest mountains (weakest edges) can be suppressed by drowning them. All the mountains below a certain drowning threshold will not be taken into account. This is shown in Fig. 3. In the implementation, the rainfalling concept is carried out by calculating the steepest descent direction for each pixel. The directions are limited to the pixels neighboring the central pixel. A visualization of the steepest descent directions for a small part of an image is given in Fig. 4. The pixels marked with a circle in the middle are pixels from where there is no descent possible. Hence, they are the local minima of the topographic landscape. The pixels with a topographic height lower than the drowning threshold will also be marked as local minima. A steepest descent direction from one to another pixel is depicted as an arrow, a segment between two circles designates two connected local minima
Fig. 2. Rainfalling concept Fig. 3. Drowning threshold Fig. 4. Steepest descent directions and local minima pixels. Both these arrows and segments will be called connections in this paper. One group of pixels that is connected must now make up one segment. We used 4-neighborhood for the steepest descent calculations and 8-neighborhood for determining the minima. Experimental results showed that this combination produces the least oversegmented image out of all 4 possible combinations. All the results presented in this paper are produced with this combination. 3 Description of the Line Segment Based Rainfalling Watershed Algorithm The traditional rainfalling and flooding watershed algorithms use the region label image as the main data structure during the algorithm and also as the output of the algorithm. A region label image is an image with for each pixel
a label of the region to which the pixel belongs. To store these labels one uses 4 bytes to be sure that a large image with a lot of regions can be segmented. So for an image with n pixels, the region label image uses up 4n bytes. Next to the region label image, all these algorithms also use a specific internal data structure that uses up at least 4n bytes and that is accessed at the same time as the region label image on at least one moment during a complete execution of the algorithm. For the classic flooding watershed algorithm [1] this specific structure is an array of pointers to pixels, for our first optimized algorithm [3] this is also an array of pointers to pixels and for our fast sequential algorithm [4] this is an additional label image. For the exact amounts we refer to the original papers. In total the traditional algorithms use at least 8n bytes. The memory access for these algorithms is unbounded, i.e. it is possible that when the bottom row of an image is processed one also needs access to the pixels of the top row (worst case). One can not limit the memory access to the last few already processed lines. For the classic flooding watershed algorithm this unbounded access can be noticed during the sorting step and during the spatially random filling of the region label image based on increasing topographic height, for example when both at the top and at the bottom there is a pixel at the same height. For our previous algorithms this unbounded access can be noticed during the tracking of the steepest descent paths, for example when there is a path that goes all the way from bottom to top. In our line segment based watershed algorithm we replaced the traditional region label image with a data structure that stores the regions in linked line segments [5]. This makes it possible to create an efficient algorithm that uses considerably less memory, that only needs one scan over the input image and that only needs access to the last 3 processed lines. A line segment is an interval of pixels on a certain row of the image. The final configuration of line segments for the steepest descent directions shown in Fig. 4 is depicted in Fig. 5. A line segment contains its row, start column and end column (all 2 bytes). It also contains a pointer to another line segment (4 bytes). These pointers are used to link all the line segments belonging to one region together with a region descriptor. The region descriptor for the region in the bottom left corner of Fig. 5 is displayed in Fig. 6. A region descriptor contains a pointer to the first line segment, a pointer to the last line segment and a pointer to one of the middle line segments of a region (12 bytes). The last line segment of a region descriptor always points to the region descriptor itself. Finally a data structure with r regions and l line segments will take up 12r + 10l bytes. Now we will describe the general steps of the line segment based watershed algorithm. We assume that the rows 0 to i 1 are already processed. Calculate the steepest descent directions of row i. Repeat the following steps until the end of the row is reached. Look for a continuous run of horizontal connections and create a new line segment from this run, i.e. as long as there is a horizontal connection between the current and the next pixel, keep extending the current line segment with an extra pixel.
Fig. 5. Line segment configuration Fig. 6. Example region descriptor, linking line segments together Look for connections with the line segments of row i 1. More specific, check for vertical and diagonal connections that connect the current line segment with any of the line segments of the previous row. No connections. Create a new region descriptor and add the current line segment. One connection. Add the current line segment to the respective region descriptor. Two or more connections with line segments of distinct region descriptors. Merge the respective region descriptors and add the current line segment. Determine the next line segment. Process the next row. To conclude the description we will sum up some important remarks: The merging of two region descriptors is done by interleaving the start to middle and middle to end parts of both region descriptors. The region descriptor to which a line segment belongs is found by going to the next line segment in the linked list until one reaches the pointer to
Fig. 7. PEPPERS 512x512, segmented with rdt = 0.001 the region descriptor. The execution time lost on this search is very limited because this search is only initiated for the line segments of the previous row and those are always kept in the middle to end part of a region descriptor (together with the line segments of the current row). If the current line segment will be added to the region descriptor where the previous line segment was added to then we can extent the previous line segment with the current line segment instead of just adding the current line segment. This situation is visualized by the vertical dotted lines in Fig. 5. This check can be done with no penalty in execution time and it will severely reduce the total amount of line segments. In the realistic example we will give later on, it reduced the amount of line segments from 67138 to 39092, about 58%. 4 Results The segmentation results are theoretically identical to our previous algorithms when using the same neighborhoods. To still give an example of how a typical low-level watershed segmentation result looks like, we tested our line segment based watershed algorithm on the standard test image PEPPERS 512x512 with a relative drowning threshold of 0.001 (the relative drowning threshold rdt is the absolute drowning threshold divided by the maximum topographic height). For a more detailed analysis of the segmentation results we refer to our previous papers on the subject [3,4]. The segmentation result is shown in Fig. 7. The line segment based watershed algorithm produced 2354 regions and 39092 line segments and thus used 419168 bytes. This is 20% of what a region label image based technique would use. To show that the line segment based watershed algorithm is still competitive speed-wise despite the higher complexity we compared it to our fast
30 25 Time taken (ms) 20 15 10 5 0 Label image Line segments 10 4 10 3 10 2 10 1 10 0 Relative drowning threshold Fig. 8. Execution times Fig. 9. Connected components test pattern region label image based technique [4]. Both algorithms were implemented in C, compiled with gcc 3.4.2 with optimization parameter -O3 and run on one core of an Intel Core 2 Duo T7400 2.16 GHz. The execution times are shown in Fig. 8. For low drowning thresholds it is not much slower and for very high thresholds it is even faster than the label image based algorithm. Our new algorithm is also a bit faster for the complex connected components test image displayed in Fig. 9, 5.9 ms vs 7.9 ms. These type of artificial images have very little pixels above the drowning threshold and almost no steepest descent calculations have to be done, thus the running time is completely dominated by the parts that have to merge the connected minima. 5 Conclusions We have developed a line segment based watershed segmentation algorithm that uses considerably less memory than the traditional region label image based watershed algorithms and that is very competitive speed-wise. The memory access is also limited to the last 3 processed lines. Future work will be the construction of a 3D version of this algorithm and the investigation if the complexity of the algorithm can be reduced. References 1. Vincent, L., Soille, P.: Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence 13 (1991) 583 598 2. Beucher, S.: Segmentation d images et morphologie mathématique. PhD thesis, School of Mines (1990)
3. De Smet, P., Pires, R.: Implementation and analysis of an optimized rainfalling watershed algorithm. In: Proc. Electronic Imaging, Science and Technology, Image and Video Communications and Processing. (2000) 759 766 4. De Bock, J., De Smet, P., Philips, W.: A fast sequential rainfalling watershed segmentation algorithm. In: Advanced Concepts for Intelligent Vision Systems, 7th international conference, Antwerp, Belgium. Volume 3708 of Lecture notes in computer science., Springer (2005) 477 484 5. Philips, W.: Weakly separable segmented image coding: Theory, results and implementation aspects. Technical report, ELIS (1996)