Multivalued image segmentation based on first fundamental form

Multivalued image segmentation based on first fundamental form P. Scheunders Vision Lab, Department of Physics, University of Antwerp, Groenenborgerlaan 171, 2020 Antwerpen, Belgium Tel.: +32/3/218 04 39 Fax: +32/3/218 03 18 Email: scheun@ruca.ua.ac.be Abstract In this paper, a new segmentation technique for multivalued images is elaborated. The technique makes use of the first fundamental form to access edge information of a multivalued image. On the obtained edge map, a watershedbased algorithm is applied. In order to remove noise or local texture, before segmentation, an anisotropic diffusion filter is applied, also making use of the first fundamental form. In this way, the entire procedure is applied using multivalued processing. Experiments are performed on colour images, medical multimodal images and multispectral satellite imagery. Segmentation results are compared to singlevalued segmentation and filtering, applied to the intensityonly or the band-average images. 1 Introduction With the evolution of imaging technology, an increasing number of image modalities becomes available. In remote sensing, sensors are used that generate a number of multispectral bands. In medical imagery, distinct image modalities reveal different features of the internal bo. Examples are MRI images, sensitive to different imaging parameters (T1, T2, proton density, diffusion,...), CT and nuclear medicine modalities. A particular type of multivalued images are colour images. A lot of attention has been devoted to the classification and/or segmentation of multimodal data. In remote sensing, classification of textured regions is performed for identification purposes [1] [2]. In medical imaging, segmentation of different tissue regions is aimed at [3] [4]. Other image processing procedures to facilitate the classification and segmentation of multivalued images include noise filtering and image enhancement. It is obvious that all these image processing and analysis techniques would benefit from the combined use of the different bands. Nevertheless, in most cases single-valued processing and analysis techniques are applied to each of the bands separately. The results for each component are then combined in a usually heuristic manner. A large part of image processing and analysis techniques makes use of the image edge information, that is contained in the image gradient. A nice way of describing multivalued edges is given in [5]. Here, the images first fundamental form, a quadratic form, is defined for each image point. This is a local measure of directional contrast based upon the gradients of the image components. This measure is maximal, at each image point, in a particular direction, that in the greylevel case is the direction of the gradient. Based on this definition, in [6], a colour edge detection algorithm was described and a colour image anisotropic diffusion algorithm was described in [7]. In this paper, we will make use of the concept of first fundamental forms, for the segmentation of multivalued images. The segmentation procedure uses the immersionbased watershed algorithm [8, 9]. That algorithm is applied to the edge map that is obtained from the first fundamental form, so that the segmentation takes into account the edge information of all bands of the multivalued image simultaneously. The watershed algorithm is known to perform oversegmentation, due to noise and local texture. To avoid oversegmentation, an anisotropic diffusion filter is performed a priori. This filtering is also performed using the edge map obtained from the first fundamental form. In this way, each component image is filtered based on the common edge information. Experiments are performed on three types of images: colour images, multimodal MRI images and multispectral satellite images. To evaluate the proposed technique, it is compared to the single-valued equivalent, performing anisotropic filtering and watershed segmentation on the intensity image or the average band image. The manuscript is organized as follows: in the next section the first fundamental form is reviewed. In the third sec- 1

tion the waterhed algorithm is explained and in the fourth section the anisotropic diffusion algorithm is explained. Finally, in section 5, the experiments are conducted. 2 Multivalued image edge representation using the first fundamental form For the derivation of the first fundamental form, we will follow [5]. Let I(x; y) be a multivalued image with components I n (x; y);n = 1; :::N. The value of I at a given point is a N-dimensional vector. To describe the gradient information of I, let us look at the differential of I. In a Euclidean space: di @I @I = + (1) and its squared norm is given by (sums are over all bands of the image): (di) 2 = T 0 @ @I @x @I @I 0 T 2 @I @I @ P @In @x P @I n @x @I @y 1 2 A 2P @I n @In P @In @In @y @y = 1 2 A This quadratic form is called the first fundamental form. It reflects the change in a multivalued image. The direction of maximal and minimal change are given by the eigenvectors of the 2 2 matrix. The corresponding eigenvalues denote the rates of change. For a greylevel image (N =1), it is easily calculated that the largest eigenvalue is given by 1 = krik 2, i.e. the squared gradient magnitude. The corresponding eigenvector lies in the direction of maximal gradient. The other eigenvalue equals zero. For a multivalued image, both eigenvalues differ from zero. When 1 fl 2, the gradients of the different bands are in the same direction (equal or opposite orientation). When 2 ' 1, there is no preferential direction. In any case, 1 will give adequate edge information about the multivalued image. 3 Multivalued Watershed Algorithm In the immersion-based watershed algorithm of [8], the gradient magnitude of an image is calculated. That image is considered as a topographic relief where the brightness value of each pixel corresponds to a physical elevation. The technique can simply be described by figuring that holes are pierced in each local minimum of the topographic relief. In (2) the sequel, the surface is slowly immersed into a lake, by that filling all the catchment basins, starting from the basin that is associated with the global minimum. As soon as two catchment basins tend to merge, a dam is built. The procedure results in a partitioning of the image in many catchment basins of which the borders define the watersheds. For multivalued images, a single-valued approach can be adopted by segmenting each band separately, or by first combining the bands into a single greylevel image. The concept of the first fundamental form however allows to access gradient information from all bands simultaneously. By analogy with greylevel images, we will adopt 1 (x; y) as the gradient magnitude image of a multivalued image, on which the watershed procedure is applied. 4 Multivalued Anisotropic Diffusion In this section we will apply an adaptive filtering technique, based on anisotropic diffusion [10]. In [7], an anisotropic diffusion technique was described for colour images, based on the first fundamental form. We will apply the following procedure. From a multivalued image, the first fundamental form is calculated using (2). The eigenvectors and corresponding eigenvalues of the first fundamental form describe an ellipse in the image plane. Since the first eigenvector is directed along the gradient, it will be directed across an edge. The second eigenvector will the be directed along the edge. Anisotropic diffusion is based on the idea to smooth an image preferably along edges while trying to keep high frequency information across the edges. Using the first fundamental form, a locally adapted Gaussian smoothing kernel is constructed (see [11] for more information): G(r) =exp with standard deviations given by: ff 1 = ff 2 = ff 1+C ff 1+C ρ 1 r:v1 ff r:v 2 2 ff1 2 + ff2 2 ψ 1 1 2 1 + 2 2! where ff is the standard deviation of the image noise and C a measure for the corner strength. v 1 and v 2 are the eigenvectors corresponding to 1 and 2. The advantage of this smoothing kernel is that it is more extended along and less extended across the edges. The algorithm was originally designed for greylevel images, where instead of the first fundamental form (2), a quadratic form was used, in which the sum was taken over (3) (4)

a local window around each pixel [12]. The obtained Gaussian kernel was then convolved with the image. In the case of multivalued images, a Gaussian kernel can be calculated for each band separately. In [7], it is argued that it is better to calculate a kernel for the complete image, based on the first fundamental form, and to apply it to each band separately. The complete multivalued procedure, including filtering and segmentation then looks as follows: ffl Calculate the first fundamental form using (2) ffl From the obtained eigenvectors and eigenvalues, calculate the local smoothing kernels using (3) ffl Smooth each band using the obtained kernels ffl Recalculate the first fundamental form of the smoothed images, using (2) ffl Perform the watershed algorithm on the obtained 1 (x; y) 5 Experiments In this section the proposed techniques are evaluated. A severe drawback of the watershed procedure is oversegmentation. All kind of postprocessing techniques exist to deal with this problem. In this paper, we concentrate on the effect of applying segmentation in a multivalued way. Therefore, no attempt is made for reducing the oversegmentation, other than the filtering. We will compare the segmentation and filtering procedure with the single-valued procedure. For this, a greylevel image is generated from the multivalued image, which is not more than the average of all bands. For a colour image, this reflects the intensity of the image. Three different experiments are discussed: segmentation on a colour image, a multimodal MRI image and a multispectral satellite image. Results are compared qualitatively, with respect to the ability of the segmentation procedure to track relevant edges, and the effectivity of the filtering procedure. gradients appear, that are opposite to eachother in different bands. These are averaged away in the intensity. When treating the intensity gradient image as an 8-bit image, the strong gradients dominate the result. Apparently, important edge information can be missed. When using the first fundamental form, the opposite gradients are enhanced, and are less dominated by the strong gradients. 5.2 Multimodal medical image segmentation In figure 2, four different MRI images are shown, obtained by different imaging modalities (SFLASH, T e = 10;T r = 60; DSE-LATE, T e = 35; 120;T r = 3300; SE, T e = 30;T r = 2000; SE, T e = 30;T r = 4000). In figure 3, the segmented results are shown using the multivalued and the single-valued approach (no filtering was applied)(the background image is the average of the 4 bands). A similar result is obtained as with the colour image, with a similar explanation. 5.3 Multispectral satellite image segmentation In figure 4, two bands from a multispectral Landsat Thematic Mapper image are shown. Since a lot of local texture is present, oversegmentation can be avoided by prefiltering the images. This is done in a multivalued way, using the proposed technique. To compare performance, a segmentation is performed where the bands are averaged and the average is prefiltered applying the same anisotropic diffusion technique (single-valued filtering). In figure 5, the segmented results are shown, using multivalued and singlevalued filtering and segmentation respectively (the background is the filtered average of the two bands). Using the single-valued procedure, one still observes oversegmentation. This can be attributed to an inadequate filtering of the local texture. In the multivalued case, the filtering is more effective, so that the local texture is smoothed away, leading to a superior segmentation result. 5.1 Colour image segmentation The original peppers image is segmented using the proposed technique. To compare, the intensity image is generated, and segmented using the same watershed technique. No filtering is applied. In figure 1, the results are shown (the background is the intensity). One can notice that important edges have been missed using the single-valued approach. To explain this result, one has to look at the difference between the squared gradient image of the intensity and the image 1 (x; y). In the original image, numerous smaller

Figure 1. Segmentation result of peppers using the watershed procedure; a: multivalued; b: singlevalued. Figure 2. 4 MRI images, obtained with different imaging modalities.

Figure 3. Segmentation result using watershed technique; a: multivalued procedure; b: single-valued procedure. Figure 4. Two bands of a multispectral Landsat Thematic Mapper image.

Figure 5. Segmentation result using watershed; a: the multivalued approach; b: the single-valued approach. References [1] I.L. Thomas, V.M. Benning, and N.P. Ching, Classification of remotely sensed images, Adam Hilger, Bristol, 1987. [2] C Lee and D.A. Landgrebe, Analyzing highdimensional multispectral data, IEEE Trans. Geosci. Remote Sensing, vol. 31, no. 4, pp. 388 400, 1993. [3] T. Taxt and A. Lundervold, Multispectral analysis of the brain using magnetic resonance imaging, IEEE Trans. Med. Imaging, vol. 13, no. 3, pp. 470 481, 1994. [4] C. Busch, Wavelet based texture segmentation of multi-modal tomographic images, Comput. & Graphics, vol. 21, no. 3, pp. 347 358, 1997. [5] S. Di Zenzo, A note on the gradient of a multiimage, Comput. Vision Graphics Image Processing, vol. 33, pp. 116 125, 1986. [6] A. Cumani, Edge detection in multispectral images, CVGIP: Graphical Models and Image Processing, vol. 53, pp. 40 51, 1991. [7] G. Sapiro and D.L. Ringach, Anisotropic diffusion of multivalued images with applications to color filtering, IEEE Trans. Image Processing, vol. 5, 11, pp. 1582 1586, 1996. [8] L. Vincent and P. Soille, Watersheds in digital spaces: An efficient algorithm based on immersion simulations, IEEE Trans. Pattern Anal. Machine Intell., vol. 13, pp. 583 593, 1991. [9] D. Hagyard, M. Razaz, and P. Atkin, Analysis of watershed algorithms for greyscale images, Proceedings of the IEEE International Conference on Image Processing, vol. 3, pp. 41 44, 1996. [10] P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, IEEE Trans. PAMI, vol. 12,7, pp. 629 639, 1990. [11] G.Z. Yang, P. Burger, D.N. Firmin, and S.R. Underwood, Structure adaptive anisotropic filtering for magnetic resonance image enhancement, Proceeding of CAIP, pp. 384 391, 1995. [12] J. Sijbers, P. Scheunders, M. Verhoye, A. Van der Linden, D. Van Dyck, and E. Raman, Watershed-based segmentation of 3d mr data for volume quantization, Magnetic Resonance Imaging, vol. 15,6, pp. 679 688, 1997.