SEGMENTATION OF IMAGERY USING NETWORK SNAKES

Transcription

1 SEGMENTATION OF IMAGERY USING NETWORK SNAKES Matthias Butenuth Institute of Photogrammetry and GeoInformation, Leibniz Universität Hannover Nienburger Str. 1, Hannover, Germany KEY WORDS: Segmentation, Imagery, Snakes, Network, Topology, Shape ABSTRACT: A new methodology for the segmentation of imagery using network snakes is presented in this paper. Snakes are a well known tehnique, but usually are limited to losed objet boundaries. Enhaning traditional snakes the fous is on objets forming a network respetively being adjaent with only one boundary in between. In addition, the fous is on linear objets with opeon-fixed endings. The internal energy ontrolling the shape of the objet ontours during the energy minimization proess is defined for nodes with different degrees to enable the exploitation of the objet topology. Exemplary results of two different appliations demonstrate the funtionality and transferability of the proposed methodology: First, field boundaries are extrated from high resolution satellite imagery. The seond example from the medial setor deals with the delineation of adjaent ells in mirosopi ell imagery. Conluding remarks are given at the end to point out further investigations. 1. INTRODUCTION The segmentation of imagery is a well known problem in image proessing and omputer vision. One important methodology to delineate objets preisely are ative ontours, first introdued by (Kass et al., 1988). Ative ontours are a sophistiated image proessing tehnique ombining image features with shape onstraints in an energy minimization proess. Parametri ative ontours, often alled snakes (Kass et al., 1988; Blake and Isard, 1998), have a rigid topology, in ontrary to geometri ative ontours (Malladi et al., 1995; Caselles et al., 1997), whih are able to hange their topology due to flexible level sets and thus allow for extrating foreground objets without prior knowledge about their shape. Numerous approahes using snakes have been presented to detet different objets in many kinds of imagery, for example refer to medial image segmentation (MInerney and Terzopoulos, 1996; Suri et al., 2002), 3D deformable surfae models (Cohen and Cohen, 1993), the extration of roads using sale spae and snakes (Laptev et al., 2000) or the displaement of lines in artographi generalization tasks (Burghardt and Meier, 1997). Most of these approahes require losed ontours, whih desribe the boundary of an objet separately a limitation onerning linear objets with opeon-fixed endings and a limitation regarding objets, whih form a network and thus interat with eah other during the optimization proess. A new methodology of parametri ative ontours to overome these restritions is presented in this paper, alled network snakes. In the literature, only a limited amount of work an be found regarding ative ontours beyond expliitly or impliitly represented losed objet boundaries. Trihedral orners imposing onstraints of 90 degree angles between the three edges ending at the orner are used to extrat buildings in (Fua et al., 2000). An extension of parametri ative ontours, whih ombines the ability to handle transiently touhing objets and exerts topologial ontrol is given in (Zimmer and Olivio- Marin, 2005). An adaptive adjaeny graph onsisting of a network of ative ontours was firstly introdued in (Jasiobedzki, 1993) and afterwards utilized in (Dikinson et al., 1994) to trak 3D objets. The authors introdue onstraints in the form of springs to onnet the ontour ends, but the approah does not enable the definition of a unique nodal point inluding a geometrial ontrol of the ontours up to the nodal point. In ontrary, the methodology presented in this paper is able to handle objets with a giveetwork topology, but without the neessity of introduing any partiular onstraints. Possible appliations using network snakes are, for example, the extration of road networks, field boundaries as well as adjaent ells. For these purposes parametri ative ontours are more appliable than level set tehniques, sine the image-dependent energy terms of parametri ative ontours are defined speifially to individual objets. Multiple level sets as well as multiple parametri ative ontours are not suitable, beause they an interset or overlay eah other losing the orret topology. However, parametri ative ontours need an initialization to start the energy minimization proess. The required information an be taken from an initial segmentation or from a GIS, as long as a orret topology an be assumed. The next setion outlines traditional snakes, while Setion 3 fouses on the enhanements onerning network snakes. In Setion 4 exemplary results of two different appliations are presented to demonstrate the potential and the transferability of the proposed methodology: At first, the extration of field boundaries from high resolution satellite imagery is depited. Field boundaries have beome objets of inreasing interest during the last few years. Appliation areas are geo-sientifi questions suh as the derivation of potential wind erosion risk fields and appliations in the agriultural setor, for instane preision farming or the monitoring of subsidies. Work on the extration of field boundaries exists but is limited, for example refer to (Torre and Radeva, 2000; Aplin and Atkinson, 2004), but neither a fully automati solutioor the exploitation of the topology is used. The seond exemplary result highlighted in Setion 4 is the extration of adjaent ells in mirosopi ell imagery. Medial image segmentation has reeived a large attention, in partiular the delineation of ells for highthroughout biologial researh and drug disovery (Suri et al., 2002; Jones et al., 2005). However, the fous is mostly on

2 single ells not taking into aount the neighborhood. Finally, onluding remarks are given and further investigations are disussed in Setion TRADITIONAL SNAKES In this setion, parametri ative ontours are summarized in order to provide a basis for a disussion of their pros and ons onerning the enhanements ontained in Setion 3. Traditional snakes (Kass et al., 1988) are defined as a parametri urve () s ( x() s, y() s ) v =, (1) where s is the ar length and x and y are the image oordinates of the 2D-urve. In the simplest way, the image energy an be written as the image intensity itself with s ) I( v( s) ) E img =, (2) where I represents the image. In the literature, the image energy is often defined as E img s ) I ( v( s) ) 2 =. (3) I(v(s)) is the norm or magnitude of the gradient image at the oordinates x(s) and y(s). In pratie, the image energy E img (v(s)) is omputed by integrating the values I(v(s)), taken from preomputed gradient magnitude images along the line segments, that onnet the verties of the ontour. The internal energy is defined as E int ( ()) 2 2 v s = 1 () s v () s + β () s v () s 2 α s ss, (4) where v s and v ss are the first and seond derivative of v with respet to s. The funtion α(s) ontrols the first-order term of the internal energy: the elastiity. When the aim is to minimize E int (v(s)) and v(s) is allowed to move, large values of α(s) let the ontour beome very straight between two points. The funtion β(s) ontrols the seond-order term: the rigidity. Large values of β(s) let the ontour beome smooth, small values allow the generation of orners. α(s) and β(s) need to be predefined based on the modeled shape harateristis of the objet of interest. The total energy of the snake defined as 1 * snake = Esnake s )ds 0 1 = img int + 0 E E * snake, to be minimized, is [ E s ) + E ( v( s) ) E ( v( s) )]ds The additional external energy E on (v(s)) is introdued in (Kass et al., 1988) as an external onstrained fore, whih provides the on. (5) opportunity for individual fores at partiular parts or points of the ontour. With onstant weight parameters α(s) = α and β(s) = β a minimum of the total energy in Equation 5 an be derived by solving the Euler equation: E img v s ) () s () s + β v () s = 0 α v. (6) ss The derivatives are approximated with finite differenes sine they aot be omputed analytially. Converted to vetor notation with v i = (x i, y i ) and with E img (v(s)) / v(s) = f v (v) the Euler equations read α ssss ( vi vi ) αi+ 1( vi+ 1 vi ) 1( vi 2 2 vi + vi ) 2βi ( vi 2vi + v + 1 ) + 1( vi 2 vi v + 2 ) () = 0 i + β i i + β i i + f v v and an be rewritten in matrix form as () = 0 Av + f v v. (8) A is a pentadiagonal matrix, whih depends only on the funtions α and β. Equation 8 an be solved iteratively by introduing a step size γ multiplied with the negative time derivatives v / t, whih are disretized by v t v t 1. It is assumed that f v (v) is onstant during a time step, i.e. f v (v t ) f v (v t-1 ), yielding an expliit Euler step regarding the image energy. In ontrast, the internal energy is an impliit Euler step due to their speifiation by the banded matrix A. The resulting equation is ( v ) = ( v v ) t + f v t t t (7) Av γ. (9) The time derivatives vanish at the equilibrium ending up in Equation 8. Finally, a solution an be derived by matrix inversion: ( A + I) ( γ v κ f ( v )) vt = t v t γ, (10) where I is the identity matrix and κ is an additional parameter in order to ontrol the weight between internal and image energy. A requirement of traditional snakes is the neessity to have an initialization lose to the true objet boundary. Additional terms to inrease the apture range of the image fores and thus bridge larger gaps between initialization and true objet boundary, for example the balloon model (Cohen, 1991), are only appliable, when the bakground is relatively homogeneous and no disturbing strutures hinder the movement of the snake. Sine these onditions aot be guaranteed in general, a solution without suh additional terms is preferred in this work. Instead, strong internal energies are used ontaining the modeled shape harateristis of the objet of interest to be relatively independent of the initialization and those parts of the image energies, whih represent disturbing strutures.

3 3. NETWORK SNAKES In order to enhane traditional snakes to be able to deal with network topologies and open endings of ontours, a loser look to the internal energy E int (v(s)) ontrolling the shape part of the urve is required. The minimization of the internal energy during the optimization proess is only defined for losed objet boundaries, i.e. v 0 = v n (Kass et al., 1988), beause the derivatives are approximated with finite differenes (f. Equation 7). Most of the approahes to be found in the literature use losed ontours or define fixed end points when using open ontours. This proess requires orret end points before starting the snake optimization, whih often aot be guaranteed. Similarly, network topologies represented by single ontours ending in ommoodal points require predefined orret nodal points. In this work a new definition of snakes is given, ahieving a solution using image features and shape onstraints without fixed end or nodal points. At first, the topology of the initial ontour has to be derived. In addition to the nodes with a degree ρ(v) = 2 of the preliminary ontour v(s) eah node with a degree ρ(v) 2 has to be set up. Nodes with a degree ρ(v) = 1 define the end points and nodes with a degree ρ(v) 3 define the nodal points of the ontour (f. Fig. 1 for an example). Consequently, the ontrol of the total energy at the ommon nodal point v n = v a n = v b n = v n is defined for network snakes as follows: β β β ( va va ) β ( v ) + ( ) = 0 a v a f 2 v v a a ( vb vb ) β ( v ) + ( ) = 0 b v b f 2 v v b b ( v v ) β ( v v ) + f ( v ) = 0 n n n n 2 v (11) All three ontours interset in the ommoodal point without interating onerning their partiular shape. The energy definition of Equation 11 allows for a minimization proess ontrolling the shape of eah ontour separately, though ending in one ommon point exploiting the network topology. The matrix A of Equation 8 is adapted aordingly at the nodal points and their neighbors to fulfill the new definition of the internal energy, i.e. omitting some parts of the banded struture and/or filling up some additional parts to build further onnetions between different parts of the ontour. The definition of the internal energy for nodes with a degree ρ(v) > 3 is straightforward to the proposed method above, i.e. adding further parts to Equation 11. Similarly, the new internal energy is defined at the end points of a ontour: only one part of Equation 11 is needed, beause only one ontour without onnetion to other parts of the ontour is available. The adaptation of the matrix A is analogous ompared to nodal points. Thus, the ontrol of the shape is feasible by the end of the ontour without fixing the end points. Snakes have the tendeny to shorten during the energy minimization due to the first term (α-term) of Equation 4. A shortening of ontours with an open ending an be avoided by haining the end points at the image border allowing for movement only along the image borders (f. Fig. 1). Alternatively, the ontour an be hained at a topologially neighbored objet allowing for movement only along the objet border. When there are no neighbored objets to allow for haining the open endings of the ontour, a possible idea ould be the introdution of an external onstraint fore regarding a onstant length of the ontour. Figure 1. Topology of a network snake Imposing the network topology in the energy minimization proess auses a problem when solving Equations 7 and 8: the derivatives approximated by finite differenes are not defined for nodes with a degree ρ(v) = 1 or ρ(v) 3, beause the required neighboring nodes are either not available (ρ(v) = 1) or existing multiple times (ρ(v) 3). Thus, the shape ontrol an not be aomplished at these parts of the ontour in a traditional way. Let v a, v b and v represent three ontours, eah ending in a ommoodal point v n with a degree ρ(v) = 3. Regarding Equation 7, the first term, weighted by the parameter α, aot support the ontrol of the internal energy during the energy minimization proess in the viinity of v n when using network snakes: the finite differenes of the first term approximating the derivatives are only available for the two nodes v n-1 and v n, but not for v n+1. Thus, no shape ontrol is possible and the first term is not onsidered. The seond term of the internal energy, weighted by the parameter β, is rewritten using the available finite differenes ontrolling the urvature of the ontour. 4. EXEMPLARY RESULTS Results onerning the funtionality and apability of network snakes are presented in this setion. Two different appliations are shown to point out the transferability of the methodology: the extration of field boundaries from high resolution satellite imagery and the extration of ells from mirosopi ell imagery. 4.1 Extration of Field Boundaries The delineation of field boundaries within the omplex environment of vegetation is aomplished with olor satellite images having a resolution of two meters. The strategy for extrating the objets of interest is divided into two parts: First, a segmentation is arried out in a oarse sale to derive the topology taking into aount a somewhat inaurate geometrial position. The topology of the segmentation, however, is assumed to be orret. In a seond step, network snakes are used to improve the preliminary results exploiting the loal image features and the objet topology.

4 The initial segmentation of the imagery is briefly outlined below, for details refer to (Butenuth and Heipke, 2005). The use of prior knowledge from a GIS enables a partition of the image sene: Field boundaries are only loated within the open landsape and, in addition, the road network desribes already fixed field boundaries, beause fields naturally end at these objets. Within these regions of interest a multi-hannel region growing is performed using the RGB- and IR-hannels of the image resulting in an initial segmentation of the image (f. Fig. 2a, blak lines). Note, that the geometrial orretness of the segmentation has been artifiially degraded to emphasize the following steps in a better way. The result of the segmentation is used to derive the topology (f. Fig. 2b) and to initialize the network snake. Sine the objets to be extrated are rather straight, the parameter β is set to a large value ompared to α. Thus, image noise and small disturbanes have relatively small effets and, in addition, relatively oarse initial values of the ontour an be used. The open endings of the ontour are hained to the image borders, and are allowed for movement only along the borderline. In Figure 2 the apability of the network snake is highlighted: the ontours and in partiular the nodal points with a degree ρ(v) 3 and the end points move to the orret result, although the initialization is rather poor. The underlying image onsists of the standard a) b) ) d) Figure 2. Extration of field boundaries from high resolution satellite imagery (400 H 400 pixels): a) initialization of the network snake (blak); b) topology; ) initialization (blak), movement (thin blak) and result of the snake (white); d) result superimposed on the intensity hannel of the CIR-image

5 a) b) ) d) e) f) Figure 3. Extration of ells from mirosopi ell imagery (135 H 135 pixels): a) ell image; b) ell nulei and derived initialization (gray); ) initialization of the network snake (blak); d) topology; e) initialization (blak), movement (thin blak) and result of the snake (white ); f) result superimposed on ell image [imagery provided by Evote Tehnologies] deviation of the image intensities of the CIR-image within a quadrati mask, beause high values typially belong to field boundaries. Regarding the area around the nodal points, in partiular the point with a degree ρ(v) = 4, the internal energy and the exploitation of the topology speify the movement of the ontour during the first iteration steps, beause the values of the image energy are without effet. Step by step, the inverted image energy helps with small values pushing the ontour respetively the nodal points to the orret solution during the minimization proess. The final result is depited in Figure 2d superimposed on the satellite image. The geometrial orretness is in most parts onvining, only the tree rows on the left side prevent a learly defined field boundary and, thus, the image energy aot support the energy minimization proess in an optimal manner. However, the example demonstrates, network snakes are a powerful methodology to delineate objets preisely exploiting their topology. 4.2 Extration of Cells The delineation of adjaent ells in mirosopi ell imagery is the seond example presented in this setion. Figure 3a shows a mirosopi image of stained ytoplasm, whih fluoresed during the data apture. The depited image has a size of about 20 H 20 mirometers. Again, the strategy for extrating the objets of interest is divided into two parts: At first, a oarse objet ontour is needed to initialize the proessing and to derive the topology. In a subsequent step the geometrial orretness of the initial objet ontour is optimized using network snakes. The ell nulei are muh easier to detet than the boundaries of the ytoplasm (ell membrane), beause they are well defined against the bakground (f. Fig. 3b). The bakground of the ell nulei image is segmented followed by a alulation of the skeleton (f. Fig. 3b, gray line). Due to the fat, that eah ell nuleus is loated within the assoiated ell membrane, the skeleton an be used to initialize the network snake having a orret topology even though the geometrial orretness is not very high. Before starting the optimization proess, the objet ontour is thinned out taking into aount an equal distane between eah node (f. Fig. 3, blak line). The topology of the objet ontour is derived to set up the network snake (f. Fig 3d). Sine the objets to be extrated have a speifi urvature, the parameter β ontrolling the internal energy of the energy minimization proess is not set to suh a large value ompared to the extration of mostly straight field boundaries (f. Setion 4.1), yet is set again larger than α. A histogram linearization of the original image is aomplished to ease the optimization, beause the enhaned ontrast of the image helps to push the ontour to the orret solution. In Figure 3e the optimization proess is highlighted: Starting from the initialization (blak line) the movement of the network snake (thin blak line) is depited resulting in extrated ell boundaries (white line). In Figure 3f the final result is

6 superimposed on the ell image. The optimization proess works well, although there is strong image noise and in parts the image intensities aot help yielding the orret result. In the lower right part of the image one ell is not delineated ompletely, beause the initialization is too far away and the image energy is not able to push the ontour to the true objet boundary. 5. CONCLUSIONS A new segmentation methodology to delineate objets preisely from imagery using network snakes is presented in this paper. Using traditional snakes, adjaent objets, whih influene eah other and objets forming a network or having open endings are not defined due to the representation of the internal energy. In ontrary, network snakes exploit the topology of the objets of interest during the energy minimization proess omprising a omplete shape ontrol of the ontours. The exploitation of the topology turns out to be a powerful method to deal with noise and disturbanes in the imagery. The obtained objet ontours represent a superior geometrial solution when interating with eah other ompared to traditional snakes. Different results onerning the extration of field boundaries from high resolution satellite imagery and the extration of ells from mirosopi ell imagery demonstrate the potential and the transferability of the proposed methodology. In addition, the two examples emphasize, the requirement of a given orret topology an be ahieved in partiular appliations. However, when the assumption of a given orret topology aot be guaranteed, an additional intervention omprising the global view of the optimized network has to be onsidered to insert or delete parts of the ontour. Possible further appliations are the delineation of other objets suh as road networks, and other topis suh as the update of GIS-data with an already given topology. In addition, the ontrol of the internal energy an be improved hoosing varying values of the parameters α and β, if the modeled objet shape has these harateristis. A further interesting question is the behavior of the iteration proess: are there dependenies of the initialization, internal energy and image harateristis and an they be exploited? For example, if the objet of interest has other shape harateristis than the image disturbanes or noise, the ontrol of the internal energy and the exploitation of the topology an allow for a relatively oarse initialization. Investigations regarding this problem an give speifi answers about the required quality of the image data and the initialization. REFERENCES Aplin, P. and Atkinson, P. M., Prediting Missing Field Boundaries to Inrease Per-Field Classifiation Auray. Photogrammetri Engineering & Remote Sensing, Vol. 70, No. 1, pp Blake, A. and Isard, M., Ative Contours. Springer, Berlin Heidelberg New York, 351 p. Burghardt, D. and Meier, S., Cartographi Displaement Using the Snake Conept. In: Förstner, Plümer (eds.), Semanti Modeling for the Aquisition of Topographi Information from Images and Maps, Basel, Birkhäuser Verlag, pp Butenuth, M. and Heipke, C., Network Snakes-Supported Extration of Field Boundaries from Imagery. In: Kropatsh, Sablatnig, Hanbury (eds.), Leture Notes in Computer Siene, Vol. 3663, Springer Verlag, pp Caselles, V., Kimmel, R. and Sapiro, G., Geodesi Ative Contours. International Journal of Computer Vision, Vol. 22, No. 1, pp Cohen, L. D. and Cohen, I., Finite Element Methods for Ative Contour Models and Balloons for 2-D and 3-D Images. IEEE Transations on Pattern Analysis and Mahine Intelligene, Vol. 15, No. 11, pp Cohen, L. D., On Ative Contour Models and Balloons. CVGIP: Image Understanding, Vol. 53, No. 2, pp Dikinson, S. J., Jasiobedzki, P., Olofsson, G. and Christensen, H. I., Qualitative Traking of 3-D Objets using Ative Contour Networks. Proeedings IEEE Conferene on Computer Vision and Pattern Reognition, Seattle, pp Fua, P., Grün, A. and Li, H., Optimization-Based Approahes to Feature Extration from Aerial Images. In: Dermanis, Grün, Sanso (eds.), Geomati Methods for the Analysis of Data in the Earth Sienes, Leture Notes in Earth Siene, Vol. 95, Springer, pp Jasiobedzki, P., Adaptive Adjaeny Graphs. SPIE, Vol Geometri Methods in Computer Vision II, San Diego, pp Jones, T. R., Carpenter, A. and Golland, P., Voronoi- Based Segmentation of Cells on Image Manifolds. In: Liu, Jiang, Zhang (eds.), Leture Notes in Computer Siene, Vol. 3765, Springer, pp Kass, M., Witkin, A. and Terzopoulus, D., Snakes: Ative Contour Models. International Journal of Computer Vision, Vol. 1, pp Laptev, I., Mayer, H., Lindeberg, T., Ekstein, W., Steger, C. and Baumgartner, A., Automati Extration of Roads from Aerial Images Based on Sale Spae and Snakes. Mahine Vision and Appliations, No. 12, pp Malladi, R., Sethian, J. A. and Vemuri, B. C., Shape Modeling with Front Propagation: A Level Set Approah. IEEE Transations on Pattern Analysis and Mahine Intelligene, Vol. 17, No. 2, pp MInerney, T. and Terzopoulos, D., Deformable Models in Medial Image Analysis: A Survey. Medial Image Analysis, Vol. 1(2), pp Suri, J. S., Kamaledin Setarehdan, S. and Singh, S., Advaned Algorithmi Approahes to Medial Image Segmentation: State-of-the-Art Appliations in Cardiology, Neurology, Mammography and Pathology. Springer Verlag, 668 p. Torre, M. and Radeva, P., Agriultural Field Extration from Aerial Images Using a Region Competition Algorithm. International Arhives of Photogrammetry and Remote Sensing, Vol. XXXIII, Amsterdam, No. B2, pp Zimmer, C. and Olivio-Marin, J. C., Coupled Parametri Ative Contours. IEEE Transations on Pattern Analysis and Mahine Intelligene, Vol. 27, No. 11, pp