Intalzng Snakes W. Neuenschwander, P. Fua y, G. Szekely and O. Kubler Communcaton Technology Laboratory Swss Federal Insttute of Technology ETH CH-892 Zurch, Swtzerland Abstract In ths paper, we propose a snake-based approach that lets a user specfy only the dstant end ponts of the curve he wshes to delneate wthout havng to supply an almost complete polygonal approxmaton. We acheve much better convergence propertes than those of tradtonal snakes by usng the mage nformaton around these end ponts to provde boundary condtons and by ntroducng an optmzaton schedule that allows the snake to take mage nformaton nto account rst only near ts extremtes and then, progressvely, towards ts center. These snakes could be used to allevate the often repettve task practtoners have to face when segmentng mages by abolshng the need to sketch a feature of nterest n ts entrety, that s, to perform a panstakng, almost complete, manual segmentaton. 1 Introducton In recent years snakes, have emerged as a very powerful tool for semautomated object delneaton. They have been orgnated by Terzopoulos, Kass, and Wtkn [1, 2] and have snce gven rse to a large body of lterature ([3, 4, 5, 6] among many others) that explores theoretcal and mplementaton ssues as well as new applcatons. In most of these papers, however, t s assumed that the ntal poston of the snake s relatvely close to the desred soluton. Whle ths s a reasonable assumpton for applcatons such as moton trackng [7, 8], t s neectve for delneatng complex objects from scratch. The optmzaton of the tradtonal snakes s typcally global and takes edge-nformaton nto account along the whole curve smultaneously. When the snake's ntal poston s far away from the desred result, ths often results n the snake gettng stuck n an undesrable local mnmum because t uses rrelevant edge nformaton. The mnmzaton problem y SRI Internatonal Artcal Intellgence Center 333 Ravenswood Avenue Menlo Park, CA 9425, USA s solved by treatng the snake as a physcal system evolvng under the nuence of the potental that s the sum of an objectve functon and a dsspaton potental that enforces convergence. Ths potental tends to prevent the snake from movng far away from ts current poston, thereby contrbutng to forcng the snake's ntal poston to be close from the desred soluton n order to acheve convergence and not get stuck n an undesrable local mnmum. In the remander of the paper, we wll refer to these snakes as \dynamc snakes" because ther poston s computed by solvng the dynamcs equaton of a physcal system. Here, we descrbe a snake approach that allows a user to specfy only the end ponts of the curve he wshes to delneate nstead of a complete polygonal approxmaton. Ths way we may abolsh the need to outlne the desred structure very precsely, that s, to perform a panstakng, almost complete, manual segmentaton. The optmzaton progresses from the end ponts towards the center thereby eectvely propagatng edge-nformaton along the curve wthout the need for a dsspaton potental. The user-suppled end ponts and the automatcally computed edge gradent n ther vcnty serve as anchors. They are rst used to compute an ntal state that s approxmately correct n the anchors' vcnty. Whle the mage term s \turned on" progressvely from the snake's extremtes towards ts mddle secton, the snake's poston s teratvely recomputed. As a result, the snake eventually nds the smooth path, as dened by the regularzaton term, that best matches the edge connectng the two end ponts and has the rght orentaton at these ponts. We wll refer to these snakes as \statc" snakes. In the followng secton, we descrbe the tradtonal or dynamc snakes. Next, we ntroduce our own breed of statc snakes. Fnally, we present results on both synthetc and real mages that demonstrate the mproved performance of the statc snakes for ntalzatons that are far away from the desred result.
(a) Fgure 1: Ill condtoned behavor of the dynamc snakes wth respect to ntalzaton. (a) Slghtly derent ntalzatons: the snake s ntalzed usng a polygon wth ve vertces. Whle the thrd vertex moves closer to the shape, the other vertces are the same for all sx stuatons. The correspondng results: the snake detects the correct contour (6th mage par) when the thrd vertex s close enough to the object's border. However, t s not ntutvely obvous what the threshold s. 2 Dynamc snakes The orgnal snakes [2] are modeled as tme dependent 2{D curves dened parametrcally as ~v(s; t) = (x(s; t); y(s; t)) s1 ; (1) where s s proportonal to the arc length, t the current tme and x and y the curve's mage coordnates. The snake deforms tself as tme progresses R so as to mnmze an mage potental E I (~v) =? P (~v(s; t)) ds, 1 where P (~v(s; t)) s a functon of the mage. One typcal choce s to take P (~v(s; t)) to be equal to the magntude of the mage gradent. However, because the gradent magntude can vary rapdly due to contrast changes and to nose, t has proven eectve to ether clp the derved potental force [4] or replace the gradent magntude by ts logarthm [3]. Alternatvely, one can take P (~v(s; t)) to be the Eucldean dstance to the closest edge-pont n an edge-map computed usng an operator such as the Canny edge-detector [9]. The results of all these approaches are very smlar. In our mplementaton, descrbed n secton 3, we use the latter where the Eucldean dstances are computed usng the Danelsson dstance transform [1]. Unfortunately, whatever the choce of P, E I (~v) s typcally not a convex functonal. To perform the optmzaton, followng Terzopoulos et al., one must mnmze an energy E(~v) that s the sum of E I (~v) and of a regularzaton term E D (~v). Usng the thn-plate model, E D (~v) s taken to be E D (~v) = 1 2 Z 1 (s) @~v(s; t) @s 2 +(s) @2 ~v(s; t) @s 2 2 ds ; (2) where (s) and (s) are arbtrary functons that regulate the curve's tenson and rgdty. In the mplementaton descrbed n secton 3, and are taken to be constant and are suppled by the user. We have shown [3] that they can be chosen n a farly magendependent way. Several technques, however, have been proposed to dynamcally adjust the values of and along the curve (see [11] for example). Varatonal calculus shows that f v mnmzes E = E D + E I and s sucently regular, that s at least C 4 (; 1), then t must be a soluton of the Euler dfferental equaton?rp (~v(s; t)) =? @ @~v(s; t) (s) @s @s + @2 (s) @ 2 ~v(s; t) @s 2 @s 2 (3) Note that, n order to have a unque soluton for ths equaton, one must specfy boundary condtons such as the values and dervatves of ~v(s; t) s2f;1g. In practce, to mnmze E(~v), one must dscretze the curve ~v by samplng t at regular ntervals. We therefore take ~v to be a polygonal curve dened by a set of vertces ~v t = (x t ; y t) n. Usng nte derences, the snake energy E(~v) becomes E(~v) = E I (~v) + E D (~v) where E I (~v t ) =? E D (~v t ) = 1 2 + 1 2 X X X P (x t ; yt ) h (x t? xt?1) 2 + (y t? yt?1) 2 h(2x t? xt?1? xt +1) 2 + (2y t? yt?1? yt +1) 2 : Note, E D (~v t ) s quadratc and can be rewrtten as E D (~v t ) = 1 2 X T t KX t + 1 2 Y T t KY t ; (4)
(a) Fgure 2: Senstvty of dynamc snakes to nearby contours. (a) Derent ntalzatons: the snake s ntalzed usng a polygon wth an ncreasng number of vertces. The correspondng results: The fact that the two objects are lyng close to each other forces the user to outlne the desred contour segment precsely. If the snake touches the nuence regon of a nearby object t wll get stuck on the wrong contour. where K s a (n+1) (n+1) penta-dagonal matrx, and X = (x ; x 1 ; : : : ; x n ) and Y = (y ; y 1 ; : : : ; y n ) are the vectors of the x and y vertex coordnates. A curve that mnmzes the energy E must be such that @E = @E D + @E I = @~v @~v @~v : (5) Snce the deformaton energy E D n equaton (4) s quadratc and decouples the x and y coordnates of the curve, equaton (5) can be rewrtten as a system of equatons K@ X Y 1 A = @ F X F Y 1 A where FX =? @E I @X ; F Y =? @E I @Y (6) whch are coupled by the \mage forces," F X and F Y. Note that F X and F Y depend on the snake's poston, makng the system sem-lnear. The matrx K, however, s not nvertble and these equatons cannot be solved drectly. Ths stems from the fact that the Euler derental equaton (3) has a unque soluton only when boundary condtons are suppled. In secton 3, we wll show how we can solve ths system of equatons by supplyng the boundary condtons. Ths wll be one of the key derences between our approach and more tradtonal snake mplementatons n whch the mnmzaton of E(~v) s acheved by embeddng the curve nto a vscous medum and solvng the equaton of the dynamcs. Ths amounts to addng a Raylegh dsspaton functonal to the energy and leads to solvng the followng derental equaton: @E + @~v d~v = dt ; where s the vscosty of the medum. As n the case of equaton (6), after tme dscretzaton, ths can be rewrtten as a set of two equatons n X and Y : (K + I)X t = X t?1 + F X (K + I)Y t = Y t?1 + F Y (7) where K, F X and F Y are dened n equaton (6). The matrx (K + I) s postve dente for >. The dynamc snakes are eectve when the ntal poston of ~v s close from the desred soluton. However, as llustrated by gures 1 and 2, they are very senstve to ntal condtons. They can easly get caught n local mnma when the desred outlne presents large concavtes that force the snake to extend tself or when there are other edges n the vcnty of the desred one that may \catch" the snake. In the next secton, we ntroduce a derent breed of snakes, the \statc snakes" that allevate these problems by replacng the vscosty term by boundary condtons that produce better solutons of equaton (3). 3 Statc snakes To mprove upon the snakes' convergence propertes, we must use constrants that better reect the mage propertes than the Raylegh dsspaton functonal of secton 2. In our mplementaton, we assume that the user speces snake end ponts n the vcnty of clearly vsble edge segments, whch mples a well dened edge drecton. It becomes natural to use these ponts and ther assocated edge drectons as anchor ponts and to propagate the edge nformaton along the curve startng from them. We use these anchor ponts to derve an ntal poston for the snake whch wll, n general, be close to the desred answer n the vcnty of those ponts but
(a) Fgure 3: Evoluton of a statc snake on the synthetc mages of gures 1 and 2. The crcles denote the farthest vertces away from the end ponts for whch the mage forces are turned on. The optmzaton stops when the two crcles meet (secton 3.3). nowhere else. We wll therefore \turn on" the mage forces (6) n those areas and compute a new poston for the snake usng the same boundary condtons as before. A longer part of the new soluton wll be closer to the actual mage edge than before; the mage forces can then turned on on ths longer part and the snake's poston recomputed. By teratng ths process, we eventually turn on the mage forces over the whole length of the snake, thereby achevng the propagaton of the edge nformaton from the anchor ponts to the snake's mddle. Our approach s closely related to perturbaton theory [12]: we start wth an unperturbed soluton of our mnmzaton problem and progressvely perturb t by consderng more and more of the mage forces. Dynamc programmng provdes an alternatve approach to nd an optmal path between selected ponts. However, there are strct contrans for the applcable cost functon and the method s bascally restrcted to 2-D path ndng problems. The lack of these lmtatons n the snake approach allows the generalzaton of our method to a broader class of problems. 3.1 Solvng the mnmzaton problem wth boundary condtons As dscussed n secton 2, mnmzng the snake's energy amounts to solvng the Euler derental equaton (3) whch leads, after dscretzaton, the semlnear system of the two equatons (6). By xng the curve's end ponts (x ; y ) and (x n ; y n ) and gvng the curve's tangent at those ponts, the above system of equatons reduces to: K X = F X ; K Y = F Y where X s the n? 1 vector (x 1 ; : : : ; x n?1 ), Y the n? 1 vector (y 1 ; : : : ; y n?1 ) and K an (n? 1)(n? 1) penta dagonal matrx that s now nvertble. The system s stll sem-lnear and cannot, n general, be solved n closed form. Instead, one must stll use a tme dscretzaton and teratvely solve the system K X t = F X K Y t = F Y 3.2 Intalzaton X =Xt?1 ;Y =Yt?1 X =X t?1 ;Y =Y t?1 In order to successfully optmze our snake, we must start from an ntal poston that s approxmately correct n the neghborhood of the end ponts. The easest way to acheve ths result s to solve the homogeneous equatons that correspond to the system of Euler equatons (3). As dscussed n secton 2, we take and to be constant, and the homogeneous system becomes? d 2 v(s) ds 2 + d 4 v(s) = (8) ds 4 where v stands for ether x or y and s 1. We assume, that the user has chosen the snake's head and tal close to domnant edge fragments. In order to nd the true edge locaton n the near neghborhood of the selected start and end ponts we rst perform a lnear search. The snake tangent at ts head and tal s then gven by the valley drecton n the potental surface correspondng to the selected contour fragments. It s obtaned from the orentaton map that can be computed at the same tme as the Canny edge-map. Whle the tangent drecton at the end ponts can be computed, ts orentaton cannot be determned. By default, the boundary condtons are chosen so that the ntal snake denes acute angles :
(a) (c) Fgure 4: Outlnng roads n an aeral mage. (a) Statc snakes ntalzatons. Fnal results. All the road edges are correctly outlned except the bottom left one. (c) The erroneous result s corrected by addng a new control pont. wth the lne jonng the two end-ponts. Snce ths heurstc may fal, we provde the user wth the possblty to p the orentaton at both ends. Ths could be automated by ntalzng the snake n the four possble ways (2 possble orentatons at each end) and retanng the best nal result. By constructon, ths soluton has the rght tangent at the end ponts and s close to the rght answer near these ponts. It can therefore serve as an ntal curve for the followng mnmzaton of the energy functonal. s the Danelsson dstance map. We shft both boundares across the contguous vertces whose potental does not exceed the one of the currently actve ones by more than 1. Ths strategy allows the snake to leave valleys and to close small gaps. However, further nvestgaton s requred to better control ths \gap closng" mechansm. We are also workng on modfyng these heurstcs to deal wth a potental surface derved drectly from the mage gradents. 3.3 Optmzaton procedure We start the optmzaton of the energy term by denng the ntal snake as the soluton of the homogeneous derental system of equaton (8). At ths stage the snake \feels" absolutely no external potental forces. Durng the ongong teratve optmzaton process the mage potental s turned on progressvely for all the snake vertces, startng from the extremtes. Assumng that the user selects the end ponts nearby domnant edge fragments n the mage, ths ntalzaton ensures that the snake les already close to ts optmal poston at both ends. We dene the \force boundares" as the locaton of the vertces farthest away from the end ponts that feel the mage forces. These boundares approach each other durng the ongong optmzaton process accordng to the followng rule: Each boundary s moved at every teraton step by at least one vertex. To speed up the convergence, we use the fact that our potental surface 4 Results In ths secton, usng both synthetc and real mages, we compare the dynamc snakes wth our statc snakes and show that the formers' ntalzaton must typcally be much closer to the desred answer to acheve comparable results. To acheve a far comparson, we use the same tenson and rgdty parameters, and dened n equaton (2), for both knds of snakes. Fgure 3 llustrates a statc snakes' behavor on two synthetc mages and ts ablty to outlne the cavty and dstngush between two nearby objects. Fgure 4 shows that our statc snakes can be used to delneate roads n an aeral mage usng very dstant end-ponts. Note, however, that our snakes can stll become confused n the presence of junctons. Ths s the case for the snake drawn n the bottom left corner of gure 4, whch s beng trapped by an undesrable local mnmum. In practce, when such problems arse, our nterface allows the user to add a new pont n the mddle of the curve, thereby splttng t nto two
snakes (see gure 4(c)). Fgures 5 and 6 show the snake's performance on an mage of an apple and llustrate smlar results on a low contrast face mage. (a) Fgure 5: Detecton of derent mage features by statc snakes. The apple's outlne s detected n (a) whereas shows the detecton of the projected shadow. (a) Fgure 6: Outlnng facal features. (a) Three pars of end ponts on a face mage (Courtesy of INRIA). Fnal results. 5 Concluson We have proposed a snake-based approach to semautomated delneaton that allows a user to outlne an open contour by only specfyng very dstant end ponts and allowng the computer to propagate edgenformaton from the extremtes towards the center. Ths yelds excellent convergence propertes for the snakes and dmnshes very substantally the probablty gettng trapped nto an undesrable local mnmum. By sequentally denng end ponts of adjonng open snake fragments we are able to segment more complex shapes. Intellgent ntalzaton of snakes s requred to make them nto an operatonal tool for mage segmentaton where exstng mplementatons leave almost all the work to an "expert user" [2]. Our method has been desgned for practtoners of mage evaluaton wthout a professonal background n mage understandng. We concentrate our future research on generalzng our approach to handle more general constrans and problems of hgher dmensonalty. References [1] D. Terzopoulos, A. Wtkn, and M. Kass. Symmetry-seekng Models for 3D Object Reconstructon. IJCV, 1(3):211{221, October 1987. [2] M. Kass, A. Wtkn, and D. Terzopoulos. Snakes: Actve contour models. IJCV,1(4):321-331,1988. [3] P. Fua and Y.G. Leclerc. Model Drven Edge Detecton. Machne Vson and Applcatons, 3:45{ 56, 199. [4] F. Leymare and M.D. Levne. Trackng deformable objects n the plane usng an actve contour model. IEEE PAMI, 15(6):617{634, 1993. [5] I. Cohen, L. D. Cohen, and N. Ayache. Usng Deformable Surfaces to Segment 3-D Images and Infer Derental Strucutres. CVGIP: IU, 56(2):242{263, September 1992. [6] L.H. Stab and J.S. Duncan. Boundary Fndng wth Parametrcally Deformable Models. IEEE PAMI, 14(11):161{175, November 1992. [7] D. Terzopoulos and R. Szelsk. Trackng wth Kalman Snakes. In A. Blake and A. Yulle, edtors, Actve Vson. The MIT Press, 1992. [8] B. Bascle and R. Derche. Stereo Matchng, Reconstructon and Renement of 3D Curves Usng Deformable Contours. In ICCV, pages 421{43, Berln, Germany, 1993. [9] J. Canny. A computatonal approach to edge detecton. IEEE PAMI, 8(6):679{698, 1986. [1] P.E. Danelsson. Eucldean dstance mappng. CVGIP, 14:227{248, 198. [11] R. Samadan. Changes n connectvty n actve contour models. In Proceedngs of the IEEE Workshop on Vsual Moton, Irvne, Calforna, pages 337{343, March 1989. [12] A. W. Bush. Perturbaton methods for engneers and scentsts. CRC Press, 1992.