4. Levelset or Geometric Active Contour

Transcription

1 32 4. Levelset or Geometric Active Cotour The sake algorithm is a heavily ivestigated segmetatio method, but there are some limitatios: it is difficult to let the active cotour adapt to the topology. This ca be see whe a sake eters a log cocavity or whe two objects are surrouded by oe sake. The sake has trouble eterig the etire cocavity or detectig the two objects as separate objects. To solve the problems of eterig cocavities ad splittig the active cotour the levelset method was proposed by Sethia ad Osher [22] ad applied o image processig by Malladi [16] ad Casselles [4]. I this chapter first the basic idea of the levelset method is explaied. The the flow fuctio that is required for image segmetatio is discussed ad the improvemet arrow bad extesio. The ext sectio discusses the additio of a extra term that attracts the cotour stroger to the edges. The additioal term ca also be replaced by a gradiet vector flow, which is discussed ext. Fially the levelset method is applied o some test images to determie the robustess of the method, the results are discussed ad some further ehacemets that ca be implemeted are give. 4.1 Theory Malladi [16] ad Casselles [4] idepedetly applied the levelset method developed by Sethia ad Osher [22] o image processig. The basic idea of the levelset method is to embed the propagatig curve i a higher dimesioal fuctio ad evolve this higher dimesioal fuctio i such a way that the propagatig curve approaches the desired features, usually edges, of the image. I this sectio the evolutio of the curve as part of a higher dimesioal fuctio is derived. Let (0) be a closed ad smooth iitial cotour evolvig accordig to the flow fuctio F i the ormal directio to the curve as depicted i figure 4.1. The flow fuctio oly works i the ormal directio as a flow or chage i the tagetial directio is ot oticeable or relevat. The curve simply slides alog it's path. The flow fuctio ca deped o several geometric parameters, as the curvature of the cotour itself, image features, etc.

2 33 image Import "C: Documets ad Settigs s My Documets Virjaad Verslag levelset cotourf.gif" 1, 1 ; ListDesityPlot image ; Figure 4.1: Curve propagatig with speed F i ormal directio. Now accordig to the levelset method by Osher ad Sethia [22] this iitial cotour is embedded as the zero levelset i a higher dimesioal fuctio. For the higher dimesioal fuctio usually the siged distace fuctio is take, which is defied i equatio 4.1. x, t 0 d (4.1) Where d is the distace from x to (0) ad is egative iside the iitial cotour ad positive outside. This is illustrated i figure 4.2 i which the iitial cotour is give by a circle. This circle is the zero levelset of a siged distace fuctio which is a coe. sigeddistace x_, y_, poits_, rad_ : Module res, res Table Mi Dot #, # rad & poits, i, 1, x, j, 1, y ; sigeddistace 128, 128, 64, 64, 30 ; Graphics`Graphics` DisplayTogetherArray ListCotourPlot, Cotours 0, Frame True, FrameTicks Noe, CotourStyle RGBColor 1, 0, 0, Thickess 0.001, ListPlot3D, Mesh False, ViewPoit 1, 0.1, 0.1 ; Figure 4.2: Circle as iitial cotour ad the siged distace fuctio, which is the iitial. The fuctio is ow evolved ad with this implicitly the cotour (t), which is represeted as the zero levelset of, is evolved. To illustrate this cosider the fuctio of figure 4.2. Whe a costat is added to

3 34 the fuctio all the values of are icreased ad the coe moves up. As a result the zero levelset shriks as a lower part of the coe is ow the zero levelset. Whe a costat is subtracted the reverse occurs. Now cosider the arbitrary cotour from figure 4.1 which evolves accordig to the flow fuctio i the ormal directio to the cotour. How must the fuctio evolve to esure that the cotour evolves as depicted? Cosider a poit x o the iitial cotour (0). This poit moves accordig to the flow fuctio i the ormal directio: x ' t F Now fill i the poit x i the 4.3 is the result. (4.2) fuctio, as the poit x(t) is o the cotour ad thus the zero levelset equatio x t, t 0 (4.3) To see how evolves i time the derivative of to time ca be take by simply applyig the chai rule: d dt t x ' t 0 (4.4) Takig ito cosideratio equatio 4.2 ad the case that the ormal of the zero levelset is give by the followig fuctio. (4.5) The evolutio equatio of becomes as give i equatio 4.6: t F 0 (4.6) 4.2 Flow Fuctio The flow fuctio determies accordig to which parameters the zero levelset evolves. Therefore it depeds o what features are to be extracted how this flow fuctio looks like. First let us cosider the flow fuctio to be a costat. F (4.7) If is positive the zero levelset moves outward ad if it is egative it moves iward. Aother possibility is to let the flow fuctio be depedet o the curvature. F (4.8) I which is a costat ad is the curvature. The curvature is give by the followig equatio. xx y 2 2 x y xy yy x 2 x 2 y 2 (4.9) i which xx ad yy are the secod derivative of to the x ad y directio respectively. Wheever is egative the curve stays smooth ad shriks, if is chose to be positive however, the solutio is highly erratic as show i figure 4.3.

4 35 DisplayTogetherArray ListCotourPlot phistore, Cotours 0, ListCotourPlot phi, Cotours 0 ; Figure 4.3: Curve evolved with = 1 (left) ad = -1 (right). The combiatio of a costat ad the curvature give a simple, but effective flow fuctio: the curve expads or collapses accordig to ad the curvature maitais its smoothess. There is however o exteral iformatio icorporated ito this equatio. I this case it is desired to extract a object from a image. There must be a way to make the evolutio stop at certai image features, preferably the boudaries of the object which has to be segmeted. Therefore a stoppig term is used. This stoppig term must have the property to become very small, possibly zero, ear the edges ad approach uity at areas with uiform itesity. A example of such a stoppig term is give i the followig equatio. g 1 1 G I (4.10) I which I represets the image which is covolved with a Gaussia operator with width. There is a objectio agaist usig equatio 4.10 as the stoppig term. The oe i the deumerator has o physical meaig. Furthermore it ca just as easy be aother value. A icer stoppig term would be the followig. g e G I (4.11) I which is a positive costat. This has the same properties as equatio 4.10, but falls faster to zero [16]. The resultig fial evolutio equatio ow becomes: t g 0 (4.12) The flow fuctio described above is a good startig poit. There are however three mai methods to improve the performace of the levelset i the literature. The first focusses o the leakage prevetig. The secod method chages the flow fuctio as give above to icorporate a drivig force that depeds o statistical meas. The fial method uses the levelset to evolve bubbles as multiple iitializatios [25]. A problem of the levelset is that at weak edges leakage occurs. This meas that the cotour crosses this border, after which it keeps evolvig outward or iward respectively. Oe way to solve this is by addig a extra stoppig term to equatio 4.12 as discussed i sectio 4.4 [14, 25, 32]. This stoppig term ca be see as a extra force poitig towards the edge, usually the gradiet of the image is used. As with the sake this gradiet ca be smoothed out over the image as discussed i sectio 4.6 [31]. I this case the cotour is attracted from further away ad gaps i the gradiet image cause less leakage.

5 36 A differet stoppig term has bee developed by Siddiqi, which ca be used i combiatio with the first stoppig term [23]. The secod stoppig term is a area miimizig term, which provides additioal attractio towards the edge. I stead of a costat that works as a costat pressure force i equatio 4.12, a statistical force ca be used. By cosiderig the mea itesity of the object to be segmeted ad the mea of the surroudig pixels a force ca be applied o the levelset that pushes the cotour away from the secod towards the first. This has bee doe for example by Zeg et al. [33]. Aother area of ivestigatio is called bubbles. I this area a set of seed poits is used as startig poits, which grow, shrik, merge or split accordig to the evolutio of the levelset [15]. By lettig the bubbles evolve uder the ifluece of the image, objects ca be segmeted. This could be a first step i creatig a automatic iitializatio. Research has also bee doe o the implemetatio of the levelset ad more specifically o the optimizatio of the implemetatio. Oe of the first optimizatio techiques implemeted was that of the arrow bad [16]. I this techique the fuctio is oly updated i a arrow bad aroud the zero levelset. This meas that less values have to be calculated ad this speeds up the process. How much the process is sped up depeds o the size of the cotour, but istead of calculatig the values for all the pixels, oly the values aroud the cotours have to be calculated. This is further discussed i sectio 4.5. Aother improvemet is to apply the fast marchig method to determie the iitial fuctio [22]. It is used to calculate the siged distace trasform from a give curve. This is a fast method that ca be performed before the levelset fuctio is implemeted ad helps the levelset reach the fial borders faster. 4.3 Implemetatio First a startig cotour is chose by the user, by selectig pixels aroud which small circles are draw. These circles make up the iitial cotours ad the siged distace fuctio of these cotours is the startig fuctio. A example of a iitial is show i figure 4.2. The stoppig term is calculated from the iput image accordig to equatio The iput image is a simple white circle o a black backgroud. The result is show i figure 4.4. image Table If x 2 y 2 300, 100, 0, x, 63.5, 63.5, y, 63.5, 63.5 ; g Exp gd image, 1, 0, 1 2 gd image, 0, 1, 1 2 ; DisplayTogetherArray ListDesityPlot image, ListDesityPlot g ;

6 37 Figure 4.4: The iput image ad the stoppig term derived from it. Now the fuctio must be updated. Equatio 4.12 is approximated i time by a forward Euler derivative. 1 t F 0 (4.13) 1 t F (4.14) i which t is the time step ad F the flow fuctio as discussed i the previous sectio. Equatio 4.14 ca ot simply be calculated like that. This is illustrated i figure 4.5. I the first graph a frot is propagated ad is calculated with cetral differece. I the secod graph a Gaussia derivative is used. I the fial graph the propagatio scheme developed by Sethia is used. DisplayTogetherArray ListCotourPlot back, Cotours 0, ListCotourPlot gaus, Cotours 0, ListCotourPlot seth, Cotours 0 ; Figure 4.5: Differet implemetatio schemes: backward differece (left), Gaussia derivative (middle) ad Sethia scheme (right). As ca be clearly see the backward differece scheme causes artifacts. The artifacts that form are caused by umerical istability. The Gaussia scheme does ot seem to have these artifacts, but further ivestigatio of this scheme is required. I this report the scheme proposed by Sethia is used. For a further discussio see Sethia [22]. I the scheme developed by Sethia forward ad backward differece of the fuctio is combied i such a way that the iformatio flows from where the curve has bee to where it is ow. To illustrate this cosider a circle as a cotour movig outward. The scheme ow esures that whe the derivative is take the poits o the iside of the curve are subtracted from the poits o the curve. If the circle is movig iward the scheme esures that the poits outside the circle are subtracted from the poits o the curve to calculate the derivative. The forward ad backward derivatives are defied as follows. D ij x i 1, j (4.15) D ij x i 1, j (4.16) D ij y 1 (4.17) D ij y 1 (4.18) With the followig scheme the iformatio flows from where the curve has bee to where it is ow [22]. 1 t max F, 0 mi F, 0 (4.19)

7 38 I which ad are defied as: max D ij x, 0 2 mi D ij x, 0 2 max D ij y, 0 2 mi D ij y, max D ij x, 0 2 mi D ij x, 0 2 max D ij y, 0 2 mi D ij y, (4.20) (4.21) This is implemeted i Mathematica as follows ad it is demostrated o the test image created above. d dt _, g_, bad_ : Module x, y, dmix x_, y_ : x, y x 1, y ; dplusx x_, y_ : x 1, y x, y ; dmiy x_, y_ : x, y x, y 1 ; dplusy x_, y_ : x, y 1 x, y ; d0x x_, y_ : 1 x 1, y x 1, y ; 2 x, y 1 x, y 1 d0y x_, y_ : ; 2 ablaplus pos_ : Max dmix pos, 0 2 Mi dplusx pos, 0 2 Max dmiy pos, 0 2 Mi dplusy pos, 0 2 ; ablami pos_ : Max dplusx pos, 0 2 Mi dmix pos, 0 2 Max dplusy pos, 0 2 Mi dmiy pos, 0 2 ; d02x x_, y_ : 1 4 d02y x_, y_ : 1 4 x 2, y 2 x, y x 2, y ; x, y 2 2 x, y x, y 2 ; d0xy x_, y_ : 1 x 1, y 1 x 1, y 1 x 1, y 1 x 1, y 1 ; 4 pos_ : If d0x pos 2 d0y pos 2 0, 0, d02x pos d0y pos 2 2 d0y pos d0x pos d0xy pos d02y pos d0x pos 2 d0x pos 2 d0y pos ; d dt Map g First #, Last # Max, 0 ablaplus # Mi, 0 ablami # # d0x # 2 d0y # 2 &, bad ; x, y Dimesios ; res Table 0, x, y ; MapThread res First #1, Last #1 #2 &, bad, d dt ; pos Flatte Table, i, 3, x 2, j, 3, y 2, 1 ; Do phi phi 0.1 d dt phi, g, pos ; If Mod k, 10 0, Show ListDesityPlot im2, PlotLabel ToStrig k, DisplayFuctio Idetity, p1 ListCotourPlot phi, Cotours 0, CotourStyle RGBColor 1, 0, 0, Thickess 0.001, PlotLabel ToStrig k, DisplayFuctio Idetity, DisplayFuctio $DisplayFuctio, k, 1, 200

8 Figure 4.6: Fial result of simple levelset o circle ( = -1, = 0.3). 4.4 Addig The Gradiet The simple levelset method as give before has the tedecy to leak out of weak boudaries. Also as ca be see i figure 4.6 it does ot completely stop at the edges, but slightly before the edges. To prevet this from happeig aother stoppig term was added to equatio 4.12 by Kicheassamy [14] ad Yezzi [32]. t g 1 c 0 (4.22) I which c is the extra stoppig term ad a costat. c is give by: c G I (4.23) This results i the additioal term pullig back the curve towards the edges. The extra stoppig term c is very egative o the edges. Whe the gradiet of c is take, the result ca be depicted as a force field i which the force poits towards the edges as ca be see i figure 4.7. Cosider a force poitig to the right ad the cotour expadig to the left. As was metioed before expadig meas that somethig is subtracted from the fuctio. Now the edge is i the other directio, as the force is poitig i the right directio, towards the edge. The force must ow be added to the fuctio at this part of the cotour. This results i this part of the cotour shrikig back towards the edge. Care must be take that whe the cotour is movig alog the force, the force is added or subtracted as the cotour is shrikig or expadig respectively. Whe the cotour is movig i the opposite directio of the force, the force is subtracted or added as the cotour is shrikig or expadig respectively. This is implemeted by approximatig c as follows [22]. max c x, 0 D ij x mi c x, 0 D ij x max c y, 0 D ij y max c y, 0 D ij y (4.24) Max px First #, Last #, 0 dplusy # Mi px First #, Last #, 0 dmiy # Max py First #, Last #, 0 dplusx # Mi py First #, Last #, 0 dmix #

9 Figure 4.7: Result of levelset with extra stoppig term. 4.5 Narrow Bad A improvemet i speed ca be obtaied by applyig the arrow bad method. I this method oly the values i a arrow bad aroud the zero levelset are updated. This reduces the umber of calculatios ad thus icreases the speed of the etire process. As the fuctio is a siged distace fuctio, the i priciple determiig the arrow bad should just be a questio of selectig the values below a certai threshold, which is the width of the arrow bad. As the fuctio evolves however the siged distace fuctio may o loger represet the distace to the zero levelset accurately. To combat this the fuctio ca be reiitialized oce i a while. Aother way of implemetig the arrow bad is by determiig the zero levelset ad takig the eighbourhood of this zero levelset with a certai width. This way reiitializatio is ot required, because the arrow bad is o loger depedet o the value of the fuctio. This is implemeted as follows. The fuctio is divided ito two segmets, the positive side ad the egative side of the fuctio. The borders betwee these segmets are determied ad these are the pixels where the fuctio is zero. A eighborhood aroud each zero levelset pixel is determied ad all these coordiates are retured as the arrow bad. makebad maskim_, w_ : Module imbrdr, x, y, posbrdr, x, y Dimesios im ; makeborders2 labelimage_ : Module left, dow, border, left labelimage RotateLeft labelimage ; dow labelimage Map RotateLeft, labelimage ; border l_, d_ : If l 0 && d 0, 100, 0 ; SetAttributes border, Listable ; border left, dow ; imbrdr makeborders2 maskim ; posbrdr Positio imbrdr, 0 ; eighb Flatte Outer #1, #2 &, Rage w Ceilig w 2, Rage w Ceilig w 2, 1 ; res posbrdr, Uio Flatte eighb. a_, b_ # a, b & posbrdr, 1 ; res 1 Map If # 0, 1, 0 &,, 2 ; pos 0, pos makebad 1, 3 ;

10 Addig A Gradiet Vector Flow As ca be see i sectio 4.4 a added gradiet improves the performace of the levelset. I chapter 3 about the sake the gradiet vector flow was itroduced to attract the cotour from further away to the edges. This ca also be applied o the levelset. I stead of the gradiet which is added as extra stoppig term, the gradiet vector flow field is added. The gradiet vector field is calculated similarly as i chapter 3. The c x ad c y i equatio 4.24 are replaced by u ad v of equatios 3.7 ad Applyig The Levelset I this sectio the basic levelset, the levelset with the extra stoppig term ad the levelset with the gradiet vector flow are applied o test images ad the results are compared. First the ifluece of oise o the performace of the levelset is tested. This is doe by creatig a image of a cross with 50% oise added. The parameters of the levelset were set as follows: = 1 ad = 0.3. For the extra stoppig term ad the gradiet vector flow = 1. The results after 3000 iteratios with timestep of 0.1 is give i figure 4.8. DisplayTogetherArray basic, xstop, gvf ; Figure 4.8: Result of 50% oise o cross image of the basic levelset (left), extra stoppig term levelset (middle) ad gradiet vector flow levelset (right). As ca be see i figure 4.8 the levelset performs well o oisy images. This ca be explaied by the way the levelset works. The cotour forms itself aroud edges of the oise ad splits off i a part that segmets the small oise particle ad a frot that cotiues movig. The part aroud the oise particle shriks i o itself as the edge is usually ot strog alog its etire border ad therefor lets the cotour leak i. The cotour the collapses ito itself ad ceases to exist. The movig frot however stays itact ad collides with the desired edge. The oise however does slow dow the speed at which the cotour moves. It takes time to move aroud every particle i the image. This is why it takes 3000 steps before the cotour coverges o the fial edge. Next the ifluece of a hole i the gradiet image is tested. I this case i stead of a hole i the gradiet image a hole i g as give i equatio 4.11 for all the levelsets was made. For the extra stoppig term levelset ad the gradiet vector flow levelset a hole was also made i c as give i equatio The same parameters were used for the levelset as i the oise images, but ow 500 iteratios with timestep of 0.1 were take.

11 42 DisplayTogetherArray basic, xstop, gvf ; Figure 4.9: Result o a hole i the gradiet image of a test image of the basic levelset (left), extra stoppig term levelset (middle) ad gradiet vector flow levelset (right). This result is very iterestig. As ca be see the first two levelset methods are subject to strog leakig, though the result of the leakig looks differet. The gradiet vector flow levelset however does ot leak at all. The differece i leakig betwee the basic levelset ad the extra stoppig term levelset ca be explaied by the extra stoppig term. This term pulls the cotour oto the middle of the edge, while the basic levelset stops evolvig oce it reaches the edge. The basic levelset stops because it is multiplied by zero or a ear zero value i g at that locatio. Because of this the levelset that has leaked out of the hole does ot merge with itself at the edge as happes i the extra stoppig term levelset. It is stopped o the iside ad the outside of the edge. The gradiet vector flow levelset however is aother story. Because the edge regios of the extra stoppig term have bee smoothed out over the image, the levelset aroud the hole is pulled to the sides by the gradiet vector flow force. This couters the term that pushes the cotour outward ad therefor the cotour does ot leak out of the hole. Now the ifluece of a added gradiet i the backgroud of a real image o the performace of the levelset is tested. The same parameters as before are used for the levelset ad 500 steps of timestep 0.1. DisplayTogetherArray basic, xstop, gvf ; Figure 4.10: Result o a real image with a added gradiet of the basic levelset (left), extra stoppig term levelset (middle) ad gradiet vector flow levelset (right). As ca be see the basic levelset is ot stopped by the stoppig term g, but the extra stoppig term keeps the cotour o the edge. The gradiet vector flow however is startig to leak out of the desired boudaries. The added gradiet makes some edges less strog. For the levelset strog edges are required. As ca be see with the basic levelset, whe the edges are too weak the stoppig term g is ot eough to stop the cotour o the edges. The added gradiet however pulls the cotour back whe it crosses over the edge, therefor the ifluece of the weaker edges resultig from a added gradiet is ot great. For the gradiet vector flow however the pull towards the edge is ot strog eough. This is a result of the smoothig. This causes the

12 43 stoppig term at the edge to lose its stregth to the eighborhood. After which the gradiet vector flow force is ot strog eough to couter the costat that drives the cotour outward. As the levelset was primarily developed to hadle cases where the cotour was required to split, or merge, it is ow tested o a image with two circles. The iitial cotour is a circle that lies aroud both the circles ad is made -1, which meas it moves iward. The other parameters are all kept the same ad 500 timesteps are take with stepsize 0.1. DisplayTogetherArray basic, xstop, gvf ; Figure 4.11: Result o a image with two circles of the basic levelset (left), extra stoppig term levelset (middle) ad gradiet vector flow levelset (right). As ca be see the levelset performs well ad splits icely i two separate cotours that detect the two circles. The basic levelset as discussed above stops just before the edge, while the extra stoppig term ad the gradiet vector flow force pull it further o the edge. 4.8 Discussio Ad Future Ehacemets The levelset method accomplishes its goal very well. As ca be see i figure 4.11 it has o problems with splittig oe cotour ito two. Noise has little effect o the ed result, but it does take the levelset loger to reach it. Oe mayor disadvatage of the levelset method is that it requires strog edges. If the edges are ot strog eough or there are gaps i the edges the cotour will leak out ad the object that is to be segmeted, is ot detected. The extra stoppig term ad the gradiet vector flow ca slow the leakage, but they have their limitatios. Oe way to prevet the leakage some more would be to implemet the secod stoppig term itroduced by Siddiqi [23]. The levelset fuctio would become as follows. t g c 2 x c 0 (4.25) I this solutio x is the positio ad 2 x c is called the area miimizig term. This stoppig term would have to be ivestigated ad implemeted to see its effect o the performace of the levelset. This secod stoppig term is reported to be able to hadle small gaps i the cotour [23]. The levelset method ca be exteded to three dimesios. This ca be doe by icorporatig derivatives of i the z directio as well ad the fuctio ow becomes a four dimesioal fuctio. I stead of a startig cotour a startig surface is required, but the evolutio equatio remais the same. Because the surface is implicitly stored i the fuctio as the zero levelset, there is o problem with keepig track of the surface as there is i the three dimesioal sake.

13 Fially the implemetatio i Mathematica is very slow. It takes about 10 miutes to segmet a image of 256 by 256. The speed is also depedet o the amout of oise preset i the image. There are methods to speed the method up, for istace by first implemetig the Fast Marchig Method before applyig the levelset method [22]. Aother way to icrease the speed of the levelset is by implemetig it i C. This would icrease the speed ad by usig Mathlik a coectio betwee Mathematica ad this program ca be made. 44