Regularized Laplacian Zero Crossings as Optimal Edge Integrators

Transcription

1 Regularize aplacian Zero rossings as Optimal Ege Integrators R. KIMME A.M. BRUKSTEIN Department of omputer Science Technion Israel Institute of Technology Technion ity, Haifa 32, Israel Abstract We view the funamental ege integration problem for object segmentation in a geometric variational framework. First we show that the classical zero-crossings of the image aplacian ege etector as suggeste by Marr an Hilreth, inherently provies optimal ege-integration with regar to a very natural geometric functional. This functional accumulates the inner prouct between the normal to the ege an the gray level image-graient along the ege. We use this observation to erive new an highly accurate active contours base on this functional an regularize by previously propose geoesic active contour geometric variational moels. 1. Introuction Ege integration for segmentation is an ol, yet still very active area of research in low-level image analysis. Textbooks in computer vision treat ege etection an ege integration as separate topics, the first being consiere one of labelling eges in the image to be followe by a process of integrating the local eges into meaningful curves. In fact one may view basic ege etection as a process of estimating the graient of the image, i.e. computing at each pixel (x; y) the values u(x; y) an v(x; y) by using the values of I(x; y) over a neighborhoo N (x; y) of (x; y) an esignating as eges the places where the length of the graient vector estimate [u; v] (ri) excee some threshol value. The more avance ege etectors such as those propose by Marr an Hilreth [8] attempt to locate points or curves efine by local maxima of the image graient. The Marr Hilreth proposal for ege etection yiel curves that elineate the zero crossing of the aplacian operator applie to a smoothe version of the image input. The smoothing propose is via a Gaussian convolution operator an its with is a parameter that can be varie proviing the opportunity to o scale space processing an vertical integration on the zero-crossing curves. In this paper we propose to regar the ege etection an integration process as a way to etermine curves in the image plane that pass through points where the graient is high an whose irection best correspon to the local ege irection preicte by the estimate graient. Inee, if we somehow estimate the graient fiel [u(x; y); v(x; y)] base on consiering I(x; y) for each pixel (x; y) over some neighborhoo N (x; y), where is a size parameter, we shall have at each point a value, given by the intensity of the graient (u 2 (x; y) + v 2 (x; y)) 1 2, that tells us how likely an ege is at this point an, if an ege exists, its likely irection will be perpenicular to the vector [u(x; y); v(x; y)]. It is therefore natural to look for curves in the image plane, (s) [x(s); y(s)], that pass through points with high intensity graients with tangents agreeing as much as possible to the ege irections there. Thus we are le to consier the following functional, evaluating the quality of (s) as an ege-curve caniate, ((s)) Z [u((s)); v((s))] 1 1 (s) T! where () is some monotonically increasing function. Here, the inner prouct of ri [u; v] with the normal 1 h i T (s) to (s), given by ~n(s) ; where 1 (s) is an arclength parameterize curve, is a measure of how well (s) is locally tracking an ege. Inee we want (s) to pass at high graient locations in the ege irection, an hence the inner prouct of its normal with the estimate graient of I shoul be high, inicating both alignment an consierable change in image intensity there. This inner prouct will also be proportional to the graient magnitue, since T 1 [u((s)); v((s))] jrij cos(); 1 where is the angle between the outwar pointing normal ~n to (s) an the graient irection. The functionals measure how well an arclength parameterize curve of length approximates an ege in the image plane. Our task, of course is to etermine several most probable ege curves in the image plane. We shall o so by etermining curves that locally maximize these functionals. 1

2 Suppose first that we are consiering close contours (s), an that (). Then, we have that ((s)) I (s) an Green s theorem yiel ((s)) ZZ u((s)) y(s) v((s)) x(s) v(x; y) + u(x; y) y x xy; where is the region insie (s). But, recalling that v(x; y) is an estimate of I(x; y) an u(x; y) an estimate y of I(x; y) we have x ((s)) ZZ ZZ 2 2 I(x; y) + I(x; y) 2 2 x y (I(x; y))xy: ; xy Therefore, the functional that we want to maximize is the integrate aplacian over the area enclose by (s). This means that if we have an area where the aplacian is positive ((s)) shoul expan from within this area to the places where I(x; y) becomes zero an subsequently changes sign. This shows that optimal ege-curves in the sense of maximizing ((s)) are the zero crossings of the aplacian. If we initialize (s) as a small circular bubble at a place where I(x; y) is positive an then let (s) evolve accoring to a rule that implements a graient escent in conjunction with the functional ((s)); i.e. we implement t ((s; t)) (s; t) ; the curve (s; t) will expan in time t to the nearest zerocrossing curve of the input image aplacian. Therefore, we have obtaine a beautiful interpretation of the classical Marr-Hilreth ege etection metho [8]. The zero-crossings of the aplacian are curves that best integrate the eges, in the sense of our functional ((s)) with (), if we wish to o so base on graients estimate for the (smoothe) input image I(x; y). While this fact is peagogically very pleasing, it oes not alleviate the notorious over sensitivity properties of this ege-etector which in noisy images yiel lots of false ege curves. However, we shall show here that this insight provies the basis for a new an practical active contour process which enhances an improves upon the previously esigne such metho for image segmentation. We next present the full erivation of the variational results leaing to the new ege integration processes an then show the performance of the resulting algorithms. 2. lose Active ontours: Derivation Motivate by the classical snakes [4], geometric active contours [5, 1, 6], an finally the geoesic active contours that were shown in [2] to be relate to the snakes, we search for simple parametric curves in the plane that map their arclength interval [; ] to the plane, such that : [; ]! IR 2, or in an explicit parametric form (s) [x(s); y(s)]. Here s is the arclength parameter, an we have the relation between the arclength s an a general arbitrary parameterization p, given by s x(p) 2 2 y(p) + jpj: We efine, as usual, ~n; ; an ~t to be the unit normal, the curvature, an the tangent of the curve. We have that ~n ss, an ~ t s p j p j. As escribe in the introuction, consier the geometric functional () I (h V ~ ; ~ni): This is an integration along the curve of a function efine in terms of a vector fiel V ~ [u(x; y); v(x; y)], where for example we can take V ~ ri(x; y) [I x ; I y ] as the gray level image graient. Our goal is to fin curves that minimize the above geometric functional. In a general parametric form, we have the following reparameterization invariant measure () (h V ~ ; ~ni)j p j: Define, h V ~ ; ~ni. The Euler agrange (E) equations () shoul hol along the extremum curves, an for a close curve these equations are () or in a more compact form () x x p y y p p! ()j p j; ()j p j; where we use the shorthan notation [x; y] T an p [x p ; y p ] T. In case of an open curve, one must also consier the en points an a aitional constraints to etermine their optimal locations. Before we work out the general () case let us return to the simple example iscusse in the introuction, where (). In this case we have that () h ~ V ; ~nij p j 2

3 ~V ; [y p; x p ] j p j j p j (y p u + x p v): The E equation for the x part is given by x x x p y p u x + x p v x v (y p u + x p v) y p u x + x p v x v x x p v y y p y p (u x + v y ) y p iv( ~ V ): In a similar way, the y part of the E equations is given by (y p u + x p v) y y y p y p u y + x p v y + u y p u y + x p v y + u x x p + u y y p x p (u x + v y ) x p iv( ~ V ): Since the E equations are erive with respect to a geometric measure, we can use the freeom of reparameterization for the curve, ivie by j p j, an obtain the geometric E equation: iv( ~ V )~n, an for ~ V ri(x; y) we have I~n, where I I xx + I yy is the usual aplacian operator. It is obvious from this that the geometric E conition is satisfie along the zero crossing curves of the image aplacian, which as escribe above explains the Marr-Hilreth [8, 7] ege etector from a global-variational point of view. Below, we shall extract further insights an segmentation schemes from this observation. We note that heuristic non-variational flows on vector fiel were presente in [14, 11]. In a recent relate result, introuce by Vasilevskiy an Siiqi [13], alignment with a vector fiel is use as a minimization criteria for segmentation of complicate close thin structures in 3D meical images. As a secon example we consier () jj p 2. The E is given by h~ V ; ~ni jh V ~ ; ~nij iv(~ V )~n sign(h ~ V ; ~ni)iv( ~ V )~n; an for V ~ ri we have sign(hri; ~ni)i~n. The new term sign(hri; ~ni), allows the moel to automatically hanle changing contrasts between the objects an the backgroun. For example, it hanles equally well an image of ark objects on bright backgroun an the negative of this image. Now, we are reay to pursue the general case for () in the functional () (where h V ~ ; ~ni). We shall use often the following reaily verifie relationships, j pj ~n ~t () s hr; ~ti j pj p ~t ~t ~n h~ V s ; ~ni + h V ~ ; ~n s i h V ~ s ; ~ni h V ~ ; ~ti j p j 1 h V ~ ;~ti~n; p an that p (j p j) j p jiv( ~ V )~n: Using these relations we have () ()j p p j p jpj + ~t p jpj + ~t j pj j pj j pj p jpj + ~t p p jpj + ~t ~t + ~t j pjiv( V ~ )~n j pj sh V ~ ;~ti~n + ~t ~t + ~t ~t + ~t j pj j pj + (iv( V ~ ) + h V ~ ; ~ni) + (h V ~ s; ~nih V ~ ;~ti h V ~ ;~ti ) 2 ~n + some aitional tangential components: Here we use the shorthan notations r [ x ; y ], an p r p. evel Set Formulation: In orer to etermine optimal curves in the plane, we nee to solve numerically the E equations. Here we shall follow the geoesic active contour philosophy, see [2], an esign a curve evolution rule that is given by t () : This is a graient escent rule with respect to the chosen cost functional, an in this flow one can consier only the normal components of (), since tangential components have no effect on the geometry of the propagating curve. Next, we can embe the curve in a higher imensional (x; y) function, which implicitly represents the curve as a zero set, i.e., f[x; y] : (x; y) g. In this way, the well known Osher-Sethian [9, 12] level-set metho can be employe to implement the propagation. Given the curve evolution equation t ~n, its implicit level set evolution equation rea t : The equivalence of these two evolutions can be easily verifie using the chain rule an the relation ~n r, t hr; t i hr; ~ni r; r : 3

4 r We reaily have that ~t iv r [y ;x] ; ; ~ Vs [u s ; v s ] [hru; ~ti; hrv; ~ti]; an sign(h ~ V ; ~ni) sign(h ~ V ; ri): Thereby, the explicit curve evolution as a graient escent flow for () jj is given by t sign(h ~ V ; ~ni)i~n; for which the implicit level set evolution is given by t sign(h ~ V ; ri)i: 3. Open Active ontours Fua an eclerc in [3], were first to propose a geometric moel for motion of open curves in the image to optimize an ege fining functional. We shall first escribe the Fua-eclerc functional an then replace the geoesic active contour part of it with our new ege integration quality measure. et R 1 () j pj; be the arclength of an open curve (p). Aing the variation (p) to the curve, such that (p) ~ (p) + (p), ifferentiating w.r.t., an letting go to zero, yiel () Z ~n + ()~t() ()~t(); where s is the arclength parameter. Also, following Fua an eclerc, consier R g () g((s)); where g is some suitably efine ege inicator function, for example g(x; y) 1(jrIj 2 + 1). The first variation of g () can be easily R shown to be given by g () (hrg; ~ni g)~n +()g(())~t() ()g(())~t(): The Fua-eclerc functional is efine as g() g() : omputing the first variation, we () g have that g g 2 ; shoul hol for any. Therefore, the following conitions must be satisfie, g g ; or explicitly, R (hrg; ~ni g)~n + ()g(()) ~t() ()g(())~t() R g ~n + () ~t() ()~t() : Thus, we shoul verify the following necessary conitions for a local extremum to hol for any, Z ((hrg; ~ni g) + g ) ~n ()g(())~t() g ()~t() ()g(())~t() g ()~t(): Therefore, the geometric conitions that must be met along the curve is g g + hrg; ~ni ~n ; an at its en points g(()) g g, an g(()). We can use these conitions to guie a graient escent process for an active contour evolution towar the local minimum of the Fua-eclerc functional. To o that we apply the following evolution equation along the curve an at its en points, t g hrg; ~ni g The first two terms epict the geoesic active contour ([2]) moel, while the thir term irects the curve to gain length by applying the inverse geometric heat equation at points where g((s)) < g. We still nee to esign the motion of the en points. onsier the en point (). The curve shoul reuce its length if g(()) > g, in which case the en point shoul move along the tangent ~ t(). Hence, for example, we can use the following evolution rules at the en points: ~n: t () (g(()) g )~t() t () ( g g(()))~t(): Optimal Ege Integration: We propose to use our measure, R () instea of g, in the Fua-eclerc functional. Here we compute the evolution equations that propagate the open curve towar a maximum of the functional () : Therefore, we are searching for arg max (). The quantity in this maximization process, penalizes the length of the curve, i.e. it plays a role opposite from its role in the minimization of the Fua-eclerc functional. We now use the expression evelope in the previous sections for the general (h V ~ ; ~ni) close curve case. We have that () Z + ~t h V ~ ;~ti~n Using these conitions in the Fua-eclerc formulae yiel along the curve, + ~n ; an (~t h V ~ ;~ti~n) ~t; at the en points () an (). For () jj, the graient escent flow of the curve is given by t sign(hri; ~ni)i~n + ~n t () ( )~t h V ~ ;~ti~n t () ( )~t + h ~ V ;~ti~n: A Simpler Optimal Ege Integration: Functionals that involve ratio of two integral measures, like the Fua-eclerc functional, require integration along the contours for a : 4

5 proper graient escent flow. Integral parts are present in the E equations which require computationally intensive global integration proceures for the computation of the proper flow. Recall however that our goal is to maximize () on one han, that lea to long curves, while also penalizing the length of the curve on the other han. We shall therefore consier the following alternative functional that woul also realize these goals, ~ () : The E equations in this case are given by () + ~n; along the curve, an ( 1)~t h V ~ ;~ti~n; at the en points. The motivation for the tangential term at the en point is obvious, it either exten or shrinks the curve. The normal term pulls it from running parallel to the vector fiel an irects the en point towar the center of the ege (where shoul be zero). These two components efine the motion at the en points. For () jj an V ~ ri we have t sign(hri; ~ni)i~n + ~n t () (jhri; ~nij 1)~t sign(hri; ~ni)hri; ~ti~n t () (1 jhri; ~nij)~t + sign(hri; ~ni)hri; ~ti~n: 4. Simulation Results We teste the ege integration metho iscusse in this paper on two simple examples. The first presente segmentation examples shown are not typical for active contours an coul be easily processe with less sophisticate metho. However, they capture the ifficulties of the existing active contour moels an therefore are useful for comparison of the ifferent metho. In the close contour cases we starte from the image frame as the initial contour, an applie a multi-resolution coarse to fine proceure, as in [1], to spee up the segmentation process. Figure 1 shows the avantage of the aplacian moel in cases where only the graient is affecte, the aplacian being invariant to an aitive intensity plane, as well as in cases where the aplacian is also change by a constant when a parabola was ae to the intensity surface. Figure 2 clearly exhibit the segmentation avantages of the aplacian active curve moel as a core with the geoesic active term as a regularization. We here use the functional () I () I g 1((s)) Z g 2(x; y)xy; where an are small positive constants, () jj, an hri; ~ni. g i, i 1; 2 are ege inicator functions with lower values along the eges. In this case, the graient escent flow for maximizing () is given by t (sign(hri; ~ni)i + (g 1 hrg 1 ; ~ni) g 2 ) ~n: Figure 1: Synthetic images with a tilte intensity plane (top) an a parabola intensity surface (bottom) ae to the original image. Mile: GA results. eft: aplacian moel results. The level set formulation for this flow is r t sign(hri; ri)i + iv g 1 g2 : Next, we applie our open contour moel for ege integration on similar images, but here we starte with short contour segments that expane an locke onto bounaries, if such existe in the vicinity of the initialize contours. If no bounaries are etecte locally, the contour segments shrink an eventually isappear. The numerical implementation for the open contour case is an explicit marker-points base moel which was easier to program in this case. At each iteration the marker-points are reistribute along the contour to form equi-istant numerical representations of the contour. A simple monitoring proceure, removes a marker point when successive marker points get too close to one another, an a a new marker point in the mile of two successive marker points when the istance between them gets larger than a given threshol. The examples show how initial segments expan an eform until they lock onto the bounaries of rather complex shapes. See onclusions In this paper we propose to incorporate the irectional information that is generally ignore when esigning ege integration metho in a variational framework. Simulations that were performe with the newly efine ege integration processes amply emonstrate their excellent performance as compare to the best existing ege integration metho. Our extene active contour moels are just a few examples of the many possible combinations of geometric mea- 5

6 sures. Other functionals that coul be consiere to either open or close curves are g, or R ()q(jrij), for an ege p inicator function like q 1 g(jrij) or q(jrij) jrij For close curves, the part is most effective when the curve is close to its final location, therefore, the functional H [()(1g)g] coul also be consiere. Acknowlegments We thank Evgeni Krimer an Roman Barsky for implementing an testing the open active contours moels. References [1] V. aselles, F. atte, T. oll, an F. Dibos. A geometric moel for active contours. Numerische Mathematik, 66:1 31, [2] V. aselles, R. Kimmel, an G. Sapiro. Geoesic active contours. IJV, 22(1):61 79, [3] P. Fua an Y. G. eclerc. Moel riven ege etection. Machine Vision an Applications, 3:45 56, 199. [4] M. Kass, A. Witkin, an D. Terzopoulos. Snakes: Active contour moels. International Journal of omputer Vision, 1: , [5] R. Mallai, J. Sethian, an B.. Vemuri. A topologyinepenent shape moeling scheme. In SPIE s Geometric Metho in omputer Vision II, volume SPIE 231, pages , July [6] R. Mallai, J. Sethian, an B.. Vemuri. Shape moeling with front propagation: A level set approach. IEEE Trans. on PAMI, 17: , [7] D. Marr. Vision. Freeman, San Francisco, [8] D. Marr an E. Hilreth. Theory of ege etection. Proc. of the Royal Society onon B, 27: , 198. [9] S. J. Osher an J. Sethian. Fronts propagating with curvature epenent spee: Algorithms base on Hamilton-Jacobi formulations. J. of omp. Phys., 79:12 49, [1] N. Paragios an R. Deriche. Geoesic active contours an level sets for the etection an tracking of moving objects. IEEE Trans. on PAMI, 22(3):266 28, 2. [11] N. K. Paragios, O. Mellina-Gotaro, an V. Ramesh. Graient vector flow fast geoesic active contours. In Proceeings IV 95, Vancouver, anaa, July 21. [12] J. Sethian. evel Set Metho: Evolving Interfaces in Geometry, Flui Mechanics, omputer Vision an Materials Sciences. ambrige Univ. Press, [13] A. Vasilevskiy an K. Siiqi. Flux maximizing geometric flows. In Proceeings IV 95, Vancouver, anaa, July 21. [14]. Xu an J. Prince. Snakes, shapes, an graient vector flow. IEEE Trans. IP, 7(3): , Figure 2: Top: Geoesic active contour results. Mile: aplacian active contours. Bottom: Using the geoesic active contours as a regularization for the aplacian active contour. time time5 time1 time15 time2 time25 time3 time35 time4 time45 time5 time55 Figure 3: Open geometric aplacian active contours: The small curves (top left frame) exten along the bounaries an capture most of the outer contours of the symbols. 6