Snakes operating on Gradient Vector Flow Seminar: Image Segmentation SS 2007 Hui Sheng 1
Outline Introduction Snakes Gradient Vector Flow Implementation Conclusion 2
Introduction Snakes enable us to find boundaries of objects Wide range of applicability Medical sciences Motion tracking Image analysis 3
Snakes First introduced in 1987 by Micheal Kass and Andrew Witkin Snakes are deformable Snakes are attracted to object boundaries through forces Snakes operate locally, thus one must provide an initial position for them 4
Snake behavior A smooth curve which matches to image data Curve is initialised near target Iteratively refined Iteratively refined Initial position Intermediate Final position 5
Snake Energies Find the best fit between the snake and object by minimizing the energy function: E snake = s [Einternal (r(s)) + Eexternal (r(s))]ds where the snake is parametrically defined as r(s) = (x (s), y(s)), and s 6
Internal Energy The internal energy can be written as E int = ( (s) r s (s) 2 + (s) r ss (s) 2 ) / 2 elasticity rigidtity where r s and r ss are the first and second derivatives of the snake with respect to s (s), (s) specify the elasticity and rigidity of the snake, respectively 7
Internal Energy (s) and (s) are often constants, independent of s Note that a large positive value of and/or means that stretching(elasticity) and/or bending(rigidity) are tightly constrained as we search for an energy minimum 8
External energy It is derived from the image The external energy computed from the image is a weighted combination of energies which attract the snake to lines or edges 9
External energy For a gray-level image I(x,y), G If the image is a line drawing (black on white) 10
Force equation Minimizing energy equation can be interpreted as a force balance equation F int + F ext = 0 where F int = E int = ( r ss + r ssss ) and F ext = E ext The internal force F int discourage streching and bending while the external potential force F ext pulls the snake toward the desired image edges 11
Numerical Methods Represent the curve with a set of n points r i = (x i, y i ), i = 0,... n-1 ( x, y 0 0) ( 3 x 3, y ) ( x, y 17 17 ) 12
Numerical Methods We can rewrite the energy equation as follows: According to E int n E snake = E int (i) + E ext (i) i=1 E int = ( (s) r s (s) 2 + (s) r ss (s) 2 ) / 2 we approximate the derivatives with finite differences and converting to vector notation with r i = (x i, y i ), we rewrite E int to E int (i) E int (i) = i r i r i-1 2 / 2 + i r i-1 2r i + r i+1 2 / 2 where r(0) = r(n). 13
Numerical Methods The corresponding Euler equations is: i (r i r i-1 ) i+1 (r i+1 r i ) + i-1 (r i-2 2r i-1 + r i ) 2 i (r i-1 2r i + r i+1 ) + i+1 (r i 2r i+1 + r i+2 ) (f x (i), f y (i)) = 0 where f x (i) = E ext / x i f y (i) = E ext / y i 14
Numerical Methods The Euler equations can be written in matrix form as AX F x (X, Y) = 0 AY F y (X, Y) = 0 where A is a matrix, X, Y, F x, F y are the Arrays and thier dimention is the number of points in snake 15
Numerical Methods We set the right-hand sides of the equations equal to the product of a step size and the negative time derivatives of the left-hand sides AX t + F x (X t-1, Y t-1 ) = ϒ(X t X t-1 ) AY t + F y (X t-1, Y t-1 ) = ϒ(Y t Y t-1 ) where ϒ The above equation can be solved by matrix inversion X t = (A + ϒI) -1 (ϒX t-1 + F x (X t-1, Y t-1 )) Y t = (A + ϒI) -1 (ϒY t-1 + F y (X t-1, Y t-1 )) 16
Traditional force for the snake The traditional force for the snake is defined as the Gradient of edge map of the image F ext = k(x, y) 17
Edge map Edge map k(x, y) is derived from the image I(x, y) having the property that it is larger near the image edges where i = 1,2,3, or 4 k(x, y) = E ext (i) (x, y) 18
Gradient of edge map as external force First, the gradient of an edge map k has vectors pointing toward the edges, which are normal to the edges at the edges Second, these vectors generally have large magnitudes only in the immediate vicinity of the edges Third, in homogeneous regions, where I(x, y) is nearly constant, k is nearly zero 19
Gradient of edge map as external force Small capture range Fig. 1. (a) (b) (c) (a) Convergence of a snake using (b) traditional potential forces, and (c) shown close-up within the boundary concavity. Difficult to pull the snake into boundary concavity 20
Gradient of edge map as external force We try to keep the highly desirable property of the gradients near the edges, but to extend the gradient map farther away from the edges and into homogenous regions using a computational diffusion proccess 21
Gradient Vector Flow A new external force for snakes Large capture range Detects shapes with boundary concavities 22
Detecting boundary concavities and expanding the capture range by GVF Traditional force GVF force The GVF vectors are pointing somewhat downward into the top of the U-shape, which should cause an active contour to move farther into this concave region 23
Gradient Vector Flow The GVF field is defined to be a vector field V(x,y) = ( (x,y), (x,y)) V(x,y) is defined such that it minimizes the energy functional = ( where k(x,y) is the edge map of the image and is noise parameter 24
Gradient Vector Flow Using the calculus of variations, GVF field can be obtained by solving following equations 2 2 where 2 is the Laplacian operator and k x, k y are the partial derivative of the edge map 25
Numerical Methods Equations can be solved by treating u and v as functions of time 26
Numerical Methods For convenience, we rewrite the equations as follows: where b(x,y) = k x (x,y) 2 + k y (x,y) 2 c 1 (x,y) = b(x,y)k x (x,y) c 2 (x,y) = b(x,y)k y (x,y) 27
Numerical Methods To set up the iterative solution, let the indices i, j and n correspond to x, y and t, respectively, and let the spacing between pixels be x and y and the time step for each iteration be t. Then the required partial derivatives can be approximated as: 28
Numerical Methods Substituting these approximations into equations gives our iterative solution to GVF as follows: where 29
Implementation The main C++ files: Snake update GVF update Two Classes: Array1D Array2D Help functions: Matrix Inversion Gaussian Filter Interpolation Distance equalization 30
Implementation The process of my implementation: 1. Import an image 2. Initialize the snake for the image 3. Compute the edge map of the image 4. Input the edge map to the GVF solver 5. Deforms the snake 31
Results Binary edge map GVF force has more capture range than traditional force U- shape initial snake traditional GVF force force 32
Results Binary edge map GVF force pulls the snake into the boundary concavtity while traditional force can not do it initial snake traditional force GVF force 33
Results Distance equalization without distance equalization with distance equalization 34
Results Gray level image edge map initial snake traditional force GVF force 35
Results Gray level image edge map initial snake GVF snake 36
Results Gray level image edge map initial snake GVF snake 37
Results Gray level image edge map initial snake GVF snake 38
Results Gray level image egde map initial snake GVF force 39
Problem with GVF snake Parameters must be adjusted. Finding GVF field is computationally expensive 40
Conclusion We have introduced the snake and his principles We have introduced a new external force model for snakes called gradient vector flow(gvf) The field is calculcated as a diffusion of the gradient vectors of a gray-level or binary edge map We have shown that it allows for flexible initialization of the snake and encourages convergence to boundary concavities 41
References Chenyang Xu and Jerry L.Prince, Snake, Shape, and Gradient Vector Flow, IEEE Transactions on Image Processing, 1998. M.Kass, A. Witkin, and D. Terzopoulos, Snakes: Active contour models., International Journal of Computer Vision. v. 1, n. 4, pp. 321-331, 1987. C.Xu and J.L Prince, Gradient Vector Flow: A new External Force for Snakes, Proc. IEEE Conf. on Comp. Vis. Pat. Recog. (CVPR), Los Alamitos: Comp. Soc. Press, pp. 66-71, June 1997. 42
Thanks for your attention!!! Questions??? 43