Level Sets & Snakes Snakes, Active Contours, and Segmentation Introduction and Classical Active Contours Active Contours Without Edges Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 1/19
Introduction and Classical Active Contours The basic idea in active contour models (or snakes) is to evolve a curve, subject to constraints from a given image u 0, in order to detect objects in that image. Ideally, we begin with a curve around the object to be detected, and the curve then moves normal to itself and stops at the boundary of the object. Since its invention this technique has been used both often and successfully. The classical snakes model involves an edge detector, which depends on the gradient of the image u0, to stop the evolving curve at the boundary of the object. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 2/19
Introduction and Classical Active Contours Let u 0 (x, y) map the square 0 x, y 1 into R, where u 0 is the image and C(s) : [0, 1] Ø R 2 is the parametrized curve. The snake model is to minimize where a, b, and l are positive parameters. The first two terms control the smoothness of the contour, while the third attracts the contour toward the object in the image (the external energy). Observe that by minimizing the energy, we are trying to locate the curve at the points of maximum u 0, which act as an edge detector, while keeping the curve smooth. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 3/19
Introduction and Classical Active Contours An edge detector can be defined by a positive decreasing function g(z), depending on the gradient of the image u 0, such that A typical example is for p 1, where J is a Gaussian of variance s. We can also define a compact version for the energy via Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 4/19
Introduction and Classical Active Contours Using the variational level set formulation of Zhao et al., we arrive at This is motion of the curve with normal velocity equal to its curvature times the edge detector plus convection in the direction that is the gradient of the edge detector. Thus, the image gradient determines the location of the snakes. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 5/19
Introduction and Classical Active Contours In a sequence of papers beginning with Chan and Vese, the authors propose a different active contour model without a stopping (i.e. edge) function, so a model that does not use the gradient of the image u0 for the stopping process. The stopping term is now based on the Mumford-Shah segmentation technique. The model these authors develop can detect contours both with and without gradients, for instance objects that are very smooth, or even have discontinuous boundaries. In addition, the model and its level set formulation are such that interior contours are automatically detected, and the initial curve can be anywhere in the image. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 6/19
Active Contours Without Edges Define the evolving curve G as the boundary of a region W. We call W the inside of G and the complement of W = W c the outside of G. The method is the minimization of an energy-based segmentation. Assume that u 0 is formed by two regions of approximately piecewise constant intensities of distinct values u 0i and u 00. Assume further that the object to be detected is represented by the region with value u 0i. Denote its boundary by G 0. Then we have u 0 u 0 i inside G0 and u 0 u 0 0 outside G0. Now consider the fitting term where G is any curve and C 1, C 2 are the averages of u 0 inside G and outside G. In this simple case it is obvious that 0, the boundary of the object, is the minimizer of the fitting term. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 7/19
Active Contours Without Edges Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 8/19
Active Contours Without Edges In the following active contour model the fitting term plus some regularizing terms will be minimized. The regularizing terms will involve the length of the boundary G and the area of W. This is in the spirit of the Mumford-Shah functional. Thus, using the variational level set formulation, the energy can be written, with f the level set function associated with G, as (m, n, l 1, l 2 0) Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 9/19
Active Contours Without Edges The classical Mumford-Shah functional is a more general segmentation defined by Here u is the cartoon image approximating u 0, u is smooth except for jumps on the set G of boundary curves, and G segments the image into piecewise smooth regions. The method differs in that only two subregions are allowed in which u is piecewise constant, so we may write Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 10/19
Active Contours Without Edges We have This expresses the fact that the best constant value for the segment u is just the average of u 0 over the subregion. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 11/19
Active Contours Without Edges In order to compute the Euler-Lagrange equations we use the variational level set approach and arrive at The non-morphological approach is more effective; i.e., f is replaced by d e (f) in the term multiplying the brackets. Here for e > 0 and small, which gives a globally positive approximation to the delta function. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 12/19
Active Contours Without Edges Thus the model is Generally, the parameters are taken to be n = 0, l 1 = l 2 = 1, and μ > 0 is the scale parameter. Although only two regions can be constructed, they can, and generally will, be disconnected into numerous components in the finescale case, with each component having one of two constant values for u. One important remark concerning this model as opposed to other level set evolutions is its global nature. All level sets have the potential to be important. Thus reinitialization to the distance function is not a good idea here. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 13/19
Active Contours Without Edges Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 14/19
Active Contours Without Edges Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 15/19
Extensions Replacing u 0 by the curvature of its level sets Replacing u 0 by orientations to do texture segmentation. Vector valued images. Removing the piecewise constant assumption and allowing piecewisesmooth solutions to the variational problem, smooth inside each zero isocontour of f, with jumps across the edges. Getting several (many!) different regions corresponding to different level set functions. Based on the four color theorem we can partition an image using only four colors such that any two adjacent regions have different colors. Therefore, using two level set functions we can identify the four colors by the four possibilities f i > 0, f i < 0, i = 1, 2. This automatically gives a segmentation of the image. Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 16/19
Extensions: two channels Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 17/19
Extensions: 3 channels Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 18/19
Extensions: 4 phases Scale Space and PDE methods in image analysis and processing - Arjan Kuijper 19/19