1. Two double lectures about deformable contours. 4. The transparencies define the exam requirements. 1. Matlab demonstration

Transcription

1 Practical information INF 5300 Deformable contours, I An introduction 1. Two double lectures about deformable contours. 2. The lectures are based on articles, references will be given during the course. 3. The transparencies will be available on the course website. 4. The transparencies define the exam requirements Lars Aurdal, Norsk Regnesentral, 9th of February 2005 Plan, lecture I Plan, lecture II 1. What is segmentation? 2. Motivation, why deformable contours? 3. The basic formalism: snakes as presented by Kass and Witkin. 1. Matlab demonstration 2. Variations of the basic snakes a. Topological adaptability b. 3D 3. The capture range problem a. Distance measure based solutions b. Gradient vector flow field based solutions

2 What is segmentation? What is segmentation? 1. Sonka: The main goal of image segmentation is to divide an image into parts that have a strong correlation with objects or areas of the world contained in the image 2. Sonka: Complete segmentation: results in a set of disjoint regions corresponding uniquely with objects in the input image. 3. Sonka: Partial segmentation: regions do not correspond directly with image objects. 1. Sonka: To achieve a complete segmentation, cooperation with higher processing levels which use specific knowledge of the problem domain is necessary. 2. Sonka: There exists a whole class of segmentation problems that can be solved using lower level processing only. What is segmentation? What is segmentation? 1. Aurdal: Segmentation is a process that maps an input image (possibly multispectral) into an output image where each pixel in the input image is given a label indicating its membership in a group of pixels sharing some common property. Group inclusion can be based on: a. Similar spectral properties. b. Similar higher order spectral properties (texture). c. etc.. 1. Segmentation is not the same thing as classification! 2. In segmentation each pixel receives a label indicating its membership in a group of pixels (group 1, group2 etc ) 3. In classification each pixel receives a label indicating its membership in a group of pixels having a sensible physical interpretation (water, ship, land etc ) 4. Classification will often use segmentation as a preprocessing step

3 Motivation Motivation 1. Classical assumption: Digital images will show real world objects as well defined regions with unique gray levels and a clear border against a uniform background. a. This assumption can hold true in certain cases, but typically it does not. 2. Real world objects seen in digital images might very well blend with the background and many parts of the objects might be more or less occluded. Beware of extreme case of blending and occlusion Motivation Motivation 1. Classical assumption: The image contains all the information necessary in order to produce a good segmentation. a. Again, this might be so in certain situations, but this is more of an exception than the rule. 2. In real world images we will typically need higher level information about the objects we are trying to segments: a. Size b. Shape c. Position 1. We know of many cases where such knowledge is used: a. Thesholding: We assume knowledge of graylevels. b. Adaptive thesholding: We assume knowledge of the size of the objects we are interested in c. Hough transform: We assume certain shapes

4 Motivation Motivation 1. The methods we will look at now are just another example of segmentation methods using models external to the image in order to obtain the best possible segmentation. 2. Typical segmentation problems solved with this kind of image are for instance ultrasound cardiac images. a. Much noise and blurred edges b. Much knowledge about shape etc. Left ventricle of the heart, apical view. GE medical 1. Snakes are model-based methods for localization and tracking of image structures. 2. The snake is defined as an energy minimizing contour (often defined using splines). 3. The energy of the snake depends on its shape and location within the image. 4. Snakes are attracted to image boundaries through forces. 1. The approach is iterative: a. The user draws an initial approximate contour. b. A dynamic simulation is started. c. The contour is deformed until it reaches equilibrium. 2. Snakes depend on: a. Interaction with the user. b. Interaction with a high level description. c. Interaction with image data adjacent in space and time.

5 1. Different types of forces will affect the snake: a. Internal forces. 1) Tension forces that make the snake act like a membrane. 2) Rigidity forces that make the snake act like a thin plate that resists bending. 2. Image forces 3. Constraint forces. a. User supplied forces or forces that come from higher level image understanding processes. 1. The contour is represented as: 2. This is a parametric representation of the contour. 3. The vector describing the position of every point on the contour makes one pass over the entire contour as s varies from its minimum to its maximum value. Typically s is normalized: What is a parametric contour? 1. Let x(s)=cos(2π s) and y(s)=sin(2π s). Furthermore, let s [0,1]. 2. Then v(s) describes a circle as s varies from 0 to The shape of the contour v(s) is dictated by a functional: 2. This functional can be seen as representing the energy along the contour. 3. The final position of the contour will correspond to a minimum of this energy function.

6 1. The first term of the functional can be viewed as a representation of the internal deformation energy: 1. The first term: 2. imposes tension, that is, we want a curve as short as possible. 1. The second term: 2. imposes rigidity, changes in direction should be smooth. 1. The weight functions w 1 (s) and w 2 (s) are used to modify the weights given to the two terms. 2. Why do you think it is a smart idea to make these a function of s? Can you think of cases where this might be a nice feature.

7 Physical analogy 1. The first term makes v act like a membrane 2. The second term makes it act like a thin plate. 3. The weighting functions will just balance these two types of behavior. 1. The second term of the functional introduces an external energy term into the expression for the functional: 1. The most common way of defining P(x,y) is the following: 1. In this expression, P(v(s)) denotes a scalar potential function defined on the image plane. 2. We obviously choose potential functions P(x,y) whose minima coincide with intensity extremas, edges or other interesting features of the image. 2. What does this expression mean in terms of the final position of the snake.

8 1. A variation of this can be obtained by letting: 2. What would be the use for this kind of definition? 1. So now we know how we measure the goodness of the current snake position, we simply measure the energy given by: 2. We are basically looking for a minimum in this energy. 3. When the minimum is reached, the snake will be in an optimal position. 4. How do we find this minimum? 1. Remember: 2. Also: 1. Let assume that the functions w 1 (s) and w 2 (s) are simply constants. 2. In practice, this is by far the most common situation. 3. Thus we have: 3. And:

9 1. The total energy can thus be written: 1. It can be shown that the v that minimizes E must satisfy the following differential equation: 2. This is the Euler-Lagrange equation for the given problem. 3. You ll find more on this in appendix A. 1. We have the equation: 1. We have: 2. This becomes: 2. In component form this becomes:

10 1. In practical life the parameter domain is discrete, that is {s 0 =0, s 1,s 2,,s n =1}. 2. The curve v(s) will be described by n+1 sample points (control points), v 0 =v(s 0 ),,v i =v(s i ), v n =v(s n ). 3. The differential equations in the previous slide must be transformed into discrete versions. 1. We must solve these equations numerically. 2. That means discretizing them. 3. First we need to find expressions for the second and fourth derivatives of x and y with respect to s. 4. The details of this are discussed in appendix B. 1. The second derivative of x(s) with respect to s, x (s) can be approximated by: 2. Similarly, the approximation to the fourth derivative is given by:

11 1. Look at the x-component of the previous equations: 1. For one particular x(s n ) the equation will be: 2. After discretizing and using the previous approximations this can be written i matrix form. 2. Introducing α=w 2, β=(-w 1-4w 2 ) and γ=(-2w 1 +6w 2 ) this can be rewritten: 1. The matrix equation for the x component becomes: 1. A more compact way of writing this is: A 2. The matrix A has special properties, it is a pentadiagonal matrix. 3. It is typically referred to as the stiffness matrix, it actually relates the positions of the points along the contour to each other. 4. So now all we need is a matrix inversion.?

12 1. Not quite that simple! 2. We look for a snake position where the internal forces are counterbalanced by the external forces. 3. If all we do is a matrix inversion in the previous eqaution we will find a curve v that locally is the best possible curve. 4. But we want a curve that is globally optimal. 5. The way we solve this is iterative. 6. We actually make the position of the snake a function of time, the snake can move. 1. Remember we had: 2. The physical interpretation of this is that we have found the wanted x and y vectors when the transformation of x and y is exactly counterbalanced by P x and P y. 3. That is, when the internal forces are counterbalanced by the external forces. 1. What if we made x and y dependent on time (or iteration), that is we write x t and y t. 2. With this change in mind, consider the following change in the previous equation: 1. The beauty of this is that it allows us to write an iteration for xt and yt. 2. Start by rewriting: 3. It is actually saying exactly the same thing as the previous equation, the wanted x t and y t is found when their transformation is counterbalanced by the external forces.

13 1. Start with: 2. A slight rearrangement gives us: 1. This was the initial idea of Kass and Witkin in their very important paper: Snakes: Active contour models, Michael Kass, Andrew Witkin and Demetri Terzopoulos, International Journal of Computer Vision, pages , This is an excellent paper that definitely deserves some study. 3. This is the basic iteration that lets us update x and y. 1. In order to obtain snakes that move in a physically more plausible way the nodes are typically given a mass and an initial speed. 2. A damping force is also often used (as if the snake was moving in some viscous medium). 1. The following slides deal with the Euler Lagrange equation and its derivation and are included primarily for the sake of completeness. 2. The notation is kept as general as possible. 3. Some of the blood-dripping details have been omitted for the sake of clarity. Return to presentation

14 1. Leonard Euler is recognized as one of the greatest mathematicians of all times. 2. Read more about him at: icians/euler.html 1. Joseph-Louis Lagrange was an outstanding French mathematician. 2. Read more about him at: aticians/lagrange.html 1. The Euler Lagrange arises within the framework of the calculus of variations. 2. Unlike the case in static optimization problems, the problem in variational calculus is that of finding some function y(x) such that some integral: 3. is either a minimum or a maximum. 1. In this equation, f() is the functional, y (x)=dy/dx, and x is the independent variable. 2. For y(x) to be an extremum it is required that any neighboring function is not an extremum. 3. We define a neighboring function of y by adding an arbitrary amount to y(x). 4. Parametrically this is written as y=y(α, x).

15 1. Observe that for α=0, y=y(0,x)=y(x). 2. Furthermore: 1. Having defined variations to our solution, let s look for conditions under which our solution is extremum by substituting y(α,x) in our functional: 3. where η(x) is some arbitrary function of x that has continuous first derivatives and vanishes at x 1 and x 2 (that is, η(x 1 )=0 and η(x 2 )=0). 4. Note that this is the way in which we define variations of our solution. 1. For J(α) to be an extremum, we must have: 1. We have: 2. for all η(x) 3. Let s look at: 2. Interchanging the order of integration and differentiation we get:

16 1. By the chain rule we get: 1. Substituting we get: 2. But: 3. and 2. Now observe that if we could reduce η/ x to g()η(x) then we could factor out η(x) and would be left with something that must be equal to zero. 1. This can be done by integration by parts. 2. Remember: 3. Applying this to the previous integral we get: 1. But we have defined η(x) to be zero at the limits of integration so the first term disappears. 2. From the second term it is possible to factor out η(x) which is what we wanted. 3. Substituting back into the original equation we get:

17 1. We have found that: 1. We have found that: 2. Thus we have: 2. Since η(x) is arbitrary the remaining part of the integrand must be equal to zero as a necessary condition for J to be an extremum. 3. Thus: 4. This is the Euler Lagrange. 1. A development very similar to what we have just gone through allows for the establishment of the Euler Lagrange equation for cases where f is a function of y and its first and second derivatives. 2. This equation is: 1. This equation is a central equation in mathematics and physics. 2. It has many important uses, one of its famous applications is to the solution of the Brachistochrone problem, see: lem.html

18 Appendix B: Numerical approximations to derivatives. Appendix B: Numerical approximations to derivatives. 1. The following slides deal with the numerical approximation to derivatives and are included primarily for the sake of completeness. 2. The notation is kept as general as possible. 3. Some of the gory details have been omitted for the sake of clarity. 1. The derivative of a function f(x), denoted f (x) is defined by: 2. It is a measure of how fast the function f changes with a change in x. 3. You all know that, for instance, the derivative f (x) of f(x)=x 3 is f (x)=3x But what if no analytical expression for f is known, what if we only have samples of f. Return to presentation Appendix B: Numerical approximations to derivatives. Appendix B: Numerical approximations to derivatives. 1. Consider a list of samples of f, let us call them f(n), n=.-2,-1,0,1,2 2. The definition of the derivative partly answers our question as to how we compute a numerical approximation to it. 1. Remember the definition of f (x): 2. If we remove the limit we get: 3. For h small, we would expect this to be fairly close to the true value for f (x).

19 Appendix B: Numerical approximations to derivatives. Appendix B: Numerical approximations to derivatives. 1. The question is obviously how large the error is when we compute the derivative tin this way. 2. In order to analyse this, consider the Taylor expansion of f(x) around the point (x+h): 3. Reorganizing this we find: 1. A simple trick allows us to improve on this. Remember first: 2. It is also a simple matter to show that: 3. The difference of these equations is: 4. Here, the term O(h) regroups terms including h and higher powers of h, it is a measure of the error we make. Appendix B: Numerical approximations to derivatives. Appendix B: Numerical approximations to derivatives. 1. Remember: 2. Rearranging we get: 3. Compare this with: 1. By similar reasoning, discrete approximations to higher order derivatives of f(x) can be found. 2. This is a complex subject and this very simplistic exposition is in no way complete. 3. The only intention is to give the reader a starting point for further study. 4. This symmetrical way of computing f (x) is better, the error we make will be much smaller.