Various Methods for Medical Image Segmentation

Transcription

1 Various Methods for Medical Image Segmentation From Level Set to Convex Relaxation Doyeob Yeo and Soomin Jeon Computational Mathematics and Imaging Lab. Department of Mathematical Sciences, KAIST Hansang Lee and So Jung Yun Statistical Inference and Information Theory Lab. Department of Electrical Engineering, KAIST

2 Outline 1. Medical Image Segmentation: An Overview 2. Level Set Method 3. Graph Cut Method 4. Convex Relaxation Method July 11,

3 Outline 1. Medical Image Segmentation: An Overview 1) What is Segmentation? 2) Segmentation of Medical Images 3) Various Segmentation Methods 2. Level Set Method 3. Graph Cut Method 4. Convex Relaxation Method July 11,

4 What is Segmentation? The purpose of image segmentation is to partition an image into meaningful regions with respect to a particular application Applications of Segmentation Object detection, Recognition, Medical Imaging July 11,

5 Segmentation of Medical Images Locate tumors and other pathologies Measure tissue volumes Diagnosis, study of anatomical structure Surgery planning Virtual surgery simulation Intra-surgery navigation Segmentation results of brain tumor segmentation T1, T1-contrast, T2, Flair, manual segmentation, automatic segmentation July 11,

6 Various Segmentation Methods Number of documents Graph cut Level set Convex relaxation Scopus query- Graph cut: TITLE-ABS-KEY(("graph-cut" OR "graph cut") AND "segmentation" AND ("CT" OR "computed tomography" OR "MRI" OR "organ" OR "cell" OR "medical")) AND (LIMIT-TO(SUBJAREA, "COMP") OR LIMIT-TO(SUBJAREA, "ENGI") OR LIMIT-TO(SUBJAREA, "MEDI")) / Level set: TITLE-ABS-KEY(("level-set" OR "level set") AND (same as above) / Convex relaxation: TITLE-ABS-KEY("convex" AND ("relaxation" OR "relaxed") AND (same as above) July 11,

7 Outline 1. Medical Image Segmentation: An Overview 2. Level Set Method 1) Variational Approach 2) Active Contours 3) Level Set Method 3. Graph Cut Method 4. Convex Relaxation Method July 11,

8 Variational Approach Overview Maximizing or minimizing functionals, which are often expressed as definite integrals involving functions and their derivatives. The interest is in extremal functions that make the functional attain a maximum or minimum value. Under ideal conditions, the maxima and minima of a given functional may be located by finding the points where its derivative vanishes. If finding the points directly is not easy, steepest descent method is used to find the points iteratively. July 11,

9 c.f.) Steepest Descent Method Steepest Descent Method is based on the observation that if the multivariable function FF(xx) is defined and differentiable in a neighborhood of a point aa, then FF(xx) decreases fastest if one goes from aa in the direction of the negative gradient of FF at aa. July 11,

10 Active contour model Example 1) Geodesic Active Contours model (1997) To detect the contours of the objects lying in the image Example 2) Chan-Vese model (2002) To find two regions which mean intensities are as disjoint as possible or, July 11,

11 Active contour model Active contour model is an energy minimizing, deformable curve influenced by constraint and image forces that pull it towards object contours. Movie Initial Curve Evolutions Detected Objects July 11,

12 How to solve the optimization problems? Level Set Method Basic Idea Using an implicit representation of the contour with a higher dimensional function Advantages Robust on topological change Easy to calculate geometric elements normal vector, curvature Limitations Slow Local solutions July 11,

13 How to solve the optimization problems? Level Set Method Basic Idea Using an implicit representation of the contour with a higher dimensional function Advantages Robust on topological change Easy to calculate geometric elements normal vector, curvature Limitations Slow Local solutions July 11,

14 Active Contour with Level Set Example 1) Geodesic Active Contours model (1997) Example 2) Chan-Vese model (2002) 2-phase model July 11,

15 Summary for Level Set Method Methods Global solutions Speed Memory Accuracy Flexibility Level Set Not guaranteed Slow Low Accurate Flexible July 11,

16 Outline 1. Medical Image Segmentation: An Overview 2. Level Set Method 3. Graph Cut Method 1) Graph Cut Overview 2) Max-flow/Min-cut Theorem 3) Graph Cut Method 4) Graph Cut Result of Medical Images 4. Convex Relaxation Method July 11,

17 Graph Cut Overview Image Segmentation: A Graph Approach Image I = {p} July 11,

18 Graph Cut Overview Image Segmentation: A Graph Approach Image I = {p} Graph G = (V,E) July 11,

19 Graph Cut Overview Image Segmentation: A Graph Approach Image I = {p} Seed S = {O,B} Graph G = (V,E) July 11,

20 Graph Cut Overview Image Segmentation: A Graph Approach Image I = {p} Seed S = {O,B} Graph G = (V,E) July 11,

21 Max-flow/Min-cut Theorem s a d b e c f t July 11,

22 Max-flow/Min-cut Theorem s a d b e c f t July 11,

23 Max-flow/Min-cut Theorem s a d b e c f t July 11,

24 Max-flow/Min-cut Theorem Min-cut s a d b e c f t NP hard July 11,

25 Max-flow/Min-cut Theorem Min-cut s s a d b e c f a d b e c f Flow t t NP hard July 11,

26 Max-flow/Min-cut Theorem Min-cut s s a d b e c f a d b e c f Flow t t NP hard July 11,

27 Max-flow/Min-cut Theorem Min-cut s s a d b e c f a d b e c f Flow t t NP hard July 11,

28 Max-flow/Min-cut Theorem Min-cut Max-flow s s a d b e c f a d b e c f Flow t t NP hard July 11,

29 Max-flow/Min-cut Theorem Min-cut Max-flow s s a d b e c f a d b e c f Flow t t NP hard Solved in polynomial time July 11,

30 Max-flow/Min-cut Theorem Min-cut Max-flow s s a d b e c f a d b e c f Flow t t NP hard Solved in polynomial time Max-flow/Min-cut Theorem. The value of the max-flow is equal to the capacity of the min-cut. July 11,

31 Graph Cut Method July 11,

32 Graph Cut Method s t July 11,

33 Graph Cut Method s EE(LL) = RR pp (LL pp + BB pp,qq δδ LLpp LL qq pp pp,qq Unary Energy t July 11,

34 Graph Cut Method s EE(LL) = RR pp (LL pp + BB pp,qq δδ LLpp LL qq pp pp,qq Unary Energy t July 11,

35 Graph Cut Method s EE(LL) = RR pp (LL pp + BB pp,qq δδ LLpp LL qq pp pp,qq Unary Energy t July 11,

36 Graph Cut Method s EE(LL) = RR pp (LL pp + BB pp,qq δδ LLpp LL qq pp pp,qq Pairwise Energy t July 11,

37 Graph Cut Method EE(LL) = RR pp (LL pp + BB pp,qq δδ LLpp LL qq pp pp,qq Pairwise Energy July 11,

38 Graph Cut Method EE(LL) = RR pp (LL pp + BB pp,qq δδ LLpp LL qq pp pp,qq Pairwise Energy July 11,

39 Graph Cut Method s t July 11,

40 Graph Cut Method s t July 11,

41 Graph Cut Method July 11,

42 Graph Cut Results of Medical Images Blood pool of a left ventricle in cardiac MRI (Boykov & Funka- Lea: IJCV 2006) July 11,

43 Graph Cut Results of Medical Images Knee cartilage in knee MRI (Shim et al. Radiology 2009) July 11,

44 Graph Cut Results of Medical Images Anterior cruciate ligament in knee MRI (Lee et al. ACCV 2012) July 11,

45 Graph Cut Summary Methods Level Set Graph Cut Global solutions Not guaranteed Guaranteed Speed Slow Fast Memory Low Huge Accuracy Accurate Less accurate Flexibility Flexible Not flexible July 11,

46 Outline 1. Medical Image Segmentation: An Overview 2. Level Set Method 3. Graph Cut Method 4. Convex Relaxation Method 1) Convex Relaxation Overview 2) Convex Relaxation for 2-phase CV Model 3) Recent Research for Convex Relaxation 4) Summary for Convex Relaxation Method July 11,

47 Convex Relaxation Overview Objective of designing an algorithm To guarantee to find global solutions To get the solution rapidly To obtain solutions accurately To be flexible to work for a large class of problems July 11,

48 Convex Relaxation Overview Non-convexity captures local solutions depending on the initialization July 11,

49 Convex Relaxation Overview Most existing objective functionals in image processing are not convex. Local solutions are captured depending on the initialization. Example 1) Geodesic Active Contours model (1997) Example 2) Chan-Vese model (2002) July 11,

50 Convex Relaxation Overview Non-convex objective functional Convex objective functional July 11,

51 Convex Relaxation for 2-phase CV model For any given fixed cc 1, cc 2, a global minimizer for MMMM(, cc 1, cc 2 ) can be found by carrying out the following convex minimization : and then setting T. F. Chan, S. Esedoglu, M. Nikolova, Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models. SIAM Journal on Applied Mathematics Vol. 66, , July 11,

52 Recent research for Convex Relaxation Fast Global Minimization of the Active Contour/Snake Model. (X. Bresson et al., 2005) : Geodesic Active Contour model A convex Relaxation Approach for Computing Minimal Partitions. (T. Pock et al., 2009) : Multi-phase Chan-Vese model An Algorithm for Minimizing the Mumford-Shah Functional. (T. Pock et al., 2009) : Mumford-Shah functional July 11,

53 Summary for Convex Relaxation Method Methods Level Set Graph Cut Convex Relaxation Global solutions Not guaranteed Guaranteed Guaranteed Speed Slow Fast Fast Memory Low Huge Low Accuracy Accurate Less accurate Accurate Flexibility Flexible Not flexible? July 11,

54 Conclusions Level set method and Graph cut method are frequently used in image segmentation. When we design an algorithm, it has to be fast, accurate and flexible. Moreover, it always provides global solutions. Level set method and Graph cut method can not guarantee these four properties simultaneously. Convex relaxation methods guarantee three properties. (fast, accurate, global solutions) We don t know how general Convex relaxation method can be. Convex relaxation methods are still developed. July 11,

55 Thank you. Q&A