Computational Radiology Lab, Children s Hospital, Harvard Medical School, Boston, MA.

Transcription

1 Shape prior integration in discrete optimization segmentation algorithms M. Freiman Computational Radiology Lab, Children s Hospital, Harvard Medical School, Boston, MA. moti.freiman@childrens.harvard.edu

2 Shape prior integration in discrete optimization segmentation algorithms This research was done at the: Computer Aided Surgery and Medical Image Processing Lab. School of Eng. And Computer Science, The Hebrew University of Jerusalem, Israel Website:

3 Outline: Introduction Local shape constraint graph min-cut for vascular lumen segmentation Latent parametric shape constraint graph min-cut for Aortic Arch Aneurysm (AAA) thrombus segmentation Latent non-parametric shape constraint graph mincut for kidney segmentation Related work

4 Introduction

5 Discrete segmentation Segmentation: A labeling map that classify each voxel to its class The classification problem can treat each voxel independently (thresholding etc.) or as a Markov Random Field (MRF, dependencies between neighboring voxels) Relaxation: We will discuss only 2 classes MRF problems, although the presented solution are extendable to problems with more than 2 classes

6 Discrete segmentation Maximum A Posteriori Estimation of Labeling map (M) given an observed image (I) is defined as: where

7 Discrete segmentation : The likelihood term, represents the likelihood of the observed information at voxel x given its label m(x) : The spatial regularization term, penalize for assigning different labels to neighboring voxels

8 Discrete segmentation The solution can be found by minimizing the negative log of this energy:

9 Discrete segmentation In case of binary problems: The optimal solution can be obtained by the graph min-cut technique in polynomial time, where edge weights are representing the model probabilities, (Boykov et al, 1999,2001). Illustration from Boykov et al, 2001

10 Intensity based probabilities Boykov et al framework used only intensity information to compute the MRF probabilities Not always sufficient to separate between objects in medical images Does not include any object shape information Estimation of the prior intensity model is usually obtained by having the user delineate foreground and background regions Energy function is biased to convex shapes, which is inappropriate for segmenting elongated objects with bifurcations such as vascular structures

11 Incorporation of fixed shape priors into the graph min-cut framework 1. Graph cut segmentation using an elliptical shape prior, Slabaugh & Unal, ICIP Interactive Graph Cut Based Segmentation With Shape Priors, Freedman & Zhang, CVPR 2005.

12 Incorporation of shape priors into the graph min-cut framework 3. OBJ-CUT, Kumar, Torr & Zisserman, CVPR Graph cut segmentation with non linear shape prior, Malcolm, Rathi & Tannenbaum, ICIP 2007.

13 Local shape constraint graph mincut for vascular lumen segmentation (Freiman et al, 3DPH 2009)

14 Shape constrained graph-cut based segmentation Global minimization of a shape constrained discrete energy model: Both the likelihood and the regularization terms depend on the shape model. Shape prior is obtained using a local shape descriptor

15 Local tubular shape descriptor (Frangi 98)

16 Local tubular shape descriptor (Frangi 98)

17 Asymmetric adaptive regularization weights boundary based regularization Encourage labeling map to include voxels nearby high vesselness response to be included in the object class Less sensitive to intensity variability inside the vessel σ is linearly depend on the vesselness shape term

18 Energy sub-modularity Energy must be sub-modular to allow polynomial optimization with the graph-cut framework is non-negative, therefore:

19 Effect of intensity and shape terms on carotid bifurcation segmentation

20 Carotid arteries segmentation results (3D)

21 Carotid arteries segmentation results (2D views) (a) Severe stenosis (b) Dental implants artifacts

22 Carotid arteriessegmentation results (2D views) (c) Vertebral arteries (d) Coronal view

23 Interactive refinement 1. Given two seed points 2. Compute the shortest-path on the image graph, based on local and global edge weights 3. Estimate vessel radius near the seed points and define the possible region for vessel surface 4. Estimate vessel intensity model, based on the computed path 5. Compute optimal cut based on smoothing and gradient terms

24 Final results

25 Latent parametric shape constraint graph min-cut for Aortic Arch Aneurysm (AAA) thrombus segmentation (Freiman et al, ISBI 10)

26 A close look at the anatomy 1) Aortic lumen 2) Aortic thrombus 3) Inferior Vena Cava (IVC) 4) Right psoas muscle 5) Left psoas muscle 6) Vertebrae 7) The small bowel

27 Abdominal Aortic Aneurysm (AAA) lumen segmentation Lumen segmentation using our method: Nearly automatic vessels segmentation using graph-based energy minimization. Proc. 3D Segmentation in the Clinic: A Grand Challenge III, Carotid bifurcation evaluation, MICCAI 2009 workshop.

28 Intensity information is not sufficient for thrombus segmentation

29 Abdominal Aortic Aneurysm thrombosis segmentation Challenge: No explicit model for the thrombosis Solution: Treat the shape constraint as a latent variable Discrete energy minimization using the Expectation- Maximization approach

30 Optimization scheme Loop until convergence: End loop. E-step: Estimation of both intensity and shape parametric models. M-step: Graph min-cut segmentation, using the assumed shape and intensity models.

31 Latent parametric shape model Thrombosis can be modeled as a set of axial ellipsoids

32 First iteration: prior intensity model without shape constraint Fixed prior intensity model No shape constraint Optimization is limited to a predefined fixed radius around the lumen

33 Robust ellipsoid fitting 1. Collect a set of points P on the segmentation surface 2. Compute the distance from each point p i to the estimated ellipsoid surface 3. Select the N closest points to current estimated ellipsoid 4. Fit a 2D parametric ellipsoid to the selected points using Taubin s least-squares method (IEEE TPAMI, 1991)

34 EM optimization: E-step For each slice ellipsoid is fitted using the proposed method 3D model is reconstructed by collecting the 2D ellipsoids Distance map is used to represent the shape model

35 EM optimization: M-step Voxel to terminal nodes edges: Intensity term: based on the previous iteration thrombosis region intensity PDF. Background probability is considered as: 1-foreground. Shape term: voxel s probability to belong to the thrombosis, based on the ellipsoids model Voxel to neighbor voxels edges: Intensity term: based on voxels contrast Shape term: spatial probability of the thrombosis surface, based on the ellipsoids model

36 Segmentation results Green contour: ground truth Red contour: our result (includes the lumen)

37 Segmentation results Green contour: ground truth Red contour: our result (includes the lumen)

38 Latent non-parametric shape constraint graph min-cut for kidney segmentation (Freiman et al, MICCAI 2010)

39 Kidney anatomy 1) Left kidney 2) Right kidney 3) Liver 4) Vertebrae Main challenge: Separation between the kidney surrounding tissue such as the liver, muscles, and spleen

40 Kidney segmentation: Intensity based graph-cuts 1) Shim, H., Chang, S., Tao, C., Wang, J.H., Kaya, D. and Bae, K.T. Semiautomated Segmentation of Kidney From High- Resolution Multidetector Computed Tomography Images Using a Graph-Cuts Technique. J Comput Assist Tomogr, 33: , 2009.

41 Non parametric latent shape prior Non parametric shape prior: Set of Kidney CT volumes, with annotated kidneys A common coordinate system is not required No parameterization of the inter-patient shape variability Required multiple registrations during the segmentation process accelerated using parallel computing

42 EM based energy minimization (1)

43 EM based energy minimization (2)

44 First iteration: E-step: model estimation The new CT volume is registered using B-Spline registration to each one of the atlas CT volumes The kidney region is a weighted average of the projected annotations from the atlas datasets, to the new volume. Weights represent the fidelity between the grayscale images Intensity model is computed based on weighted histogramming of the assumed kidney region Subsequent iterations: The binary result from previous iteration is used for intensity model computation The kidney region is a weighted average of the projected annotations from the atlas datasets, to the new volume. The weights represent the fidelity to current segmentation

45 M-step: Graph min-cut optimization Voxel to terminal nodes edges: Intensity term: Foreground: based on the kidney region intensity PDF (computed from the kidney region histogram) Background probability is considered as: 1-foreground. Shape term: Voxel s probability to belong to the kidney, based on the atlas model:

46 M-step: Graph min-cut optimization Voxel to neighbor voxels edges: Intensity term: based on voxels contrast Shape term: spatial probability of the kidney surface, based on the atlas model. More sensitive to contrast changes on the expected object boundary

47 Examples

48 Results

49 Conclusions 1. A local shape constraint graph min-cut approach for vascular lumen segmentation. 2. A global parametric shape constraint approach for AAA thrombosis segmentation. 3. General non-parametric shape constraint graph mincut approach for organs segmentation with application to kidney.

50 Shape constraints integration in graph structure 1. S. Vicente, V. Kolmogorov, and C. Rother, Graph cut based image segmentation with connectivity priors, in CVPR A. Besbes, N. Paragios, N. Komodakis, and G. Langs, "Shape Priors and Discrete MRFs for Knowledge-based Segmentation, In CVPR C. Wang, O. Teboul, F. Michel, S. Essafi and N. Paragios, 3D Knowledge-Based Segmentation Using Pose-Invariant Higher-Order Graphs, In MICCAI D.R. Chittajallu, S.K. Shah, and I.A. Kakadiaris, A shape-driven MRF model for the segmentation of organs in medical images, In CVPR I. Ben Ayed, K. Punithakumar, G. Garvin, W. Romano, and S. Li, Graph Cuts with Invariant Object-Interaction Priors: Application to Intervertebral Disc Segmentation, in IPMI 2011

51 Shape constraints integration in graph structure NP hard problems - require complex optimization schemes to achieve approximate solutions Enforce Discretization of the shape models

52 Acknowledgements Prof. L. Joskowicz, M. Natanzon, N. Boride, J. Frank, L. Weizman, A. Kronman (School of Eng. and Computer Science, The Hebrew Univ.) Dr. J. Sosna, S.J. Esses, P. Berman (Dept. of Radiology, Hadassah Medical Centre). O. Shilon, E. Nammer (Simbionix LTD). This research is supported in part by MAGNETON grant from the Israeli Ministry of Trade and Industry and by the Hoffman Hebrew Univ. Responsibility and Leadership program.

53 Thank you!