Medical Image Analysis Active Shape Models

Transcription

1 Medical Image Analysis Active Shape Models Mauricio Reyes, Ph.D. ISTB - Institute for Surgical Technology and Biomechanics University of Bern

2 Lecture Overview! Statistical Shape Modeling! Quick summary/review! Further considerations! Reference selection! Modeling scaling! Outlier detection! Modeling Physiology! Statistical Shape Modeling for Segmentation: Active Shape Models 2

3 Segmentation Overview Big Picture Simple Methods! > Thresholding! > Region-Growing! >! Classification! + Clustering! > knn! > SVM! >! Deformable Models!! > Snakes! > Level-Sets! >! Active Models! > ASM > AAM 3

4 Statistical distribution of shape How?!

5 Summary Statistical Shape Modelling! Model: Mean Shape Statistical Analysis via Principal Component Analysis (PCA) Eigen-decomposition of matrix provides eigenvectors/eigenvalues (λ,φ) Sφ k = φ k λ k With k, the kth eigen-vector/value

6 Summary Statistical Shape Modelling! -Common reference space - Find anatomical corresponding points amongst shapes

7 Summary Statistical Shape Modelling! - Find anatomical corresponding points amongst shapes Principal modes of deformation - Model variability

8 Further considerations > What if scaling is included (e.g. bone length variability is considered)? Non-rigid transf. Affine transf. Direct morphing of these two might cause problems! What we look for Affine transf. Non-rigid transf. 8

9 Further Considerations! - Modelling Schemes! Mesh to Mesh Registration Reference mesh Isosurface Image volume

10 Further Considerations! - Modelling Schemes! Mesh to Volume Registration Reference mesh Image volume

11 Further Considerations! - Modelling Schemes! Volume to Volume Registration Reference mesh Reference volume Image volume

12 Further Considerations! - Modelling Schemes! * = Masking out soft tissues and other non-desired structures: Improved registration->improved modeling!

13 Further Considerations! - Modelling Schemes! Parameterization to Parameterization Registration Common base domain Surface and parameterization Image volume Common reference space: No bias! J

14 Bias in reference selection and natural coordinates reference! Not the same! L Rueckert et al. Miccai 2001

15 Bias in reference selection and natural coordinates reference! Computing mean image: Guimond et al. CVIU 2000

16 Bias in reference selection and groupwise modelling! Computing mean image: Guimond et al. Miccai 2000

17 Bias in reference selection and groupwise modelling! Computing mean image: Guimond et al. Miccai 2000 Reference 1 Reference 2

18 Bias in reference selection and groupwise modelling! Computing mean image: Guimond et al. Miccai 2000 Average model using ref 1 Average Intensity using ref 2

19 Further considerations! - Detecting Outliers through SSM! Sometimes the input data for modeling is not as expected.. Coronal slide from a 4D CT scan

20 Further considerations! - Detecting Outliers through SSM! Contribution measure (Cr): Cr(i,k) = 1 n (b i k ) 2 λ k bb: modal amplitude for subject i on mode k. Lamba_k: kth eigenvalue m1 m2 m3

21 Further considerations! - Detecting Outliers through SSM! Example: Image artefact, segmentation problem, deformity, etc.. Synthetic deformation added to instance number 13.

22 Further considerations! - Detecting Outliers through SSM! Example

23 Statistical Modelling of Physiology!

24 Statistical Modelling of Physiology!

25 Statistical Modelling of Physiology! First mode of variation Mean transformations

26 Statistical Modelling of Physiology! Second mode of variation Mean transformations

27 SSM for Segmentation: Active Shape Models

28 Active Shape Models ASM > General idea Start with SSM and image Initially place SSM in image Sample along SSM surface normals, find largest gradient (= edge) Deform pose (scale, rotation, translation) and shape (PCA modes) of model to best fit to extracted image features (largest gradient) Cootes et al

29 Initialization A Crucial Step > Approximately align SSM with structure of interest > Many different methods, choose the one for your application User interaction Centroid alignment Chamfer matching Atlas registration Global search on entire image (evolutionary algorithms) 29

30 Initialization A Crucial Step Importance of Initialization Initial 2 Iterations 20 Iterations 30 Image Source: Cootes et. al. SPIE Medical Imaging, 2001

31 Feature Extraction > Edge detection methods.. 31

32 Normals > SSM points + connections give mesh So we know the neighbors of each point of two vectors yields orthogonal vector This is the surface normal 32

33 Sampling along profiles > Given: Point on SSM Normal at that point > Sample along normal profile > Parameters Sampling range Sampling step size Yields 2k+1 samples > Choose point with largest feature value (gradient) > Model point should be transformed to that point 33

34 Pose Estimation > Goal: Rough alignment of both shapes! > Here: similarity transform in 2D! > Optimize registration process according to certain cost function (e.g. mean Euclidean distance)! X = T X t ICP, s, Θ( x) T X t, s, Θ x x = y y t t Θ + s cos ssin Θ ssin Θ x s cosθ y minimize Y X = Y T, s, Θ( x) X t 34

35 Shape Estimation > PCA-based deformation > Keep only k most important PCA modes (e.g. so that k modes account for 90% of variance) > Threshold b to allow for plausible shapes only, e.g. b i < 3 σ i (3 std.dev. capture more than 95% of variation in Gaussian distribution) b b x x + = = P P T T Pb ( x x) ( Y x) 35

36 Optimization Schemes > Optimize pose and shape part separately or jointly > Optimization Choose optimizer for pose estimation (e.g. gradient descent) Closed form solution for shape > Iterative method: Compute profile along normals Adjust pose Adjust shape Iterate from beginning until convergence b = P X = TX t, s, Θ( x + Pb) Y = TX t, s, Θ( x + Pb) T ( T X t 1, s, Θ ( Y) x) 36

37 Summary The Complete ASM Algorithm 37

38 Advanced Methods > Multiresolution pyramid > Handling noise by averaging along normal points > Intensity profiles 38

39 Advanced Methods > Intensity profiles: Modeling local structure 1. Samples along profile for ith image 2. Avoid intensity changes-> use gradient values 3. Normalize 4. Repeat for each data point 5. Assume Gaussian distribution: (g,s g ) 6. Quality of fit for new sample s: Mahalanobis distance 7. Pick lowest values (higher probability) 39

40 Examples ASM 2D Initial 2 Iterations 6 Iterations 18 Iterations 40 Image Source: Cootes et. al. SPIE Medical Imaging, 2001

41 Examples ASM 2D Clinical example Initial 1 Iteration 14 Iterations 41 Image Source: Cootes et. al. SPIE Medical Imaging, 2001

42 Examples ASM 3D [Fripp 2007] [Bauer 2012] 42

43 Example in CMF 43

44 Example in CMF ASM enhanced with muscle information Model Patient 44

45 Example in CMF Model enhanced with: Facial muscles Muscle fibers (anisotropy of tissue properties) Predefined surgical approaches Data preprocessing (e.g. cropping) Simulation-specific (e.g. sliding contact areas) 45

46 Statismo > Statismo provides a high level API. > Users works with shapes, not vectors. > Core functionality implemented only once. > Exchanging models becomes easy. Application 1 Sampling High level API Probabilities Representer (model semantic) Portable File (HDF5) Statismo Application n Conditional distributions 46

47 Image-guided soft tissue deformation for CMF surgery Statismo is Open Source Get it at Statismo - A framework for PCA based statistical models Lüthi M., Blanc R., Albrecht T., Gass T., Goksel O., Büchler P., Kistler M., Bousleiman H., Reyes M., Cattin P., Vetter T. Insight Journal, 47

48 Building texture models > Warp each training sample to mean shape to obtain meanfree patch > Sample intensities from shape-normalized image to form a texture vector > Normalize intensity to eliminate *lighting* variations > Start from the beginning, same as SSM PCA on normalized texture data g g + P g b g > We want to obtain a combined appearance model, which contains both shape and texture variations 48

49 Combined models of appearance > Shape and texture summarized by parameter vectors b > Handle correlations between shape and texture parameters b s and b g > Apply PCA on combined b to get eigenvectors P c and eigenvalues c > c is vector of appearance parameters controlling both shape and texture b Wsb = bg b = s P c c Weighting to account for unit differences 49

50 Combined Models of Appearance > Linear model > We can express shape and texture directly as function of c > How to generate a new image: Generate shape-free intensity image from g Warp it using the control points described by x Q s x = x + g = g + = PW s Q c Q 1 s Q = P g g s g P P c cs cg 50

51 Model Fitting > Minimize difference between new image and the synthetic AAM image > Key step: estimate parameter update from current image sample > Difference vector δi = I I i m 51

52 Model Fitting More Detailed 1 > Estimate update step using Jacobian > Residual difference r(p) > p is parameter vector p(c,t,u) > Minimize error E(p) > Taylor expansion of r(p+δp) > Choose δp to minimize E(p+ δp) > Need estimate of Jacobian J > Estimate J by small displacements on training set r( p) = g s g m E ( p) = r( p) r( p + δp) = r( p) + 2 r δp p E( p + δ p) = r( p + δp) δp = Rr( p) r R = p T r p 1 r p 2 T r J = p 52

53 Model Fitting More Detailed 2 > Iterative algorithm: Measure residual error r(p) Predict correction δp=-rr(p) p=p+k δp Repeat to convergence r( p) = g s g m δp = Rr( p) 53

54 Summary The Complete AAM Algorithm 9. Repeat until convergence 54

55 Examples AAM 55

56 Blanz &Vetter et al. 99 Example: AAM in face synthesis > Parameters: Pose, shape, illumination

57 Examples AAM [Cootes 2004] [Dornaika 2003] 57

58 Comparison ASM AAM Differences! > ASM searches around current position along normals, AAM searches at current position! > ASM minimizes distance between model and corresponding points, AAM minimizes difference between synthetic and target image!! Evaluation > ASM usually has larger capture range > ASM is faster > ASM usually has more accurate feature point location > AAM gives better match to image texture 58

59 References > [1] Cootes T.F., Taylor C.J., Cooper D.H., Graham J., Active Shape Models Their training and application Computer Vision and Image Understanding, 1995 > [2] Cootes T., Taylor C.J., Statistical Models of Appearance for Computer Vision, TechReport University of Manchester, 2004 > [3] Heimann T., Meinzer H., Statistical Shape Models for 3D Medical Image Segmentation: A Review, Medical Image Analysis,

60 Evaluating segmentation

61 Validation of Image Segmentation > Digital phantoms. Ground truth known accurately. Not so realistic. > Acquisitions and careful segmentation. Some uncertainty in ground truth. More realistic. > Autopsy/histopathology. Addresses pathology directly; resolution > Clinical data? Hard to know ground truth. Most realistic model. Source: S. Warfield

62 Validation of Image Segmentation > Comparison to digital and physical phantoms: Excellent for testing the anatomy, noise and artifact which is modeled. Typically lacks range of normal or pathological variability encountered in practice. Most common atlas: BrainWeb MRI of brain phantom from Prastawa et al. MedIA 2010 MRI of brain phantom from Styner et al. IEEE TMI 2000

63 Comparison To Higher Resolution MRI Photograph MRI Provided by Peter Ratiu and Florin Talos. Source: S. Warfield

64 Comparison To Higher Resolution Photograph MRI Photograph Microscopy Provided by Peter Ratiu and Florin Talos. Source: S. Warfield

65 Comparison to Autopsy Data > Neonate gyrification index Ratio of length of cortical boundary to length of smooth contour enclosing brain surface Source: S. Warfield

66 Validation of Image Segmentation > Comparison to expert performance; to other algorithms. Experts are currently doing the segmentation tasks that we seek algorithms for. Surgical planning. Neuroscience research. > What is the appropriate measure for such comparisons? Source: S. Warfield

67 Measures of Expert Performance > Repeated measures of volume Intra-class correlation coefficient > Spatial overlap Jaccard coefficient: Area of intersection over union. Dice coefficient: increased weight of intersection. Vote counting: majority rule, etc. > Boundary measures Hausdorff, 95% Hausdorff. > Bland-Altman methodology: Requires a reference standard. > Measures of correct classification rate: Sensitivity, specificity ( Pr(D=1 T=1), Pr(D=0 T=0) ) Positive predictive value and negative predictive value (posterior probabilities Pr(T=1 D=1), Pr(T=0 D=0) )

68 Measures of Expert Performance > Measures of correct classification rate: Sensitivity, specificity ( Pr(D=1 T=1), Pr(D=0 T=0) ) Positive predictive value and negative predictive value (posterior probabilities Pr(T=1 D=1), Pr(T=0 D=0) ) Wikipedia

69 Thanks for your attention!