Image Segmentation and Validation

Transcription

1 Computational Radiology Laboratory Brigham and Women s Hospital Boston, Massachusetts USA a teaching affiliate of Harvard Medical School Image Segmentation and Validation Simon K. Warfield Assistant Professor of Radiology Harvard Medical School Department of Radiology, MRI Division Computational Radiology Laboratory An NCRR National Resource Center

2 Overview Applications: Abdominal lesion ablation assessment. Prostate brachytherapy and biopsy. Image Segmentation: Combining registration and classification. Capturing pathological anatomy. Extending algorithms to abdominal and orthopedic anatomy. Validation approaches: Assessment of accuracy and precision, comparison of imaging and analysis methods. Application: Knee cartilage assessment. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 2

3 Application Abdominal lesion ablation assessment: Warfield et al. MICCAI 2000 Butz et al. MICCAI 2000, M.S. thesis 2000 Nakamura Ph.D. thesis 2002 Silverman et al. Radiology 2000 Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 3

4 MRI-Guided Cryotherapy Vertically open 0.5T MRI scanner provides access to patient for percutaneous thermal ablation; gas-based cryo technology allows for multiple, small diameter cryo-needles to freeze tumors. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 4

5 Planning Computer-assisted planning of cryotherapy can assist in the optimization of probe placement. Multiple elliptical volumes are used to approximate coverage of a tumor (green), A and C. For research, such idealized plans can be compared retrospectively against actual ice formation (yellow), A, B and D. A global view of the anatomy provides visualization of possible probe trajectories (liver, pink; vessels, blue), E and F. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 5

6 Liver Motion Due to Breathing Provided by Nobuhiko Hata. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 6

7 Targeting Real-time Targeting In-Plane In-Plane 90 Perpendicular Initial Probe 3 Probes 2 of 3 Probes MRI-GUIDED CRYO RENAL bowel Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 7

8 Monitoring MRI-GUIDED CRYO LIVER 3 CRYONEEDLES AT 4 TIME-POINTS DURING A 15 MINUTE FREEZE ADJACENT TO KIDNEY Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 8

9 Control Computer-assisted control monitors and predicts iceball extent. The 3D computation augments the physicians visualization. Shown is an image from a 10 min freeze, MRIguided cryo experimental liver. Top left: 3D view; top right: in plane view of actual iceball; bottom left: OF vectors identify growth; bottom right: predicted iceball. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 9

10 Assessment LIVER CRYONECROSIS TUMOR GALLBLADDER Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 10

11 Rigid Registration: MRI to CT Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 11

12 Pre- to Post- Therapy Registration Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 12

13 MRI Guided Cryo - Liver COR Pt. J.G. # ser21 i005 PRE INTRA POST SAG Pt. J.G. # ser03 i003 Pt. J.G. # ser09 i080 Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 13

14 MRI Guided Cryo - Renal Pre Procedure Post Procedure MRT CRYO Proc#104 AXIAL CORONAL Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 14

15 Application Prostate Brachytherapy and Biopsy Bharatha et al. Med Phys 2001 Ferrant et al. IEEE TMI 2001 Zou et al. Acad Radiology 2003 Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 15

16 Nonrigid Registration - Prostate Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 16

17 Multi-parametric Maps Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 17

18 Nonrigid Registration - Prostate Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 18

19 Surface Matcher Elastic Model Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 19

20 Intraoperative Display Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 20

21 Intraoperative Display Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 21

22 Intraoperative Display Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 22

23 Image Segmentation Paradigm Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 23

24 Contrast Optimization Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 24

25 Statistical Classification Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 25

26 Estimation of Class Distributions P = p( x') dx' R p( x) V Select n samples: n k n k k P = C P (1 P) k E[ k] = np An estimator for the local probability is then (Duda,Hart 1973): k/ n p( x) = V Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 26

27 Supervised Statistical Classification Error rate with unlimited training data is (1+1/k)R*, where R* is the minimum possible. Good stability with respect to prototype selection. Good noise rejection properties. Fast algorithms are known. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 27

28 Multiple Sclerosis PDw T2w Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 28

29 Multiple Sclerosis PDw T2w knn Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 29

30 Overlapping Distributions Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 30

31 Segmentation by Alignment Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 31

32 Explicit Anatomical Models Seek a segmentation by ``anatomy, not ``tissue class. For example: lateral ventricles, not cerebrospinal fluid. Explicit models have been used to: Guide low-level image processing, Recognize structures by recognizing edges, Segment structures by boundary projection. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 32

33 Explicit Anatomical Models Different approaches can be characterized by: The representation of the anatomical model. The method used to establish correspondences. The representation of the transform for projecting the model onto a subject. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 33

34 A Volumetric Template 1. Construct the template. 2. Project the segmentation from the template onto subject scans. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 34

35 Spatially Varying Classification New idea : integrate 'segmentation by alignment' with 'segmentation by classification.' Exploit anatomical context to increase the ability to distinguish between different classes. Increase the dimensionality of the feature space in which we carry out classification by using robust anatomical localization as new features. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 35

36 Anatomical Localization Nonrigid registration of template of normal anatomy to match subject. Model anatomical localization as a spatially varying penalty for a classification different from that indicated by the matched template. Iterate registration and classification. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 36

37 Anatomical Localization Matched atlas White matter localization Gray matter localization Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 37

38 Overlapping Class Distributions Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 38

39 Anatomical Localization Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 39

40 Multiple Sclerosis PDw T2w knn SVC Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 40

41 MS Lesion Classification Atlas EM SVC Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 41

42 SVC Summary More accurate and robust segmentations obtained through classification with local anatomical context. Successful applications: Normal neonate brain MRI, cartilage of the knee (Warfield et al. Med Imag Anal 2000). White matter disease: Multiple sclerosis, gait disorder and normal aging in the elderly, Alzheimer s disease. Brain tumor segmentation: meningiomas and gliomas (Kaus et al. Radiology 2001). Image guided surgery: brain tumor surgery, liver cryotherapy, prostate brachytherapy. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 42

43 Validation of Image Segmentation STAPLE (Simultaneous Truth and Performance Level Estimation): An algorithm for estimating performance and ground truth from a collection of independent segmentations. Warfield, Zou, Wells MICCAI Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 43

44 Validation of Image Segmentation Comparison to digital and physical phantoms: Excellent for testing the anatomy, noise and artifact which is modeled. Typically lacks range of normal or pathological variability encountered in practice. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 44

45 Comparison To Higher Resolution MRI Photograph MRI Provided by Peter Ratiu and Florin Talos. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 45

46 Comparison To Higher Resolution Photograph MRI Photograph Microscopy Provided by Peter Ratiu and Florin Talos. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 46

47 Validation of Image Segmentation Comparison to expert performance; to other algorithms: What is the appropriate measure for such comparisons? Our new approach: Simultaneous estimation of hidden ``ground truth and expert performance. Enables comparison between and to experts. Can be easily applied to clinical data exhibiting range of normal and pathological variability. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 47

48 How to judge segmentations of the peripheral zone? 1.5T MR of prostate Peripheral zone and segmentations Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 48

49 Estimation Problem Complete data density: ( D, T p, q) Binary ground truth T i for each voxel i. Expert j makes segmentation decisions D ij. Expert performance characterized by sensitivity p and specificity q. We observe expert decisions D. If we knew ground truth T, we could construct maximum likelihood estimates for each expert s sensitivity (true positive fraction) and specificity (true negative fraction): pˆ, qˆ = arg max ln f ( D, T p, q) Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 49 f p,q

50 Expectation-Maximization General procedure for estimation problems that would be simplified if some missing data was available. Key requirements are specification of: The complete data. Conditional probability density of the hidden data given the observed data. Observable data D Hidden data T, prob. density Complete data (D,T) f ( T D, θˆ ) Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 50

51 Expectation-Maximization Solve the incomplete-data log likelihood maximization problem pq ˆ ˆ E-step: estimate the conditional expectation of the complete-data log likelihood function. M-step: maximize, = argmaxln f ( D p,q) p,q ˆ ( ) ln ( ) Q θ θ = E f ˆ D,T θ D,θ Q( θ θˆ ) Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 51

52 Expectation-Maximization Since we don t know ground truth T, treat T as a random variable, and solve for the quality parameters that maximize: ˆ Q( θ θ ) ) = E ln f( ˆ D,T θ D,θ Parameter values θ j =[p j q j ] T that maximize the conditional expectation of the log-likelihood function are found by iterating two steps: E-step: Estimate probability of hidden ground truth T given a previous estimate of the expert quality parameters, and take expectation. M-step: Estimate expert performance parameters by comparing D to the current estimate of T. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 52

53 To Solve for Expert Parameters: k+ 1 k+ 1 k k p, q = argmaxe ln f( D, T p,q) D, p,q p,q k = arg max E ln f( D T, p,q) f( T) D, p,q p,q where k k p,q k k f ( T D,p,q ) = = i [ are the previous estimates of the quality parameters. f( D k k f( D T,p,q ) f( T) k k f( D T,p,q ) f( T) T k k ij Ti, pj, qj ) f( Ti ) i j k k [ T f( Dij Ti, pj, qj ) f( Ti )] i j k k f( Dij Ti, pj, qj ) f( Ti ) k k j i D,p i,q = k k T f( Dij Ti, pj, qj ) f( Ti ) i j Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 53 where i indexes over voxels and j over experts. For each voxel i f( T ) ] k

54 Expectation-Maximization Solve the incomplete-data log likelihood maximization problem E-step: estimate the conditional expectation of the complete-data log likelihood function. M-step: maximize θˆ = arg max ln f ( D θ) θ ˆ ( ) ln ( ) Q θ θ = E f ˆ D,T θ D,θ Q( θ θˆ ) Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 54

55 STAPLE: Expectation-Maximization Since we don t know ground truth T, treat T as a random variable, and solve for the quality parameters that maximize: ˆ Q( θ θ ) ) = E ln f( ˆ D,T θ D,θ Parameter values θ j =[p j q j ] T that maximize the conditional expectation of the log-likelihood function are found by iterating two steps: E-step: Estimate probability of hidden ground truth T given a previous estimate of the expert quality parameters, and take expectation. M-step: Estimate expert performance parameters by comparing D to the current estimate of T. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 55

56 To Solve for Expert Parameters: k+ 1 k+ 1 k k p, q = argmaxe ln f( D, T p,q) D, p,q p,q k = arg max E ln f( D T, p,q) f( T) D, p,q p,q where k k p,q k k f ( T D,p,q ) = = i [ are the previous estimates of the quality parameters. f( D k k f( D T,p,q ) f( T) k k f( D T,p,q ) f( T) T k k ij Ti, pj, qj ) f( Ti ) i j k k [ T f( Dij Ti, pj, qj ) f( Ti )] i j k k f( Dij Ti, pj, qj ) f( Ti ) k k j i D,p i,q = k k T f( Dij Ti, pj, qj ) f( Ti ) i j Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 56 where i indexes over voxels and j over experts. For each voxel i f( T ) ] k

57 True Segmentation Estimate W f( T = 1 D,p, q ) k k k i i i α β = j f D T = p q f T = k k ( ij i 1, j, j ) ( i 1) k k T f( Dij Ti, pj, qj ) f( Ti ) i j α k = α k β k + k k k = f Ti = p jd : 1 j p = jd : = 0 j ( 1) (1 ) ij = f ( T = 0) q (1 q ) k k k i jd : = 0 j jd : = 1 j ij f ( T = 1) : prior probability true label at voxel i is 1. W k i i : conditional probability that true label is 1. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 57 ij ij

58 Expert Performance Estimate Now we seek an expression for the conditional expectation of the complete-data log likelihood function that we can maximize. k+ 1 k+ 1 k k p, q = argmaxe ln f(, ) f( ), D T p,q T D p,q p,q arg max ln (,, ) ln ( ) k = E f Dij Ti pj qj + f Ti Dp,,q p,q ij i k = arg max E ln f( Dij Ti, pj, qj ) Dp,,q p,q j i k k Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 58

59 Expert Performance Estimate Now, consider each expert separately: p q E f D T p q k+ 1 k+ 1 k k j, j = arg max ln ( ij i, j, j), pj, q Dp,q j i k = arg max Wi ln f( Dij Ti = 1, pj, qj) + p j, q j i k (1 Wi )ln f( Dij Ti = 0, pj, qj) = arg max ln + (1 ) ln(1 ) k k Wi pj Wi qj pj, qj id : = 1 id : = 1 ij + + Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 59 ij k k Wi ln(1 pj) (1 Wi )ln qj id : = 0 id : = 0 ij Differentiate this with respect to p,q and solve for zero. ij

60 Expert Performance Estimate p q k 1 : 1 i k + idij = j = k k W id : 1 i + W = id : = 0 i ij (1 W ) k 1 : 0 i k + idij = j = k k W id : 1 i + W = id : = 0 i ij W ij (1 ) (1 ) p (sensitivity, true positive fraction) : ratio of expert identified class 1 to total class 1 in the image. q (specificity, true negative fraction) : ratio of expert identified class 0 to total class 0 in the image. ij Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 60

61 Extension to Unordered Categories Complete data density: f ( D,T θ) True segmentation T i for each voxel i May be binary T i {0,1} May be categorical Ti {0,1,..., N 1} Expert j makes segmentation decisions D ij Expert performance θ s s characterizes probability of deciding label s when true label is s. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 61

62 Probability Estimate of True Labels k si W f( T = s D,θ ) = s i i k f( T = s) f( D T = s, θ ) i ij i j f( T = s ) f( D T = s, θ ) i ij i j k k Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 62

63 Expert Performance Estimate Now, consider each expert separately: θ k+ 1 k j = arg max E ln f( Dij Ti, j), j θ θ D θ j i k = arg max Wsi ln f( Dij Ti = s, θ j ) θ j i s k = Wsi f Dij = s Ti = s θ j θ j id : = s s s arg max ln (, ) ij Model for truth consisting of finite set of unordered categories (e.g. binary truth, labeling from segmentation): θ k+ 1 k j arg max Wsi lnθ jss θ j id : = s s s = ij Note constraint on sum of parameters. Solve for maximum. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 63

64 Parameter Estimation Noting that = 1 s θ js s We can formulate the constrained optimization problem: 0= ln + θ k Wsi θ js s λ θ js s jn n id : = s s s s lnθ = + k js s js s Wsi λ s i: D = s s θ jn n s θ jn n k 1 = Wni + λ θ id : ij = n ij ij jn n θ Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 64

65 Parameter Estimation Therefore θ k + js s And noting that We find that θ id : 1 ij = s k + js s = = s λ W k si k Wsi id : ij = s = id : 1 ij = s i λ W W k si k si s i: D = s Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 65 1 λ = = i W ij k si W k si

66 Results: Synthetic Experts Several experiments with known ground truth and known performance parameters. Goal: Determine if STAPLE accurately identifies known ground truth. Determine if STAPLE accurately determines known expert performance parameters. Understand sensitivity of STAPLE with respect to changes in prior hyper-parameters; requirements for number of observations to enable good estimation; convergence characteristics. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 66

67 Synthetic Experts 10 observations of segmentation by expert with p=q=0.99 STAPLE p,q estimates: mean p std. dev p mean q std. dev q Four segmentations of ten shown. STAPLE ground truth. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 67

68 Synthetic Experts 10 segmentations by experts with p=0.95, q=0.90 STAPLE p,q estimates: mean p std. dev p mean q std. dev q Four segmentations of ten shown. STAPLE ground truth. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 68

69 Expert and Student Segmentations Test image Expert consensus Student 1 Student 2 Student 3 Student 4 Student 5 Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 69

70 Phantom Segmentation Image Expert Students Expert + Students STAPLE Estimate Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 70

71 Phantom Segmentation STAPLE performance estimates for each segmenter: Expert Sensitivity Specificity Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 71

72 Clinical Applications Prostate peripheral zone. Cryoablation of kidney tumor. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 72

73 Prostate Peripheral Zone p j q j Dice Frequency of selection by experts. STAPLE truth estimate Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 73

74 Model Spatial Homogeneity Markov Random Field model of spatial homogeneity. Application to assessment of cryoablation of kidney tumor. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 74

75 A Binary MRF Model for Spatial Homogeneity. Include a prior probability for the neighborhood configuration: k k f ( T D,p,q ) = i = i T k k [ f( Dij Ti, pj, qj ) f( Ti ) f( T i )] i j k k [ T f( Dij Ti, pj, qj ) f( Ti ) f( T i )] i j where i indexes over voxels and j over experts. For each voxel k k f( T D,p,q ) = i where f ( T i k k f( D T,p,q ) f( T) k k f( D T,p,q ) f( T) i k k f ( D T, p, q ) f( T) f( T ) ij i j j i i j k k T f( Dij Ti, pj, qj ) f( Ti ) f( T i ) i j ) is the prior probability of the spatial configuration of the true segmentation of the neighbors of voxel i. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 75

76 MAP Estimation With MRF Prior f T f D T p q f T f T k k k k ( i D,p i,q ) ( ij i, j, j) ( i) ( i) j + k k k k log f ( Ti D,p i,q ) log( f ( Dij Ti, pj, qj) f ( Ti)) j k l β ( TT + (1 T )(1 T)) kl k l k l whereβ kl > 0 iff voxels kl, are neighbors. Greig et al :Solve for binary T i with Ford-Fulkerson 1 λiti+ βkl( TT k l+ (1 Tk)(1 Tl)) 2 k l where λ = log( f( D T = 1, p, q) / f( D T = 0, p, q)) i i i i i k k f( Ti = 1) p (1 ) : ij 1 j p jd = jd : ij = 0 j = log k k f( Ti = 0) q (1 jd : ij 0 j q ) = jd : ij = 1 j = log( W /(1 W)). i i Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 76 i

77 Synthetic Experts Only three segmentations by different quality experts. p=0.95,q=0.95 p=0.95,q=0.90 STAPLE p,q estimates: p1, q , p2, q , p3, q , p=0.90,q=0.90 With MRF prior STAPLE ground truth. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 77

78 Cryoablation of Kidney Tumor Segmentations before training session with radiologist: Rater frequency. STAPLE with MRF. After training session: Based on the STAPLE performance assessment, we found the training session created a statistically significant increase in performance of the raters. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 78

79 STAPLE Summary Key advantages of STAPLE: Estimates ``true segmentation. Assesses expert performance. Principled mechanism which enables: Comparison of different experts. Comparison of algorithm and experts. Extensions: Prior distribution or extended models for expert performance characteristics. Generalize to ordered categories. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 79

80 Segmentation of Knee Cartilage Grau-Colomer et al. IEEE TMI in press. Warfield et al. Med Imag Analysis Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 80

81 Cartilage of the Femur of the Knee 26 expert segmentations Knee MRI STAPLE truth estimate. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 81

82 Segmentation of Knee Cartilage Spatial context in an improved watershed algorithm: MARKER SELECTION Class prototypes Markers FILTERING PROBABILITY CALCULATION WATERSHEDS DETECTION ORIGINAL IMAGE SEGMENTED IMAGE Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 82

83 Segmentation of Knee Cartilage Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 83

84 Validation of Cartilage Segmentation 7 scans acquired with the same imaging parameters: Scans 1,2,3 from different subjects, Scans 4,5,6,7 all from the same subject with controlled motion of the knee joint: 4 Patient movement ½ voxel shift 5 6 ½ voxel shift 7 Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 84

85 Intra- and Inter-rater Assessment For scans 4-7: Segmented manually by two experts, 5 times each Segmented 10 times using our algorithm STAPLE Ground truth calculated for each expert Ground truth calculated for both experts combined Ground truth calculated for our algorithm Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 85

86 Intra- and Inter-rater Assessment Intra-observer Assessment Inter-observer Expert 1 Expert 2 Assessment Our algorithm Scan Sens Spec Dice Sens Spec Dice Sens Spec Dice Sens Spec Dice Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 86

87 Comparison: Raters, Watershed Standard watersheds Our algorithm Subject Sens. Spec. Dice Sens Spec. Dice scan scan scan scan Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 87

88 Consistency of Volume Estimation Coefficient of variation Expert 1 Expert 2 Experts Our method ½ voxel shift 5 Patient movement 6 ½ voxel shift 7 Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 88

89 Summary Image segmentation key to: quantitative monitoring and assessment in ablative therapy (liver, kidney). radiation dose planning and alignment of multi-modality data for improved targeting for prostate therapy. Knee cartilage assessment. Validation strategies: Digital and physical phantoms. STAPLE. Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 89

90 Acknowledgements Contributors to this research: Peter M. Black. Ferenc A. Jolesz. Ron Kikinis. Lawrence Panych. Kelly H. Zou. Steve Haker. Vicente Grau-Colomer. Martha Shenton. Clare Tempany. Carl Winalski. Michael Kaus. William M. Wells. Stuart G. Silverman. Paul Morrison. This research was supported by: The Whitaker Foundation Center for the Integration of Medicine and Innovative Technology NIH P41 RR13218, P01 CA67165, R33 CA99015, R21 CA89449, R01 NS Unifying Statistical Classification and Geometrical Models - MICCAI2003. Image Segmentation and Validation. Slide 90