Correlation between Model and Human Observer Performance on a Lesion Shape Discrimination Task in CT

Transcription

1 Correlation between Model and Human Observer Performance on a Lesion Shape Discrimination Task in CT Yi Zhang, Shuai Leng, Lifeng Yu and Cynthia McCollough Department of Radiology Mayo Clinic, Rochester MN 2012 MFMER slide-1

2 Background The golden-standard assessment for medical image quality relies on costly and laborious human observer studies. Model observers have been successfully used to predict human-observer performance in different imaging modalities. A channelized Hotelling observer (CHO) has been previously validated in a lesion detection task with location-known exactly (1) and a location-uncertainly detection/localization task (2) in CT. Morphologic characteristics of lesions (e.g. the boundary shape) often play an important role in accurate diagnosis. (1) Yu, L., S. Leng, et al. (2013). Medical physics 40: (2) S Leng, L. Y., L Chen, JCR Giraldo, CH McCollough (2012). SPIE proceeding 2012 MFMER slide-2

3 Purpose To investigate how well a channelized Hotelling observer can predict the human observer performance A lesion shape discrimination task Realistic CT scans For both filtered backprojection (FBP) and iterative reconstruction (IR) 2012 MFMER slide-3

4 Methods Phantom cm stadium water phantom 8 cylindrical rods mimic lesions 2 sizes: 7/16, 1/2 2 materials: acrylic, Lexan 2 shapes: circle, hexagon, the same cross-sectional area. CT scan 128-slice Definition Flash (Siemens Healthcare) 120 kvp, mm, 0.5 s rotation time, 0.75 pitch CareDose4D, quality reference mas: 120, 180, 240, 360 and repetitions Reconstruction kernels: B40 (FBP), I40 (SAFIRE) 5 mm slice thickness, 100 mm zoom-in FOV ( matrix size) 2012 MFMER slide-4

5 2-Alternative Forced Choice (2AFC) Study Hexagon, 100 images Circle, 100 images 100 2AFC realizations Total 4000 trials 5 doses 2 sizes 2 contrasts 2 reconstruction algorithms 2012 MFMER slide-5

6 Human Observer Study Study design 3 trained readers read all 4000 trials Lesion characteristics (size, shape and contrast ) were known Calibrated monitor, darkened reading room Standard abdomen display window (40, 400 HU) An interface (Matlab) to assist readers Percent correct (PC) was calculated 2012 MFMER slide-6

7 Model Observer Channelized Hotelling observer Linear model De-correlation the noise prior to the matched template (1) Gabor channel Describes the spatial and spatial frequency selectivity of simple cortical cells (2) 60 channels (6 passbands 5 orientations 2 phases) Center frequencies: 3/128, 3/64, 3/32, 3/16, 3/8 and 3/4 cycles/pixel Bandwidth: 1 octave Orientations: 0, /5, 2 /5, 3 /5, and 4 /5 radians Phases: 0 and (1) Barrett, H. H., J. Yao, et al. (1993). Proc Natl Acad Sci U S A 90(21): (2) Marčelja, S. (1980). J. Opt. Soc. Am. 70(11): MFMER slide-7

8 CHO Template artificially enhanced boundary Signal (hexagon) Background (circle) Template correct boundary 2012 MFMER slide-8

9 Edge Emphasis Rationale Human observers paid more attention to the edge of the rod when performing shape discrimination task Original Channel input Signal Binary mask = Background = 2012 MFMER slide-9

10 Procedure Edge emphasized test image 1 60 Gabor channels... Channel output Intra class scatter matrix CHO template Test variable λ 1 Edge emphasized template 2012 MFMER slide-10

11 Procedure 60 Gabor channels Edge emphasized test image 2... Channel output Intra class scatter matrix CHO template Test variable λ 2 Edge emphasized template 2012 MFMER slide-11

12 Internal Noise Image 1 Image 2 Test variable λ 1 Test variable λ 2 Decision maker if λ 1 > λ 2, then image 1 is hexagon; if λ 1 < λ 2, then image 2 is hexagon Noise only image Internal noise proportional to the stanardard deviation of 2012 MFMER slide-12

13 Results 2012 MFMER slide-13

14 Edge Emphasis PC of CHO performance without edge emphasis Percent correct Human observer, FBP Human observer, IR Model observer w/o edge emphasis, FBP Model observer w/o edge emphasis, IR Quality reference mass (mas) 7/16, 90HU-contrast lesion, no internal noise was added. PC of human observer performance were shown as the reference MFMER slide-14

15 Edge Emphasis PC of CHO performance with edge emphasis Percent correct Human observer, FBP Human observer, IR Model observer w/ edge emphasis, FBP Model observer w/ edge emphasis, IR Quality reference mass (mas) Edge emphasis improved CHO performance which can be degraded subsequently to match human observer performance MFMER slide-15

16 Human and CHO Performance Spearman s rank correlation coefficient were 0.88 for FBP and 0.92 for IR (p<0.01) MFMER slide-16

17 Performance Agreement (Bland-Altman Plot) Mean absolute PC difference between human and model observers was 0.5±3.6% for FBP (95% limits of agreement: -6.6% to 7.7%), and 0.2±3.1% for IR (-5.8% to 6.2%) MFMER slide-17

18 FBP v.s. IR Mean PC differences between FBP and IR was 3.0% for human observers, and was 2.6% for CHO (Wilcoxon signed rank test, p<0.01) MFMER slide-18

19 Discussions and Conclusions Employing the edge emphasis mask at the input end of CHO better represents the human s psychophysical process involved in the shape discrimination task. CHO highly predicted the human observer performance for the lesion shape discrimination task for both FBP and IR. CHO model has the potential to be used to evaluate IR performance. The SAFIRE IR algorithm significantly improved observer performance compared to FBP for lesion shape discrimination task. The improved performance of IR compared to FBP depends on lesion sizes, contrasts and doses MFMER slide-19

20 Acknowledgements Matthew Kupinski, Ph.D. University of Arizona 2012 MFMER slide-20

21 Thank you! CT Clinical Innovation Center MFMER slide-21