8/2/2016. Acknowledgement. Common Clinical Questions. Presumption Images are Good Enough to accurately answer clinical questions

Transcription

1 Image Quality Assessment using Model Observers: Clinical Implementation and Practical Considerations Shuai Leng, PhD Associate Professor, Department of Radiology Mayo Clinic, Rochester MN Acknowledgement Lifeng Yu, PhD Chris Favazza, PhD Cynthia McCollough, PhD Baiyu Chen, PhD Chi Ma, PhD Yi Zhang, PhD NIH Funding Support R01 EB U01 EB Common Clinical Questions How much can IR reduce dose? What is the lowest dose for this patient? What is the optimal scanning protocol? Presumption Images are Good Enough to accurately answer clinical questions 1

2 Image Quality Assessment Standard Physical Metrics Contrast Noise CNR MTF NPS Limitations for assessing non-linear algorithms IR or non-linear denoising algorithms Contrast Dependent MTF Fessler and Rogers, TMI, 1996 Li et al, Med. Phys Chen et al, Med. Phys Yu et al, Med. Phys Limitation of CNR All three have similar noise: HU FBP 400 ma IR 200 ma IR 100 ma McCollough et al. Degradation of CT Low-Contrast Spatial Resolution Due to the Use of Iterative Reconstruction and Reduced Dose Levels Radiology (2015) Task-based Image Quality Assessment Human observer Labor intensive and costly Intra- and Inter- observer variability Model observer Composite image quality metric Relevant to the clinical imaging task Simulates human observer 2

3 Clinical Implementation of Model Observer Diagnostic Task Image generation: signal and background Phantom and ROI selections. Model observer type Test statistics and Figure of merit Compare to human observer performance * Eckstein et al, A practical Guide to Model observers for visual detection in synthetic and natural noise images. Handbook of Medical Imaging (Ch 10), 2000 * Barrett and Myers, Foundations of Image Science, 2004 * Vaishnav et al, Objective assessment of image quality and dose reduction in CT iterative reconstruction, Med. Phys * Verdun et al, Image quality in CT: From physical measurements to model observers, Phys. Med Diagnostic Task Detection Low contrast detection Low contrast Detection and Localization Classification Shape, Texture Estimation Quantification, e.g. lung nodule volume Yu et al. Med. Phys Leng et al. Med. Phys Richard et al, Med. Phys Zhang et al,. PMB, 2014 Chen et al,. SPIE 2014 Image Generation: Phantoms Physical phantoms are used in most studies Needed to recon on scanners to assess the non-linearity of commercial algorithms not available for simulations. Ma et al, JMI, 2016 Yu et al, Med. Phys Yu et al, RSNA 2014 Wilson et al. Med. Phys Popescu et al, Med. Phys

4 Knowledge of Noise 8/2/2016 Image Generation Scan with relevant protocols and dose levels Reconstructed with algorithms to be investigated Leng et al. Med. Phys Observer Model Types of Models Full Hotelling Channelized-Hotelling None Sum of Channel ROI Best Channel Channelized-SNR Channelized-NPW NPW NPW-HVS Knowledge of Signal Full * Eckstein et al, A practical Guide to Model observers for visual detection in synthetic and natural noise images. Handbook of Medical Imaging (Ch 10), 2000 NPWEi Non-prewhitening matched filter model observer with eye filter and internal noise (NPWEi)* *Burgess et al., JOSA A Opt Image Sci Vis 14, (1997). Multiple recent studies in CT Boedeker and McNitt-Gray, PMB, 2007 Richard et al, Med. Phys Gang et al, Med. Phys Richard et al, Med. Phys Chen et al, Med. Phys Li et al, Med. Phys Image quality NPS MTF: contrast dependent in IR Observer - Internal noise (N i ) - Eye filter (E) Object - Task function (W) 4

5 Channel Output Channel Template /2/2016 Channelized Hotelling Observer (CHO) A linear model observer with a scalar response t ω To mimic the existence of spatial frequency selective channels in human visual processing. Multiple recent studies in CT g N CHO c n 1 1 ω CHO Sc g sc n g cn g bc Barrett, HH, J Yao, JP Rolland and KJ Myers (1993). Proc Natl Acad Sci U S A 90(21): Wunderlich and Noo, PMB, 2008 Yu et al, Med. Phys Leng et al, Med. Phys Popescu and Myers, Med. Phys Zhang et al, PMB, 2014 Ma et al, JMI, 2016 Model Development: Channel Type Band Pass: Gabor Laguerre-Gauss: Diff. of Gaussians Abby and Barrett, 1995 Eckstein et al, 1998 Test Statistics Calculation covariance matrix S c g s - g b Template ω CHO Sc g sc g bc Channel Index Multiply: Channel Index Test Statistics: 1 t ω CHO g c N n 1 n g cn 5

6 100 2AFC trials 8/2/2016 Localization Task Apply template at each possible location Final test statistics: highest value at all possible locations (1,1) (1,n) max (n,1) (n,n) 1 Whitaker et al, Opt. Exp. 2008; Gifford et al, TMI, 2005 Leng et al, Med. Phys Popescu and Myers, Med. Phys Figure of Merit Percent correct (Pc) For MAFC tasks Area under ROC curve Detectability index Figure of Merit Area under LROC curve True positive: ROC: Correct detection LROC: Correct detection & Correct localization FROC, EFROC Popescu, Med. Phys Wunderlich et al, IEEE TMI, 2012 Leng et al, Med. Phys

7 Validation with Human Observer Model observer usually outperforms human observer Internal noise added to final test variable. final + a * s( bg ) * rand(-1,1) Internal noise added to channel responses. N n=1 w n (g cn + n ) Zhang et al, Evaluation of internal noise methods for Hotelling observer models, Med. Phys Validation with Human Observer Human Observer vs Model Observer A LROC CTDI vol (mgy) Pearson Correlation Coefficients: for 3mm and 5 mm lesions Leng et al, RSNA Validation with Human Observer Bland-Altman plot comparing human and model observer performance Mean of percent correct * Yu et al, Med. Phys

8 Practical Considerations Fourier domain vs spatial domain Number of images 2D/3D observers Homogeneous and heterogeneous background Fourier Domain vs Spatial Domain Fourier-domain implementations (e.g. NPWEi) Fewer scans required Assumption required: locally linear and shift-invariant Spatial-domain implementations (e.g. CHO) No linear-shift-invariant assumption Larger number of images required to get a good estimation of image statistics Studies comparing Fourier-domain and spatial domain implementations. Solomon et al, SPIE 2015 Chen et al, SPIE 2016 Fourier (NPWEi) vs spatial (CHO) 8

9 Fourier (NPWEi) vs spatial (CHO) * Chen et al, SPIE 2016 Practical Considerations Fourier domain vs spatial domain Number of images/scans Single-slice vs Multi-slice observers Homogeneous and heterogeneous background Number of Samples Theoretical work on the estimation of bias and variance. The goodness of correlations depends on the number of repeated scans used in the training and testing. Clinical CT Labor and machine usage limitations. Important to quantify the statistics associated with limited but reasonable number of repeated scans. Conceptual plot from Barrett and Myers, P 972; Resubstitution method Gallas et al. SPIE Barrett and Myers 2004, Wunderlich et al. TMI 2009, 9

10 Area under ROC curve (Az) vs Scan # Az calculated using subsets of 10 to 100 scans. Ma et al,. J Med Imaging Dependence on Number of Channels Ma et al,. J Med Imaging Dependence on Object Size and Contrast Minimum number of repeated scans increased when the radiation dose level decreased, object size and contrast level decreased (more challenging tasks) Ma et al,. J Med Imaging

11 Average Pixel Correlation Detectability Index Mean Pixel Correlation 8/2/2016 Repeat scan reduction: Increase images/scan N = number of scans 1 Slice: N = Slices: N = 50 3 Slices: N = 33 : : n Slices: N = 100/n Favazza et al, AAPM, 2015 Repeat scan reduction: Increase images/scan Slice Separation (mm) Favazza et al, AAPM, 2015 Repeat scan reduction: Increase images/scan single slice value stand. dev. Detectability index: 2 slice locations 100 training images Separation of 2 nd slice image data (mm) Favazza et al, AAPM,

12 Signal-present Signal-absent 8/2/2016 Number of Scans Needed Example 6 mm cylindrical object (small) 10 HU contrast (low contrast) 60 mas (low dose) 20 channels Each Scan 6 Every 1cm (6 cm) 60 images 10 scans Practical Considerations Fourier domain vs spatial domain Number of images/scans Single-slice vs Multi-slice observers Homogeneous and heterogeneous background Multi-slice CHO Multi-slice CHO to incorporate multi-slice viewing model during clinical reading 2D static viewing Multi-slice viewing Ensemble average Yu et al, RSNA

13 Model Observers: MS_CHO Single-slice preprocessing Integrate decision variables from multiple slices g 1 g c1 =U T g 1 T λ 1 =ω CHO1 g c1 g 2 g c2 =U T g 2 T λ 2 =ω CHO2 g c2 λ z T λ = ω zho λ z where ω zho = K z 1 Δλ z g N g cn =U T g N T λ N =ω CHON g cn Multi-slice integration Single-slice pre-processing Plastisa et al, JOSA, 2011 Ba et al, SPIE 2015 Yu et al, RSNA 2015 Ba et al, JMI 2016 Correlation between Human and MS_CHO Lesion size Ba et al, SPIE 2015 Ba et al, JMI 2016 Yu et al, RSNA 2015 Practical Considerations Fourier domain vs spatial domain Number of images/scans Single-slice vs Multi-slice observers Homogeneous and heterogeneous background 13

14 Phantom Patient 8/2/2016 3D Printing Phantoms Solomon et al, SPIE, 2014 Leng et al, JMI, 2016 Michael. A. Miller and G. Hutchins, IEEE NSSC 2007, M26-28 Yoo, T., Hamilton, T. et al, IEEE ISBMI, 2011, D Printing Phantoms Phantom Patient Phantom Patient Leng et al, Construction of realistic phantoms from patient images and a commercial three-dimensional printer. Journal of Medical Imaging 3(3), (2016) Lesion Insertion Image based method 1 Projection based method 2 Patient projections Previously segmented lesion Forwardproject Desired insertion location Patient projections w/ inserted lesions Lesion projections Reconstruct on a CT scanner CT images 1. Solomen and Samei. PMB ;59(21): Chen et al.. Med Phys. 42, 7034 (2015). 14

15 Projection-domain lesion insertion Chen et al., Acad. Radiol Ma et al., SPIE Sample Applications Samei and Richard, Assessment of the dose reduction potential of a model-based iterative reconstruction algorithm using a task-based performance metrology, Med. Phys Sample Applications Li et al, Statistical model based iterative reconstruction in clinical CT systems. Part III. Task-based kv/mas optimization for radiation dose reduction. Med. Phys

16 Sample Applications Gang et al, Task-driven image acquisition and reconstruction in cone-beam CT. PMB, 2015 Standardize Protocols in Large Practice Standardize Protocols in Large Practice Stage 1: FBP Standardize Protocols with physical IQ metrics Stage 2: IR Standardize Protocols with task-based image quality metrics Default setting resulted in 60-75% dose reduction How does IR perform? What is the appropriate amount of dose reduction? Favazza et al, A cross-platform survey of CT image quality and dose from routine abdomen protocols and a method to systematically standardize image quality, PMB,

17 Image noise vs Tube Current Noise(25%-33% dose) Noise(100% dose) C Favazza et al, Low-Contrast Detectability Vs. Dose for CT Images Reconstructed Using FBP and IR: Assessment with a Model Observer. SU-G (Sunday, July 31, 2016) AUC vs Tube Current Object 4: 25 HU Contrast 4.8 mm diameter IR STD 25% dose FBP 100% dose -Δ18% AUC 50% dose 78% dose 100% dose 25% dose 33% dose 18% AUC Reduction AUC vs Tube Current Object 4: 25 HU Contrast 4.8 mm diameter IR STD 78% dose FBP 100% dose AUC 78% dose 100% dose 17

18 Summary Multiple studies showed correlation and validation with human observers Methods have been investigated to make model observer studies more practical Various applications have used model observers for taskbased image quality assessment, especially in iterative reconstruction Studies with textured background, 3D (peudo-3d) observers are actively investigated 18