Disclosures 7/21/2014. AAPM 2014 Challenges and opportunities in assessment of image quality

Transcription

1 7/2/24 AAPM 24 Challenges and opportunities in assessment of image quality Ehsan Samei Department of Radiology Duke University Medical Center Disclosures Research grant: NIH R EB838 Research grant: General Electric Research grant: Siemens Medical Research grant: Carestream Health

2 7/2/24 Credits Jeff Siewerdsen, Johns Hopkins Grace Gang, Johns Hopkins Guanghong Chen, UW Ke Li, UW Duke CIPG Clinical Imaging Physics Group 4 Imaging quality and safety Quality: Imaging provides a clinical benefit Diagnostic information Image quality: Universally-appreciated Enhancement dependent on illusive criteria Safety: Imaging involves a level of cost Monitory cost Information excess Radiation cost Optimization involves 4 steps:. Reasonable measures of risk 2. Reasonable measures of image quality 3. Justified balance between the two by targeted adjustment of system parameters 4. Consistent implementation We cannot optimize imaging without quantification 2

3 7/2/24 Optimization framework Benefit regime Quality indices (d, A z, ) Different patients and indications Optimization regime Scan factors (mas, kvp, pitch, recon, kernel, ) Risk regime Safety indices (organ dose, eff. dose, ) Outline. Quality Characterization 2. Quality assessment 3. Quality implementation 3

4 7/2/24 Model-based recons +s Decades in the making Enabled by high-powered computers Significant potential for dose reduction Potential for improved image quality Model-based recons -s Speed Increased vendor-dependence Unconventional image appearance Limited utility of prior quality metrics Need for nuanced implementation for effective improvement in patient care ASiR (GE) Veo (GE) IRIS (Siemens) SAFIRE (Siemens) AIDR (Toshiba) idose (Philips) Iterative Reconstruction Reconstruction Noise texture? IR Images courtesy of Dr de Mey and Dr Nieboer, UZ Brussel, Belgium Low Dose 4 DLP,.9 msv 4

5 7/2/24 Reconstruction Iterative Reconstruction courtesy of University of Erlangen, Germany Reconstruction Iterative Reconstruction courtesy of Mayo Clinic Rochester, MN Reconstruction Iterative Reconstruction Low Dose 338 DLP,.8 msv Images courtesy of Dr de Mey and Dr Nieboer, UZ Brussel, Belgium 5

6 7/2/24 Qualimetrics.: CNR st order approximation of image quality Related to detectability for constant resolution and noise texture (Rose, 948) Task-generic CNR = Q L -Q B s B Parameters that affect IQ. Contrast 2. Lesion size 3. Lesion shape 4. Lesion edge profile 5. Resolution 6. Viewing distance 7. Display 8. Noise magnitude 9. Noise texture. Operator noise Feature of interest Image details Distractors Parameters that are measured by CNR. Contrast 2. Lesion size 3. Lesion shape 4. Lesion edge profile 5. Resolution 6. Viewing distance 7. Display 8. Noise magnitude 9. Noise texture. Operator noise Feature of interest Image details Distractors. Contrast? 8. Noise magnitude 6

7 MTF NPS MTF NPS 7/2/24 Why CNR is not enough: Noise texture IR Solomon, AAPM 22 Resolution and noise, eg Comparable resolution Lower noise but different texture IRIS -SAFIRE x IRIS -SAFIRE Spatial frequency Spatial frequency Resolution and noise, eg 2 Higher resolution Lower noise but different texture x MBIR -ASIR MBIR -ASIR Spatial frequency Spatial frequency 7

8 NPS (HU 2 mm 2 ) PSF width (mm) Peak frequency (mm - ) Normalized NPS (A.U.) 7/2/24 Nonlinearity NPS vs dose NPS (Veo) 5 mas 2.5 mas 25 mas 5 mas mas 2 mas 3 mas Normalized NPS (Veo) 5 mas 2.5 mas 25 mas 5 mas mas 2 mas 3 mas f xy (mm - ) f xy (mm - ) Li, Tang, and Chen, Statistical Model Based Iterative Reconstruction (MBIR) in clinical CT systems: Experimental assessment of noise performance, Med. Phys. (24) NPS peak frequency vs dose.8.4 (fitting) Veo Veo (fitting).2. f peak (mas) mas Li, Tang, and Chen, Statistical Model Based Iterative Reconstruction (MBIR) in clinical CT systems: Experimental assessment of noise performance, Med. Phys. (24) Resolution s joint dependence on contrast and dose.8.6 Veo % % Contrast (HU) 75% Dose % 8

9 Iodine concentration 7/2/24 Detectability index Resolution and contrast transfer Attributes of image feature of interest ' ( d NPWE ) 2 = òò Image noise magnitude and texture [ òò MTF 2 (u,v)w 2 Task (u,v) E 2 (u,v)dudv] 2 MTF 2 2 (u,v)w Task (u,v) NPS(u,v)E 4 (u,v) + MTF 2 2 (u,v)w Task (u,v)n i dudv Richard, and E. Samei, Quantitative breast tomosynthesis: from detectability to estimability. Med Phys, 37(2), (2). Chen et al., Relevance of MTF and NPS in quantitative CT: towards developing a predictable model of quantitative... SPIE22 Task-based quality index Fisher-Hotelling observer (FH) ' ( d FH ) 2 MTF 2 2 (u,v)w = Task (u,v) òò dudv NPS(u,v) Non-prewhitening observer (NPW) ' [ òò MTF ( d NPW ) 2 (u,v)w 2 2 Task (u,v) dudv] 2 = MTF 2 2 (u,v)w Task (u,v) NPS(u,v)dudv òò NPW observer with eye filter (NPWE) ' ( d NPWE ) 2 = òò [ òò MTF 2 (u,v)w 2 Task (u,v) E 2 (u,v)dudv] 2 MTF 2 2 (u,v)w Task (u,v) NPS(u,v)E 4 (u,v) + MTF 2 2 (u,v)w Task (u,v)n i dudv Task function: C (r ) = C peak ( - r 2 ç èr ø Task characteristics for detection and estimation Size æ ö ) n F Chen, SPIE 23 9

10 AUC (NPWE) AUC (NPWE) 2% 29% 7/2/24 GE MBIR Dose Reduction Potential Large feature detection task.95 MBIR ACR Acrylic insert % Eff dose (msv).2 msv 3.7 msv Richard, Li, Samei, SPIE 2 GE MBIR Dose Reduction Potential Small feature detection task.95 MBIR ACR.5 lp/mm bar pattern % Eff dose (msv).2 msv 3.7 msv Richard, Li, Samei, SPIE 2 Christianson, AAPM 22 Duke-UMD-NIST study Parameter GE Discovery CT 75 HD Siemens Flash Philips ict kvp Rotation time SFOV DFOV Recon Algorithm Standard / ASIR 5 B3f / I3f Safire 3 B / idose5 Recon Mode Helical Helical Helical Collimation Pitch Slice Thickness.25 Dose levels: 2, 2, 7.2, 4.3,.6,.9 mgy For anonymity will be referred to as vendor A, B, and C

11 7/2/24 Observer study design Observer study design 3 scanner models 3 dose levels 2 reconstruction algorithms slices x 5 repeated exams Total of 9 images 2 expert observers from two institutions How much can IR reduce dose? Default clinical protocol at 2 mgy 23% dose reduction

12 7/2/24 How much can IR reduce dose? Default clinical protocol at 2 mgy 6% dose reduction How much can IR reduce dose? Default clinical protocol at 2 mgy 33% dose reduction CNR vs observer performance 2

13 HU 7/2/24 d vs observer performance Our contrast detail phantom 64.5 mm 25% mm 2 mm 33 mm 3 mm 2 mm 3 mm 5 mm 8% 4 mm TangoPlus 9 36% Tango Black + DM 853 DM mm 36% 25% 8% 8% 3% 65 mm DM 852 DM 855 DM 85 DM 855 Vero White DM 843 DM 8425 DM 9795 DM 9785 DM 977 DM 976 DM975 DM 974 Tango + Vero Black Vero Blue Vero Grey Vero Clear Tango Grey RGD 56 FC 72 Durus White CT images of the low-contrast phantom B3s I3s-5 3

14 IR Strength NPS NPS (mm 2 HU 2 ) TTF 7/2/24 Acquired CT data Siemens SOMATOM Force 4 doses (.7,.4, 2.9, 5.8 mgy) 2 slice thicknesses (.6, 5 mm) 4 Recons (, ADMIRE 3-5) 5 TTF and NPS across recons (b) Spatial Frequency (/mm) ADMIRE 4, STD = 2 HU ADMIRE 5, STD = 5 HU ADMIRE 3 ADMIRE 4.5 ADMIRE 5. (a) Spatial Frequency (/mm) (c) 5 ADMIRE 3 ADMIRE 4 ADMIRE 5 5 (b) Spatial Frequency (/mm) CT images across dose and recon Dose 4

15 VIG score.6 Observer Performance VIG score VIG score VIG score VIG score VIG score Observer Performance Observer Performance Observer Performance Observer Performance Observer Performance VIG score VIG score /2/24 Insert Counting Experiment Visible groups across: Dose (.74,.4, 2.9, 5.8 mgy) Slice Thickness (.6,.8, 5 mm) Recon (, ADMIRE 3-5) Total number of discernable objects ADMIRE 3 ADMIRE 4 ADMIRE 5 Slice Thickness: Slice.6 Thickness: mm.6 mm Slice Thickness: Slice.8 Thickness: mm.8 mm Slice Thickness: Slice 5 Thickness: mm CTDI vol (mgy) CTDI vol (mgy) CTDI vol (mgy) CTDI vol (mgy) CTDI vol (mgy) CTDI vol (mg.9 ADMIRE3 ADMIRE3 y=.938*x+., R 2 = IQ metrics vs human performance y=.6*x+.82, R 2 = CNR AUC CNR 2 d ' NPW AUC SNR 2 d 'NPWE.9 ADMIRE3 y=.938*x+., R 2 = ADMIRE3 y=.84*x+.494, y=.6*x+.82, RR 2 2 = = ADMIRE 3 y=2.9*x+.5, R 2 = AUC CNR AUC NPW SNR AUC NPWE.9 ADMIRE3 y=.84*x+.494, R 2 = ADMIRE 3 y=2.9*x+.5, R 2 =.77 5

16 7/2/24 Task-Based Detectability Extended to quadratic PL image reconstruction Line Pair Detection Task.6 Detectability Map d (x,y).4.2 G. Gang et al. Med Phys 4 (24) Justification of Assumptions: Local Stationarity and Linearity % difference between Fourier and spatial domain calculation of d Ellipse phantom Cylinde r Ellipse Thorax PL Similar levels of d computed via Fourier and spatial domain approach (~5-5% agreement) Applicable to various objects and imaging tasks. Local Fourier metrology applicable to quadratic PL reconstruction, gives a useful framework for system design and optimization. G. Gang et al. Med Phys 4 (24) Estimability index (e ): prediction of the quantification precision capturing the interaction between the imaging system, the segmentation software, and the lesion 2 e ' Noise TTF ( u, v, w ; C, N ) Wtask ( u, v, w ) dudvdw 2 [ NPS ( u, v, w ; N ) N ( u, v, w )] W ( u, v, w ) dudvdw i Resolution task Tumor characteristics 3D Noise power spectrum (NPS) Texture and magnitude of the noise Internal noise (N i) Inconsistency of the quantification software due to placement of random seeds 3D Task transfer function (TTF) System resolution with respect to the contrast of the object and image noise 3D Task function (W task) Size, shape, and contrast of the nodule being quantified 6

17 7/2/24 Empirical precision measurement LUNGMAN, Kyoto Kagaku, Japan LungVCAR GE Healthcare Predicted precision matches empirical precision! PRC Wj / Vtrue j 2.77 ˆ / V Wj true _ ( Vijk Vij ) Vtrue 2 n K i k 2.77 / nk ( ) Outline. Quality Characterization 2. Quality assessment 3. Quality implementation 7

18 7/2/24 Task-based assessment metrology Mercury Phantom 3. Diameters matching population cohorts Depths consistent with cone angles Straight-tapered design enabling evaluation of AEC response to discrete and continuous size transitions 55 Design: Base material Polyethylene 8 2 KVp Near patient equivalent Affordable Easy to machine 56 Design: Size Pediatric representation percentages MP 3. section Water size equivalent 2 mm 2 mm 85 mm 77 mm Abdomen Chest Head Age Percentile Age Percentile Age Percentile mm 22 mm mm 29 mm mm 355 mm

19 7/2/24 Design: Size Adult representation percentages MP 3. section Water size equivalent Abdomen Chest Head M F M F M F 2 mm 2 mm mm 77 mm mm 22 mm mm 29 mm mm 355 mm Design: Resolution, HU, noise Representation of abnormality-relevant HUs Sizes large enough for resolution sampling Maximum margin for individual assessment Iso-radius resolution properties Matching uniform section for noise assessment 59 Non-Stationary Noise and Resolution The Local NPS and MTF NPS MTF 2.5 x -6 Noise and resolution model for Penalized Likelihood (PL) model-based reconstruction.* Predictive framework for NPS, MTF, and detectability index (d ) enables task-based design and optimization of new systems using iterative reconstruction. *Fessler et al. IEEE-TIP (996) G. Gang et al. Med Phys 4 (24) 9

20 % Noise Reduction From IR 7/2/24 Design: Resolution, HU, noise Air - HU 44 Air - HU Water HU.5.2 Water HU Iodine 8.5 mgi/cc 225 HU Iodine 8.5 mgi/cc 225 HU Bone 95 HU 26. Polystyrene -4 HU 26 Bone 95 HU Polystyrene -4 HU Sections 2-5 Smallest Section imquest (image quality evaluation software) HU, Contrast, Noise, CNR, MTF, NPS, and d per patient size, ma modulation profile Noise texture effect 8% 7% 6% 5% 4% 3% 2% % % Water Water + Sponge Water + Acrylic mas IR noise reduction: 65% in uniform bkgd 2% in textured bkgd Solomon, SPIE 22 2

21 7/2/24 Mercury phantom 3. Added contrast detail and texture Textured lung phantom Generation Textured soft-tissue phantom: clustered lumpy background* *Bochud, F., Abbey, C., & Eckstein, M. (999). Statistical texture synthesis of mammographic images with super-blob lumpy backgrounds. Optics express, 4(), Retrieved from 2

22 7/2/24 Textured soft-tissue phantom: clustered lumpy background* *Bochud, F., Abbey, C., & Eckstein, M. (999). Statistical texture synthesis of mammographic images with super-blob lumpy backgrounds. Optics express, 4(), Retrieved from Anatomically informed heterogeneity phantoms 3 mm 3 mm 65 mm 65 mm 68 Lung texture phantom 3 mm 65 mm 69 22

23 IR 7/2/24 Soft-tissue texture phantom 3 mm 65 mm 7 How was the data analyzed? = + = STD Noise in the uniform phantom 2 3 STD (HU) 2 3 STD (HU) 3 STD (HU) 23

24 IR IR IR 7/2/24 Noise in the soft-tissue phantom 2 3 STD (HU) 2 3 STD (HU) 3 STD (HU) Noise in the lung phantom 2 3 STD (HU) 2 3 STD (HU) 3 STD (HU) Comparing noise across textures 2 3 STD (HU) 2 3 STD (HU) 2 3 STD (HU) 2 3 STD (HU) 2 3 STD (HU) 2 3 STD (HU) 24

25 NPS nnps (HU (mm 2 *mm 2 ) 2 ) 7/2/24 What about noise texture? Lung Texture NPS IR : Red Pixels IR: Red Pixels IR: Blue Pixels Spatial Frequency (cycles/mm) Outline. Quality Characterization 2. Quality assessment 3. Quality implementation Components of quality assurance System performance assessment Quality by inference Prospective protocol definition Quality by prescription Retrospective performance assessment Quality by outcome 25

26 7/2/24 kv IR optimization Feature with no iodine A Z A Z Detectability trends with dose/size ACR phantom Task Optimal technique % dose reduction (wrt 2 kvp ) No Iodine 8/ kvp with IRIS 36% With Iodine 8 kvp with IRIS 4% Feature with iodine Eff dose (msv) Eff dose (msv) Samei, Richard, Med Phys, in press, 24 Smitherman, AAPM, 24 8 Protocol optimization Setting dose to achieve a targeted task performance for a given size patient Smitherman, AAPM, 24 26

27 7/2/24 Protocol optimization Setting dose to achieve a targeted task performance for a given size patient Smitherman, AAPM, 24 Quality-dose dependency Quantitative volumetry via CT PRC: Relative difference between any two repeated quantifications of a nodule with 95% confidence Noise texture vs kernel GE Siemens Solomon, Samei, Med Phys, 22 27

28 GNI 7/2/24 Texture similarity Sharpness Solomon, Samei, Med Phys, 22 CT image quality monitoring GNI Christianson, AAPM, y = x R 2 = Subtraction Method Noise per slice ASiR 28

29 SSDE (mgy) GNL (HU) SSDE (mgy) GNL (HU) SSDE (mgy) GNL (HU) 7/2/24 Trends in dose and noise Scanner Scanner 2 Scanner 3 Abdomen Pelvis Exams (n=2358) 5 Scanner 45 Scanner 2 4 Scanner Effective Diameter (cm) Effective Diameter (cm) Christianson, AAPM, 24 Trends in dose and noise Scanner Scanner 2 Scanner 3 Abdomen Pelvis Exams (n=2358) 5 Scanner 45 Scanner 2 4 Scanner SSDE varies with size Effective Diameter (cm) Effective Diameter (cm) Christianson, AAPM, 24 Variability in dose and noise 3 25 Abdomen Pelvis Exams (n=2358) 3 ACR DIR 25 th - 75 th Target noise level Scanner Scanner 2 Scanner 3 Scanner Scanner 2 Scanner 3 Christianson, AAPM, 24 29

30 7/2/24 Conclusions New technologies necessitate an upgrade to performance metrology towards higher degrees of clinical relevance: Physical surrogates of clinical performance Taskful metrology and dependencies Incorporation of texture into image quality estimation Extension of quality metrology to quality monitoring and quality analytics Thank you! 3