Clinical Prospects and Technological Challenges for Multimodality Imaging Applications in Radiotherapy Treatment Planning Issam El Naqa, PhD Assistant Professor Department of Radiation Oncology Washington University, School of Medicine, St. Louis, MO SWAAPM Austin, TX, Spring 2008
Why Multimodality Image Analysis? Motivation Increase usage of multimodality imaging (CT,PET,MRI,MRS,US) in diagnostic, image-guided intervention, daily localization Complementary effect Anatomical, physiological, soft tissue structures Improved target definition (Bradley et al 04, Rasch et al 05, Zangheri et al 05, Milker-Zabel et al 06) Objectives Integrate multiple information streams from different imaging technologies to automatically define biophysical target (normal structure) volume
Multimodality Image Integration Anatomical Imaging Functional Imaging US MRI CT PET/ CT PET SPECT MRS Biophysical Target Biophysical Target= f ( CT, PET, MRI,...)
H&N Example: CT/MRI/PET Milker-Zabel et al., IJROBP 06
Prostate Example: CT/MRI/3D-TRUS Smith et al., IJROBP 07
Clinical Application Challenges Increased acquisition time Efficiency and automated delineation Co-registration and fusion of different imaging data PET/CT, but how about other modalities?
Image Registration
Image registration Single modality deformable In PET/CT registration of transmission images instead of emission images Multimodality Rigid PET to CT using normalized mutual information (NMI) Deformable Multimodality Registration Feature based Volume Intensity based
Deformable Registration (Level set) 1 3 5 2 4 6 Yang et al., SPIE 07
Improved Optical Flow Deformable Registration Multigrid Multipass Yang et al., ICCR 07
Deformable Registration (Optical flow) Before registration After registration Yang et al., AAPM 07
Deformable Registration Tool
NMI Rigid Registration of Multimodality Images MAX(NMI) where NMI = H ( A) + H( B) H( A, B) and H ( ) is image entropy
Example of NMI Registration (MR/CT)
Surface matching and FEM (Finite Element Method) FEM-based multi-organ deformable image registration (Brock et al., IJROBP 05)
Intensity Remapping Define the intensity mapping function Finding function f through regression T(i) = f (s(i))+η(i) f(s) = a 0 +a 1 *s+a 2 *s 2 +a 3 *s 3 + +a n *s n Bi-functional dependence: allow to remap one intensity value to two intensity values in the second image
Multimodality optical flow For any image registration: J(h) measure the distance (difference) between the moving image and the fixed image. R(h) measure the variations of the motion field General solution: Similarity metrics Mutual information Cross-correlation Correlation ratio
Adaptive Radiotherapy Application: KVCT-MVCT Registration Yang, Chaudhari, Goddu, Khullar,. Deasy, El Naqa
MVCT KVCT Registered w/o correction Registered w/ correction
Validation of Deformable Registration using a biomechanical phantom Courtesy Deshan Yang (AAPM 07)
Image Segmentation
Coronary stenosis detection (Edge detection and linking) (El Naqa et al 96) Examples Microcalcification detection (Supervised Machine learning) (El Naqa et al. 02) MR cardiac classification (Unsupervised learning) (Zheng, El Naqa, 05) PET/CT NSCLC delineation (Active Contours)
Methods I: Clustering Zheng et al., MRI 05
II. Active Contour Deformable Models Definition: Geometric representations for curves or surfaces that are defined explicitly or implicitly in the imaging domain. These models move (deform) under the influence of internal forces, which are defined within the curve or surface itself, and external forces, which are computed from the image data Pros Boundary smoothness (continuity) Subpixel accuracy Prior information (atlas-based) Mathematically tractable (PDE) 2D curves are easily generalized to 3D surfaces Cons PDE! (Numerical instability)
Parametric models--cont Problems non-convex optimization problem in (2) sensitivity to contour initialization dependency on parameterization inability to account for topological adaptation
Geometric models Contour = cross section at L = 0 (i.e., {(x,y,z) Φ (x,y,z;t) = 0}) Evolution in the normal direction L=+1 L=0 L=-1 L=-1 L=+1 L=0
PET Segmentation Examples
Active Contour Segmentation (Synthetic Data) Gradient-based Region-based
Active Contour Segmentation (Clinical Data) Gradient-based Region-based El Naqa et al., ICCR 04
3D Active Contour Segmentation
Multimodality Image Analysis
Algorithm to Apply to Multimodality
Pre-processing: Motion-based Compensation in PET Motion Blur 12 Superior-Inferior (mm) 10 8 6 4 2 0 10 8 6 Anterior-Posterior (mm) 4 2 4 5 6 Lateral (mm) 7 8 Deconvolution-corrected El Naqa et al., Med Phys. 06
Method II: Active Contours GTV-CT GTV-PET GTV-PET/CT (a) GTV-CT GTV-PET GTV-PET/CT Initialization MVLS CT PET (b) (c) (d)
GTV-PET (40% SUVmax) Initialization MVLS CT PET (b) (c) (d)
(b) (a) (c)
(a) (b) (c) (d) (e)
El Naqa et al., AAPM 06
Phantom Validation of Multimodality Concurrent Segmentation I (a) Courtesy Sasa Mutic
Phantom Validation of Multimodality Concurrent Segmentation II CT PET MR
Phantom Validation of Multimodality Concurrent Segmentation III PET/CT/MR CT only PET/CT/MR CT only Overlap Index 1 0.8 0.6 0.4 0.2 0 1 2 3 4 % Error in volume estimate 8 6 4 2 0 1 2 3 4 Balls Balls El Naqa et al, ICIP 07
Multimodality Image Analysis Tool GUI Screen shot of the software tool. (1) image selector, (2) manual registration control, (3) window level control, (4) zoom control (5) 3D slice number control, (6) status information, (7) the working image panel, (8) ROI region contour, (9) not confirmed segmentation result, (10) right mouse click context menu, (11) menu, (12) the result display panel, zoomed in to ROI, (13) confirmed segmented regions, (14) separated 3D rendering window.