SIGMI. ISL & CGV Joint Research Proposal ~Image Fusion~

Transcription

1 SIGMI ISL & CGV Joint Research Proposal ~Image Fusion~

2 Introduction Research Diagram What CGV Lab is interested in What ISL is interested in Research Plan

3 Research Diagram Medical Imaging and Application CGV 3D Organ Modeling 3D/4D registration ISL Model-based Simulation Model-based Quantification Interactive Medical Application Registration X-ray/CT Image Volume Rendering Image Reconstruction Multimodal Image Fusion MR/PET/US Image

4 Research Diagram Medical Imaging and Application CGV 3D Organ Modeling 3D/4D registration ISL Model-based Simulation Model-based Quantification Interactive Medical Application Registration X-ray/CT Image Volume Rendering Image Reconstruction Multimodal Image Fusion MR/PET/US Image

5 What CGV Lab is interested in (Statistical) Shape Modeling Model-based medical simulation Haptic simulation Surgical planning Model-based medical diagnosis Deformities/Volume Etc. Model-based medical visualization

6 Progressive Template Surface Deformation Organ Shape Modeling Template Initialization Multi-level Neighborhood Non-rigid model deformation Laplacian deformation framework Minimizing energy function E 1-ring neighbor 2-ring neighbor Local Shape detail restoration Rotation-and Scale Invariant (RSI) Transformation Non-rigid deformation w/o RSI transformation with RSI transformation J. Kim, M. C. Valdes-Hernandez, N. A. Royle, J. Park, LBC 1936, and ADNI, IEEE Transactions on Medical Imaging, 2014

7 Interactive Surgical Planning/Simulation Surgical Planning Tool for Broken Bones Haptic Dental Simulator K. Kim, J. Cho, J. Kim and J. Park, EuroHaptics, pp , 2012 K. Kim and J. Park, ACM Symposium on Virtual Reality Software and Technology, pp , 2009 T. Kim and J. Park, International Forum on Medical Imaging in Asia(IFMIA), 2012 T. Kim, J. Kim and J. Park, The 8th International Conference of ISSiS, pp. 90, 2012

8 3D Model-based Morphometric Analysis Medical image 3D Organ Modeling Model-based Measurement Talonavicular distance map SU WB SU WB Articular Surface on Navicular Articular Surface on Talus J. Kim, S. Seo, D. Lee, and J. Park, Journal of Foot and Ankle Research (i-fab Community), 7(Suppl 1):A42, 2014 D. Lee, T. Kim, S. Seo, E. Kim, W. Lee, K. Wapner, J. Park, the annual meeting of American Orthopedic Foot and Ankle Society (AOFAS), 2015 J. Kim, M. C. Valdes-Hernandez, N. A. Royle, J. Park, LBC 1936, and ADNI, IEEE Transactions on Medical Imaging, 2014

9 What ISL Lab is interested in Image registration Uni-modal registration X-ray CT registration MR MR (CT CT) registration Multi-modal registration US MR (CT) registration PET CT registration Image reconstruction Motion compensated reconstruction Cardiac X-ray CT Respiratory gated PET Super resolution on PET image Image-based Sinogram-based US image Registered MR (CT) image MR (CT) image Uncompensated PET Motion compensated PET Conventional Image-based SR result

10 Registration Feature-based registration Minimize distances between the corresponding features Internal liver vessels [1] Volume rendering of US vessels Volume rendering of MR vessels Volume rendering of matched vessels IVC & liver surface [2] Volume rendering of US IVC & diaphragm 3D US 3D MR Regidered MR Volume rendering of MR IVC & liver surface Volume rendering of matched features W. H. Nam, D. -G. Kang, D. Lee, J. H. Lee, and J. B. Ra, Physics in Medicine and Biology, vol. 57, pp , Jan C. Weon, W. H. Nam, D. Lee, J. Y. Lee, and J. B. Ra, Medical physics, vol. 42, no. 1, pp , Jan. 2015

11 Registration Image voxel-based registration Maximize an voxel information (e.g. entropy) based similarity measure. Normalized mutual information (NMI) Measure based on the joint entropy Y ( A, B) = H ( A) + H ( B) H ( A, B) H(A), H(B): Marginal entropy H(A,B): Joint entropy M(A,B): Mutual information Propose to use spatial information, edge orientation coincidence, in addition to joint entropy. E(A,B) = H(A B) + H(B A) + H(O A,B) = H(A,B,O) M(A,B) Proposed similarity measure F(A,B) = W(A,B) {H(A,B,O) - M(A,B)} W(A, B): Weighting function to handle the partial overlap problem Y. S. Kim, J. H. Lee, and J. B. Ra, Pattern Recognition, vol. 41, issue 11, pp , Nov B o( θ ) J. H. Lee, Y. S. Kim, D. Lee, D. -G. Kang, and J. B. Ra, IEEE Signal Processing Letters, vol. 17, no. 4, pp , Apr US b O MR a 3D joint histogram A

12 Applications Generation of 4D image 3-respiratory-phase CT data NRR End inspiration Neutral End expiration Non-rigid registration (NRR)

13 Applications US-MR (CT) registration Indirect lesion positioning system Viewing angle US z xx yy Registered MR yyz xx zz yyxx Registration θ 2 θ 1 3D US ROI 2D US θ1 (with sufficient features) Generated 4D MR (CT) (including lesion information) Registered 2D MR (CT) slice Lesion (visible or invisible) Lesion (visible) Estimate relative lesion position & the corresponding viewing angle difference Corresponding 2D US θ2 (with insufficient features) Determined MR slice including a lesion D. Lee, W. H. Nam, J. Y. Lee, and J. B. Ra, Physics in Medicine and Biology, vol. 56, no, 1, Jan C. Weon, W. H. Nam, D. Lee, J. Y. Lee, and J. B. Ra, Medical physics, vol. 42, no. 1, pp , Jan

14 Applications PET motion compensated reconstruction Uncompensated PET images PET w/o motion PET w/ motion Gated PET CT Motion compensated PET W. H. Nam, I. J. Ahn, K. M. Kim, B. I. Kim, and J. B. Ra, Physics in Medicine and Biology, vol. 57, no. 1, pp , Jan

15 Research directions C G V I S L Option #1 Model-based registration (Brain, bone, etc.) Application of the registration Option #2 Interactive Visualization Registration based image fusion (US, MR, PET) Option #3 Model-based deformities measure Feature & image voxel based registration measure Improved registration performance

16 THANK YOU