Medical Image Processing: Image Reconstruction and 3D Renderings 김보형 서울대학교컴퓨터공학부 Computer Graphics and Image Processing Lab. 2011. 3. 23 1
Computer Graphics & Image Processing Computer Graphics : Create, store and manipulate models and images using computers Image Processing : any form of signal processing for which the input is an image, the output of image processing can be either an image or a set of characteristics or parameters related to the image. 2
Medical Applications Multi-modality images CT MR PET CONFOCAL TEM 3
4 Capability of Radiologists
Evolution of Spiral CT Scanners TOSHIBA Philips Siemens Siemens GE TOSHIBA Brilliance ict Dual Source Dual Source LightSpeed 64 750HD Row thickness Detector rows Spatial resolution 0.5 mm 0.625 mm 0.6 mm 0.6 mm 0.625 mm 0.5 mm 64 128 32 64 64 320 0.35 mm 0.34 mm 0.4 mm 0.4 mm 0.24 mm 0.35 mm Detector Coverage 32 mm 80 mm 19.2 mm 38.4 mm 44 mm 160 mm Rotation Speed 350 ms 270 ms 330 ms 280 ms 350 ms 350 ms 5
6
Need of 3D Imaging Limitations of diagnosis by radiologist according to a large of amount of images Additional cost due to filming of a large of amount of images Data overload due to multi-slice CT/MR Need 3D imaging 7
8
General CT System X-ray Tube Scan (Data acquisition) Console Monitor Detector Reconstructor Data Acquisition System 9
3D Reconstruction Sampling point During the rotation 3D Volume Data reconstruction Re Filtering & Back projection Projected images 10
Back-projection Every voxel in the object is calculated using geometric information. r is a position vector in object space P Φ is a position vector in projection space r is back-projected from the value at P Φ of projected image Projection space ry Y( r) D, Z( r) d ry P ( r) P ( Y( r), Z( r)) rz d r z D z P Φ Y r(x,y,z) Source to object : d O x Z Beam Source y Object space Source to detector : D 11
Back-projection Feldkamp, Davis, and Kress Practical cone-beam algorithm 12
GPU based 3D Reconstruction Projection image : 512 512, 360 slices Volume Size : 512 512 512 Test machine CPU : Intel Core2Quad Q6600 GPU : NVIDIA GTX 280 1GB CPU Backprojection (BP) 352 unit : second GPU Upload projections BP on GPU Download result 0.3 3.5 3 6.8 13
How we see 3D Volume Data? DICOM PACS Server CT Non-DICOM DICOM Gateway (Converter) MR 3D Visualization & CAD 2D MPR VR MIP/MinIP SSD Endoscopy Colonoscopy BSA Fusion CAD 14
How we get 3D images from a stack of many slices? MPR VR MIP MinIP 15
Thin- vs. Thick-section Scans Scans with a large section thickness (e.g., 5mm) Excellent low-contrast detectability For primary interpretation Scans with a small section thickness (e.g., 0.67, 1mm) Excellent through-plane spatial resolution But, grainy noise For Multiplanar refomations (MPR) or 3D renderings (VR, MIP/MinIP, SSD) routinely reconstruct images with two or more different STs from a single multi-helical projection data 16
Thin- vs. Thick-section Scans Thick-section scan image Thin-section scan image 17
Multiplanar Reformation (MPR) Reconstruction planes : Axial (Transverse), Coronal, Sagittal Interactive viewing : Freehand drawing Axial Coronal Sagittal 18
Multiplanar Reformation (MPR) Window level: image brightness Window width: image contrast Lung window Window level: -600 HU Window width: 1600 HU Window level: 40 HU Window width: 400 HU Bone window Window level: 300 HU Window width: 2000 HU 400-1024 40 3071 19
Multiplanar Reformation (MPR) Interactive viewing : Curve, Freehand drawings 20
Multiplanar Reformation (MPR) MPR images imposed on volume rendering (+) Real-time & entire information without loss of data (-) 2D image & lack of overall feature A: Anterior P: Posterior R: Right L: Left H: Head F: Foot 21
Maximum Intensity Projection (MIP) How do we see only organs with high density values such as bones, contrasted vessels, and calcification? Vessels, Calcifications Bones 22
Maximum Intensity Projection (MIP) Maximum attenuation value is mapped to a 2D gray scale image Entry point Exit point 23
Minimum Intensity Projection (MinIP) How do we see only organs with low density values such as lung and airway? Line of Sight Minimum attenuation value is mapped to a 2D gray scale image 24
Ray Sum Technique Entry point Adds the pixel values along the line of sight Resultant display is similar to an X-Ray image Excellent for displaying overlapping vessels Exit point MIP Ray Sum 25
MIP / MinIP / Ray Sum Pros Good for representing organs with high/low/average intensity values Cons 2D image Lack of depth information Overlap of objects with similar intensities Contrasted vessels, parenchyma, bones Low performance Need to process the whole data sets 26
Volume Rendering Data Set Volume Data Image A method of displaying volumetric data as a two-dimensional image Render volume without extracting any surfaces (Direct Volume Rendering, DVR) Surface Shaded Display, SSD 27
Theoretical Background on Volume Rendering Refer to Chapter 1 of the book Real-Time Volume Graphics by K. Engel et al. 28
29 Theoretical Background on Volume Rendering Opacity Correction Compositing schemes assumes an equidistant sampling When the sampling rate needs to be changed The discretized opacity and color need to be modified accordingly. t k dt t k e e T t s i i s ) ( t k e T ~ ~ t t T T ~ ~ t t ~ ) (1 1 ~ A segment length of t A segment length of t ~ t q t t q C ) ( t q C ~ ~ A segment length of t A segment length of t ~ t t C C ~ ~
30 Volume Rendering Parameters
Volume Rendering Pros High quality Natural representation of CT/MRI images. Good detectability of subtle lesions Minimal editing Cons Huge data sets Computationally expensive Operator dependent Cannot be embedded easily into a polygonal scene 31
GPU based Volume Rendering Texture-mapping Ray Casting 32
Perspective Volume Rendering Perspective vs. Non-perspective Non-perspective : parallel ray Perspective : divergent ray Perspective viewing Inside (Virtual Endoscopy) Applied fields Angioscopy Bronchoscopy Gastroscopy Colonoscopy Cisternoscopy Non-perspective Perspective 33
GPU based Perspective VR Texture-based VR with perspective view Set the viewing matrix for perspective view Proxy geometry is changed to hemisphere mesh for uniform sampling interval eye proxy geometry mesh proxy geometry perspective texture-based VR 34
35 Virtual Angioscopy
36 Virtual Gastroscopy
Virtual Colonoscopy Virtual colonoscopy A non-invasive computerized medical procedure for examining the entire colon to detect polyps. More comfortable than conventional colonoscopy because it does not use a colonoscope. Low risks, low cost, unlimited number of viewing angles Convenient to localize the three dimensional position of polyps 37
The Future of Medical Imaging More advanced visualization techniques Large volume data processing Real time rendering 4D rendering Computer Assisted Diagnosis (CAD) Mammography CAD Colon CAD Lung CAD Image processing Volume measure Automatic segmentation Fusion Real time 3D reconstruction 38
39 Thanks!