Overview Non-rigid Image Registration Introduction to image registration - he goal of image registration - Motivation for medical image registration - Classification of image registration - Nonrigid registration approaches My research method ransformation properties - Symmetry - ransitivity Validation of image registration algorithms Future work Department of Biomedical Engineering Georgia Institute of echnology Yi-Yu Chou March, 004 he Goal of Image Registration Motivation for Medical Image Registration Image registration consists in finding a transformation that realigns two or several images (in D) or image volumes (in 3D). o fuse information from multiple imaging devices to correlate different measures of structures and function Beonit M. Dawant, Non-Rigid Registration of Medical Images: Purpose and Methods, A Short Survey, IEEE ISBI, pages 46 468, 00. he goal of image registration is to determine a common coordinate system in which images can be compared or fused on a pixel-by-pixel basis. PE image with MRI Motivation for Medical Image Registration o measure dynamic patterns of structure change during brain development, tumor growth, degenerative disease processes or pre- and post intervention images. Normal brain image / Alzheimer s brain image Pre- and post-surgery of brain MRI Head and neck MRI-C image Motivation for Medical Image Registration Passing segmentation or labeling information from the atlas to subject image Brain atlas and MRI
Classification of Image Registration Rigid ransformation Geometric transformations:. Rigid. Affine 3. Projective 4. Non-rigid Original Rigid Affine Projective Nonrigid Rotation(R) ranslation(t) x x p r p r t = = t r = y y t r r = Rp + t r p cos( θ ) R = sin( θ ) sin( θ ) cos( θ ) Preservation of length and angle Affine ransformation Projective ransformation Rotation ranslation Scale Shear x a = y a a x a + a y a 3 3 No more preservation of lengths and angles Parallel lines are preserved Straight lines are preserved Nonrigid ransformation Gemeral Registration Approaches Elastic transformation Nonlinear Curved Can not generally be expressed in a matrix notation Involve a large number of parameters Can map straight lines into curves Feature based (points, edges, surfaces) Intensity based (work directly with image intensity value) Hybrid
Nonrigid Registration Approaches Basis function expansions ( Fourier basis, wavelet basis, radial basis, B-splines, etc) Physical models Optical flow-based methods My Research Method Non-rigid : Rigid or affine registration do not have enough degrees of freedom or flexibility to accommodate local shape differences. Intensity based : Feature based registration require extra work (segmentation, feature extraction, etc ), and requires user interaction to specify landmarks unique correspondences can not always be specified, and such methods usually only provide coarse registration due to the small number of correspondences specified. Spline warps. Cubic B-Spline. hin-plate Spline Registration Algorithm Registration Algorithm I Initial ransform Update Nonrigid image registration is an optimization problem, where the goal is to optimize an image similarity measure with respect to the transformation parameters. I I* Measure Similarity Optimal? he most widely used image similarity measures are mean square difference and normalized mutual information. Final Optimization Methods Registration Procedures Gradient Descent : ISM Γ( I, I ) = arg max[ ISM ( I, I opt opt )] In theory, similarity measures have more local optimum as the dimension of the transformation increases. ransformation Multi-resolution search : Coarse to fine strategy
Registration Procedures Registration Procedures D Experimental Results Case - weighted coronal brain image D Experimental Results Case - hin-plate Spline + Mean Square Difference Original image Deformed image x size = 6 mm Computed image Displacement Field y size = 86 mm maximal displacement :.6 mm Max Error :.9 mm D Experimental Results Case - Short axis MR cardiac image D Experimental Results Case - Cubic B-Spline + Normalized Mutual Information Original image x size = 386 pixel Deformed image Computed image Displacement Field y size = 33 pixel Max Error :.4 pixel maximal displacement : 3.3 pixel
3D Experimental Results Cubic B-Spline + Mean Square Difference 3D Experimental Results Cubic B-Spline + Mean Square difference Original image Image Dimensions: (6, 6, 0) Voxel Size: (.38,.38, ) mm maximal displacement : 9.93 mm Deformed image rue vector field maximal error :.09 mm Computed vector field Symmetry ransitivity ransformation Properties Most image registration algorithms do not produce transformations with these properties. Satisfying the symmetry and transitivity properties are necessary but not sufficient conditions for establishing whether or not a registration algorithm produces biologically meaningful transformations. Symmetry When an image registration operator is applied to two (different) images, the obtained transformation should be the inverse of the transformation obtained, when the order of images is reversed. his symmetry property can be formalized as : Γ ( I, I ) = [ Γ ( I, I )] Symmetry Many nonrigid image registration algorithms have difficulty producing symmetry property because numerical optimization techniques used to find the optimal image transformation often get struck in local minima. Symmetry In 999 Christensen [] proposed the consistent linear-elastic image registration algorithm that minimizes the pairwise inverse consistency error between pairwise transformation.. Jointly estimate the forward and reverse transformations.. Constrain the forward and reverse transformations to be inversed. [] Gary E. Christensen, "Consistent Linear-Elastic ransformations for Image Matching", IPMI 999: 4-3
Symmetry Symmetry est for Affine Registration In this research, we proposed : Γ( I, I ) = arg min[ ISM ( I, I opt opt ) + ISM ( I, I opt )] a b c d e f g (a) Original image (b) Deformed image (c) Computed image using affine registration (d) Computed image using affine registration with symmetry property (e) Differences image of a and b (f) Difference image of b and c (g) Difference image of b and d ransitivity Image registration algorithms that have a difficult time producing symmetry property have an even harder time producing transformation that satisfy the transitivity property. ransformations with the transitivity property allow a corresponding point to be mapped from A to B to C to A z= AC (x) y= AB (x) x= BA (y) y= CB (z) A B C BC [ AB (x)] = AC (x) A B C - - - ransitivity M AB BC AC = = = ransitivity D ransitivity est y= AB (x) z= AC (x) x= BA (y) y= CB (z) A B C BC = [ = = AC AB ] o[ ] a b c d e f hree randomly selected images from a sequence of D short axis cardiac MR images are shown in (a), (b), and (c). he computed displacement field from image (a) to image (b) is shown in (d), the computed displacement field from image (b) to image (c) is shown in (e), and the computed displacement field from image (a) to image (c) is shown in (f).
D ransitivity est (cont.) 3D ransitivity est a b c he transitivity error (max, mean, std) for random triples of images (I, I, I 3 ) with a given model (M) of D short axis cardiac MR images. he units for the errors are pixels. Oskar Skrinjar, Yi-Yu Chou, and Hemant agare, ransitive Nonrigid Image Registration: Application to Cardiac MR Image Sequences, SPIE Medical Imaging, February 004, San Diego, CA. d e f hree randomly selected images from a sequence of 3D short axis cardiac MR images are shown in (a), (b), and (c). he computed displacement field from image (a) to image (b) is shown in (d), the computed displacement field from image (b) to image (c) is shown in (e), and the computed displacement field from image (a) to image (c) is shown in (f). I 3 6 4 I 6 3D ransitivity est (cont.) 3 4 4 I 3 3 6 8 6 M 8 3 6 3 4 9 max.x0-4.x0-4.x0-4.3x0-4.x0-4.4x0-4.0x0-4.4x0-4 mean.x0 -.9x0 -.x0 -.x0 -.x0 -.x0 -.x0 -.x0 - std 3.x0-3.4x0-3.x0-3.6x0-3.x0-3.x0-3.x0-3.6x0 - he transitivity error (max, mean, std) for random triples of images (I, I, I 3 ) with a given model (M) of 3D short axis cardiac MR images. he units for the errors are voxels. Related Issues: Validation Validation strategies: Visual assessment : contour overlays; difference images Simulation : artificially deformed image; biomechanical model Gold Standard : implanted markers; (only suitable for rigid registrations) Consistency : (A,B) (B,C) = (A,C) Future Work D symmetry test 3D symmetry test