Image/Geometry Registration (Mesh Correction) C. BAJAJ, Y. ZHANG and the DDDAS team

Image/Geometry Registration (Mesh Correction) C. BAJAJ, Y. ZHANG and the DDDAS team

What is Image/Geometry Registration? Image/Geometry registration (sometimes called fusion) is to determine geometrical transformations to align and match two or more Images/Geomety acquired at different times, from different sensors, or from different view points. Determine G and L s.t. I 2 ( x,y ) = G( I 1 ( L (x,y) ) where (x, y ) = L(x, y) Co-registered 3D viz of the brain and basal ganglia. The brain viz is from an MRI study, while the basal ganglia viz is depicting PET of some metabolism rate using F-18 FDG (Courtesy of PET Laboratory at Mount Sinai Medical Center, New York).

Image Registration (contd) Two major classes of change [Brown 92]: 1. Variations due to the differences in acquisition which cause the images to be misaligned. 2. Variations that are of interest such as object movements, growths, or other scene changes. MRI and SPECT head sagittal slices of the same patient and the co-registered (MRI + SPECT) image. The lesion on the top of the skull is more prominent in the composited image, although it can be visualized in both modalities. Diastole ventricle Systole ventricle Matching between the diastole and L.G.Brown. A Survey on Image Registration Techniques. ACM Computing Surveys. 24(4)):325-376, 1992. systole contours

Image Registration/Reconstruction of Hippocampal Neurons Transmission Electron Microscopy, Thin Sections: Kristen Harris, ILM, University of Texas at Austin Addtl. Collab: Tom Bartol, Salk Institute

Image Registration Techniques Correlation analysis Fourier based matching Point Mapping Elastic Model-Based Match & Fit

Image Co-Registration 3D image registration using fast Fourier transformation: find the shift, rotation, and scaling transformation to co-register two images. R. Araiza et al. 3-D Image Registration Using Fast Fourier Transformation: Potential Applications to Geoinformatics and Bioinformatics.

Some Characteristics of Registration Methods Feature space: intensities, edges, contours, surfaces, salient features (corners, line intersections, regions of high curvatures), statistical features (moment invariants, centroids), higherlevel structural and syntactic descriptions. Similarity measure: Cross-correlation with/without prefiltering Norm of absolute differences Fourier based correlation scoring Search space strategies: Relaxation/evolution based matching level set evolution Dynamic programming

Level Set Method Previous Work A fast multiscale and multigrid method for the matching of images in 2D and 3D [Clarenz, Droske, Rumpf 1991]. 1 2 E( φ) = f1 oφ f2 dx 2 Ω f1, f2 are the intensity maps of the two images to be matched, and Фis the deformation/transformation. Original image f1 Distorted image f2 Application of the Matching result deformation to a uniform grid

Mesh Correction Problem Description Given a 3D Image sequence <I 1, I 2, I n > generate a sequence of finite element meshes <M 1, M 2, M n > of the ROI, via mesh updates for efficiency.

Image/Geometry Processing for Finite Element Meshing of ROI MRI Data 2D/3D Image Processing Contrast Enhancement Filtering Classification Segmentation Surface from Voxel Map 2D Image Tracing/Lofting NURBS/A-Patch Construction www.tbts.org/assets/images/brainmap.gif Geometry Curation Geometry Medial Axis/ Segmentation Tet/Hex Meshing (LBIE Mesher) Quality Improvement MRI: 512x512x27 volume rendering segmentation surface model w. tumor 10264 hexes

Collaborators: Prof. Guoliang Xu (Chinese Academy of Sciences, China) Quality Improvement with Geometric Flow and Pillowing Quality Improvement surface diffusion flow x t Properties: r r = ΔH ( x) n( x) + v( x) T ( x) 1. Denoising the surface mesh normal movement 2. Improving the aspect ratio of the surface mesh -- Tangent movement doesn t change the shape The surface diffusion flow is volume preserving 3. Improving the aspect ratio of the volumetric mesh vertex adjustment inside the volume Pillowing: Discretized Laplace-Beltrami operator over quad [1] Before quality improvement, the mesh has 5 elements with negative Jacobian After quality improvement, the mesh has no elements with References: negative Jacobian. 1. Y. Zhang, C. Bajaj, G. Xu. Surface Smoothing and Quality Improvement of Quad/Hex Meshes with Geometric Flow. 14 th Meshing Roundtable, 449-468, 2005.

Pillowing Results Human Brain Data Original Pillowing # Vertices 8767 7120 # Hexes 11918 10264 Worst Jacobian Negative 0.129

Mesh Correction Free form deformation (no topology changes) Level set methods (with topology changes) (1.a) Initial conditions, (1.b) Global alignment using mutual information and implicit representations, (2.a) Establishing correspondences using FFD, (2.b) Registering locally using FFD, (2.c) Grid deformation. X. Huang, N. Paragios, D. Metaxas. Registration of Structures in Arbitrary Dimensions: Implicit Representations, Mutual Information & Free Form Deformation. MICCAI 2003. From top left to bottom right: saggital slice through the original 3D image, second image generated by reflection, deformation applied to a uniform grid, matching result. U. Clarenz, M. Droske, M. Rumf. Towards Fast Non-Rigid Registration. Image Analysis and Medical Imaging, 2002.

Free-Form Deformations 1. Barr Technique [BARR84] - uses a set of hierarchical transformations for deforming an object, including stretching, bending, twisting and taper operators. 2. Sederberg and Parry Technique [SEDE86] deforms solid geometric modes in a free-form manner. This technique is based on trivariate Bernstein polynomials, and it enables deformation objects by manipulating the control points. 3. Coquillart Technique [COQU90] is an extension of Sederberg and Parry s technique. The goal is to change the shape of an existing surface either by bending it along an arbitrary shaped curve or by adding randomly shaped bumps to it. 4. Chang and Rockwood Approach [CHAN94] deforms an object by repeatly applying affine transformation in space. [BARR84] Barr, A. H., Global and Local Deformations of Solid Primitives, Proceedings of SIGGRAPH 84, Computer Graphics 18, 3, 21-30. [CHAN94] Chang, Y.and Rockwood, A. P., A Generalized de Casteljau Approach to 3D Free-form Deformation, SIGGRAPH 94, Computer Graphics, 257-260. [COQU90] Coquillart, S., Extended Free-Form Deformation: A sculpturing Tool for 3D Geometric Modeling, SIGGRAPH 90, Computer Graphics 24, 4, 187-196. [SEDE86] Sederberg, T. W. and Parry, S. R., Free-Form Deformation of Solid Geometric Models, SIGGRAPH 86, Computer Graphics 20, 4, 151-159.

Level Set Methods Prior Work The Level set method [Osher & Sethian 88, Osher & Edkiw 03] allows one to dynamically deform and track an implicit surface using a governing PDE, which describes various laws of motion depending on geometry, external forces, or a desired energy minimization. T(x) is the target image, S(x) is the initial image, the energy function is: Elastic registration: Viscous fluid registration: I. Yanovsky, P. Thompson, S. Osher, A. Leow. Large Deformation Unbiased Diffeomorphic Nonlinear Image Registration: Theory and Implementation. UCLA CAM Report 06-71, 2006.

Level Set Method Previous Work To find a registration between two given 2D/3D images based on a matching of the edges in the images [Mumford & Shah 89, Droske & Ring 05] E( u, Γ) = μ Ω ( u u d ) 2 dx u d is from the start image defined on an open bounded set Ω and u is aimed to be a approximation of u d which should be smooth on Ω\Γ, where Γ is the set of potential edges (i.e. subsets of Hausdorff dimension n 1 located at singularities of the given image). Here H n 1 denotes the n 1 dimensional Hausdorff measure. + Ω\ Γ u 2 dx + αη n 1 ( Γ) Top row: start image Bottom row: target image White curve is the edge set of reference image M. Droske and W. Ring. A Mumford-Shah Level Set Approach for Geometric Image Registration. D. Mumford and J. Shah. Comm. Pure Appl. Math, 42:577-685,1989.

Mesh Correction Possible Algorithm M 1 is the static mesh generated from the image data I 1 φ is the deformation from I 1 to I 2, the energy function is defined as 1 2 E( φ) = I1 oφ I2 dx 2 Ω Therefore M 2 M 1 oφ Find the difference of the boundary voxels between I 1 and I 2 using φ, remesh M 1 locally to obtain M 2. If topology changes, then generate a local re-mesh via level set update. Quality improvement

Higher Order Level Set (HLS) Method Given a non-negative function g(x) over a domain. Find a surface Γ in, such that the energy functional Ω E ( Γ) = g( x) dx + ε h( x, n) dx Γ is minimal, where h(x, n) is another non-negative function for regularizing the constructed smooth surface [Bajaj et al. 2006]. φ = L( φ) + H ( φ) t φ L( φ) = ( g + εh) div( ) φ, φ H ( φ) = ε div( P n h) φ + 2 Γ T [ ( g + εh) ] φ Ω R 3 C. Bajaj, G. Xu and Q. Zhang. Smooth Surface Constructions via a Higher-Order Level-Set Method. ICES report, 2006-18. (a) shows the point set data produced by a 3D surface scanner. (b) shows the the reconstruction result using our C 2 tri-cubic spline level-set method. (c) shows the reconstruction result for a C 0 tri-linear levelset method.

Higher Order Level Set (HLS) Method Four main steps to compute the spline level set φfunction 1. Initialization: Given a initial Γ, construct a piecewise tri-linar level-set function φ over the grid G l. Convert φ to Φ. 2. Evolution: Resampling Φ to obtain a new φ over the grid G l. 3. Re-initializtion: Applying re-initialization step to φ. 4. Iso-contouring; Extract 3-sided or 4-sided iso-surface patches. G 1 approximate these patches by parameter surfaces.

Smooth Biomolecular Surface Construction To construct the SES for molecules, we minimize the energy function where Γ Γ + = Γ dx dx x g E ε 2 ) ( ) ( φ φ φ φ φ ε φ φ φ φ = + = + = T g g H div g L H L t ) ( 4 ) (, ) ( ) ( ) ( ) ( ) ( 2 (a) shows the van der Waals surface of a molecule. (b) shows the corresponding solvent excluded molecular surface constructed using our C 2 tri-cubic spline level-set method. (c) illustrates that the smooth solvent excluded surface constructed tightly encloses the van der Waals surface (a). = = m i r x x C i i e x g 1 ) ( 2 2 1 ) (

Smooth Biomolecular Surface Construction

Extra Slides

Proposed Mesh Correction Algorithm Static meshing from I 1 to obtain M 1 Dynamic updating of M 1 obtain M i, i=2,3 Image registration Define a function to measure the difference of imaging data between two adjacent time steps. Geometry registration remesh locally, dynamically and efficiently 1. Free form deformation (no topology changes) 2. Level set method (with topology changes) Quality Improvement

Search Space Strategies Relaxation Matching is a standard method for quasioptimally solving the local correspondences of the template and input images. Dynamic Programming is to efficiently find the optimal time matching for two images/patterns.

Spatial Transformations Rigid: translation, rotation, scale. Affine: A transformation is affine if T(x) T(0) is linear Projective and perspective: 3D --> 2D Global polynomial transformation