A non-rigid registration method for mouse whole body skeleton registration

Transcription

1 A non-rigid registration method or mouse whole body skeleton registration Di Xiao* a, David Zahra b, Pierrick Bourgeat a, Paula Berghoer b, Oscar Acosta amayo a, Catriona Wimberley b, Marie Claude Gregoire b, Olivier alvado a a he Australian e-health Research Center, IC, CIRO, Australia b Radiopharmaceutical Research Institute, ANO, Australia ABRAC Micro-C/PE imaging scanner provides a powerul tool to study tumor in small rodents in response to therapy. Accurate image registration is a necessary step to quantiy the characteristics o images acquired in longitudinal studies. mall animal registration is challenging because o the very deormable body o the animal oten resulting in dierent postures despite physical restraints. In this paper, we propose a non-rigid registration approach or the automatic registration o mouse whole body skeletons, which is based on our improved 3D shape context non-rigid registration method. he whole body skeleton registration approach has been tested on 21 pairs o mouse C images with variations o individuals and time-instances. he experimental results demonstrated the stability and accuracy o the proposed method or automatic mouse whole body skeleton registration. Keywords: 3D non-rigid registration, mouse whole body skeleton registration, 3D shape context model 1. INRODUCION mall animal imaging is increasingly used as a pre-clinical tool to identiy new imaging agent, or assess eectiveness o therapy o diseases through their MRI, micro-c and Position Emission omography (PE) images in vivo. his involves scanning a cohort o small rodents (typically mice and rats), and computing population statistics o speciic organs or measuring and quantiying temporal changes in a region o interest (ROI). One challenging step to study a large numbers o individual animals is perorming spatial normalization beore any subsequent processing. Common tasks include the tracking o tumor size variations and shape in a longitudinal experiment o rats or mice C/PE images acquired over time or mapping speciic organs rom an atlas to any new scanned images. For small animals, because o the articulated joints and their anatomical structures, it is challenging to position the animals in a same position with a same posture or each scan. he use o physical support or the animals reduces large posture dierences but a signiicant deormation o the animal body still exists rom one scan to another and between dierent animals. he only easily identiiable and robust anatomical eatures present in C images are the skeletons, lungs and skin. Recently, a ew methods have been proposed to address the automatic registration o small animal whole body skeletons. Baiker et al. proposed an automatic articulated registration method or mouse skeleton registration by identiying joints and individual bones and traversing a hierarchical mouse skeleton tree predeined and registering each part by iterative closest point (ICP) rom a coarse to ine method [1]. Li et al. [2] proposed using robust point-based registration [3] and sotassign algorithm [4] or mouse whole body skeleton registration in their two-step registration process and tested it on a mouse s serial C images. Hesheng et al. [5] proposed a deormable image registration method, consisting o a global aine transormation and a local B-splines deormation, or mouse whole body skeleton registration. he deormation model was incorporated into robust point matching (P-RPM) [3] method or estimating point correspondences and surace deormation. he irst method is a combination o piecewise rigid registration methods or automatically labeling o whole body skeletons. he later two methods aim at a whole body registration by non-rigid registration method without skeleton structure labeling. *Di.Xiao@csiro.au; phone ; Medical Imaging 2010: Biomedical Applications in Molecular, tructural, and Functional Imaging, edited by Robert C. Molthen, John B. Weaver, Proc. o PIE Vol. 7626, 76261N 2010 PIE CCC code: /10/$18 doi: / Proc. o PIE Vol N-1

2 In this paper, we aim at addressing the whole body skeleton registration by point matching based surace registration. Point matching based surace registration, using 3D shape context has recently emerged as an alternative to intensity bases non-linear registration [6]. Given a set o points on a source surace, 3D shape context can provide the corresponding point locations on a target surace. Once the matching is established, warping o one surace to the other can be perormed with a smooth deormation model. hape context was originally proposed by Belongie et al. or 2D graph matching [7], as it provided an eective way to compute the similarity between two point clouds. A irst 3D shape context was built in [8] or measuring 3D shape similarity. Urschler et al. [5, 9, 10] extended 2D shape context to 3D and applied it in image registration area or lung suraces and thoracic registration rom C images. Di et al. [11] added surace curvature inormation rom shapes to improve the perormance o 3D shape context based surace registration. Point mismatching has been reported as an issue in 2D object recognition [12, 13] and or 3D point matching [9]. In 3D shape context based registration application, a common method is to remove a percentage o highest cost correspondences [5, 9]. Because the joint structure o mouse skeleton and individual dierences o mouse posture during C scans, point mismatching is also present when applying 3D shape context model or inding point correspondences between mouse skeletons. Long distance point mismatching can cause severe surace distortion ater registration and neighboring point mismatching within a small region can cause surace local stretching and olding. hereore, a robust point matching by 3D shape context model is important or a correct mouse skeleton registration which is described in a separate publication at this conerence. In this paper, we propose a ramework o 3D shape context based non-rigid registration method or mouse whole body skeleton registration. A skeleton extracting and processing method is proposed or identiying a clean skeletons or templates or registration. An improved 3D shape context model is applied as a kernel o the non-rigid registration. A speciic iterative registration procedure is developed or mouse whole body skeleton registration. 2. MEHOD In this paper, all mouse C images were acquired by a same micro-c/pe scanner. he mouse C images were scanned or early stage tumor location and growth estimation. With a pre-deined scan protocol, the mice were positioned with a similar prone posture and their ore limbs were stretched orward along two sides o their heads. heir hind limbs were stretched backward with toes acing up. 2.1 keleton extraction and pre-processing A pre-processing ramework is proposed or processing mouse C images and obtaining clean and usable mouse skeletons. Firstly, the raw mouse C image noise is reduced by curvature low denoising method, which uses level set algorithm [14], or preserving sharp boundaries and smoothing homogeneous regions. hen, the image is thresholded into a binary image with a ixed threshold (threshold value = 1950), obtained ater experimentation and related to the Hounsield units o the bone. Finally, a mesh set o mouse raw skeletons is generated by applying marching cubes surace construction algorithm [15] and triangulation method on the binary image. A respiratory sensor is an accessory that is oten needed or monitoring respiration o a mouse during C/PE scan. Because o the similar image intensity values o the respiratory sensor and bones, the extracted meshes by abovementioned binary thresholding method can contain the sensor (as shown in Fig. 1(a)). By applying mesh searching and sorting on all meshes, the second largest mesh representing respiratory sensor can be identiied and removed (Fig. 1(b)). Ater urther removing some small trivial meshes, a clean mouse whole body skeleton can be obtained as shown in Fig. 1(c). By applying the same mesh cleaning method, the largest mesh, which contains skull, spine and hind limbs, can be labelled. his largest mesh, which is named as major skeleton (Fig. 1(d)), is used in an initial step o mouse wholebody skeleton registration. By applying mesh shrinking method on the clean whole-body skeleton and projecting all points on the processed skeleton to the skeleton s longitudinal axis (direction rom its inerior to superior), a 2D histogram (distribution o the points on the longitudinal axis) can be generated (Fig. 2). he minimal value on the histogram at lumbar vertebrae can be automatically detected and used as a eature point to cut the whole-body skeleton into two parts: hind-body skeleton (Fig. 1(e)) and ore-body skeleton (Fig. 1()). he hind-body skeleton is one single connected mesh. he ore-body skeleton consists o our independent meshes: vertebrae and head, let ore limb, right ore limb and sternum. Proc. o PIE Vol N-2

3 (a) A raw skeleton extracted rom a binary image. (b) A skeleton ater respiratory sensor removal. (c) A clean mouse whole body skeleton. (d) A major skeleton ater (e) A ore-body skeleton. largest mesh selection. Fig. 1. keleton pre-processing and each part used in skeleton template. () A hind-body skeleton. Fig. 2. 2D histogram o skeleton mesh point projection on skeleton longitudinal axis (rom inerior 0 to superior 100) and lumbar eature point. 2.2 Preparation o skeleton emplate By using the pre-processing method, a scanned mouse C image can be processed to generate a skeleton template or an atlas, which includes the ollowing parts: a clean whole body skeleton (multiple meshes) without respiratory sensor and trivial meshes, a major skeleton (one mesh) which includes connective skull, vertebrae, hips and hind limb skeletons, a hind body skeleton which including hind part o vertebrae, hips and hind limb skeletons, a ore body skeleton which including head, ore part o vertebrae, sternum and ore limb skeletons, a lumbar eature point which separates the hind body and ore body skeletons rom their whole body skeleton. Figs. 1(c) to 1() show each part o a complete mouse skeleton template. he template obtained by the method is used as a source skeleton to register to other mouse skeletons (target skeletons) in the experiments o the paper. Proc. o PIE Vol N-3

4 2.3 3D shape context based non-rigid registration I M represents source surace and M represents target surace, the goal o a registration is to ind a transorm t : M M. A triangular mesh representation M ( V, F) is used here, where V is the set o vertices (points in a mesh) and F the set o triangular aces o the mesh. uraces are usually high resolution meshes; thereore, landmarks (as low resolution vertices selected rom the suraces) are used to represent the suraces. he transorm is computed rom the landmarks correspondences. In this paper, the landmarks are vertices rom decimated triangular meshes, which are obtained rom their high resolution triangular meshes by mesh decimation approach. A landmark mesh L has the relationship with its high resolution mesh Μ as { V L} { V M} and { F L} { F M}. he decimation method ensures that the landmark vertices are uniormly distributed on the original high resolution meshes. We apply an improved 3D shape context based non-rigid registration method in the ramework o mouse whole body skeleton registration. he method was based on 3D shape context model but provided eective point mismatching correction algorithm, which uses topological structure correction (C) method or correcting long geodesic distance mismatch (LGDM) points and correspondence ield smoothing (CF) method or correcting neighbors crossing mismatch (NCM) points. Please see corresponding paper presented at this conerence or details [16]. Here, the improved 3D shape context method is used or correct corresponding points matching between the source and target landmark meshes L and L by their landmark vertices V and V. A inal source landmarks V L and target landmarks V are obtained, and the correspondences between them are constructed. Ater obtaining the correspondences between the two sets o landmark vertices V and V, the mapping o the source surace is modeled with a thin-plate spline (P) [17] or high resolution vertices mapping and constrained by the source surace s topology. he P model, describing the transormed point ( x ', y', z' ) independently as a unction o original point ( x, y, z) has the orm: n 1 + a2x + a3 y + a4z + wu i ( V ( i) ( x, y, z) ) i= 1 ( x', y', z') = t( x, y, z) = a (1) 2 U ( r) = r logr is the basis unction, a= a, a, a, } { a4 V (i where the global aine parameters o the transormation and w= { w 1,..., w n } the additional non-linear deormation. ) ( i = 1,..., N ) are source landmark vertices. N is the number o the source landmark vertices. Landmark vertices V (i) and V (i) are used to compute the coeicients a and w in the unction by minimizing its bending energy [14]. With computed coeicients and source landmark vertices, a non-linear mapping unction is built between the source and target vertex sets. Combined with the topology o the source mesh Μ, a transorm t : M M is constructed. 2.4 An iterative procedure o non-rigid registration Considering the relative large posture dierence in mouse C scan, a 4-step iterative procedure has been developed or mouse skeletons registration during implementing the improved 3D shape context non-rigid registration method. he iterative procedure is illustrated in the diagram in Fig. 3. he initial source landmark mesh L and the target landmark mesh L are generated by mesh decimation algorithm rom their respective high resolution meshes M and M. At the irst iteration, the deormed landmark meshes are initialized rom the original decimated meshes. During the steps 2 to 4, various matching correction techniques and applied successively. During all the steps the number o vertices and the topology are preserved In the procedure, the correspondence ield 1 is generated rom the basic 3D shape context algorithm with one-to-one landmark vertex match between the deormed landmark set 1 and matched target landmark set 1. As described in section 2.3, the C algorithm uses mesh topology rom source landmark mesh. In order to avoid re-computing the topology, the Proc. o PIE Vol N-4

5 topology o all vertices is preserved based on the source landmark mesh. A straightorward solution is to let the number o vertices on the source landmark mesh to be less than that o vertices on the target landmark mesh ater their respective decimation processes. his avoids removing any vertices in the source landmark mesh during the process o 3D shape context computation. he correspondence ield 1 preserves 100% o the cost, and the input o the C algorithm is the deormed landmark mesh with correspondence ield 1. ource high resolution mesh arget high resolution mesh ource landmark mesh Ite = 1 Mesh decimation method arget landmark mesh P matrix Ite = 2,3,4 Deormed landmark mesh Deormed landmark mesh with matched landmark set 1 3D shape context calculation or correspondences Correspondence ield 1 Matched target landmark set 1 opological structure correction (C) algorithm 2% highest costs removal algorithm ame vertex Ids Deormed landmark mesh with matched landmark set 2 Matched source landmark set 2 Correspondence ield 2 Matched target landmark set 2 Correspondence ield smoothing (CF) algorithm Matched source landmark set 3 Correspondence ield 3 Matched virtual target landmark set 3 P transorm computation between source and virtual target landmarks Iterative no. = 1, 2, 3, 4 Fig. 3. An iterative procedure o the improved 3D shape context non-rigid registration. Ater the C, long geodesic distance mismatch correspondences are removed rom two landmark sets. In addition the 2% highest cost correspondences are removed as they correspond to obvious mismatching errors rom our experiments. he result is a robust correspondence set between the two meshes. he deormed landmark mesh has the same topology (same vertices with their Ids in the mesh) as the source landmark mesh, a matched source landmark set 2 can be identiied rom the source landmark mesh by using the matched vertex Ids rom the deormed landmark set 2. A correspondence ield (correspondence ield 2) between the matched source landmark set 2 and the matched target landmark set 2 is constructed. A CF method is applied or smoothing the correspondence ield 2 and generating correspondence ield 3. he smoothing ilter kernels are set to 5, 4, 2 and 1 or the dierent iterations. he irst iteration with large kernel size estimates a very smooth correspondence ield that is subsequently relaxed to allow more precise matching o individual points. he very small kernel size o the last step ensures mostly that the deormation ield does not contain any olding. In our experience, we ound that the whole procedure had a stable convergence with excellent matching results. he P transorm computed ater the iterative procedure can be directly applied or the transormation o the source high Proc. o PIE Vol N-5

6 resolution mesh M, thereby interpolating the transormation to the whole image domain rom the set o matching points. 2.5 A ramework o mouse whole-body skeleton registration o limit computation time and keep the memory requirement compatible with standard desktop computer memory we used 800 landmarks or the improved 3D shape context based non-rigid registration method. he registration uses a coarse-to-ine approach with the ollowing steps: (1).keleton extraction and pre-processing method on a mouse C to obtain a clean mouse whole body skeleton. (2) Initial registration o major skeletons rom a template to that o the study by using the 3D shape context based non-rigid registration. Use the registration transorm to map the lumbar eature points rom the template to the clean skeleton and clipping a hind-body skeleton rom the clean skeleton. (3) Iterative ine-tuning registration rom the hind-body skeleton template to the study s hind-body skeleton. (4) Apply the registration transorm rom step (3) to map the lumbar point rom the template to the clean skeleton and clipping a ore-body skeleton rom the study. he remapping o the lumbar point provides an accurate ore-body clipping at the cutting point. (5) Iterative ine registration rom the ore-body skeleton template to the study s hind-body skeleton, using decreasing kernel sizes. (6) Generation o the inal P transorm rom the correspondence ields obtained rom steps (3) and (5), and mapping o the whole-body skeleton rom the template to the study. 3. EXPERIMEN AND DICUION he proposed method was implemented using C++ language. In the experiments, we used the method in [18] and a surace measurement tool [19] to calculate mean absolute dierence (MAD) error (mean absolute distance o all sampled points) and ace root mean square (RM) error (root mean square error o all sampled triangle aces) between two suraces. 3 mice were scanned by micro-c scanner. Each mouse was imaged with C scan once every 4 or 5 days. 12 C images (each mouse with 4 time points) were chosen or testing our method. For each C image, the original image size was 384, 384, and 461 with an isotropic pixel resolution o mm. 3.1 Mouse hind limb skeleton registration and intra- and inter-group comparison he proposed method was applied or mouse hind limb skeleton registration. he skeleton rom each mouse s irst C scan was used as the source skeleton. he registration experiments were grouped into 1) intra-subject registration: the irst scan rom each mouse was registered to its other three scans; and 2) inter-subject registration: each mouse s irst scan was registered to a dierent mouse s three scans. For each registration process, α =0.6, β =0.55 and high cost preservation 98%. We used 800 landmarks, 4 iterations o registration process with respective median ilter kernel 4, 4, 2 and 1. Fig. 4 gives two examples o registered results rom the intra-subject and inter-subject groups (Figs. 4(a) and 4(b)). Even with large posture dierence as shown in Fig. 4(a), the method can obtain very good registration results. Fig. 5 shows the means and standard deviations o MAD and ace RM errors rom the intra-subject group and inter-subject group. he mean MAD error rom the intra-subject group is 0.16 mm with maximum 0.20 mm and minimum 0.13 mm, and the mean ace RM 0.20 mm with maximum 0.27 mm and minimum 0.16 mm. he mean MAD error rom the inter-subject group is 0.17 mm with maximum 0.26 mm and minimum 0.13 mm, and the mean ace RM 0.23 mm with maximum 0.37 mm and minimum 0.17 mm. he large errors usually happened in the joint parts, as patella and calcaneus. tatistical results o t-est based on MAD errors and ace RM errors did not ind signiicant dierences or the mean o errors rom the inter-subject group compared to the intra-subject group. Proc. o PIE Vol N-6

7 (a) One registration result rom intra-subject group (side view and top view). (b) One registration result rom inter-subject group (side view and top view). Fig. 4. Results o mouse hind limb skeletons registration (red-source skeleton; yellow-target skeleton; blue-registered skeleton). (Please reer to color picture o the paper on CD-ROM rom proceedings volumes). Mean and DEV Mean MAD Face RM MAD Face RM Intra-subject Inter-subject Fig. 5. Mean and standard deviation o registration errors rom inter-subject and inter-subject groups 3.2 Perormance o iterative registration procedure Fig. 6. ource surace approaching the target surace in an iterative procedure ((a) beore iteration; (b) registered surace ater iteration 1; (c) registered surace ater iteration 2; (d) registered surace ater iteration 3; (e) registered surace ater iteration 4. red source, yellow target, and blue registered surace). (Please reer to the color picture in the paper on CD-ROM rom proceedings volumes). MAD errors in registration process Face RM erros in registration process MAD error (mm) Intra-subject 1 Intra-subject 2 Inter-subject 1 Inter-subject Iterative number Fig. 7. Iterative registration process rom inter-subject and inter-subject groups Face RM error (mm) Iterative number Intra-subject 1 Intra-subject 2 Inter-subject 1 Inter-subject 2 Proc. o PIE Vol N-7

8 An experiment was designed to test the convergence o the proposed iterative registration procedure. 4 iterations were adopted in the experiment with Gaussian kernel 5, 4, 2 and 1 respectively. wo pairs o samples each rom the same mouse were used as intra-subject group and the other two pairs o samples rom dierent mice as inter-subject group. Fig. 6 shows one example o registered suraces (blue color) in each step o the iterative procedure. Fig. 7 shows the MAD and ace RM errors o the registered suraces during the iterative process. It shows, ater one iteration, there is an obvious error decrease then ollowed by a stable error decrease process in the rest o 3 iterations. 3.3 Mouse whole body skeleton registration he proposed method was applied or mouse whole body skeleton registration. From the 12 C images rom 3 mice, We selected 8 pairs o C images (each pair rom same mouse) as intra-subject (IA) group and 13 pairs o C images (each pair rom dierent mice) as inter-subject (IR) group. he iterative number or all three processes was 4 with Gaussian kernel 5, 4, 2 and 1. Fig. 8 and Fig. 9 show two examples o registration results. able 1 and able 2 give the MAD and ace RM errors or total 21 pairs o registered skeletons. he registration results demonstrate that the proposed registration ramework provides a robust and accurate registration or mouse skeletons. able 1 MAD and ace RM errors o mouse whole body skeleton registration in intra-subject group. Hind body skeleton Fore body skeleton Errors (mm) IA1 IA2 IA3 IA4 IA5 IA6 IA7 IA8 MAD Face RM MAD Face RM able 2 MAD and ace RM errors o mouse whole body skeleton registration in inter-subject group. Hind body skeleton Fore body skeleton Errors (mm) IR1 IR2 IR3 IR4 IR5 IR6 IR7 IR8 IR9 IR 10 IR 11 IR 12 IR 13 MAD Face RM MAD Face RM Fig. 8. A mouse whole body skeleton registration result rom inter-subject group (red source; yellow target; blue registered skeleton). (Please reer to the color picture in the paper on CD-ROM rom proceedings volumes). Fig. 9. A mouse whole body skeleton registration result rom intra-subject group (red source; yellow target; blue registered skeleton). (Please reer to the color picture in the paper on CD-ROM rom proceedings volumes). 4. CONCLUION We proposed an automatic mouse whole body skeleton registration method based on our improved 3D shape context non-rigid registration. he registration ramework or mouse whole body skeleton registration demonstrated the coarseto-ine step and iterative procedure in each registration step achieved a stable registration process and an accurate Proc. o PIE Vol N-8

9 registration result. he registration method was robust or both inter-subject registration and intra-subject registration. he registration method can be applied or other organs registration o small animals, as lung and skin and the registration results can be used as an initial step or small animal whole body organs registration or volume image registration. REFERENCE 1. M. Baiker, J. Milles, A.M. Vossepoel, I. Que, E.L. Kaijzel, C.W.G.M. Lowik, J.H.C. Reiber, J. Dijkstra and B.P.F. Lelieveldt, Fully Automated whole-body registration in mice using an articulated skeleton atlas, IBI, (2007). 2. X. Li,.E. Peterson, J.C. Gore, B.M. Dawant, Automatic registration o whole body serial micro C images with a combination o point-based and intensity-based registration techniques, IBI 2006, (2006). 3. H. Chui and A. Rangarajan, A new point matching algorithm or non-rigid registration, Computer vision and Image Understanding, (2003). 4.. Gold, A. Rangarajan, C.P. Lu,. Pappu, E. Mjolsness, New algorithms or 2D and 3D point matching: pose estimation and correspondence, Pattern Recognition 31(8), (1998). 5. H. Wang and B. Fei, A robust B-plines-based point match method or non-rigid surace registration, Proc. o the 2 nd International Conerence on Bioinormatics and Biomedical Engineering, (2008). 6. M. Urschler, H. Bischo, Registering 3D lung suraces using the shape context approach, Proc. Medical Image Understanding and Analysis, (2004). 7.. Belongie, J. Malik and J. Puzicha, Matching hapes, Proc. Eighth Int l. Con. Computer Vision, (2001). 8. M. Kortgen, G. Park, M. Novotni, R. Klein, 3D hape Matching with 3D hape Context, eventh Central European eminar on Computer Graphics (2003). 9. M. Urschler, J. Bauer, H. Ditt, H. Bischo, IF and hape Context or Feature-Based Nonlinear Registration o horacic C Images, Proc. Computer Vision Approaches to Medical Image Analysis, (2006). 10. M. Urschler, H. Bischo, Assessing breathing motion by shape matching o lung and diaphragm suraces, Proceedings o the PIE Medical Imaging, 5746, (2005). 11. D. Xiao, P.. Bourgeat, J.E. Fripp, O.A. amayo, M. Gregoire, O. alvado, Non-rigid registration o small animal skeletons rom micro-c using 3D shape context, Proc. o PIE Medical Imaging, 7259, 72591G (2009). 12. A. hayananthan, B. tenger, P.H.. orr, and R. Cipolla, hape Context and Chamer Matching in Cluttered cenes, Proc. IEEE Con. Computer Vision and Pattern Recognition, 1, (2003) Giannarou, and. tathaki, Object Identiication in Complex cenes using hape Context Descriptor and Multi-stage Clustering, 15 th International Conerence on Digital ignal Processing, (2007). 14. J.A. ethian, Level set methods and ast marching methods, Cambridge Press, chapter 16, econd edition (1999). 15. W. Lorensen and H. Cline, Marching cubes: a high resolution 3D surace construction algorithm, Computer Graphics, 21, (1987). 16. D. Xiao, D. Zahra, P. Bourgeat, P. Berghoer, O. Acosta amayo, C. Wimberley, M. Gregoire, O. alvado, An improved 3D shape context registration method or non-rigid surace registration, Proc. o PIE Medical Imaging, (2010) (Accepted). 17. F.L. Bookstein, Principal warps: hin-plate splines and the decomposition o deormation, IEEE rans. PAMI, 11(6), (1989). 18. N. Aspert, D. anta-cruz,. Ebrahimi, MEH: Measuring Errors between uraces using the Hausdor distance, the proceedings o the IEEE Int. Con. on Multimedia and Expo (ICME), I, (2002) Proc. o PIE Vol N-9