Iterative Estimation of 3D Transformations for Object Alignment

Transcription

1 Iterative Estimation of 3D Transformations for Object Alignment Tao Wang and Anup Basu Department of Computing Science, Univ. of Alberta, Edmonton, AB T6G 2E8, Canada Abstract. An Iterative Estimation Algorithm (IEA) of 3D transformations between two objects is presented in this paper. Skeletons of the 3D objects are extracted using a fully parallel thinning technique, feature point pairs (land markers) are extracted from skeletons automatically with a heuristic rule, and a least squares method and an iterative approach are applied to estimate the 3D transformation matrix. The algorithm has three advantages. First of all, no initial transformation matrix is needed. Secondly, user interaction is not required for identifying the land markers. Thirdly, the time complexity of this algorithm is polynomial. Experiments show that this method works quite well with high accuracy when the translations and rotation angles are small, even when noise exists in the data. 1 Introduction 3D alignment or registration algorithms have many applications in medical image processing. Consider two objects O1 and O2, such that O2=M*O1, where M is the 3D transformation matrix. In this context, the objective of 3D alignment or registration is to estimate the 3D transformation matrix M. Surveys [1-2] of 3D alignment methods are available in the literature. Since the previous decades, a lot of 3D alignment algorithms, including land marker based algorithm [3], have been proposed. This paper focuses on land marker based algorithms. The mean square distance () [3] is used as the metric. Point correspondence [4], which relates a land marker with its counterpart, is a crucial step for algorithms in this category. The advantage of this technique is that the set of land markers is relatively sparse compared to the original 3D object, so that the optimization process is relatively fast. However, this technique has some disadvantages. First of all, some algorithms, for instance, the ICP [3] algorithm, needs to know the initial transformation matrix. Secondly, user interaction is usually required for identifying the land markers. Last but not the least, the complexity of point correspondence [4] is non-polynomial since it is a combinatorial optimization problem. In this paper, an automatic Iterative Estimation Algorithm (IEA) is proposed. It has three advantages: (i) no initial transformation matrix is needed; (ii) user interaction is not required for identifying the land markers; (iii) the complexity of this algo-

2 rithm is polynomial. The disadvantage of this algorithm is that it does not guarantee achieving global optimization. We applied IEA in 3D medical image processing. We are interested in defining and tracking volume changes of airways caused by surgery, which increases volume. Doctors need to track the changes of airways for administering effective treatments. In our application, doctors take MRI scans of patients and save them in the DICOM [5] format. A commercial software ScanIP/FE [6] is used to segment the airways and remove noise semi-automatically. The skeletons are created to depict the segmented airway using a fully parallel 3D thinning algorithm developed by us. Then, we applied a heuristic algorithm to automatically extract feature point pairs (land markers) from skeletons. Since false point correspondence is inevitable, we use IEA to remove some point pairs and use the remainder point pairs to achieve smaller. The whole procedure has 5 main steps: segmentation, noise removal, thinning, finding feature point pairs and iterative transformation estimation. In the following sections, the algorithm is described in detail. In Section 2, we introduce the data collection and segmentation procedures. Section 3 will briefly discuss how to artificially create data for comparison. The fully parallel 3D thinning algorithm and the skeletonization process is summarized in Section 4. Section 5 focuses on the acquisition of feature point pairs (land markers), and the IEA. Results of experiments are presented in Section 6, before the work is concluded in Section 7. 2 Data collection, segmentation and noise removal A small portion of the upper airway is scanned for a child as an MRI image and saved in the DICOM format, which is the industry standard for MRI. ScanIP/FE is used to segment airways, remove noise, and semi-automatically create an airway model of 3D images. A 3D image is a mapping that assigns the value of 0 or 1 to each point in the 3D space. Points having the value of 1 are called black (object) points, while 0 s are called white (background) ones. Black points form objects of the image. Our test data has about 37,000 object points. Fig. 1 shows some image slices and the segmented airway (in red). The 3D models from different viewpoints are displayed in Fig. 2. Fig. 1. image segmented red) (Left) image slice center slice last slice. Some slices and airway (in The first (Middle) A (Right) The

3 Fig. 2. 3D airway model from 3 different points of view. 3 Data creation for estimation To verify our method, we artificially created some airway models for estimation. We applied 3D homogeneous transformations [7] to the 3D image. We use (alpha, beta, gama, dx, dy, dz) to represent the rotation angles and translations for x-, y- and z-axis of the Homogeneous Matrices. We also added some random noise to the data to create models for comparison. 4 Fully parallel 3D thinning algorithm and skeletonization 3D thinning for medial lines or surface approximation is a useful approach for many potential applications. The approach has been extensively researched in the last decade [8]. In the experiments, we applied our fully parallel 3D thinning algorithm to extract the skeletons from 3D images after a noise removal step. Our method is an improvement on Ma and Sonka s algorithm [8]. The algorithm in [8] has a number of advantages; however, we found that it cannot preserve connectivity of 3D objects. Chaturvedi et al. [9] also found this problem. We improved [8] by changing some masks in Class D to preserve connectivity. The skeletons are used to represent the airway models. The skeletons have about 500 object points, a large reduction from the 37,000 object points. Fig. 3 (Left) and (Middle) show the skeletons of the original model and the artificially created model with (alpha, beta, gama, dx, dy, dz) = (1, 1, 1, 0, 0, 0). We notice that: 1. The thinning algorithm is sensitive to rotations. Even when the rotation angles are very small, the two skeletons look quite different. 2. The thinning algorithm cannot provide unit-width structures. We can see that some regions on the skeletons are quite dense. However, we will show that these two drawbacks do little harm to the estimation of 3D transformations. Thus, we can still estimate the transformations precisely based on the thinning algorithm.

4 Fig. 3. (Left) The skeleton of the original model. (Middle) The skeleton of the transformed model with (alpha, beta, gama, dx, dy, dz) = (1, 1, 1, 0, 0, 0). (Right) Line points (in red). 5 Feature point pair acquisition and the IEA Land marker based estimation algorithms have three main disadvantages as we discuss before. In this section, an Iterative Estimation Algorithm (IEA) is proposed to solve these problems. The disadvantage of this algorithm is that it does not guarantee global optimization. Definition 1: Connectivity number If P is an object point in a 3D image the connectivity number of P is the number of object points except itself in P s (3*3*3) neighborhood. In Fig. 4 (Left), the connectivity number of P is 4. Definition 2: Line point A line point is an object point with connectivity number equal to 2. In Fig. 3 (Right), the line points are displayed in red. Fig. 4. (Left) Connectivity number of point P is 4. A is an object point, A o is a background point. (Right) Point P and its 8 sub-neighborhood. Definition 3: Feature point (land marker) candidate A feature point (land marker) candidate is a line point with at least one non- line point in its (3 * 3 * 3) neighborhood. Definition 4: Feature point pair (land marker) I1 and I2 are two 3D images of the same object. I2=M*I1, where M is a 3D transformation matrix. A feature point pair (land marker) contains two object points P1 and P2, where P1 is on the skeleton of 3D image I1 and P2 is on the skeleton of 3D image I2, and P2=M*P The overdetermined linear system In general, M is a 4 * 4 Homogeneous Transformation Matrix. For point P1= (x1 y1 z1 1) T in 3D space, P2 = M * P1= (x2 y2 z2 1) T. The goal of our work is to estimate the 16 variables of matrix M in certain conditions. Since we need to calculate

5 16 variables, mathematically, we need 16 equations for this linear system. For each feature point pair, we can have 4 equations. So we need 4 feature point pairs in total to solve this problem. However, in most cases in our experiment, we can find more than 4 feature point pairs. This is a typical case of an over-determined linear system. An overdetermined linear system can be solved following well-established methods [10]. 5.2 The heuristic rule In our experiments, a heuristic rule is used to identify the feature point pairs. It requires two points in a feature point pair to have same "configurations" in its 8 subneighborhood. That is, the local topologies of two points in a feature point pair should be same. We used an adaptive method to search for the best neighborhood scale. Fig. 4 (Right) shows a point p and its 8 sub-neighborhood. Heuristic Rule: If two feature point candidates on different skeletons have the same number of object points and the same number of background points in all 8 subneighborhoods, these two points form a feature point pair. 5.3 The iterative estimation method Feature point correspondences is an open problem in Computer Vision. No general method can solve it in polynomial time. There are mainly two difficulties. Firstly, it is a combinatorial optimization problem. The time complexity is very high because of the large search space. Secondly, point correspondence is not one-one mapping. Some points, which are called outliers [4, 11], may not have counterparts because of occlusions, out of image transformations and errors in feature selection. A brief survey of previous work is available in the literature [4]. For several decades, researchers have proposed a number of point correspondence algorithms [4, 11]. Point correspondence algorithms have many applications [4, 12-13] including image alignment. The point correspondence algorithms are classified into two categories, globally optimal and non-globally optimal. A globally optimal algorithm [4] guarantees global optimality. However, the time complexity is non-polynomial. A non-globally optimal algorithm [11] is faster than a globally optimal algorithm. However, it does not guarantee global optimality. Another restriction is that some unwanted assumptions and constrains [11] are often required to detect the outliers. The IEA is a non-globally optimal algorithm. The complexity of this algorithm is polynomial. No user interaction, initial transformation knowledge and outlier detection are required. The motivation behind our work is that we realized that the characteristics of our application, as well as many other medical imaging applications, are: the number of feature point pairs is large, user interaction is unwanted, initial transformation knowledge is not easy to know and the outlier detection, which requires some unwanted assumptions, constrains, and is not very reliable, should be avoided. In these cases, we need a polynomial algorithm that requires no user interaction, no initial transformation information, implicitly removes outliers without any unwanted assumptions and constrains.

6 We notice that incorrect point correspondences and incorrect outlier detection negatively affect the estimation accuracy. However, we also notice that incorrect point correspondences and incorrect outlier detection are unavoidable. Therefore, in our algorithm, we use a very simple Heuristic Rule to do the point correspondences. There are some incorrect point correspondences and some outliers exist in the feature point pairs set. We do not know which point pair is correctly related and which point pair is incorrectly related. Since the incorrect point correspondences and incorrect outlier detection negatively affect the estimation, i.e., the is large, we iteratively remove one point pair from the whole set and test if the new is smaller. If the new is smaller than the old, we accept this removal. Otherwise, we undo the removal. The iteration stops when a preset threshold is achieved or only 4 pointpairs remain. This approach is very simple and easy to implement. It removes the incorrect point correspondences and removes outliers implicitly. The pseudo code of the Iterative Estimation Algorithm (IEA) is as follows: INPUT: 3D image I 1 and 3D image I 2, feature point candidate sets C 1 and C 2, and the threshold THRESHOLD OUTPUT: 3D transformation matrix M and the 1. Read I 1, I 2, C 1 and C 2 2. Select feature point pairs P = (P 1, P 2 ) with the Heuristic Rule, where P 1 C 1 and P 2 C 2. Denote by N FP the number of feature point pairs. 3. Initialize the mean square distance and the transformation matrix M with a predefined value. If N FP < 4, then go to Step Use least square method to solve the over-determined linear system with P, and get the 3D transformation matrix M. Create a new 3D image I 3, where I 3 =M *I 1 5. Calculate the between I 2 and I If < THRESHOLD, then go to Step 10. FOR (INDEX = 0; INDEX < N FP ; INDEX ++) 7. Remove a feature point pair with index = INDEX from P to create feature point pairs P, calculate the 3D transformation matrix M, create a new 3D image I 3, where I 3 = M * I 1 and calculate mean square distance between I 2 and I 3.

7 8. If < THRESHOLD or N FP <4, then M=M and go to Step If <, then =, P = P. Otherwise, undo the remove operation in Step 6. END FOR 10. Output the M and. The time complexity of the IEA is polynomial. Step 1, 3, 6 and 10 of the IEA has constant complexity. Step 4 and 5 has polynomial complexity. The complexity of Step 2 is O(N C1 * N C2 ), where N C1 and N C2 are the number of feature point candidates in C 1 and C 2. The complexity of Step 7-9 is O(N FP ) * (T LS +T ), where T LS is the complexity of least square method and T is the complexity of calculating. Both of T LS and T are polynomial. So the total time complexity of IEA is polynomial. To test our approach, we integrate it into our previous framework of 3D transformation estimation. Experiments in Section 6 show that this approach with IEA can estimate 3D transformation with higher accuracy. The new framework is described as follows: INPUT: 3D image I 1 and I 2, the threshold THRESHOLD, and the range of neighborhood searching NB MIN and NB MAX. OUTPUT: 3D transformation matrix between I 1 and I 2 Read I 1 and I 2, create their skeletons S 1 and S 2, Init M EST and EST with a predefined value FOR (NB = NB MIN ; NB <= NB MAX ; NB += 2) 1. Calculate the feature point candidate sets C 1 from S 1, C 2 from S 2, within current neighborhood scale NB 2. Call the IEA to get M and 3. If < THRESHOLD, then M EST = M and go to Step If EST >, then EST = and M EST = M. END FOR 5. Output the estimated 3D transformation matrix M EST In our experiments, NB MIN = 3, NB MAX = 21 and THRESHOLD =1.0. The experiments and the results are described in Section 6.

8 6. Experimental results We first applied our new method with IEA to estimate the 3D translations along the x-, y- and z-axes. All the translations are positive integers in [0, 5]. Fig. 5 shows the for 3D translations. Result shows the is always 0 in both of the previous method and the new method. Both method can locate feature point pairs precisely and estimate the translations without error. Next, we use our new method with IEA to estimate the 3D rotations in x-, y- and z-axis. All the rotation angles are positive integers in [0, 5]. Fig. 6 shows the for 3D rotations. Result shows that the new method with the IEA has smaller. We also applied our new method with IEA to estimate the combinations of translations and rotations with some random noises (Figs. 7-10). All the translations and rotation angles are positive integers in [0, 5]. The random noise is in the range of [0%, 5%]. Results show that the new method with IEA has smaller. Translations dx + dy + dz Fig. 5. for 3D translations Rotations alpha + beta + gama Fig. 6. for 3D rotations

9 Trans. and Rots. in x, y and z axis without noise dx + dy + dz + alpha + beta + gama Fig. 7. for 3D translations and rotations without noise Trans. and Rots. in x, y and z axis with 1% random noise dx + dy + dz + alpha + beta + gama Fig. 8. for 3D translations and rotations with 1% random noise 3 Trans.and Rots. in x, y and z axis with 2% random noise dx + dy + dz + alpha + beta + gama Fig. 9. for 3D translations and rotations with 2% random noise

10 Trans. and Rots. in x, y and z axis with 5% random noise dx + dy + dz + alpha + beta + gama Fig. 10. for 3D translations and rotations with 5% random noise 7 Conclusions and future work In this paper, we demonstrated how to use an iterative algorithm to improve the estimation accuracy of 3D transformations. Result shows that this method works quite well when the differences in orientation are small. This work is a preliminary approach for defining and tracking the volume changes of airways with surgery. Because of constraints on time, we do not have airway model after surgery. Therefore, we had to create some models artificially to test our algorithm. In the near future, we will obtain real models after surgery and then compare our results using data from before and after surgery. Acknowledgements The support of ASRA and NSERC in making this work possible is gratefully acknowledged. References 1. J.B.A. Maintz and M.A. Viergever, A Survey of Medical Image Registration, Medical Image Analysis, vol.2, p.1-36, L. G. Brown, A survey of image registration techniques, ACM Computing Surveys (CSUR), 24(4), p , P. J. Besl and N. D. Mckay, A method for registration of 3D shapes, IEEE Trans. Pattern Anal. Machine Intell. 14(2), p , João Maciel, Global matching: optimal solution to correspondence problems, PhD Thesis, Universidade Técnica de Lisboa,

11 7. D. Hearn and M. Baker, Computer Graphics, Pearson Education. 8. C. M. Ma and M. Sonka, A fully parallel 3D thinning algorithm and its applications, CVIU, 64 (3), p , A. Chaturvedi, Z. Lee, Three-dimensional segmentation and skeletonization to build an airway tree data structure for small animals, 50(7), Physics in Medicine and Biology, p , J. J. Leader, Numerical analysis and scientific computation, Pearson Addison Wesley, Boston, P. Torr. Motion Segmentation and Outlier Detection. PhD thesis, U. Oxford, Justin Domke and Yiannis Aloimonos, A Probabilistic Notion of Correspondence and the Epipolar Constraint, Third International Symposium on 3D Data Processing, Visualization and Transmission, June Wei Zhang and Jana Kosecka, Image based Localization in Urban Environments, Third International Symposium on 3D Data Processing, Visualization and Transmission, June 2006.