A 2D-3D Image Registration Algorithm using Log-Polar Transforms for Knee Kinematic Analysis

Transcription

1 A D-D Image Registration Algorithm using Log-Polar Transforms for Knee Kinematic Analysis Masuma Akter 1, Andrew J. Lambert 1, Mark R. Pickering 1, Jennie M. Scarvell and Paul N. Smith 1 School of Engineering and Information Technology, The University of New South Wales, Canberra, Australia. masuma.akter@student.adfa.edu.au The Trauma and Orthopaedic Research Unit in the Department of Surgery The Canberra Hospital, Canberra, Australia. Abstract In medical imaging, D-D registration approaches have been used for image guided surgery (IGS) and pre-, post-operative measurement of joint kinematics. The application addressed in this paper is the registration of single-plane D fluoroscopy images to D Computed Tomography (CT) data. A limitation of previously proposed D-D registration algorithms has been an inability to register images with large initial displacements. The new algorithm proposed in this paper uses log-polar transforms in the frequency domain to extend the range of initial displacements that can be registered automatically. Our experiment results show that the proposed approach can increase the range of initial displacements up to ± mm and ±9. This range of motions is significantly greater than any previously proposed multi-modal D-D registration algorithm. I. INTRODUCTION Multi-modal image registration is the process of spatially aligning two images regardless of the sensors used, the time of capture, position and angle. This type of registration has been used in many applications such as in tools for medical diagnosis, computer vision and remote sensing. In medical imaging applications, D fluoroscopy to D CT image registration algorithms have been adopted for radiotherapy, radiosurgery, orthopaedic surgery, interventional neuroradiology, vascular interventional radiology and kinematic analysis. The application for D-D registration we are addressing in this paper is the kinematic analysis of knee joints. The outcomes of this type of analysis are useful for comparing the D motion of bones in the knee joint before and after total knee arthroplasty surgery and also for surgery to repair ruptured ligaments. The registration procedure is required to estimate the D pose of the bones in the knee that best matches the D projection of the same bones captured by the fluoroscopy equipment. This D pose is described using six rigid-body parameters (three translations and three rotations). To date the algorithms which provide the best performance for estimating the optimum set of these six parameters are iterative in nature [1] []. These algorithms follow a gradient-descent based approach and gradually translate and rotate the segmented bones depicted in the CT scan. The algorithm continues until a good match is found between the actual D projections and simulated D projections created from the bone at its current pose. These simulated projections are known as digitally reconstructed radiographs (DRRs). The quality of the match is determined using a multi-modal similarity measure such as mutual information (MI) normalized cross correlation (NCC) or the sum of conditional variances (SCV) [1]. However, a major limitation of the gradient-descent based approach is the limited range of initial displacements for which the algorithm is able to successfully converge to the global minimum of the similarity measure. For large initial displacements these algorithms tend to converge to a local minimum of the similarity measure which may provide a very poor match between the DRR and the fluoroscopy frame. This limitation affects the performance of kinematic analysis algorithms since the algorithms cannot automatically follow the position of the bones in a knee joint from one frame to the next in a fluoroscopy video. Instead the registration must be manually initialized for each frame of the video. One option for increasing the range of initial displacements for which successful registration can be achieved is to adopt a feature-based registration approach []. In feature matching approaches, initially isolated feature keypoints (such as Harris corners) or descriptors (as in the scale invariant feature transform, (SIFT)) are extracted from the images, then the correspondences of the descriptors in each image are measured using a distance metric such as the Hausdorff or Euclidian distance. However, feature-based D-D registration approaches are very sensitive to the accuracy of the corner and descriptor selection process. An additional disadvantage is the higher computational complexity required to find the points in D that correspond to anatomical landmarks in D as this step requires exhaustive searches to obtain correspondence between features. Furthermore, the accuracy of this type of methods is not sufficient for high precision medical application, since the location of the features is typically limited to a precision of a single pixel or greater. A second alternative to gradient-based approaches is the use of the log-polar transform (LPT) in the spectral domain to estimate the scale and rotation between images. Spectralbased phase-correlation D-D image registration has been shown to be robust to noise and intensity variations in illumination or radiation exposures thanks to the Fourier magnitude normalization [4]. However this approach has not previously /1/$1. 1 IEEE

2 been applied to real multi-modal D-D registration problems. The approach proposed in [5] provided some initial promising results for the registration of pre-segmented D fluoroscopy images of a human vertebra to a segmented CT volume of the same vertebra. However, in practical scenarios for kinematical analysis of knee joints, the fluoroscopy image cannot be segmented to provide only the bones of the knee. Hence the presence of other bones and soft tissue in the fluoroscopy frame and the different imaging modalities pose a significant challenge to applying the LPT approach for multi-modal D- D registration. In this paper we will present a new registration algorithm that uses a combination of the LPT in the spectral domain and a gradient-descent based approach. By using the LPT in the spectral domain the algorithm is able to converge to the global minimum of the similarity measure for large initial in-plane displacements. The out-of-plane displacements are then estimated using a gradient-descent based method. A novel approach to pre-processing the images allows the algorithm to successfully register real-world images where the fluoroscopy frame is unsegmented and contains extra spectral information due to the presence of the soft-tissue of a normal knee joint. Experimental results show that the new algorithm is able to successfully register images with much larger initial displacements than the standard gradient-descent based algorithm. An additional benefit of our approach is the reduced computational complexity required since the initial inplane displacements are effectively estimated using only one iteration. The remainder of this paper is organized as follows: Section II discusses the spectral domain registration technique using the Fourier-Mellin theorem to show the property of invariance to rotation and scale. Section III describes our new methods using the log-polar approach. Experimental results are provided in the section IV to evaluate the performance of our proposed algorithm. Finally, section V contains our conclusions. II. REGISTRATION USING THE LOG-POLAR TRANSFORM Fourier domain registration has been shown to be invariant to scale, rotation and translations as well as providing good robustness to noise and computational efficiency [6]. A. Fourier-Mellin Theory According to the Fourier-Mellin Theory (FMT) [4], the Log-Polar Transform (LPT) [7] can be applied to convert the magnitude spectrum of an image from Cartesian space to Log- Polar (LP) space. In this space, rotation and scale changes are indicated by vertical and horizontal shifts. Assume F 1 (α, β) and F (α, β) are the spectra of I 1 (x, y) and I (x, y). Here, I (x, y) is a rotated, scaled and translated version of I 1 (x, y) with a scale factor of s, rotation r and translation (t x, t y ) such that I 1 (x, y) and I (x, y) are related by the following equation: I 1 (x, y) =I (sx cos r + sy sin r t x, sx sin r + sy cos r t y ) (1) X-ray Source Fig. 1. Y X D Obj ect Z X-ray Calibration grid Image Intensifier (XY image plane) Perspective projection model of the Fluoroscopy generation Taking the Fourier Transform of both sides of (1) results in the following equation: F 1 (α, β) = 1 s e jπ(αtx+βty) F ( α s x cos r + α s y sin r, α s x sin r + α s y cos r ) () From (), we can see that the translation in the spatial domain only affects the phase of the spectra and rotation and scale affects the magnitude of the spectra. After taking the magnitude of () to suppress the phase component, the magnitude spectra as a function of rotation and scale is given by: F 1 (α, β) = 1 s F ( α s x cos r + α s y sin r, α s x sin r + α s y cos r ) () Converting the spectra in () into polar co-ordinates results in the equation: F 1 (ρ, θ) = F ( ρ s,θ r ) (4) ρ = (x x c ) +(y y c ) (5) θ = y y c (6) x x c The polar coordinates (ρ, θ) are defined by (5) and (6), where θ and ρ denote the angle and radial distance from the center (x c,y c ) of the image respectively. From (4), we can see that the angular displacement between the images in the spatial domain is represented as a shift in the frequency domain polar image. Finally, transforming (4) into the log-polar space results in the equation: F 1 (log(ρ),θ) = F (log(ρ) log(s),θ r ) (7) and from (7) we can see that a change in scale is now also represented by a shift in log-polar space.

3 Two Images Estimate R z Estimate T x and T y GD SCV GD: Gradient Descent process SCV: Sum of Conditional Variance No Search Compete? Yes Global Minimum T x, T y, T z, R x, R y, R z Fig.. The proposed D-D registration algorithm. D Volume D DRR FFT LPT Scale P 1 =Peak θ =P1-P Target Image FFT LPT Scale P =Peak Update DRR Fig.. The process for estimating the in-plane rotation, R z. B. DRR Generation Fig. 1 shows the mechanism for capturing fluoroscopy images. The x-ray beams in the fluoroscopy unit diverge from a common point as they pass through the object and arrive at the image intensifier. Consequently, as an object moves closer to the x-ray source, it will produce a larger image at the image intensifier. This means that translation in the z-direction will result in a change in scale of the object in the fluoroscopy frame. Hence, the out-of-plane translation can be estimated by measuring the small amount of zoom that must be applied to the DRR to register it with the fluoroscopy frame. This change in scale is usually difficult to measure precisely as a large translation in the z-direction will result in only a very small scale change in the fluoroscopy frame. In our procedure, to simulate this process, the CT volume is first transformed using a projective transform which simulates the effect of the diverging x-rays in the fluoroscopy capturing process. The pixel values of the transformed volume are then summed along the z-direction to produce a D projected DRR. III. THE PROPOSED D-D REGISTRATION ALGORITHM The CT volume is first converted into a D DRR that is to be registered with the fluoroscopy image. The x-y plane is considered to be the projection plane of the DRR with the origin in its center. Hereafter, the motion parameters required for kinematic analysis are expressed as translation in the T x (anterior-posterior), T y (proximal-distal), T z (medial-lateral) directions and rotation about the x, y, and z axes denoted by R x (abduction/adduction), R y (internal/external) and R z (flexion/extension) respectively. Fig. shows the overall registration algorithm that combines translation estimation in the

4 Fluoroscopy (tibia) D DRR image (tibia) Fluoroscopy (tibia) D DRR image (tibia) Fluoroscopy (femur) D DRR image (femur) Fluoroscopy (femur) D DRR image(femur) Fig. 4. Input images (tibia and femur) used in this paper. Fig. 5. LoG filtered versions of the input images. log-polar frequency domain (for estimating R z ), translation estimation in the frequency domain (for estimating T x, T y ) and a gradient-descent approach in the spatial domain (for estimating the three out-of-plane parameters T z, R x and R y ). In the following subsections, a detailed explanation of each block in this diagram is provided. A. Pre-processing Before the registration algorithm is performed, the images must be pre-processed to enhance the registration performance. After generating the DRR from the segmented CT scan, the images are thresholded to remove noise and highlight the most dominant edges and then filtered using a Laplacian-of- Gaussian (LoG) filter. The registration is performed in three stages and the width of the LoG filter is reduced in each stage to provide progressively sharper edges in the images. A Kaiser window is applied to the images to reduce the effect of the image boundary on the spectra of the images. Fig. 4 shows the fluoroscopy and DRR images used as inputs into the registration algorithm and Fig. 5 shows these input images after the pre-processing operations have been applied. B. Estimate R z Fig. shows a block diagram of the process for estimating the in-plane rotation, R z. The LPT is first applied to the magnitude spectrum of each image. The magnitude spectra were mapped to the LP domain using bi-cubic interpolation. In our experiments we found that the location of the NCC peak for the LP spectra was sensitive to out-of-plane rotations R x and R y. So, to increase the stability of this stage, we developed a new process to compute the rotational difference of the LP mapped images. Since we only need to estimate the in-plane rotation at this stage, instead of finding the peak location of the NCC surface, the LP spectrum is summed along the scale direction for the DRR and the fluoroscopy images using equations (8) and (9) respectively N s G 1 (θ) = F 1 (log(ρ k ),θ) (8) k=1 Ns G (θ) = F (log(ρ k ),θ) (9) k=1 where F 1 (log(ρ),θ) and F (log(ρ),θ) are the log-polar mapped spectra for the DRR and the fluoroscopy image respectively and N s is the size of F 1 and F in the scale direction. Then, the difference between the locations of the maximum peaks in G 1 and G is taken as the estimated in-plane rotation for the two images. The position of the maximum peaks of G 1 and G are found using (1) and (11) as follows: P 1 = arg max(g 1 ) (1) θ P = arg max(g ) (11) θ Finally, the estimated in-plane rotation between the two images in the Cartesian space is given by: θ d = P 1 P (1) Fig. 6 shows an example of the values for the summation in the scale direction, G 1 and G for the DRR and fluoroscopy images shown in Fig. 5 when the in-plane rotation was. Our experimental results showed better stability for both tibia and femur images using this approach than when finding the location of the NCC peak. The DRR image is updated according to this value of rotation.

5 Sum of LP spectra Method of the Rz computation Fluoroscopy DRR Rotation in degree Fig. 6. C. Estimate T x and T y Summed up images in the scale axis After rectification of the in-plane rotation, the masked registration approach proposed by Dirk Padfield in [8] is used to estimate the translation between the magnitude spectrum of both images. Let I 1 (x, y) and I (x, y) denote the DRR and fluoroscopy images respectively and assume m 1 (x, y) and m (x, y) are binary mask images of the same size as I 1 (x, y) and I (x, y). The mathematical model used to calculated the masked NCC surface in [8] is given by: where NCC = N D 1 D (1) N = F 1 (F 1.F ) F 1 (F 1.M ).(F 1 (M 1.F )) F 1 (M 1.M ) D 1 = D = (14) F 1 (F (I 1.I 1 ).M ) (F 1 (F 1.M )) F 1 (M 1.M ) (15) F 1 (M 1.F (I.I )) (F 1 (M 1.F )) F 1 (M 1.M ) (16) and F 1, F, M 1, M are the spectra of I 1 (x, y), I (x, y), m 1 (x, y) and m (x, y) respectively. I (x, y) is I (x, y) rotated by 18, and. indicate the complex conjugate of the spectra and the point-to-point multiplication between the two spectra respectively. F () and F 1 () denote the FFT and inverse FFT operations. The offset of the peak from the center of the NCC surface given by (1) represents the translational shift between the two images in the spatial domain. Using these measured values of T x and T y, the D DRR image is transformed to remove the residual translation shift between the images. D. Gradient Descent Optimization Procedure and Similarity Measure In any D-D registration implementation, the most computationally expensive aspect is the updating of the D volume for each estimate of a new D position. This part of the algorithm typically involves interpolation to find the pixel values at every position in a large D volume. Estimating the values of T x, T y, and R z as described in our proposed approach is very efficient as only a single iteration is required and the D volume is only updated once. After this step, a gradientdescent approach similar to that proposed in [] is applied for iterations to estimate the out-of-plane parameters (R x, R y and T z ) as well as fine-tuning the in-plane parameters. For each iteration, the new DRR image is compared to the reference image using the SCV multi-modal similarity measure [] between Laplacian-of-Gaussian (LoG) filtered versions of the two images as shown in Fig. 5. Observe from this figure that the strong edges contain the majority of the information required for the successful similarity measurement between the images. If the value of SCV for this iteration is less than the current minimum, the value of SCV is stored together with the values of T x, T y, T z, R x, R y, and R z as the optimal set of motion parameters. IV. EXPERIMENTAL RESULTS To evaluate the performance of the new registration algorithm we compared the algorithm with the approach which used only Gauss-Newton gradient-descent based optimization to minimize the SCV similarity measure for D-D fluoroscopy to CT registration. This approach was shown in [] to provide the best performance among several competing approaches [9] [1] for the application addressed in this paper. The range of initial displacements tested in [] however was limited to ±5 mm for all translations and ±5 for all rotations. The algorithms were used to register segmented D CT data of a tibia and a femur to synthetic fluoroscopy images that were generated using the CT data. The synthetic fluoroscopy images were generated using a logarithmic attenuation function on the sum of the pixel values of the un-segmented CT data along rays that simulated the path of a point source of X- rays. The use of the logarithmic attenuation function and the inclusion of soft tissue and other bones produces a synthetic fluoroscopy image that has almost identical properties to a real fluoroscopy frame. Fig. 4 shows the synthetic fluoroscopy frames and the DRR produced from the CT data at the same position for femur and tibia bones. The exact D position of the CT data that was used to produce the synthetic fluoroscopy image is known and is used as the gold standard for measuring registration errors. To test the algorithm, a known set of D rigid body transforms were applied to the segmented CT data of the bones. Each of the algorithms was then used to register this data to the synthetic fluoroscopy frame. The true D rigid body transform parameters that align the CT data with the fluoroscopy frame are all exactly zero, so the goal of the registration algorithms was to produce a final set of transform parameters that are

6 7 6 LPT_translation_error GD_translation_error 6 5 LPT_rotation_error GD_rotation_error Translation in mm 5 4 Rotation in degree Number of iteration Number of iteration Fig. 7. Average translation error at each iteration for the femur. Fig. 9. Average rotation error at each iteration for the femur. 9 8 LPT_translation_error GD_translation_error 8 7 LPT_rotation_error GD_rotation_error Translation in mm Rotation in degree Number of iteration Number of iteration Fig. 8. Average translation error at each iteration for the tibia. Fig. 1. Average rotation error at each iteration for the tibia. as close to zero as possible. Hence, the registration error was calculated as the root-mean-square (RMS) error between the known and estimated parameters and is given in (17) and (18) below: Tx + Ty + Tz E t = (17) Rx + Ry + Rz E r = (18) The values for the translational and rotational parameters were measured in mm and degrees respectively. A set of 1 initial D positions for each bone was used with transform parameters uniformly randomly distributed between ± mm for T x, T y and T z, ± for R z and ±1 for R x and R y. The registration algorithm was assumed to have failed if any of the final parameters had a magnitude greater than 1.. The proposed approach was successful for 94% and 86% of the initial positions for the femur and tibia respectively, while the gradient-based approach using SCV was successful for only 48% and 1% for the femur and tibia respectively. Fig. 7, 8, 9 and 1 show the average registration error of the best estimate of the D position for each D volume update for the two algorithms. Since the success rate of the gradient based procedure for the tibia is only 1%, the results for only this case are shown. These graphs show that using the LPT approach in the first iteration allows the proposed algorithm to converge much more rapidly from large initial displacements than when using only a gradient descent approach. Table I shows the final average error for each of the six rigid body parameters for the proposed LPT algorithm and the gradient descent (GD) only approach proposed in []. Again these results are shown only for the cases when the algorithms were successful. These results show that, for the

7 TABLE I AVERAGE FINAL REGISTRATION ERROR. Translation(mm) Rotation( ) T x T y T z R x R y R z LPT (femur) GD (femur) LPT (tibia) GD (tibia) TABLE II REGISTRATION ERROR FOR LARGE INITIAL DISPLACEMENTS. Error (tibia) Error (femur) initial value LPT GD LPT GD +1 (-1) () 9.67 (-1.8) () 8.58 (-19.8) T x + (-) (-1) 9.68 (19.7) () (-19.87) (mm) + (-) 8 (-1). (19.9) () (-.) +1 (-1) () -.4 (-.4) () 9.18 (-9.9) T y + (-) (-1) (-.) () (-19.18) (mm) + (-) - (-1) (.) (7) 19.7 (-19.) +1 (-1) 1 (-1) 4.4 (-5.65) (1).9 (-.95) + (-) 1 (-).11 (-.8) (1) 19. (-16.75) + (-) (-) 19.9 (-19.59) () 19.7 (-17.18) +4 (-4) 1 (-) 19.5 (-1.79) (-7) (-17.1) R z +5 (-5) 1 (-) (-19.49) -1 (-7) 19. (-17.) ( ) +6 (-6) 4 (-4) 19. (-19.18) -1 (-8) 19.1 (-17.44) +7 (-7) (-4) 19. (-18.86) -1 (-9) 19.8 (-17.47) +8 (-8) (-4) (-18.65) - (-9) (-17.96) +9 (-9) (-4) (-18.48) - (-9) 19.9 (-17.88) cases when the algorithms were successful, the final error of the proposed algorithm was not significantly greater than the gradient descent approach for any of the six parameters. These results demonstrate that using the LPT approach to increase the range of initial displacements for which the proposed algorithm is successful has not reduced the overall accuracy of the algorithm. Table II shows the performance of the two algorithms for large initial values of the in-plane parameters. The error for the gradient descent approach is the error after ten iterations of the first stage of this algorithm. While the error for the proposed LPT approach is the error after a single iteration. These results show that the proposed LPT approach is robust to large initial displacements and can estimate in-plane rotations of up to 9 and in-plane translations of up to mm with acceptable error in a single iteration. Alternatively the gradient descent based approach fails to converge to the global minimum for in-plane rotations of more than 1 and in-plane translations of 1 mm or more. Instead this approach converges to a local minimum for these large initial displacements. V. CONCLUSION In this paper, we have presented a D-D image registration technique for D kinematic analysis of knee joints. The rotation and scale invariant characteristics of the Log- Polar Fourier domain allow us to develop a robust D-D registration algorithm that can produce accurate estimates for three out of the six rigid body motion parameters in a single iteration. Our experiment results show that the proposed approach can successfully register images with a range of initial displacements of up to ± mm and ±9. This range is significantly greater than any previously proposed multi-modal D-D registration algorithm. The proposed algorithm will allow faster and more robust kinematic analysis for designing and monitoring artificial knee replacements and pre- and postoperative diagnosis of knee dysfunction. REFERENCES [1] A. Muhit, M. Pickering, T. Ward, J. Scarvell, and P. Smith, A comparison of the D kinematic measurements obtained by single-plane D-D image registration and RSA, in Engineering in Medicine and Biology Society (EMBC), 1 Annual International Conference of the IEEE, 1 1-sept. 4 1, pp [] M. Pickering, A. Muhit, J. Scarvell, and P. Smith, A new multi-modal similarity measure for fast gradient-based D-D image registration, IEEE Annual International Conference of Engineering in Medicine and Biology Society, pp , 9. [] D. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, vol. 6(), pp , 4. [4] B. Reddy and B. Chatterji, An FFT-based technique for translation, rotation, and scale-invariant image registration, IEEE Transactions on Image Processing, vol. 5(8), pp , 1996.

8 [5] M. Freiman, O. Pele., A. Hurvitz, M. Werman, and L. Joskowicz, Spectral based D/D X-ray to CT image registration, proceddings of SPIE, pp , March 11. [6] G. Tzimiropoulos, V. Argyriou, S. Zafeiriou, and T. Stathaki, Robust FFT-Based Scale-Invariant Image Registration with Image Gradients, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. (1), pp , 1. [7] J. Sarvaiya, S. Patnaik, and S. Bombaywala, Image registration using log-polar transform and phase correlation, IEEE Region 1 Conference, 9. [8] D. Padfield, Masked Object Registration in the Fourier Domain, IEEE Transactions on Image Processing, vol. 1(5), pp , 1. [9] F. Wang and B. Vemuri, Non-Rigid Multi-Modal Image Registration Using Cross-Cumulative Residual Entropy, International Journal of Computer Vision, vol. 74, pp. 1 15, 7. [1] F. Maes, D. Vandermeulen, and P. Suetens, Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximization of mutual information, Medical Image Analysis, vol., pp. 7 86, 1999.