Using fractional gradient information in non-rigid image registration: application to breast MRI

Using fractional gradient information in non-rigid image registration: application to breast MRI A. Melbourne 1, N. Cahill 2,C.Tanner 1,M.Modat 1,D.J.Hawkes 1,S.Ourselin 1 1 Centre for Medical Image Computing, University College London, UK. 2 School of Mathematical Sciences, Rochester Institute of Technology, NY, USA. ABSTRACT This work applies fractional differentiation (differentiation to non-integer order) to the gradients determined from image intensities for enhanced image registration. The technique is used to correct known simulated deformations of volumetric breast MR data using two algorithms: direct registration of gradient magnitude images and an extension of a previously published method that incorporates both image intensity and image gradient information to enhance registration performance. Better recovery of known deformations are seen when using non-integer order derivatives: half-derivative breast images are better registered when these methods are incorporated into a standard diffusion-based registration algorithm. Description of Purpose Techniques making use of fractional differentiation are increasingly applied in biomedical applications: when modelling biomechanical deformations, 1 for edge detection in image processing 2, 3 and also for image registration regularisation. 4 6 Motivated by this work, we apply the technique to non-rigid image registration in order to explore the utility of fractional gradient information for image enhancement prior to and incorporated within a standard diffusion-based registration algorithm. Fractional differentiation extends the differential operator to non-integer and complex order between standard integer differentiation for example d 1/2 /dx 1/2 (d 1/2 /dx 1/2 )=d/dx. Results of the operator are easily obtained for many cases: generating phase shifts in periodic functions e iωt and ratios of generalised factorials for polynomials, x n. 7 d α dt α sin(ωt) = ωα sin(ωt + απ 2 ) (1) d α dt α eiωt = (iω) α e iωt (2) d α Γ(n +1) dx α xn = Γ(n α +1) xn α (3) d α dx α a = ax α Γ(1 α) (4) Automatic image registration algorithms typically optimise a correspondence by minimising a measure of image dis-similarity. A regularisation term is often used to smooth and propagate the local values of the image dis-similarity measure. The calculus of variations is often used to transform the registration problem into a system of partial differential equations (PDE s) for optical flow, diffusion, elastic and fluid equation based registration. 8 10 An alternative approach is to optimise the similarity measure and regulariser directly over a set of basis functions; locally supported splines have proved popular due to ease of implementation and computational efficiency. 11 A general image registration algorithm acts to minimise a total cost function d T combining a metric of image similarity d S, between two images A and B, weighted against a distance of the transformation d R from Send correspondence to Andrew Melbourne. E-mail: a.melbourne@cs.ucl.ac.uk. Medical Imaging 2012: Image Processing, edited by David R. Haynor, Sébastien Ourselin, Proc. of SPIE Vol. 8314, 83141Z 2012 SPIE CCC code: 0277-786X/12/$18 doi: 10.1117/12.911173 Proc. of SPIE Vol. 8314 83141Z-1

some ideal set of transformations with weighting μ. An explicit example for the sum-of-squared difference image similarity measure and an l2-norm regularisation penalty for a smooth velocity field v is given in (5) over a region of image overlap Ω. Throughout this work we solve the PDE for the velocity field v and this result is used to incrementally update the deformation field u using a forward finite difference in time. This cost-function is typically minimised using steepest descent, resulting in a PDE. Alternative regularisers result in a PDE for the curvature equation with the integrand of d R given by: 2 v 2 2 and for fluid regularisation with the integrand of d R = v 2 2 + v 2 2. [ d T = (A B(u)) 2 + μ v 2 2] (5) Ω Previously, a handful of authors have published image registration algorithms that make use of gradient images instead of registering intensity directly: the use of image intensity gradient information has been shown to aid the alignment of simple transformations of cranial T1, T2 and PET images by reducing local 12, 13 minima in the registration cost function. It can also be used when implicit directionality is present in the data (e.g. ultrasound data). 14 We present two methods for determining the fractional derivative of image intensities: the first an approximation using kernel filtering and the second an implementation of the G2-algorithm 7 and apply these within two algorithms for image registration: registration of fractional gradient images and a fractional implementation of a previously published algorithm, 12 originally applied to rigid registration and here extended to the non-rigid case. Fractional derivatives have a number of important properties: they are distributive, commutative (provided that indices have the same sign) and there exists a chain rule and product rule. In addition we can generalise the relationship of derivatives of functions to their Fourier transforms. Since imaging modalities generally provide discrete sets of data, it is useful to have finite difference approximations for fractional differentiation. By extending the notion of forward and central finite differences it is possible to obtain a generalised central difference formula, Δ α (6) for a given function f(j) whereα represents the order of differentiation over a possibly infinite number of terms n where ( ) α n is the generalised binomial coefficient. 7 ( ) α Δ α = ( 1) n f(j +( α n)) (6) n 2 n=0 Method Determining the fractional image gradient A kernel filtering method for determining fractional image gradients may be built around a Gaussian function: in 2D, g(x, y) exp( a(x 2 + y 2 )); this is often used to carry out image smoothing or whole-integer differentiation as an alternative to finite differencing. Figure 1 shows the smoothly varying family of functions for d α g(x, y)/dx α. We assume that the fractional image gradient may be determined using a standard convolution of the fractional-gaussian kernel with the image. Figure 1. Fractional differentiation of a 2D Gaussian kernel in d α /dx α : α =[0, 1 3, 2 3, 1, 4 3, 5 3, 2] Proc. of SPIE Vol. 8314 83141Z-2

An alternative method of discrete fractional differentiation is given by the G2-algorithm. 7 This recursive algorithm incorporates the non-local characteristic of fractional differentiation that may otherwise be suppressed using a narrow kernel by implementing a generalised finite difference formula. Figure 2 illustrates filters derived from the two methods. For 0 α 2, the convolution based method is likely to be reasonable approximation of the G2 algorithm. Figure 2. Comparison of image gradient finding methods applied to a slice from an example breast MRI. Top row: for kernel filtering (σ =0.67pixels). Bottom row: implementation of the G2 algorithm. 7 Labels denote order of differentiation in d α /dx α. In this work we attempt to register the fractional-gradient images directly using the magnitude of the gradient components: α B, and compare this with a method that weights the image intensity driving forces (for example from the mutual information or sum-of-squared differences) by a gradient dependent term. 12 The image gradient information is used to weight the standard image-similarity driving forces F(A, B(u)) for floating image B deformed by transformation u toward fixed image A (7). F new = ωf(a, B(u)) (7) Where the weighting function ω is given by (8). ( α A α B(u) ω = α A α B(u) 1 ) min[ α A, α B(u)] (8) 2 Throughout this work we restrict ourselves to the mono-modal case and use the sum-of-squared differences as the driving similarity measure. The driving gradients are thus given by (9). F(A, B(u)) = (A B(u)) B(u) (9) A registration algorithm is formed using the driving image forces above (9) which are subsequently regularised using a Fourier transform based diffusion equation solver 15 to produce a smooth velocity field that is used to update the displacement field using the material derivative; we then loop the steps until convergence. A fractional diffusion equation We solve the standard diffusion equation for driving image forces F = d S. Making use of the methodology in 15 we use the calculus of variations to find the discrete PDE: ˆ 2 v = F(A, B(u)), where ˆ denotes the discrete differentiation operator. We solve for the regularised velocity field v, so that we may update our Proc. of SPIE Vol. 8314 83141Z-3

displacement field u with larger deformations. We generalise the diffusion equation by substituting the general fractional central difference formula (6) to yield (10). ˆ α v = F(A, B(u)) (10) By assuming that both the force field F and the resulting velocity field v can be represented as linear combinations of eigenvectors φ k of the discrete fractional diffusion operator ˆ α such that v = k c kφ k and F = k d kφ k, the solution may be written as a linear equation (11). ( ) ˆ α c k φ k = λ k d k φ k (11) k k We now substitute the fractional central difference formula (6) in place of ˆ α in (11). For each individual eigenvector, φ k over spatial index j and α-dependent eigenvalue, λ k (α) the result is given in (12). ( ) α ( 1) n φ n k (j +( α 2 n)) = λ k(α)φ k (j) (12) n=0 We now seek the eigenvectors φ k and α-dependent eigenvalues λ k (α) of (12). To find a solution we use complex exponentials (13), enforcing periodic boundary conditions on potential solutions. Eigenvectors are represented as a product of a normalising scalar value A with exponential terms along each dimension d over spatial index j and eigenvalue index k with length of each dimension N d.forthis work we assume that in multiple dimensions we have a separable function and apply the fractional derivative independently in each dimension. 3 ( ) 2πikd j d φ k (j) = A exp (13) d=1 The corresponding eigenvalues of (13) are given in (14). Noting that the coefficients of ( α n) fall off rapidly for non-integer α, practically we truncate to the first 30 terms of the summation. ( ) [ α 3 ( λ k (α) = ( 1) n 2πikd exp ( α ) ] n N d 2 n) (14) n=0 Noting that (14) may be written as a binomial series, (15), it is possible to use the Taylor series of (1 + x) α to demonstrate the equivalence of this method with other regularisation methods. 4, 6 ( ) α (1 + x) α = x n (15) n We now follow the work of Cahill et al 15 in noting that the representation of the force and velocity fields as linear combinations of complex exponentials means that the coefficients, c k,d k, can be found by carrying out a Fourier transform, Φ, resulting in a simple division of each coefficient of Φ(F x ) by the corresponding eigenvalue λ k (α) for each spatial dimension d. Carrying out the inverse Fourier transform, Φ 1,oftheresult yields the solution of the velocity field (16). A consistent direction of the solution is ensured by taking the magnitude of the corresponding eigenvalues. v d = Ψ 1 (Ψ(F d )/ λ α ) (16) The registration algorithm is formed by iterating this process in time: driving image forces at a given time t, F t are found and regularised to produce a smooth velocity field, v t (16), the displacment field is then updated using the material derivative over a time step Δt, u t+1 = u t +Δt (v v u t ), and we may loop the steps until convergence. At each time point t, the velocity field is normalised by the maximum value of F t in order to compare results for different values of α. d=1 n=0 N d Proc. of SPIE Vol. 8314 83141Z-4

Data The resulting registration algorithms are tested on both simulated biomechanical deformations of 3D breast magnetic resonance (MR) data and simulated deformations applied to 3D brain MR. 16 Three-dimensional simulated deformations are generated using a finite element model of breast MRI data and deformations are modelled over five boundary displacement types to simulate differences due to a range of common breast imaging deformations: a simulation of a push-like force over a small area (AP); a simulation of single and double-sided plate-like compressions (OP, TP), and simulations of pectoral muscle movement from tension to relaxation and vice versa (PR, PT). Biomechanical deformations are applied at a maximum displacement magnitude of 10mm pixel dimensions are 1.37 1.37 4.2mm. We also test the algorithm on 12 simulated deformations of a T1 weighted brain volume from the Brainweb database. Test deformations are generated by registering this volume to 12 normal anatomical brain models using a free-form deformation algorithm. 11 For each breast dataset, each biomechanical simulation is applied in turn; for the brain volume we apply each registration transformation: in each case a known deformation u sim is applied to an undeformed image B, to produce a fixed image A. A low-level of Rician noise, σ, is also added to the deformed images (thus, A = B(u sim )+σ). Images are resampled using cubic interpolation. The residual deformation after registration of B A is measured by: R = 1 N N i u sim (i) u reg (i) 2 2 between u sim and the final registration transformation, u reg and is averaged over N displacement vectors within the masked breast or brain. Periodic boundary conditions may be used since typical breast and brain MR volumes are surrounded by large regions of background on five sides of the imaging volume; thus any influence from a wrapped transformation is likely to be constrained to these regions. Results Fractional gradient images We investigate the use of fractional gradient images for registration in place of direct intensity image registration. The progressive highlighting of features related to edges (see Figure 2) may guide the registration to a more accurate solution since it has the advantage of maintaining both gradient and image intensity information, similar to (7). 12 For each pair of gradient-images, we attempt to correct the known deformations using a standard diffusion-based registration algorithm using the SSD as detailed above. For this section we use the gradient magnitude applied to both the fixed and floating image and test the algorithm over a range 0 α 2 (with Gaussian width, σ = 1.33pixels). Alternative definitions will be investigated: using the gradient mean, maximum gradient component or other normalisations. Figure 3a-b show results when using the TP simulation. The residual deformations after 3D registration are shown in Figure 3b: the deformation residual for three cases is most reduced when using 0 <α<1, whilst for the remaining case (S2) the registration does not recover the original transformation. For all five deformation models and each of the four subjects: the median (and standard deviation) value of α with the lowest residual transformation is α =0.5 ± 0.25. Figure 3a shows both derivative and final difference images for the central slice of each volume after registration for α =[0, 0.6, 1], the optimal value for this subject is found to be α = 0.6; there are visible differences between this result and that applied to image intensities only α = 0. Failure of the algorithm for α>1 might be attributed to the increasing loss of image intensity information and it s replacement by pure gradient information. Figure 3c shows comparable results for the force-weighting method in (7-8): deformation residuals are shown for the four subjects under the AP force model: in this case the force weighting method progressively limits the convergence of the registration with increasing values of α. This is an unexpected result since the method visually highlights edge features in the forces found from (9) which would be expected to be important in driving the subsequent alignment. Fractional diffusion registration We now investigate the performance of the fractional diffusion registration algorithm when varying α. The range of α is chosen to be 0.6 α 4: the lower limit is chosen empirically since registration performance Proc. of SPIE Vol. 8314 83141Z-5

Figure 3. Using fractional image derivatives for registration. a) images after standard diffusion registration for α =[0, 0.6, 1] for the TP simulation (centre slice). α =0.6 is found to be the optimal value for image registration whilst α = 1 does not converge. b) residual transformations after registration for TP model for subjects S1-4 and c) residual transformations for the AP model using registration force weighting (7-8) for subjects S1-4. Left-most results are the unregistered deformation residuals. is monotonically worse for α<0.6 whilst the upper limit is chosen to be the standard curvature regulariser. In practice this upper limit gives a highly smooth velocity field and there is increasing influence from the periodic nature of the eigenvectors. Visually we see important differences between standard diffusion registration and the fractional case. For the breast deformation data, figure 4 shows three examples showing difference images before and after registration. The difference images for standard diffusion registration (α = 2) are visibly poorer when compared to the optimal cases as measured by the lowest deformation field residual. Non-integer α is better able to recover the known deformations in these cases. The advantages of more flexible choice of PDE are also seen for the brain data. Figure 5 shows three examples for the brain data, showing difference images before and after registration. As for the breast data, the difference images for standard diffusion registration (α = 2) are visibly poorer when compared to optimal cases selected using the lowest deformation field residual. Non-integer α is once again better able to recover the known deformations in these cases. We also compare the reduction in the deformation residual and change to the similarity measure for the 3D registrations. Figure 6a-c show the reduction in the deformation residual for three breast deformation models. Standard diffusion registration (α = 2) is seen to be close to an optimal result in most cases whilst residual values found for α<2 often show progressively worse correction of the applied deformation. For all four subjects and five deformation models, the median (and standard deviation) value of α with the lowest transformation residual is α =2.2 ± 0.44, although this result is not found to be significant in a simple t-test. Proc. of SPIE Vol. 8314 83141Z-6

Figure 4. Results after registration of three breast models (TP, AP & PT simulations) using the fractional diffusion algorithm (central slice of volume shown). Columns: reference image, difference image before and after registration for α =2andα corresponding to the lowest residual deformation (marked with *). Figure 7 shows three representative convergence plots of the SSD with iteration number for varying α using three breast deformation simulations. There is a general trend for more rapid convergence using larger α, thus non-integer α might also be used to improve computational performance. Non-integer values of α are also shown to yield improved registration when applied to deformed brain images. For all twelve cases the optimal α =1.8 ± 0.18 (median and standard deviation) and this is found to be significantly different from a distribution of mean α = 2. The optimal values of α for brain and breast data, although different, indicate that improved results can be obtained when using non-integer PDE s for image registration. Explicit differences are likely to be due to the nature of the applied deformations in each case. Conclusion & Future Work This work has shown that fractional gradient images can allow improved registration performance. Applied to breast MR data, improvements in performance are shown when using the half-derivative image, α =0.5±0.25 (for all 20 cases tested). The mode of action is likely to be the same as the second method tested (7), 12 although the computational performance is greater, since the gradient image calculation is a single preprocessing step. This method weights the image intensity driven forces by information from the gradient of the images; in the same way the half-derivative image illustrates a combination of image intensity and image gradient information (Figure 2). The figure of α =0.5 found in this work might be reflective of this combination of information, although it is not clear that 0.5 will be the optimal value in all circumstances. One method for avoiding the advance prediction of α would be to vary the order during the registration; thus adapting the extent of the edge-enhancement effect seen in Figure 2. Empirically, the choice of fractional parameter could be used to enhance the images when intensity contrast is insufficient or confounding. 2 We plan to extend the method to the non-rigid registration of serial or inter-subject brain MR and further test the effect of fractional gradient images on both objective and computational registration performance. Proc. of SPIE Vol. 8314 83141Z-7

Figure 5. Results after registration of three brain deformations (4, 8 & 9) using the fractional diffusion algorithm (central slice of volume shown). Columns: reference image, difference image before and after registration for α = 2 and α corresponding to the lowest residual deformation (marked with *). Figure 6. Deformation residual after registration for the fractional diffusion algorithm for each subject, S1-4, under the PR, OP and PT force models for varying α. This work further presented a unification of common regularisers used in medical image registration. The extension of the diffusion PDE to the fractional case has allowed improved registration in both applications tested. Applied to breast MR data, improvements in performance are shown when using α = 2.2±0.44whilst for the brain data all twelve cases are better registered using lighter regularisation, α = 1.8 ± 0.18. Although the optimal α for breast and brain data are different, the important result is that the standard diffusion regulariser is sub-optimal in these cases. In particular, a non-integer value of α gives improved performance when compared to the widely used diffusion and curvature regularisers. In practice, the differences in optimal α are likely to be correlated with the type of deformation applied. In the data used here, the registration transformation applied to the brain data is likely to be more convoluted than the smooth single-material biomechanical deformation applied to the breast data, thus a value of α<2 is better able to recover a registration-like transformation and vice versa. Hence, it is important to tune the algorithm according to the expected transformation and not restrict the choice of α to integer values. Proc. of SPIE Vol. 8314 83141Z-8

Figure 7. Convergence properties for the first 100 iterations during registration using the fractional diffusion algorithm for three breast registration examples when varying α. Three force models are shown for subjects labelled S1 & S3. For each case, larger α allows more rapid convergence. Interestingly the lowest SSD does not always correlate with the lowest deformation residual. It is conceivable that the value of α may be varied during the registration: varying the value of α for the fractional diffusion solution during the registration would be an elegant multi-resolution approach employing a smooth family of PDE based regularisers. Derivative order could be set by inspecting the deformation transformation during the registration; this would add an extra degree of flexibility to the algorithm and one that has not yet been fully explored. We also plan to extend this work to a derivation of a fractional fluid registration algorithm. The enhanced flexibility of a fully fractional algorithm could be used to improve population based studies making use of segmentation propagation, since the flexibility of the transformation can be dynamically adjusted during the propagation. The techniques described here may be applied to other areas. Fractional techniques could provide additional information when intensity contrast is insufficient or confounding. 2 The Jacobian matrix of first-order derivatives and the Hessian matrix of second-order derivatives are often exploited to improve registration performance, thus there should be a natural generalised description for non-integer order. There are many future possibilities for the technique developed here. The use of non-integer order differentiation when constructing PDE s for registration regularisation has revealed not only that it can be used to improve registration performance, but that traditional integer order regularisation may be sub-optimal for medical image registration. Acknowledgements This work was funded by the European 7th Framework Program, HAMAM, ICT-2007.5.3 and EPSRC grant EP/E031579/1 and the Computational Biology Research Centre (CBRC) Strategic Investment Award (Ref. 168). REFERENCES [1] F. Meral, T. Royston, and R. Magin, Fractional calculus in viscoelasticity: An experimental study, Communications in Nonlinear Science and Numerical Simulation 15, p. pp. 939945, 2010. [2] B. Mathieu, P. Melchior, A. Oustaloup, and C. Ceyral, Fractional differentiation for edge detection, Signal Processing 83, p. pp. 24212432, 2003. [3] A. C. Sparavigna and P. Milligan, Using fractional differentiation in astronomy, arxiv 1, pp. 1 8, 2009. [4] J. Larrey-Ruiz, J. Morales-Sanchez, and R. Verdu-Monedero, Generalized regularization term for nonparametric multimodal image registration, Signal Processing 87, pp. 2837 2842, 2007. Rapid Communication. [5] K. Kashu, Y. Kameda, M. Narita, A. Imiya, and T. Sakai, Continuity order of local displacement in volumetric image sequence, in Workshop on Biomedical Image Registration, B. Fischer, B. Dawant, and C. Lorenz, eds., LNCS 6204, pp. 48 59, Springer-Verlag, 2010. Proc. of SPIE Vol. 8314 83141Z-9

[6] A. Melbourne, N. D. Cahill, C. Tanner, and D. J. Hawkes, Image registration using an extendable quadratic regulariser, in Proc. IEEE Int Biomedical Imaging: From Nano to Macro Symp, pp. 557 560, 2011. [7] K. Oldham and J. Spanier, The Fractional Calculus, Dover Publishing, 2006 (1974). [8] J. P. Thirion, Image matching as a diffusion process: an analogy with Maxwell s demons., Medical Image Analysis 2(3), pp. 243 260, 1998. [9] B. Fischer and J. Modersitzki, Curvature based image registration, Journal of Mathematical Imaging and Vision 18, p. 81:85, 2003. [10] G. Christensen, R. Rabbitt, and M. Miller, Deformable templates using large deformation kinematics, IEEE Transactions on Image Processing 5, pp. 1435 1447, 1996. [11] M. Modat, G. Ridgway, Z. Taylor, M. Lehmann, J. Barnes, D. Hawkes, N. Fox, and S. Ourselin, Fast free-form deformation using graphics processing units, Computer Methods and Programs in Biomedicine 98(3), pp. 278 284, 2010. [12] J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever, Image registration by maximization of combined mutual information and gradient information, 19(8), pp. 809 814, 2000. [13] E. Haber and J. Modersitzki, Intensity gradient based registration and fusion of multi-modal images, in MICCAI, Springer-Verlag, ed., LNCS 4191, pp. pp. 726733,, 2006. [14] A. Roche, X. Pennec, M. Rudolph, D. P. Auer, G. Malandain, S. Ourselin, L. M. Auer, and N. Ayache, Generalized correlation ratio for rigid registration of 3d ultrasound with mr images, in MICCAI, LNCS 1935, p. pp. 567577, Springer-Verlag, 2000. [15] N. D. Cahill, J. A. Noble, and D. J. Hawkes, Fourier methods for nonparametric image registration, in Computer Vision and Pattern Recognition, 2007. [16] C. Tanner, J. A. Schnabel, D. L. G. Hill, D. J. Hawkes, A. Degenhard, M. O. Leach, D. R. Hose, M. A. Hall-Craggs, and S. I. Usiskin, Quantitative evaluation of free-form deformation registration for dynamic contrast-enhanced MR mammography., Medical Physics 34(4), pp. 1221 1233, 2007. Proc. of SPIE Vol. 8314 83141Z-10