Non-Parametric Bayesian Registration (NParBR) on CT Lungs Data - EMPIRE10 Challenge

Non-Parametric Bayesian Registration (NParBR) on CT Lungs Data - EMPIRE10 Challenge David Pilutti 1, Maddalena Strumia 1, Stathis Hadjidemetriou 2 1 University Medical Center Freiburg, 79106 Freiburg, Germany 2 UMIT, Institute for Biomedical Image Analysis, Eduard-Wallnoefer-Zentrum 1, A-6060 Hall in Tirol, Austria david.pilutti@uniklinik-freiburg.de Abstract. The alignment of intra-patient lung CT data is a challenging task due to the highly deformable nature of the lungs as a result of breathing motion on their fine structure. This requires non-rigid registration techniques to obtain satisfactory alignment. We propose a novel Non-Parametric Bayesian Registration (NParBR) method to efficiently perform non-rigid registration while operating at full spatial resolution. The NParBR method assumes that the spatial misregistration causes a Gaussian smoothing on the joint intensity statistics. This is incorporated in a Bayesian formulation and the statistics are restored with a Wiener filter. The restored statistics are back-projected into spatial domain and also spatially regularized with Gaussian filtering. To evaluate the stateof-the-art non-rigid registration methods, the EMPIRE10 challenge [4] has been proposed. The NParBR method has been tested on the EM- PIRE10 datasets and resulted in good alignment in several cases. Keywords: Non-rigid registration, joint statistics restoration, lungs registration, multi-contrast registration 1 Introduction Computed Tomography (CT) images are often used in clinical procedures to highlight internal structures of the human body with high detail. The visual analysis of CT images of the lungs can be complicated by the highly deformable nature of the organs. Breathing can also affect the fine texture of the alveoli structure in the lungs with complex displacement. Therefore, non-linear registration approaches are necessary to align lung images that can be subsequently analyzed for tasks such as disease progression, pulmonary functionality analysis, and image guided treatments in case of 4D images. We propose a novel image registration method based on the assumption that the spatial misregistration smooths the joint intensity statistics. This is incorporated in a Bayesian formulation, where the joint intensity statistics are restored non-parametrically and then back-projected to the image domain to estimate the registration. Our formulation performs a dense estimation of the

Fig. 1: Diagram describing the registration of two images with the proposed registration method. A preliminary rigid and affine registration is performed and the result is used to initialize the iterative non-rigid registration step until the stop criterion is met. spatial transformations and spatial smoothness only once per iteration, making the time performance orders of magnitude more efficient in comparison to other state-of-the-art methods [6]. The non-parametric restoration of the joint intensity statistics removes the effect of the misregistration, and can also preserve the form of the statistics. Thus, the resulting continuous spatial registration can be approximately volume preserving [1]. This property is desirable when registering organs or lesions whose volume is not expected to be changed from motion such as breathing, heart beat, and peristalsis. The proposed method

produced good qualitative results on several EMPIRE10 datasets [4] with an efficient computational time. The general methodology has been also submitted for publication [5]. 2 Method A pairwise registration is between a reference image I ref and a moving image I mov. A spatial transformation T = (u x, u y, u z ) from I ref to I mov is estimated to provide a registered image as I reg = I mov (T 1 (x)) where x = (x, y, z) are the spatial coordinates. The registration can accommodate a variable contrast. The method allows the registration over a limited Region Of Interest (ROI) over the image for which the contrast is intended for and is meaningful. In this work the spatial misregistration is assumed to cause a distortion of the joint intensity statistics that is considered to be Gaussian smoothing, which is deconvolved with Wiener filtering. An analogy can be drawn between the effect of spatial misregistration and the physical phenomenon of the ferromagnetic hysteresis [2, 6, 11]. Similarly to the lack of cross distributions in ferromagnetic hysteresis or their secondary role, any cross distribution is also ignored in our method. Their magnitude is lower compared to the effect of noise, spatial resolution, and other imaging artifacts. This assumes a smoothness for anatomy in space and a larger size for anatomic structures compared to that of the extent of the misregistration. Misregistration is also assumed to be spatially smooth. As a pre-processing step, the two images I ref and I mov are normalized in terms of their dynamic range to a maximum value of 255. This preserves the form of the histogram, expedites the transformations estimation, and it is convenient for the application of the fast Fourier transform used for applying the Wiener filter to the statistics. The method then performs an additional pre-processing step concatenating a rigid and an affine registration. The result is then used to initialize the subsequent non-rigid registration. The implementation of the non-rigid registration of pairs of images involved the deconvolution of the joint intensity statistics with the Wiener filter, its backprojection to the spatial domain that gives the maximum likelihood Bayesian estimation of the spatial transformation, and the spatial regularization obtained with a Gaussian filter. The implementation interleaves between the maximum likelihood and the spatial smoothness iteratively, k = 0,..., K 1 for a total of K iterations. An overview of the registration is given with the diagram shown in fig. 1. 2.1 Computation of Joint Intensity Statistics A neighborhood is defined as N (x) = x + x, where x represents all possible shifts within N. The joint intensity statistics are calculated by relating the intensities of each voxel x in I ref to each voxel in I mov at the corresponding spatial locations x + x and accumulating the occurrences of each intensity pair throughout the images. This results in a bidimensional statistics, which

can also differentiate between different tissue types, grouped by intensity change properties. It is assumed that two images I ref and I mov of the same anatomy under perfect alignment would give rise to the joint histogram H ideal. The joint statistics H actual of the misregistered images are considered to result from the convolution of H ideal with a 2D Gaussian filter G δ (i; 0, σ δ ): H actual = H ideal G δ (i; 0, σ δ ) + n δ, (1) where is the convolution operator, i = (i, j) is an intensity pair, σ δ standard deviation of G δ, and n δ is the noise. is the 2.2 Bayesian Restoration of Joint Intensity Statistics The total registration vector v(x) at x is assumed to consist of the underlying correct registration vector field u(x) and the misregistration vector field d(x), v(x) = u(x) + d(x). (2) Under the assumption that p(u(x)) and p(d(x)) are independent, where p denotes a probability distribution, it follows that: p(v(x)) = p(u(x)) p(d(x)), (3) where is the convolution operator. Each spatial location x in I ref is linked to a cubic spatial neighborhood N (x) in I mov. The assumption of texture uniformity within tissues leads to the probability distributions of the actual displacement v(x) and the ideal displacement u(x) as: p(v(x)) = H actual (I ref (x), I mov (x + v(x))) (4) p(u(x)) = H ideal (I ref (x), I mov (x + u(x))), (5) where H actual and H ideal are the joint histograms of two images I ref and I mov in the actual case and under assumed correct alignment, respectively. The distortion p(d(x)) assumed Gaussian can also be expressed as p(d(x)) = G(i; 0, σ δ ), (6) which is the distortion of the joint intensity statistics modeled by a Gaussian distribution. The conditional expectation of the assumed correct displacement u(x) given the initial displacement v(x) using Bayes law for p(u v) [9, 10] is: p(v u)p(u)udu E[u v] = p(u v)udu =. (7) p(v u)p(u)du It is assumed that the Fourier transform F( ) of the probability of the correct displacement F(p(u)) can be estimated from F(p(v)) by deconvolution with a Wiener filter R [10]. In the Fourier domain the Wiener filter R is defined as R = G G 2 + ɛ, (8)

where G is the Fourier transform of a Gaussian distribution for p(d(x)) and ɛ is the inverse of the signal to noise ratio of the statistics. That is Moreover, F(p(u)) = RF(p(v)). (9) p(v u) = p(v u) = p(d) = G(i; 0, σ δ ) (10) is the Gaussian distribution that models the distortion of the joint intensity statistics. The inverse Fourier transform of the Wiener filter R to the intensity range domain is defined as r = F 1 (R). Substituting Eq. (9) and Eq. (10) into Eq. (7) gives: p(v u)r p(v)udu G(i; 0, σδ )r p(v)udu E[u v] = = p(v u)r p(v)du G(i; 0, σδ )r p(v)du. (11) The substitution of Eq. (4) into Eq. (11) gives G(i; 0, σδ )r H actual udu E[u v] = G(i; 0, σδ )r H actual du. (12) Eq. (12) can be discretized with u x N to give: E[u v] = Σ N G(i; 0, σ δ )r H actual x Σ N G(i; 0, σ δ )r H actual. (13) 2.3 Backprojection for Spatial Misregistration The expected value E[u v] of the distortion allows the computation of the expected distortion d(x) at x as: E[d(x)] = E[d(x) v(x)] = E[v(x) u(x) v(x)] = E[v(x) v(x)] E[u(x) v(x)] = v(x) E[u(x) v(x)] = v(x) Σ N G(i; 0, σ δ )r H actual x Σ N G(i; 0, σ δ )r H actual. (14) This is an initial estimate of the misregistration calculated using the spatial neighborhood N. 2.4 Estimation of Smooth Spatial Transformation At the starting iteration k = 0 the vector v(x) is initialized to 0 everywhere. Following Eq. (13) and Eq. (14), the restored joint intensity statistics are backprojected to space to give an initial spatial transformation E[d(x)] k at iteration k > 0 to obtain T k(x) that is a rough estimate of the spatial transformation T k(x) = T k 1 (x) + E[d(x)] k, (15) where T k 1 (x) is the displacement from the previous iteration. The cumulative Bayesian estimate of the transformation T k is smoothed with a 3D Gaussian

filter G(x; 0, σ S ), where σ S is the standard deviation of the smoothing of the spatial transformation T k. The final estimate of the spatial transformation at iteration k is obtained with T k (x) = T k(x) G(x; 0, σ S ). (16) The value of T k (x) is calculated for all x in the ROI of I ref (x). 2.5 Implementation The method has been implemented in C ++ and operates over the 3 spatial dimensions of the data. The pairwise non-rigid registration developed is preceded by the rigid and affine registration methods provided by ITK [3] using their default settings such as for subsampling and interpolation. The Wiener filtering in Eq. (8) and Eq. (14) of the joint intensity statistics was performed in the Fourier domain using the forward and backward FFT provided by ITK [3]. The value of σ δ has been set for all datasets to 3 and is accumulated along the iterations to give the total Gaussian distortion of the joint intensity statistics. The value of ɛ of the Wiener filter has been set to 0.1 for all datasets. The spatial regularization G(x; 0, σ S ) has been performed using the efficient separable implementation of the 3D Gaussian filter from ITK [3]. The value of σ S has been set equal to the length of the side of an in-plane voxel in 3D for all datasets. A maximum number of k max = 100 iterations is enforced for the non-rigid registration. The non-rigid registration has been performed within the ROIs provided by the EMPIRE10 challenge [4] segmented using the method of van Rixoort et al. [7]. 3 Experiments and Results The NParBR method has been tested on all 30 CT image pairs of lungs provided by the EMPIRE10 challenge [4]. Each dataset consist of two images taken from the same subject, and requires intra-patient registration. The 30 datasets are subdivided into 6 different categories to cover a representative range of practical cases. Eight image pairs consist of two inspiration scans (I/I), eight image pairs consist of breathhold inspiration/expiration (I/E), four cases consist of two individual phases of a 4-D dataset (4D), four ovine datasets (Ov), two contrast/non-contrast scan pairs (Co), and finally four artificially warped scan pairs (Wa). Due to the possible large displacements of the fine structures of the lungs, which can be smaller than the displacement, the NParBR can be applied only to a subset of the provided datasets where the displacements of the fine structures are small enough. The resulting alignment for the aforementioned datasets was satisfactory after visual inspection. In Fig. 2 is an example of the application of

(a) Reference (b) Moving (c) Before Registration (d) After rigid/affine (e) After NParBR Fig. 2: An axial section from the volumetric registration of the dataset 05 from the EMPIRE10 challenge [4]. In (a) and (b) are the input reference and moving images, respectively. Finally in (c), (d), and (e) are the resulting overlaps before the registration, after rigid/affine registration, and after NParBR non-rigid registration, respectively. The red arrows point at structures that are better aligned after the NParBR non-rigid registration. the NParBR method over dataset number 05. The lung structures highlighted with the arrows show an improved alignment after the non-rigid registration step compared to both non registered and only rigid/affine registered images. Further validation is based on the prescribed evaluation protocol of the EMPIRE10 challenge [4] and will appear in the EMPIRE10 challenge website. 4 Summary and Discussion The registration of CT images of lungs is a challenging task mainly due to the highly deformable nature of the considered organs. Breathing can affect the fine structure of the alveoli with complex and potentially large displacement. The alignment of intra-patient lung images can be very useful for medical procedures such as the evaluation of disease progression, pulmonary functionality analysis, and image guided treatment in case of 4D images. The main contribution of this work has been to develop a novel non-rigid registration method based on a nonparametric Bayesian formulation for the estimate of the misregistration and its removal. The presented method has been developed to efficiently perform dense non-rigid registration on images representing objects with volume preserving transformations such as many of the motions of the organs in a body as well as for possible tumor lesions.

The proposed method can accommodate datasets of both same as well as variable image contrast and more general contrast inversion cases. The implementation is iterative and results in an effective deconvolution of the joint intensity statistics that only requires a single computation of the joint intensity statistics and spatial smoothing of the estimated registration per iteration. The method restores the joint intensity histogram non-parametrically by removing the effect of the misregistration and preserving the form of the joint intensity statistics. The restored statistics are then back-projected to the image domain. The statistical restorations together with spatial smoothness provide a spatially continuous registration that can accommodate approximate volume preservation for different anatomic regions inherently without retrospectively imposing a volume preservation constraint that can still suffer from local shearing. This approach reduces artifacts such as the shrinking and sinking effect on local regions, which can help to maximize a global measure such as MI [8], while avoiding excessive regularization such as volume preserving constraints. A misregistration particularly multicontrast can be accounted for if it gives rise to a distribution in the statistics whose size is sufficiently large compared to the value of ɛ of the Wiener filter. This made the NParBR method applicable only on a subset of the EMPIRE10 datasets. This formulation gives a performance for the NParBR method that can be orders of magnitude faster in comparison to other commonly used non-rigid registration methods while operating at full spatial resolution with comparable or better accuracy [6]. The method has been tested on the particular case of lungs registration proposed by the EMPIRE10 challenge and has shown good results in several datasets. References 1. Hadjidemetriou, E., Grossberg, M.D., Nayar, S.K.: Histogram preserving image transformations. International Journal of Computer Vision 45(1), 5 23 (2001) 2. Hill, D.L., Studholme, C., Hawkes, D.J.: Voxel similarity measures for automated image registration. In: Proc. of SPIE, vol. 2359. pp. 205 216. International Society for Optics and Photonics (1994) 3. Ibanez, L., Schroeder, W., Ng, L., Cates, J.: The ITK Software Guide. Kitware, Inc. ISBN 1-930934-15-7, second edn. (2005) 4. Murphy, K., Van Ginneken, B., Reinhardt, J.M., Kabus, S., Ding, K., Deng, X., Cao, K., Du, K., Christensen, G.E., Garcia, V., et al.: Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge. IEEE Trans. on Medical Imaging 30(11), 1901 1920 (2011) 5. Pilutti, D., Strumia, M., Büchert, M., Hadjidemetriou, S.: Non-parametric Bayesian registration (NParBR) of body tumors in DCE-MRI data. IEEE Trans. on Medical Imaging, under revision 6. Pilutti, D., Strumia, M., Hadjidemetriou, S.: Bi-modal non-rigid registration of brain MRI data with deconvolution of joint statistics. IEEE Trans. on Image Processing (2014) 7. van Rikxoort, E.M., de Hoop, B., Viergever, M.A., Prokop, M., van Ginneken, B.: Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection. Medical Physics 36(7), 2934 2947 (2009)

8. Rohlfing, T.: Image similarity and tissue overlaps as surrogates for image registration accuracy: widely used but unreliable. IEEE Trans. on Medical Imaging 31(2), 153 163 (2012) 9. Simoncelli, E.P., Adelson, E.H.: Noise removal via Bayesian wavelet coring. In: Proc. of IEEE ICIP. vol. 1, pp. 379 382 (1996) 10. Sled, J.G., Zijdenbos, A.P., Evans, A.C.: A non-parametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Trans. on Medical Imaging 17(1), 87 97 (1998) 11. Studholme, C., Hill, D.L., Hawkes, D.J.: Using voxel similarity as a measure of medical image registration. In: Proc. of BMVC. pp. 1 10 (1994)