Doctoral Thesis Proposal. A High Performance Computing Approach to Registration in Medical Imaging

Transcription

1 Doctoral Thesis Proposal A High Performance Computing Approach to Registration in Medical Imaging Gabriel Mañana Guichón National University Colombia gjmananag@unal.edu.co Director: Prof. Eduardo Romero Castro National University Colombia edromero@unal.edu.co November 10, 2005 i

2 Abstract The proposed research will be devoted to the study of the performance related issues associated to automated software registration systems in medical imaging. Registration, in this context, is the determination of a spatial transformation that aligns points in one view of a region of the anatomy, with corresponding points of the same region in another view. One major problem with most image registration techniques is their high computational cost. Because of this, these methods have found limited application to clinical situations where fast execution is required, e.g., intra-operative imaging. High performance can be achieved by reduction in data space, reduction in solution search space and parallel processing. This research aims to obtain high performance by taking advantage of grid computer architectures and exploiting inherent parallel techniques such as those found in genetic and evolutionary computation. ii

3 Contents 1 Introduction and Objectives Background Image Registration Process Grid Computing Problem Definition Objectives Review, Synthesis and Significance Image Registration Similarity Measures Registration Methods Registration Using Basis Functions Registration Using Splines Registration Using Thin-Plate Splines Registration Using B-Splines Elastic Registration Fluid Registration Registration Using FEM and Mechanical Models Registration Using Optical Flow Non-physical Models Synthesis Significance Research Method Algorithm Parallelization Experimental Design Projected Timeline and Resources Activities Completed Projected Timeline Available and Required Resources Expected Results Computational Framework Practical Implementations Expected Limitations Clinical Applicability iii

4 1 Introduction and Objectives Technological advances in medical imaging in the past two decades have enabled doctors to create images of the human body and its internal structures with unprecedented resolution and realism. Conventional radiography, state-of-the-art Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Single Photon Emission Computed Tomography (SPECT), Positron Emission Tomography (PET), and other imaging devices can quickly acquire two and three-dimensional images, and these images can further be computed to merge into a single image, volume or sequence, thus combining the information of all modalities. This combination or fusion of images requires the images to be previously aligned or registered. By registering two images, the fusion of multimodality information becomes possible, changes in the body can be detected, and therefore clinical activities like diagnosis, treatment planning and follow-up can be enhanced. The present proposal is structured as follows: in this Introduction, the image registration process is presented as an optimization problem and a brief review of grid computing is given. Next, a formulation of the research problem is established. General and specific thesis objectives follow. Next section, Review, Synthesis and Significance, presents a classification model based on eight criteria and briefly reviews the different registration methods. Next, a synthesis of the existing image registration algorithms is made along with a short analysis of their computational cost. This section ends with a discussion about the relevance of registration in medical imaging and some of its most important applications. The third section, Research Method, presents the methodology that has been followed so far and that will be applied for the rest of this research. Section fourth, Projected Timeline and Resources, describes the plan for the research, including activities already completed, as well as the resources required for its completion. Finally, sections fifth and sixth, Expected Results and Expected Limitations respectively, present the results that are expected to be achieved as well as the limitations that may result. Figure 1: PET, CT, PET/CT fusion. 1

5 1.1 Background This research deals with the high computational cost of image registration and the relevance of this restraint in medical imaging. This section reviews the overall registration process and introduces grid computing as a cost-effective alternative to overcome this difficulty Image Registration Process Any image registration technique can be described by three main components [1]: 1. a transformation which relates the reference and floating images, 2. a similarity measure which measures similarity between reference and transformed image, 3. an optimization scheme which determines the optimal transformation as a function of the similarity measure. Geometrical transformation refers to the mathematical forms of the geometrical mapping used in the registration process and can be classified by complexity into rigid transformations, where all distances are preserved, and deformable or non-rigid transformations where images are stretched or deformed. While the first is ideal for most fusion applications, and accounts for differences such as patient positioning, non-rigid transformations are used to take into account more complex motions, such as breathing or heart beating. The similarity measure is the driving force behind the registration process, and it aims to maximize the similarity between both images. From a probabilistic point of view, it can be viewed as a likelihood term that expresses the probability of a match between the reference and transformed image [2]. The main similarity measures used for image registration will be briefly reviewed below. Like many other problems in computer vision and image analysis, registration can be formulated as an optimization problem whose goal is to minimize an associated energy or cost function [3]: C = C similarity + C transformation (1) where the first term characterizes the similarity between the images and the second term characterizes the cost associated with particular deformations. From a probabilistic point of view, the cost function in Equation 1 can be can be explained in a Bayesian context. In this framework, the similarity measure can be viewed as a likelihood term which expresses the probability of a match between the two images, and the second term can be interpreted as prior which represents a priori knowledge about the expected deformation. In the case of rigid or affine registration this term is normally ignored and only plays a role in non-rigid registration. Several approaches can be used to optimize this function. They go from the use of standard numerical methods to the use of evolutionary methods, including some hybrid approaches. No matter what method is used, this always implies an iterative process whose computational cost is so high that prevents most applications from performing appropriately in real time situations. One possible way to solve this issue is to devise faster algorithms. Another way is to exploit the intrinsic parallelism that most methods convey. 2

6 1.1.2 Grid Computing The last decade has seen a considerable increase in commodity computer and network performance, mainly as a result of faster hardware and more sophisticated software. Nevertheless, there are still algorithms associated to problems in the fields of science, engineering and business, which cannot be dealt effectively with the current generation of supercomputers [4]. In fact, due to their size and complexity, these problems are often numerically and/or data intensive and require a variety of heterogeneous resources that cannot be provided by a single machine. A number of teams have conducted studies on the cooperative use of geographically distributed resources conceived as a single powerful virtual computer [5]. This alternative approach is known by several names, such as, metacomputing, seamless scalable computing, global computing, and more recently by Grid Computing. Internet and Grid based systems, whether their purpose is computation, collaboration or information sharing, are all instances of systems based on the application of fundamental principles of distributed computing. Grid computing is a set of standards and technologies that academics, researchers, and scientists around the world are developing to help organizations take collective advantage of improvements in microprocessor speeds, optical communications, raw storage capacity, and the Internet. By using the technique to disaggregate their computer platforms and distribute them as network resources, researchers can vastly increase their computing capacity. Linking geographically dispersed and heterogeneous computer systems can lead to staggering gains in computing power, speed, and productivity. For the last four years, the author of this proposal has been working on the development of a grid computing infrastructure that profits from idle CPU cycles of the workstations that are part of the National University campus at Bogotá, Colombia [7]. Given that the campus comprises several thousand computers with high heterogeneity, Jini TM [6] was chosen as the architectural foundation of the grid. Jini is a platform agnostic network architecture for the construction of distributed systems based on network-centric services that are highly adaptive to change. The features and properties of this technology, such as spontaneous networking and service discovery, leasing, distributed events and transactions, security and service-oriented programming models, make it a very suitable base for creating dynamic, reliable grid systems. Currently, the grid is in operational status and has been used to work on multiple scientific problems whose computational cost would have been prohibitive otherwise. More detailed information can be found at Problem Definition As mentioned, one major problem with advanced image registration techniques is their high computational cost. Because of this restraint, these methods have found limited application to clinical situations where real time or near real time execution is required, e.g., intra-operative imaging or image guided surgery. High performance in image registration can be achieved by reduction in data space, reduction in solution search space and parallel processing. Reduction in data space can be achieved, for instance, by performing registration using only subimages or exploiting multiresolution representations such as those provided by the wavelet transform. Reduction in solution search space can be achieved by using an iterative refinement search such as hill-climbing or simulated annealing. The previous techniques can reduce significantly the registration time without compromising registration accuracy. However, to obtain a significant increase in per- 3

7 formance, these approaches must be complemented with parallel processing. At the same time, parallel processing has always been associated with extremely expensive supercomputers or high performance clusters, almost equally expensive and unaffordable for most medical institutions. 1.3 Objectives The primary objective of this research is to devise a distributed computational framework that would allow the parallelization and execution of existing sequential registration algorithms, such as those developed in [8] and [9]. Particular effort will be placed on achieving this in an affordable way, i.e., avoiding the necessary use of high-performance hardware or costly hybrid machines, and taking advantage of existing computational infrastructure present in most medical institutions. More specifically, several registration algorithms will be studied further over the parallelization framework of a functional computational grid. This study will produce a set of the most effective strategies of parallelization of these and similar algorithms. In order to validate the usefulness and effectiveness of the implemented framework, three main different problems will be addressed: Affine registration in subtraction radiography: dental follow-up Non-rigid registration for monomodal images: subtraction in mammography Non-rigid registration for multimodal images: fusion of MRI and SPECT data Some questions that should be addressed are: What are the basic principles behind image registration? What could be the most appropriate similarity measures for the previous registration problems? What could be the most appropriate geometrical transformation methods for the previous registration problems? What could be an appropriate and adaptive optimization method for the different registration problems? What is the actual speed-up obtained for the parallelized algorithms? What are the basic problems behind the parallelization of registration algorithms? 2 Review, Synthesis and Significance 2.1 Image Registration From an operational point of view, the goal of image registration is to produce, as output, a geometrical transformation that aligns corresponding points of two given views. Generally, this transformation is then used as input to an image fusion system. In this section, we review the different 4

8 registration methods and present a reasonable classification. Registration may be classified in several different ways. Maintz [10] proposes a nine-dimensional scheme, that can be condensed into the following eight categories [11]: image dimensionality, registration basis, geometrical transformation, degree of interaction, optimization procedure, image acquisition modalities, subject, and object. Image dimensionality refers to the number of geometrical dimensions of the image spaces involved, which in medical applications are typically two and three-dimensional, but may include time as a fourth dimension. The registration basis is the aspect of the two views used to effect the registration. For example, the registration might be based on a given set of point pairs that are known to correspond (landmarks) or the basis might be a set of corresponding surface pairs. In some cases, these correspondences are derived from objects that have been attached to the anatomy, expressly to facilitate the registration process. Registration methods that are based on such attachments are termed prospective or extrinsic methods, and in contrast, those which rely on anatomic features only are termed retrospective or intrinsic. When there are no known correspondences as input, intensity patterns in the two views are used for alignment, a basis known as intensity that has recently become the most widely used registration basis in medical imaging. The category geometrical transformation refers to the mathematical forms of the geometrical mapping used to align points in one space with those in the other. These include rigid transformations, which preserve all distances, i.e., transformations that preserve the straightness of lines and hence planarity of surfaces, and all angles between straight lines. Images are rotated and translated in two or three dimensions in the matching process, but not stretched or deformed in any way. This is ideal for most fusion applications, and accounts for differences such as patient positioning. Registration problems that are limited to rigid transformations are called rigid registration problems. In deformable or non-rigid registration images are stretched or deformed to take into account complex motions, such as breathing, and any changes in the shape of the body or organs, which may occur following after surgery, for example. Non-rigid transformations are important not only for applications to non-rigid anatomy, but also for interpatient registration of rigid anatomy and intrapatient registration of rigid anatomy when there are non-rigid distortions in the image acquisition procedure. These include scaling transformations, with a special case when the scaling is isotropic, known as similarity transformations; the more general affine transformations that preserve the straightness of lines and planarity of surfaces, as well as parallelism, but change the angles between lines; the even more general projective transformations that preserve the straightness of lines and planarity of surfaces, but no parallelism; perspective transformations, a subset of the projective transformations, required for images obtained by techniques such as x-ray, endoscopy or microscopy, and the curved transformations which do not preserve the straightness of lines. In two-dimensional problems, the most used curved transformation is known as the thin-plate spline. It was first proposed by Harder[16] for designing aircraft wings and first employed in 1988 by Goshtasby [15] to describe deformations within the plane. These thin-plate splines are examples of a more general category of radial basis functions, which in the case of having compact support, are used to rectify deformations in two or three-dimensional spaces. Degree of interaction refers to the control exerted by a human operator over the registration algorithm. The ideal situation is the fully automatic algorithm, which requires no interaction, and that is a central focus of this research. The optimization procedure is the method by which the function that measures the alignment of 5

9 the images, is maximized. The parameters that make up the registration transformation can either be computed directly, i.e., determined in an explicit fashion from the available data, or searched for, i.e., determined by finding an optimum of some function defined on the parameter space [10]. The more common situation here is that in which a global extremum is sought among many local ones by means of iterative search. Popular techniques include standard numerical methods like Powells method (Levin et al., 1988), the Downhill Simplex method (Hill et al., 1991) and gradient descent methods (Zuk et al., 1994), as well as evolutionary methods like genetic algorithms (Hill et al., 1993) and simulated annealing (Liu et al., 1994). Modalities refers to the means by which the images to be registered are acquired. Twodimensional images are acquired for instance by x-ray projections captured on film or digitally, and three-dimensional images are acquired by tomographic modalities such as CT, MR or PET. In medical applications the object in each view is some anatomical region of the body. In all cases we are concerned primarily with digital images stored as discrete arrays of intensity values. Registration methods used for like modalities are typically distinct from those used for differing modalities. They are called monomodal and multimodal registration, respectively. Subject refers to patient involvement and comprises three categories: intrapatient, interpatient and, atlas. If the views are acquired from the same patient, the problem is that of intrapatient registration, when the views come from different patients, the problem is that of interpatient registration. Atlas refers to registrations between patients and atlases, which are themselves derived from patient images. Finally, object refers to the particular region of anatomy to be registered (head, lungs, vertebrae, etc.). 2.2 Similarity Measures As mentioned before, when there are no known correspondences between the images, registration is accomplished using the intensity patterns present in the views. To calculate the optimum transformation, intensity-based registration uses only pixel (2D) or voxel (3D) values of the images to be registered. This transformation is determined by iteratively optimizing some similarity measure calculated from all pixel or voxel values. Intensity-based registration algorithms can be used for several applications: registering images with the same or different dimensionality, registering using rigid and/or non-rigid transformations, and registering both intramodality and intermodality images. One of the aims of recent research in this area has been to devise general algorithms that will work on a wide variety of image types, without application-specific preprocessing [11]. All these algorithms are iterative, so there exists a chain of similarity measures that need to be optimized. In this section, the main similarity measures used for intensity-based image registration will be briefly reviewed and the applications in which they are used will be described. In cases when the images to be registered are expected to be identical, except for misalignment, or when they differ only by Gaussian noise, Viola [14] shows that the sum of squares of intensity distance (SSD) is the optimum similarity measure. The SSD will be zero when the images are aligned and will increase with misalignment. Problems like serial registration of MR images or functional MR experiments, where only a small number of voxels are expected to change during the study, can be considered reasonably close to this ideal case. However, this approach can fail if the data diverges too much from the ideal case. When a functional correlation can be assumed, i.e., when the intensities in the images are 6

10 linearly related, then the correlation ratio (CR) can be shown to be the ideal similarity measure[14]. Examples of intramodality applications like subtraction radiography come sufficiently close for this to be a useful measure, e.g. [33]. The Radio-Image Uniformity (RIU), initially devised by Woods [12] for registration of multiple PET images, is an alternative intramodality registration measure that calculates a radio image by dividing each voxel of the reference image by each voxel of the transformed image. The uniformity of this image is then determined by calculating its normalized standard deviation. The algorithm that uses RIU iteratively determines the transformation that minimizes the normalized standard deviation, i.e., maximizes uniformity. This measure is widely used in PET-PET and serial MR applications. Woods has also proposed a modified version of the previous algorithm for MR-PET registration [13]. The Partioned Intensity Uniformity (PIU), makes the assumption that all pixels with a particular MR pixel value represent the same tissue type so that values of corresponding PET pixels should also be similar to each other. The algorithm therefore minimizes the normalized standard deviation of PET voxel values for each MR intensity value (or partitioned intensity value). Statistical classifiers have been widely used in MR image analysis for segmentation of multispectral data for many years. In these approaches, a joint histogram is constructed from two images that are correctly aligned (e.g.: the first and second echo images from a spin-echo acquisition). A joint histogram is n-dimensional where n is the number of images used to generate it. The axes of the histogram are the intensities (or intensity partitions) in each image, and the value at each point in the histogram is the number of voxels with a particular combination of intensities in the different spectral components. If the joint histogram is normalized, it becomes an estimate of the joint probability distribution function (PDF) of intensities in the n images. The Shannon entropy H, widely used as a measure of information in many branches of engineering, was originally developed as part of information theory in the 1940s and describes the average information supplied by a set of symbols {s} whose probabilities are given by {p(s)}. H = s p(s) log p(s). Entropy will be maximum when all symbols s have equal probability. The use of entropy and other information-theoretic measures for image registration came about after inspection of joint histograms and PDFs. When the images are correctly aligned, the joint histograms have tight clusters, surrounded by large dark regions. These clusters disperse as the images become less well registered (Figure 2). Misregistration, therefore, results in an increase in histogram entropy. As a consequence of this observation, it was proposed that the entropy of the PDF calculated from the reference and floating images should be iteratively minimized to register them. Entropy minimization just described is not a robust voxel similarity measure for all types of image registration. The problem is that the PDF from which the joint entropy is calculated is defined only for the region of overlap between the two images. The range and distribution of intensity values in the portion of either image that overlaps with the other is a function of the transformation. The change in overlap produced by the transformation can lead to histogram changes that mask the clustering effects previously described. The solution to this difficulty is to use the informationtheoretic measure mutual information (MI) instead of entropy H [14]. MI normalizes the joint entropy with respect to the partial entropies of the contributing signals. In terms of image registra- 7

11 Figure 2: Joint histogram of two identical images: (a) without translation, (b) 1mm. horizontal translation, (c) 4mm. horizontal translation. tion, this measure takes account of the change in the intensity histogram of the images produced by the transformation. 2.3 Registration Methods In order to determine the transform parameters that best align two images, various methods have been developed. Since determining transform parameters plays a dominant role in image registration, the methods determining transform parameters are popularly recognized as image registration methods. Classified by complexity, two broad kinds of methods are applied: rigid and non-rigid registration methods. The main difference between rigid and non-rigid registration methods is in the nature of the transformation. An image coordinate transformation is called rigid when only translations and rotations are allowed [10]. The goal of rigid registration is then to find the six degrees of freedom (three rotations and three translations) of transformation T : (x, y, z) (x, y, z ) which maps any point in the reference image into the corresponding point in the floating image. An extension of this model is the affine transformation model which has twelve degrees of freedom (DOF) and allows for scaling and shearing: x a 00 a 01 a 02 a 03 x y y T (x, y, z) = z 1 = a 10 a 11 a 12 a 13 a 20 a 21 a 22 a By adding additional degrees of freedom, such a linear transformation model can be extended to nonlinear transformation models. For example, the quadratic transformation model is defined z 1 8

12 by second order polynomials: x a a 08 a 09 T (x, y, z) = y z = a a 18 a 19 a a 28 a whose coefficients determine the 30 DOF of the transformation. In a similar fashion this model can be extended to higher order polynomials such as third (60 DOF), fourth (105 DOF), and fifth-order polynomials (168 DOF). However, their ability to recover anatomical shape variability is often quite limited since they can model only global shape changes and cannot accommodate local shape changes. In addition, higher order polynomials tend to introduce artifacts such as oscillations, and therefore, they are rarely used Registration Using Basis Functions To describe the deformation field, instead of using a polynomial as a linear combination of higher order terms, a linear combination of basis functions θ i can be used: x a a 0n θ 1 (x, y, z) T (x, y, z) = y z = a a 1n a a 2n. θ n (x, y, z) Orthogonal basis functions are independent of each other and each of them contribute to the function definition in a certain unique way. Orthonormal basis functions can be viewed as unit orthogonal vectors in the function space and they satisfy { 0 for j k Ψ j (x) Ψ k (x) = 1 for j = k x Ω where Ω is the complete domain. A common choice is to represent the deformation field using a set of orthonormal basis functions such as Fourier (trigonometric) basis functions or wavelet basis functions. In the case of trigonometric basis functions this corresponds to a spectral representation of the deformation field where each basis function describes a particular frequency of the deformation Registration Using Splines The term spline originally referred to the use of long flexible strips of wood or metal to model the surfaces of ships and planes. These splines were bent by attaching different weights along their lengths. A similar concept can be used to model spatial transformations. For example, a 2D transformation can be represented by two separate surfaces whose height above the plane represents the displacement in the horizontal or vertical direction [16]. x 2 y

13 Many registration techniques using splines are based on the assumption that a set of corresponding points or landmarks can be identified in the source and target images. These corresponding points are often referred to as control points. At these control points, spline-based transformations either interpolate or approximate the displacements which are necessary to to map the location of the control point in the target image into its corresponding counterpart in the source image. The interpolation condition can be written as: T (φ i ) = φ i i = 1,..., n (2) where φ i denotes the location of the control point in the target image and φ i denotes the location of the corresponding control point in the source image. There are a number of different ways to determine the control points. For example, anatomical or geometrical landmarks which can be identified in both images can be used to define the spline-based mapping function. In addition, Meyer et al. suggested updating the location of control points by optimization of a voxel similarity measure such as mutual information. Alternatively, control points can be arranged with equidistant spacing across the image, forming a regular mesh. In this case the control points are only used as a parameterization of the transformation and do not correspond to anatomical or geometrical landmarks (pseudo- or quasi-landmarks) Registration Using Thin-Plate Splines Thin-plate splines are part of a family of splines that are based on radial basis functions. They have been formulated by Duchon and Meinguet for the surface interpolation of scattered data. Radial basis functions (RBFs) are the natural generalization of coarse coding to continuous-valued features. We want to approximate a real valued function f(x) by s(x) given the set of values F = (f 1,..., f N ) at the distinct points X = (x 1,..., x N ) R d. We can choose s( x) to be a Radial Basis Function of the form s(x) = p(x) + N λ i φ ( x x i ), i=1 x R d where p is a polynomial of degree at most k, λ i is a real-valued weight, denotes the Euclidean norm, φ is a basic function, φ : R + R, and x x i is simply a distance. An RBF is a weighted sum of translations of a radially symmetric basic function augmented by a polynomial term. The basic function φ, in this context, is a real function of a positive real r, where r is the distance (radius) from the origin. Radial basis function splines can be defined as a linear combination of n radial basis functions θ (s): t (x, y, z) = a 1 + a 2 x + a 3 y + a 4 z + n b j θ ( φ j (x, y, z) ) Defining the transformation as three separate thin-plate spline functions T = (t 1, t 2, t 3 ) T yields a mapping between images in which the coefficients a characterize the affine part of the spline-based transformation, while the coefficients b characterize the nonaffine part of the transformation. 10 j=1

14 The interpolation conditions in Equation (1) form a set of 3n linear equations. To determine the 3(n + 4) coefficients uniquely, 12 additional equations are required. These 12 equations guarantee that the nonaffine coefficients b sum to zero and that their crossproducts with the x, y and z coordinates of the control points are likewise zero. In matrix form this can be expressed as: ) ( ) Φ ( Θ Φ Φ T 0 ) ( b a Here a is a 4 x 3 vector of the affine coefficients a, b is a n x 3 vector of the nonaffine coefficients b, and Θ is the kernel matrix with Θ ij = θ ( φ i φ j ). Solving for a and b yields a thin-plate spline transformation which will interpolate the displacements at the control points. The radial basis function of thin-plate splines is defined as: { s 2 log (s) in 2D θ (s) = s in 3D Modeling deformations using thin-plate splines has several advantages: They can be used to incorporate additional constraints such as rigid bodies or directional constraints. They can be extended to approximating splines where the degree of approximation at the landmark depends on the confidence of the landmark localization Registration Using B-Splines = In general radial basis functions have infinite support. Therefore, each basis function contributes to the transformation and each control point has a global influence on the transformation. In a number of cases the global influence of control points is undesirable since it becomes difficult to model local deformations. Furthermore, for a large number of control points the computational complexity of radial basis function splines becomes prohibitive. An alternative is to use free form deformations (FFDs) which have been widely used for animations in CG. FFDs based on locally controlled functions such as B-splines are a powerful tool for modeling 3D deformable objects. The basic idea of FFDs is to deform an object by manipulating an underlaying mesh of control points. The resulting deformation controls the shape of the 3D object and produces a smooth and C 2 continuous transformation. Unlike radial basis function splines which allow arbitrary configurations of control points, spline-based FFDs require a regular mesh of control points with uniform spacing. A spline-based FFD is defined on the image domain Φ = {(x, y, z) 0 x X, 0 y Y, 0 z Z} where Φ denotes a n x x n y x n z mesh of control points φ i,j,k with uniform spacing δ. In this case the displacement field u defined by the FFD can be expressed as the 3D tensor product of the 1D cubic B-splines: 0 u (x, y, z) = θ l (u) θ m (v) θ n (w) φ i+l, j+m, k+n l=0 m=0 n=0 where i = x δ 1, j = y δ 1, k = z δ 1, u = x represents the l-th basis function of the B-splines: 11 δ x δ, v = y y δ δ, w = z z δ δ, and θl

15 θ 0 (s) = (1 s) 3 /6 θ 1 (s) = (3s 3 6s 2 + 4)/6 θ 2 (s) = ( 3s 3 + 3s 2 + 3s + 1)/6 θ 3 (s) = s 3 /6 As mentioned, FFDs are controlled locally, which makes them computationally efficient even for a large number of control points. In particular, the basis functions of cubic B-splines have a limited support, i.e., changing control point φ i, j, k affects the transformation only in the local neighborhood of that control point Elastic Registration Elastic registration techniques were proposed by Bajcsy et al. [17] for matching a brain atlas with a CT image of a human subject. The idea is to model the deformation of the source image into the target image as a physical process which resembles the stretching of an elastic material such as rubber. This physical process is governed by two forces: 1. The internal force which is caused by the deformation of the elastic material (i.e., stress) and counteracts any force which deforms the elastic body from its equilibrium shape. 2. The external force which acts on the elastic body (i.e., strain). As a consequence, the deformation of the elastic body stops if both forces acting on it form an equilibrium solution. The behavior of the elastic body is described by the Navier linear partial differential equation (PDE): µ 2 u (x, y, z) + (λ + µ) ( u (x, y, z)) + f (x, y, z) (3) where u describes the displacement field, f is the external force acting on the elastic body, denotes the gradient operator, and 2 denotes the Laplace operator. The parameters µ and λ are Lamé s elasticity constants which describe the behavior of the elastic body. These constants are often interpreted in terms of Young s modulus E 1, which relates the strain and stress of an object, and Poisson s ratio E 2, which is the ratio between lateral shrinking and longitudinal stretching: E 1 = µ(3λ+2µ) (λ+µ) E 2 = λ 2(λ+µ) The external force f is the force which acts on the elastic body and drives the registration process. A common choice for the external force is the gradient of a similarity measure such as a local correlation measure based on intensities, intensity differences, or intensity features such as edge and curvature. An alternative choice is the distance between the curves and surfaces of corresponding anatomical structures. The PDE in Equation (3) may be solved by finite differences and successive over relaxation (SOR). This yields a discrete displacement field for each voxel. An extension of elastic registration framework has been proposed by Davatzikos [18] to allow for spatially varying elasticity parameters. This enables certain anatomical structures to deform more freely than others. 12

16 2.3.6 Fluid Registration Registration based on elastic transformation is limited by the fact that highly localized deformations can not be modeled, since the deformation energy caused by stress increases proportionally with the strength of the deformation. In fluid registration these constraints are relaxed over time, which enables the modeling of highly localized deformations including corners. This makes fluid registration especially attractive for intersubject registration tasks (including atlas matching) which have to accommodate large deformations and large degrees of variability. At the same time the scope for misregistration increases, as fluid transformations have a vast number of degrees of freedom. Elastic deformations are often described in a Lagrangian reference frame, i.e., with respect to their initial position. In contrast to that, fluid deformations are more conveniently described in a Eulerian reference frame, i.e., with respect to their final position. In this Eulerian reference frame, the deformations of the fluid registration are characterized by the Navier-Stokes PDE: µ 2 u (x, y, z) + (λ + µ) ( v (x, y, z)) + f (x, y, z) (4) similar to Equation (3) except that differentiation is carried out on the velocity field v rather that on the displacement field u and is solved for each time step. The relationship between the Eulerian velocity and displacement field is given by: u (x, y, z, t) v (x, y, z, t) = + v (x, y, z, t) u (x, y, z, t) (4) t Christensen et al. suggested to solve Equation (4) using successive over relaxation (SOR). However, the resulting algorithm is rather slow and requires significant computing time. A faster implementation has been proposed by Bro-Nielsen et al. Here, Equation (4) is solved by deriving a convolution filter from the eigenfunctions of the linear elasticity operator. However, the solution of the Equation (3) by convolution is only possible if the viscosity is assumed constant, which is not always the case. Lester has proposed a model in which the viscosity is allowed to vary spatially, and therefore allows for different degrees of deformability for different parts of the image. In this case Equation (3) must be solved using conventional numerical schemes such as SOR Registration Using FEM and Mechanical Models In this scheme a three component model is proposed to simulate the properties of rigid, elastic and fluid structures. For this purpose the image is divided into a triangular mesh with n connected nodes φ i. Each node is labeled according to the physical properties of the underlying anatomical structures (f.i., bone is labeled as rigid, soft tissues as elastic, CSF (cerebral spinal fluid) as fluid). While nodes labeled as rigid are kept fixed, nodes labeled as elastic or fluid are deformed by minimizing an energy function. Edwards et al. proposed a number of different energy terms to constrain deformations: for example nodes labeled as elastic can be constrained by a tension energy E tension (φ i, φ j ) = φj φ i φ 0 2 i, j where φ 0 i, j corresponds to the relaxed distance between two nodes. An alternative choice for nodes labeled as elastic is a stiffness energy term: 13

17 E stiffness (φ i, φ j, φ k ) = φ j φ k 2φ i 2 Nodes labeled as fluid do not have any associated tension or stiffness energy. Instead, they have an associated folding energy: E fold (φ i, φ j, φ k ) = { A 2 + γ2 A 2 γ 2 A 2 0 if A 0 A 2 A 0 γ, 2 otherwise where A 0 is the area of the undeformed triangle, A is the area of the deformed triangle, and γ is a threshold for the triangular area above which the energy contribution is constant. This energy term prevents the development of singularities in the transformation, i.e., the the collapsing or folding over of triangles. In the implementation proposed by Edwards et al. the registration is driven by a similarity measure which minimizes the distance corresponding landmarks Registration Using Optical Flow A well known registration technique which is equivalent to the equation of motion for incompressible flow in physics is the so-called optical flow. The concept of optical flow was originally introduced in computer vision to recover the relative motion of an object and the viewer in between two successive frames of a temporal image sequence. Its fundamental assumption is that the image brightness of a particular point stays constant, i.e., dρ dt = 0 This optical flow equation can be rewritten as (chain rule): Alternatively this can be written as I dx x dt + I dy y dt + I dz z dt + I t = 0 I + I u = 0 where the I is the temporal difference between the images, I is the spatial gradient of the image, and u describes the motion between the two images. In general, additional smoothness constraints are imposed on the motion field u in order to obtain a reliable estimate of the optical flow Non-physical Models Separate from the non-rigid registration algorithms discussed previously, several non-physical methods have been proposed. In this section we give two examples of recent approaches in this field. A commonly applied non-rigid registration algorithm is the statistical parametric mapping (SPM) algorithm, originally developed for the analysis of PET data [19]. In this approach, the deformation field is constrained to be a linear combination of smooth basis functions. The optimal 14

18 parameters, i.e. the optimal coefficients of the basis functions, are computed by locally minimizing the sum of squared differences between the template image and the reference image while simultaneously maximizing the smoothness of the transformation. Further regularization terms were added in order to constrain the deformations The algorithm runs iteratively using a local search method. A different approach was proposed by Collins et al. (see [20] for a detailed description). Their non-rigid registration algorithm was based on an iterative refinement of a local similarity measure using a simplex optimization. As this approach is constrained only by smoothing after correspondence estimation, the derived deformation field can only be accurate for specific regions of the brain. To achieve better results, the method was refined by introducing various gyri and sulci of the brain as geometrical landmarks [21]. This method is similar to the demons algorithm proposed by Thirion [22]. Recent advances of this method were published by Pennec et al. in [23]. 2.4 Synthesis Patients are often given more than one exam, be it on separate systems or at different times during follow-up procedures. The most accurate approach to interpret the information contained in the resulting images is to transform them into a common coordinate system such that the organs get aligned and can be jointly reviewed. In simple terms, image registration combines the best features from multiple sources, and discards low-grade information before combining the available information to form a synthetic image. After registration, images can be fused and the resultant image should have more information and be more useful for human or machine perception. Since image registration may be used to produce fused images that contain more information than any single acquired view, high-performance image registration is a crucial component in clinical situations where fast execution is required, e.g., intra-operative imaging and surgical guidance. The performance problem of image registration has been approached from two different fronts: the production of efficient registration algorithms and the development of costly equipment, such as combined PET/CT scanners, which cost in the order of millions of dollars. Even with fast and efficient algorithms, advanced image registration requires high-performance computing systems, such as supercomputers (that may be costlier than combined scanners), high-performance dedicated clusters or loosely coupled clusters such as computational grids built from existent infrastructures. In this thesis, algorithms whose computational cost can be reduced by the use of supercomputers will be considered. However, instead of working with supercomputers or dedicated clusters, a much less expensive alternative will be used in form of a computational grid. Efficient ways of deploying these algorithms over the grid will be devised and implemented. 2.5 Significance Applications of image registration are myriad and range from image guided surgery [25] to the analysis of functional images [26], and include the characterization of normal and abnormal anatomical variability (brain mapping) [27], the detection of change in disease state over time [28], the visualization of multimodality data sets [29], and modelling anatomy in the process of segmentation [30, 31]. Medical imaging techniques in common use today show very different aspects of the anatomy examination. For example, computed tomography (CT) shows mostly information on dense matter, 15

19 while magnetic resonance imaging (MRI) shows information on softer tissue types. Both modalities clearly show anatomical morphology, while single photon emission computed tomography (SPECT) and positron emission tomography (PET) show functional aspects of the anatomy. When several imaging modalities are used in a single patient s case, correct registration, i.e. determining the transformation to bring one of the acquired images into agreement with the other(s), may facilitate correct images and/or treatment. Besides, monomodality registration is also important, e.g. using time series of MRI scans on tumors. Registration is the first of two steps of an integration process, and the second being imaging fusion (integrated or combined display), which mainly concerns with the proper visualization of useful image information. This fusing of two views into one may be accomplished in diverse ways. Regardless of the method employed, image registration is a crucial first step that still represents a challenging problem. 3 Research Method This section presents the methodology that has been followed so far and that will be applied in the course of this research. The methodology begins with a problem and experiments with existing technologies, and generally prototypes new ones, in an effort to devise an innovative solution. This methodology is known as problem-oriented research [32]. 3.1 Algorithm Parallelization According to the research method proposed, the procedure to follow starts with the analysis and performance evaluation of existing registration algorithms. Before parallelizing an algorithm, an essential first step is to improve its efficiency as much as possible. To do this, aggressive precomputation of elements of the transformation and multiresolution techniques will be applied when possible. Once the selected sequential algorithms are optimized, they will be parallelized following the replicated-worker design pattern implemented in ungrid that supports the partitioning of data as well as the partitioning of tasks. The following list shows the candidate algorithms that will be analyzed and serve as the basis for parallel implementations: Non-rigid registration using free-form deformations: application to breast MR images (15-30 min.) [8] Fast parametric elastic image registration (10 min. on reduced volumes 128 x 128 x 45) [34] 3.2 Experimental Design Research work proposed in this thesis will be particularly implemented at the National University Hospital. To accomplish the objectives a computational grid must be deployed with the existing computational infrastructure. Registration algorithms commonly used for the type of images produced in the hospital will be studied. These include, conventional radiographs, T1 and T2- weighted magnetic resonance images and gamma camera images (SPECT). The first approach to partitioning the algorithms to obtain computational optimization will be via evolutionary programming techniques. This will be done in two parts, one of which will be the division of the input data 16