Transformation Functions

Transcription

1 Transformation Functions A. Ardeshir Goshtasby 1 Introduction Transformation functions are used to describe geometric differences between two images that have the same or overlapping contents. Given the coordinates of a point in one image, a transformation function will determine the coordinates of the same point in the other image. We will call one of the images the reference and the other image the target. Reference image is usually given and is kept unchanged. Target image is either generated from the reference image or is given but needs to be deformed to have the geometry of the reference image. The transformation functions for image warping and image registration are determined using the coordinates of a number of corresponding points in the images, either selected manually or determined by an automatic process. In image warping, the reference image is given and the target image is generated. In image registration, both reference and target images are given, and the target image is deformed to overlay the reference image. We will first review transformation functions used for image warping. Then we will describe applications of the same transformation functions in image registration. In image warping, a mapping is needed to generate the target image from the reference image. In image registration, a mapping is required to transform the target image to overlay the reference image. We will denote coordinates of points in the reference and target images by (x; y) and (X; Y ), respectively. In 2-D, a transformation function will have two components, while in 3-D it will have three components. First, we will discuss transformation functions for warping and registration of 2-D images. Then, we will extend the concepts to 3-D images. 2 Transformation Functions for Image Warping In this section, various linear and nonlinear transformation functions will be reviewed and their properties will be examined. We assume a reference image is given and resampling it to a target geometry using an appropriate transformation function is required. The transform ation is often given, and it is required only to determine its parameters using a number of corresponding points in the images. The points used in the images to determine the parameters of a transformation are called the control points. 2.1 Transformation of the Cartesian Coordinate System This transformation is defined by x = s x X cos? s y Y sin + t x (1) 1

2 y = s x X sin + s y Y cos + t y : (2) s x and s y are the scaling factors in x and y directions; is the counter-clockwise rotation about the origin; and t x and t y are translations along x and y axes, respectively. This transformation will scale the reference image with (s x ; s y ), rotate it by, and translate it by (t x ; t y ) to obtain the target image. The target image is generated by scanning it from left to right and top to bottom and at each pixel position (X; Y ), determining the corresponding pixel (x; y) in the reference image, reading the intensity or color at (x; y), and saving it at (X; Y ). The transformation of Cartesian coordinate system preserves the angle between lines. Knowing the coordinates of a pair of corresponding points in the images, we can determine the translational difference between the images. Knowing the coordinates of two corresponding points in the images, we can also determine the scaling and rotational differences between the images. The scaling factor is obtained from the ratio of line segments connecting the points in the images. The rotational difference between the images is determined from the dot product of vectors defined by the two line segments. Examples of the transformation of Cartesian coordinate system are shown in Fig. 1. Figure 1a shows a reference image, and Figs. 1b-1d show pure rotation, pure scaling, and a combination of rotation and scaling of the reference image. 2.2 Affine Transformation The affine transformation is defined by: x = ax + by + c (3) y = dx + ey + f: (4) In the affine transformation, angles are not preserved, but parallelism is. This transformation includes the transformation of the Cartesian coordinate system and approximates the projective transformation for images obtained of a flat scene by a camera that is very far from a scene (camera distortions also assumed not to exist). For instance, satellite images of a flat area can be mapped to each other by the affine transformation [2] Examples of the affine transformation are shown in Fig Projective Transformation The projective transformation is defined by x = y = ax + by + c dx + ey + 1 (5) fx + gy + h dx + ey : + 1 (6) Projective transformation represents the true geometric difference between two images obtained of a flat scene with an ideal camera. In projective transformation, parallelism is not preserved, but straight lines remain straight. This transformation contains the affine transformation. Examples of the projective transformation are shown in Fig. 3. Since this transformation function has eight parameters, coordinates of four corresponding points are needed to determine its parameters. 2

3 (a) (b) (c) (d) Figure 1: (a) Reference image. (b) (d) Pure rotation, pure scaling, and a combination of rotation and scaling of the reference image. 3

4 (a) (b) Figure 2: Examples of the affine transformation using image of Fig. 1a as the reference. (a) (b) Figure 3: Examples of projective transformation using image of Fig. 1a as the reference. 4

5 2.4 Nonlinear Transformations When an image is transformed with a nonlinear transformation, straight lines may not remain straight, but the adjacency relation between the points will be preserved. Nonlinear transformation functions can be used to produce the effect of lens distortion or other global or local deformations. Depending on the amount of deformation desired, different transformation functions may be needed. Nonlinear transformation functions can, in general, be grouped into global and local. A global transformation maps one image in its entirety to another image, whereas a local transformation maps only a part of an image into a part of another image. Therefore, to map two images to each other, a combination of local transformations will be needed. Various global and local transformation functions can be formulated. In the following, some of the well-known ones are reviewed Global Transformation Functions Global transformation functions include polynominals, splines, weighted mean, and multiquadrics. A polynomial transformation function of degree M is defined by x = y = MX jx a jk X k Y j?k (7) j=0 k=0 MX jx b jk X k Y j?k : (8) j=0 k=0 Usually polynomials of degree larger than 2 are not advised for image geometric transformation because prediction of their behaviors from their coefficients is very difficult. Since polynomials are globally defined, changing a single coefficient will affect the entire resampled image. Parameters a and b are determined by selecting a number of points in the reference image, specifying or determining their correspondences in the target image, and using the coordinates of corresponding points in the images to solve two systems of linear equations. A polynomial of degree 2 in X and Y has 6 parameters; therefore, the coordinates of 6 corresponding points are needed to determine the parameters of the transformation. A global transformation that has a more predictable behavior is the surface splines, defined by [10] [7]: x = a0 + a1x + a2y + F i r 2 i ln r2 i (9) y = b0 + b1x + b2y + G i r 2 i ln r2 i ; (10) where r 2 i = (X? X i ) 2 + (Y? Y i ) 2 and (X i ; Y i ) is the ith control point in the target image. Parameters a, b, F and G are determined by requiring point (X i ; Y i ) in the target image to map to point (x i ; y i ) in the reference image, for i = 1; : : : ; n. Therefore, the reference image is deformed so that the control points move to desired positions in the target image. The control points can be selected manually or by an automated process. By knowing the coordinates of corresponding points in the images, the parameters of the transformation can be determined. Note that if n 5

6 corresponding points are given, we will have 2n + 6 unknown parameters. Since n corresponding points produce 2n equations, we will need 6 more equations to determine all unknown parameters. The other 6 equations are obtained from the following constraints [10]: F i = 0; X i F i = 0; Y i F i = 0; G i = 0; (11) X i G i = 0; (12) Y i G i = 0: (13) A surface spline represents the surface obtained by a thin-plate deforming under the influence of point loads. Parameters F and G show the magnitudes of the point loads. The first two relations, set the sum of the loads applied to each surface equal to zero. This will ensure that the surfaces will not move up or down under the point loads. The remaining four relations make sure that the inertia obtained from the point loads do not rotate the surfaces. These six constraints, therefore, ensure that the obtained surfaces will remain stable under the influence of the point loads [10]. In contrast to polynomial functions, surface spline are locally sensitive transformation functions. Therefore, deformations can be controlled to affect mostly local image areas. However, since the transformation functions are globally defined, there will be some effect on points far away. This requires resampling the entire target image after each local editing of the control points. Another global transformation function is the weighted mean [4] [16] defined by: x = P n W i (X; Y )x i Pn W i (X; Y ) (14) y = P n W i(x; Y )y i Pn W i (X; Y ) (15) The W i s, which are called the weights, are defined with respect to the control points in the target image. The weights are defined as inverse distances between points in the target image to the control points in that image: W i = f(x? X i ) 2 + (Y? Y i ) 2 g? 1 2. Note that at points (X = X i ; Y = Y i ) the weights become undefined, but the limiting values for x and y are x i and y i, respectively. Therefore, this transformation function maps corresponding control points to each other, and maps the rest of the points by interpolation. Another global transformation function is the multiquadrics defined by [11]: x = y = a i h i (X; Y ) (16) b i h i (X; Y ) (17) where h i is the ith basis function of multiquadrics and is defined by h i = f(x? X i ) 2 + (Y? Y i ) 2 + R 2 1 g 2. Parameter R is to be specified by the user and controls the localness of the function. Parameters a s and b s are determined by setting x = x i and y = y i when X = X i and Y = Y i for i = 1; : : : ; n and solving two systems of linear equations. Multiquadric 6

7 (a) (b) (c) (d) Figure 4: (a) (d) Images obtained using polynomial, surface spline, weighted mean, and multiquadric transformation functions. 7

8 surfaces obtained in this manner will map corresponding points exactly to each other and will map the rest of points in the images by interpolation. A few examples of image deformation using polynomial, surface spline, weighted mean, and multiquadric are shown in Fig. 4. Among these transformation functions, surface splines has the most local control, while polynomial and weighted mean have the least local control. Multiquadrics allow some local deformation, but large local deformations result in large global deformations Local Transformations Local transformation functions map local areas in the images to each other. Therefore, they make it possible to deform an image locally and resample only the revised area. This not only results in a faster speed, but it provides more control to the user. Some of the local transformation functions are piecewise linear, piecewise cubic, and local weighted mean. Piecewise linear transformation is achieved by first triangulating the control points in the reference image, identify the corresponding triangles in the target image, and defining a linear transformation as given by (3) and (4) to map corresponding triangular regions in the images to each other. The coefficients of the linear functions are determined by substituting the coordinates of the vertices of corresponding triangles into (3) and (4) and solving the obtained systems of equations. Areas outside the convex hull of the points can be mapped to each other by extrapolation [5]. When the user revises the position of a control point in the target image, the revision will influence the shapes of only those triangles that share the control point as a vertex. Therefore, only those triangles need to be resampled to obtain the revised target image. Since functional change along x and y could change sharply across a triangular edge, although intensity mapping across triangular edges will be continuous, change in intensity mapping may not be smooth. To produce a smooth deformation within as well as across triangles, it is necessary to use a piecewise cubic transformation function. Typically, the cubic functions are formulated such that functions that map adjacent triangles in the images to each other have the same derivative across the triangular edges. An interpolating smooth piecewise cubic function is given in [3] [6]. In piecewise cubics, when a control point is moved in the target image, the move will affect the tangents across all triangles having that point as a vertex. Therefore, not only triangles having that point as the vertex need to be resampled, the triangles adjacent to them need to be resampled also. Although the resampled area is larger than the piecewise linear transformation, the effect is still local and the affected region can be a small part of an entire image. Examples of piecewise linear and piecewise cubic transformations are shown in Fig. 5. Another local transformation that provides smooth transition across adjacent areas in a resampled image is the local weighted mean [13]. Suppose two polynomials, each with n parameters map an area in the reference image to an area in the target image. Suppose the area in the target image contains the ith control point and n? 1 of its nearest control points. Let the coordinates of the ith control point in the reference and target images be (x i ; y i ) and (X i ; Y i ), respectively. Then the coefficients of the polynomials can be determined by substituting the coordinates of the n corresponding control points into the equations of the polynomials and solving two systems of n linear equations. If we use the weighted mean method mentioned above, we can obtain each component of the transformation function from the weighted mean of the local polynomials. To make the transformation function local, the weight assigned to the ith polynomial is selected such 8

9 that it becomes zero at a sufficiently large distance. Maude proposed the following weights [13]. W i (R) = 1? 3R 2 + 2R 3 ; 0 R 1 (18) W i (R) = 0; R > 1; (19) where R = f(x? X i ) 2 + (Y? Y i ) 2 1 g 2 =R n and R n is the distance of point (X i ; Y i ) to its (n?1)th nearest point in the target image. The weights in equations (18) and (19) ensure that the ith polynomial will have no influence on points that are away by more than R n from the ith control point in the target image. Note that since " # " # dw dw = = 0; (20) dr R=0 dr R=1 the weighted sum of the polynomials is smooth everywhere, including across boundaries of polynomial functions, making the weighted sum of the polynomials continuous and smooth everywhere. Using the weights as defined above, the transformation function that will map the reference image to the target image can be written as: x = P n W n [(X? X i ) 2 + (Y? Y i ) 2 ] 1 2 =R n o Pi (X; Y ) P n W n [(X? X i ) 2 + (Y? Y i ) 2 ] 1 2 =R n o (21) y = P n W n [(X? X i ) 2 + (Y? Y i ) 2 ] 1 2 =R n o Qi (X; Y ) P n W n [(X? X i ) 2 + (Y? Y i ) 2 ] 1 2 =R n o : (22) P i (X; Y ) and Q i (X; Y ) are the components of the local transformation functions that map the ith point and (n? 1) of its nearest control points in the target image to the corresponding control points in the reference image. Although polynomials of arbitrary degrees can be used, polynomials of degrees higher than 2 are not advised, as small changes in the positions of control points could drastically change the resampled local areas. Local weighted mean can extraoplate beyond the convex hull of the control points, but if the control points are not uniformly spaced, holes may appear within the resampled image. When the local weighted mean is used, efforts should be made to select the control points from all over an image rather than from only limited areas. If the user has no control over selecting the control points, the holes can be removed by making the polynomials cover larger areas, which can be achieved by using a larger R n or using a polynomial of a higher degree. An example of local transformation using polynomials of degree 2 is shown in Fig. 6. Although the transformation is local, the transformation may not behave well as observed in Fig. 6b. Even though only one control point is moved, a rather large area is affected. 3 Transformation Functions for Image Registration In image registration, the reference and target images are given and it is required to determine a transformation function that will deform the target image to overlay the reference image. Assuming this transformation function is defined by f x and f y, we can write X = f x (x; y) (23) Y = f y (x; y): (24) 9

10 (a) (b) (c) (d) Figure 5: (a) Control points in the reference and target images. Only one of the control points near the center of the image is moved. (b) Trinagulation of the control points in the two images. (c), (d) Image warping using piecewise linear and piecewise cubic transformations. 10

11 (a) (b) Figure 6: (a) Control points in reference and target images. Only one of the control points near the center of the image is displaced. (b) Image warping by local weighted mean using polynomials of degree two. Note that the transformation functions used in image registration are the inverses of the transformation functions used in image warping. In image registration, the reference image must be scanned, and for each pixel the corresponding pixel in the target image must be determined. Then the color or intensity in the target image is read and saved at the pixel in the reference image. In image warping, it is required to generate the target image; therefore, the target image needs to be scanned, and for each pixel, the corresponding pixel in the reference image should be found and its value should be read and saved at the pixel in the target image. In this manner the reference image is resampled to create the target image. We see that the transformation functions used for image warping can be used to register images if (x; y) and (X; Y ) are switched in the given formulas. The type of transformation function selected significantly affects the registration result. Sometimes the type of the transformation function is given and it is only necessary to determine its parameters. For instance, satellite images of a rather flat area can be registered with linear transformation functions [2]. If the scene is flat but the camera is not very far away, then projective transformation can be used to register the images. If camera distortions exist or the scene is not flat, registration of images requires nonlinear transformation functions [8]. Often registration of images is required to determine geometric differences between them. For example, if the images represent different views of a 3-D scene, the geometric differences describe the 3-D structure of the scene [15]. If some information about geometric differences between two images is known, then an appropriate transformation function for registration of the images can be selected. Otherwise, one should choose a transformation function that can adapt to the geometric differences between images. If a large number of control points is available, it is preferred to use a local transformation function. This has two effects. First, if the images have considerable geometric differences, the 11

12 local functions can represent them. Second, if some of the correspondences are inaccurate, the inaccuracies will not spread to the entire resampled image and will be confined to local areas. If a rather small number of correspondences is given, a global transformation such as surface splines is more appropriate. If the number of points given is more than what is necessary to determine the parameters of a transformation function, least squares may be used to determine the coefficients. When least-squares method is used to determine the parameters of polynomial transformation functions, a local error will spread to an entire image. To reduce the effect of local errors to far away points, weighted least squares may be used. In weighted least squares, a weight is assigned to a point in the reference image proportional to the inverse distance of the point to the control points in that image. The following weight has been suggested by McLain [14]: W i (x; y) = n + [(x? xi ) 2 + (y? y i ) 2 ]o? 1 2 : (25) Parameter is set to a small number such as 1.0 in order to avoid division by zero. The smaller the value of, the smaller the effect of far away control points on the coefficients of the polynomial, and the larger the value of the larger the influence of far away points on the coefficients. When the control points are known to be accurate, a smaller value for should be selected, while if the correspondences are known to contain errors, a larger value of should be used in order to smooth local inaccuracies. In the weighted least squares, the coefficients of the polynomials are determined by minimizing the following two error terms: X E 2 n (x; x y) = [f x (x i ; y i )? X i ] 2 W i (x; y) (26) X E 2 n (x; y y) = [f y (x i ; y i )? Y i ] 2 W i (x; y); (27) where f x (x; y) and f y (x; y) are the polynomials. This minimization results in the following systems of equations: MX jx j=0 k=0 MX jx j=0 k=0 a jk h Wi (x; y)x l iy m?1 i b jk h Wi (x; y)x l iy m?1 i for l = 0; : : : ; M and m = 0; : : : ; l. x k i y j?k i x k i y j?k i i i = = NX NX W i (x; y)x i x l iy m?kl i (28) W i (x; y)y i x l iy m?kl i (29) 4 Transformations for Warping and Registration of 3-D Images Many of the transformation functions described for warping and registration of 2-D images can be easily extended to warping and registration of 3-D images. In 3-D, a transformation function has three components. By 3-D images we mean isotropic volumetric images that are represented by 12

13 3-D arrays. Points in 3-D images with linear geometric differences, including translation, rotation, and scaling, can be related to each other by x = ax + by + cz + d (30) y = ex + fy + gz + h (31) z = ix + jy + kz + l: (32) The parameters of the transformation can be determined by knowing the positions of four corresponding points in the reference and target images. If more than four points are available, the parameters can be determined by the least-squares method. If the images have nonlinear geometric differences, a nonlinear transformation will be needed to warp or register the images. Again, we can use global or local transformation functions. Examples of global transformation functions are, polynomials, volume splines, and weighted mean; local transformation functions are piecewise linear, piecewise cubic, and local weighted mean. If polynomials of degree M are used as components of the transformation, we will have x = y = z = MX jx kx a jkl X l Y k?l Z j?k (33) j=0 k=0 l=0 MX jx kx b jkl X l Y k?l Z j?k (34) j=0 k=0 l=0 MX jx kx c jkl X l Y k?l Z j?k : (35) j=0 k=0 l=0 When M = 2, there will be 30 unknown parameters, which can be determined if the coordinates of 10 corresponding points in the reference and target images are known. If more than 10 correspondences are available, the parameters can be determined by the least-squares method. Volume splines are the extensions of surface splines: x = a0 + a1x + a2y + a3z + F i r 2 i ln r2 i (36) y = b0 + b1x + b2y + b3z + G i r 2 i ln r2 i (37) z = c0 + c1x + c2y + c3z + H i r 2 i ln r2 i ; (38) where r 2 i = (X? X i ) 2 + (Y? Y i ) 2 + (Z? Z i ) 2. The parameters of the volume splines can be determined if coordinates of at least four corresponding points in the images are known. When solving for the parameters from the given corresponding points, the following constraints must be used also: F i = 0; X i F i = 0; G i = 0; Y i F i = 0; 13 H i = 0 (39) Z i F i = 0 (40)

14 X i G i = 0; X i H i = 0; Y i G i = 0; Y i H i = 0; Z i G i = 0 (41) Z i H i = 0: (42) These constraints ensure that the volumes remain stable under the application of the point loads. The weighted-mean method in 3-D can be written as x = P n W i (X; Y; Z)x i Pn W i (X; Y; Z) (43) y = P n W i (X; Y; Z)y i Pn W i (X; Y; Z) (44) z = P n W i(x; Y; Z)z i Pn W i(x; Y; Z) : (45) where W i (X; Y; Z) = [(X? X i ) 2 + (Y? Y i ) 2 + (Z? Z i ) 2 ]? 1 2. The piecewise linear and piecewise cubic transformations in 3-D are obtained by partitioning the space of the images into tetrahedra [12] and mapping corresponding tetrahedral regions into each other using the trilinear transformation function shown in equations (30) (32), or the piecewise cubic functions given by Alfeld [1]. The local weighted mean in 3-D is the same as in 2-D except that R = [(X? X i ) 2 + (Y? Y i ) 2 + (Z? Z i ) 2 1 ] 2 =R n, and P i (x; y; z) and Q i (X; Y; Z) are polynomials mapping the ith control point and (n? 1) of its nearest control points in the target image to the corresponding control points in the reference image. To evaluate the effectiveness of a transformation function in image registration, we can overlay the reference and resampled target images and compare their geometries. If the reference and target images are obtained with the same sensor and the intensity difference between them is not large, the quality of registration can be assessed by simply determining the intensity difference between the images. 5 What Does a Transformation Function Tell Us About the Geometric Differences between Two Images? In image registration, there often is a need to determine geometric differences between two given images. A transformation function can tell us about geometric changes that have occurred between the times the images were obtained. For instance, when given images are brain images obtained before and after surgery, one can determine the amount of brain shift that has occurred as a result of the surgery. If the images show two stages of a treatment, it becomes possible to evaluate the effectiveness of the treatment. Geometric changes in a tumor can determine the effectiveness or ineffectiveness of the treatment. How can we detect and quantify geometric differences between two images? By a closer examination of the transformation functions described for warping and registration of 2-D images we see that the functions basically represent surfaces. Given a set of corresponding points 14

15 f(x i ; y i ); (X i ; Y i ) : i = 1; : : : ; ng in the reference and target images, the x and y components of the transformation function can be considered surfaces that fit to two sets of scattered 3-D points: f(x i ; y i ; X i ) : i = 1; : : : ; ng and f(x i ; y i ; Y i ) : i = 1; : : : ; ng. The shapes of these surfaces contain information about the position and amount of geometric differences between the images [15]. Let s suppose the components of a transformation function for registration of two images are represented by X = f x (x; y) (46) Y = f y (x; y): (47) By plotting the surfaces, we can observe the x component and the y component geometric differences between the images. If the surfaces appear as planes, we can conclude that no nonlinear geometric differences exist between the images. Local bumps and intrusions in the surfaces correspond to local geometric differences between the images. Deformation appearing in one of the surfaces is evidence of difference between the images along the x or the y axis. If the images do not have rotational and scaling differences, such as medical images of a patient taken by the same scanner at different times, by plotting x?x and y?y we will see that local geometric differences will appear as bumps and intrusions in otherwise horizontal planes. The positions and sizes of the bumps and intrusions show the positions and amounts of local geometric differences between the images. Since the transformation functions are determined using the coordinates of a number of corresponding points in the images, if the correspondences are accurately determined, the transformation function can tell us about the geometric differences between the image. If the correspondences are not accurate, but it is known that the images do not have nonlinear geometric differences, bumps and intrusions in the surfaces can be the result of inaccuracies in point correspondences. We can identify the inaccurate point correspondences and remove them, and determine a more accurate transformation function from the remaining correspondences. As an example, the control points in two images are shown in Figs. 7a and 7b. The components of the transformation function mapping the images to each other are shown in Figs. 7c and 7d. We see a bump near control point 7 in the surface representing the components of the transformation. If the images are known to have no nonlinear geometric differences, this can be a sign that control point 7 in the images do not accurately correspond to each other. This control point should be removed from the set of control points and the transformation function should be recomputed 6 An Adaptable Transformation Function From the above discussions we see that there may be a need for different transformation functions when warping or registering different images. This means that the procedure that warps or registers different images requires access to different transformation functions. Rather than implementing a large number of transformation functions, we can use a transformation function with free parameters that can adjust to different types of geometric difference between images. An adaptive transformation function should have the following properties: 1. It should have the ability to represent linear as well as nonlinear geometric differences between images. 15

16 (a) (b) (c) (d) Figure 7: (a),(b) Control points in reference and target images. Only control point number 7 is different in the images. (c),(d) x and y components of the transformation function using surface splines. 16

17 2. It should have the ability to represent local as well as global geometric differences between images. 3. It should not require a specific arrangement of the control points to find the parameters of the transformation. Since each component of the transformation function in 2-D is represented by a surface, an adaptive transformation function should be able to represent planes as well as free-form surfaces. In addition, the surface should be obtainable from the coordinate of corresponding control points that may be irregularly spaced. A formulation that provides these properties is the rational Gaussian (RaG) surface [9]: p(u; v) = g i (u; v)p i u; v"[0; 1]; (48) where p(u; v) = [x(u; v)y(u; v)z(u; v)], p i = [x i ; y i ; z i ], and g i (u; v) is the ith basis function of the surface: G i (u; v) g i (u; v) = P n j=1 G j (u; ; (49) v) and G i (u; v) is a 2-D Gaussian centered at (u i ; v i ): G i (u; v) = exp n [(u? u i ) 2 + (v? v i ) 2 ]=[2 2 ]o : (50) (u i ; v i ) is the ith node of the surface and is estimated from u i = [X i? min j (X j )]=max j (X j ) and v i = [Y i? min j (Y j )]=max j (Y j ). Since an obtained surface approximates 3-D points, RaG surfaces do not map corresponding control points exactly to each other, but rather they map them approximately to each other. This approximation can be made more accurate by decreasing the standard deviation of the Gaussians. As the standard deviation of the Gaussians is reduced, the approximation converges to the interpolation, and corresponding control points will be mapped to each other. If the correspondences are not accurate, it may not be desirable to map corresponding control points exactly to each other; rather, it may be more appropriate to map corresponding control points approximately to each other. This approximation will in effect smooth noisy errors in the positions of corresponding control points. Having the ability to provide different smoothness levels, this transformation function can reduce a desired level of noise in the correspondences. If the control points accurately correspond to each other, we must make the surfaces interpolate rather than approximate the points. This can be achieved by determining new p i in equation (48) such that p(u i ; v i ) = given p i. This requires solving three systems of n linear equations to determine the transformation function. Because of the nature of Gaussians, the obtained system of equations will have a diagonally dominant matrix of coefficients, guaranteeing a solution. Figure 8 shows examples of image warping using control points shown in Fig. 5a at different smoothness levels. 7 Concluding Remarks Transformation functions are the heart of image warping and image registration. A transformation function should be selected to reflect geometric differences between two images in image regis- 17

18 (a) (b) Figure 8: Two image warping examples using rational Gaussian (RaG) surfaces with two different standard deviations. The control points shown in Fig. 5a were used to obtain the transformations. tration and desired deformations in image warping. Often, selection of the right transformation function for image registration and image warping is a challenge. To make this selection as effective as possible, an evaluation mechanism should be put in place to determine the errors in image registration or appropriateness of a generated image in image warping. If the adaptive transformation function is used, the number of control points and the standard deviation of Gaussians can be selected to minimize an objective or subjective error criterion. In addition to accuracy, speed is a critical factor, especially in image warping where the result of a modification should be generated in real time. The number of control points used to define a transformation function should be selected taking into consideration the speed and accuracy of image registration or image warping. References [1] P. Alfeld, A trivariate Clough-Tocher scheme for tetrahedral data, Computer Aided Geometric Design, vol. 1, pp , [2] R. Bernstein, Digital image processing of earth observation sensor data, IBM J. Research and Development, pp , Jan [3] R. W. Clough and J. L. Tocher, Finite element stiffness matrices for analysis of plates in bending, Proc. Conf. Matrix Methods in Structural Mechanics, pp , [4] I. K. Crain and B. K. Bhattacharyya, Treatment of non-equispaced two-dimensional data with a digital computer, Geoexploration, vol. 5, pp ,

19 [5] A. Goshtasby, Piecewise linear mapping functions for image registration, Pattern Recognition, vol. 19, pp , [6] A. Goshtasby, Piecewise cubic mapping functions for image registration, Pattern Recognition, vol. 20, no. 5, pp , [7] A. Goshtasby, Registration of images with geometric distortions, IEEE Trans. Geoscience and Remote Sensing, vo. 26, no. 1, [8] A. Goshtasby, Image registration by local approximation methods, Image and Vision Computing, vol. 6, no. 4, pp , [9] A. Goshtasby, Geometric modeling using rational Gaussian curves and surfaces, Computer- Aided Design, vol. 27, no. 5, pp , [10] R. L. Harder and R. N. Desmarais, Interpolation using surface splines, J. Aircraft, vol. 9, no. 2, pp , [11] R. L. Hardy, Theory and applications of the the multiquadric-biharmonic method, Comput. Math. Appl., vol. 19, no. 8/9, pp , [12] B. Joe, Construction of three-dimensional Delaunay triangulation using local transformations, Computer Aided Geometric Design, vol. 8, no. 2, pp , [13] A. D. Maude, Interpolation-mainly for graph plotters, Comput. J., vol. 16, pp , [14] D. H. McLain, Interpolation methods for erroneous data, in Mathematical Methods in Computer Graphics Design, K. W. Brodlie (ed.), pp , [15] M. Satter and A. Goshtasby, Registration of Deformed Images, Image Registration Workshop, NASA Goddard Space Flight Center, Greenbelt, MD, Nov , pp , [16] D. Shepard, A two-dimensional interpolation function for irregularly spaced data, Proc. 23rd National Conference ACM, pp ,