BLUR INVARIANT REGISTRATION OF ROTATED, SCALED AND SHIFTED IMAGES

Transcription

1 BLUR INVARIANT REGISTRATION OF ROTATED, SCALED AND SHIFTED IMAGES Ville Ojansivu and Janne Heikkilä Machine Vision Group, Department of Electrical and Information Engineering University of Oulu, PO Box 45, 94, Finland ABSTRACT In this paper, we propose a blur invariant image registration method that can be used to register rotated, scaled and shifted images. The method is invariant to centrally symmetric blur including linear motion and out of focus blur. The method correlates log-polar sampled phase-only bispectrum slices, which are modified for blur invariance, to estimate rotation and scale parameters. Translation parameters are estimated using a blur invariant version of phase correlation. An additional advantage of using the phase-only bispectrum is the invariance to uniform illumination changes. We present also results of numerical experiments with comparisons to similar registration methods which do not possess blur invariance properties. The results show that the image registration accuracy of our method is much better when images are blurred.. INTRODUCTION Registration of 2-D images acquired from the same scene at different times, from different viewpoints, or different sensors is a fundamental problem in image processing. Registration is needed before further analysis and fusion of the images. Typical applications include image mosaicing, superresolution, and the fusion of multimodal images in fields like remote sensing, medical imaging, and computer vision. Comprehensive surveys about the widely studied 2-D image registration can be found in [, 2]. Image registration methods can be divided into the feature and area based methods. The former attempts to match the features of salient details of images, while the latter attempts to match whole images, also called template matching. In practical applications, images contain various degradations due to imperfect imaging conditions including blur, which can result from atmospheric turbulence, out of focus, or relative motion between the camera and the scene. When the degradation process is modeled as a linear shift-invariant system, the relationship between an ideal image f(x) and an observed image g(x) is given by g(x) = f(x) h(x)+n(x), () where x is a 2-D spatial coordinate vector, h(x) the point spread function (PSF) of the system, n(x) additive noise, and denotes 2-D convolution. The point spread function h(x) represents blur, while other degradations are captured by the noise term n(x). With few exceptions, all the image registration methods are sensitive to blur, which may result in inaccurate registration. To the best of our knowledge, the only existing blur invariant 2-D registration methods are founded on the blur invariant features based on spatial image moments [3] or discrete Fourier transform (DFT) [4]. These methods are invariant to centrally symmetric blur. In the first paper, Flusser and Suk used moment invariants for matching a template to a larger blurred image. In [5], complex forms of the moment invariants were used as descriptors representing the neighborhood of a control point in feature based registration. In both papers, also rotation invariance was demonstrated. In [6], the invariants of [3] are used similarly for feature based registration of X-ray images. In that paper, rotation invariance is obtained by calculating the features for every rotation angle. The main shortcomings of these approaches based on the moment invariants are that one image must be fully included into the other, and their computation becomes slow if the template has to be matched for every possible location. Invariance to scaling with rotation and translation invariance is achieved only by normalization of the images [5]. The blur invariant phase correlation (BIPC) image registration method proposed in [4] is based on forms of DFT phase which are invariant to centrally symmetric blur. As with ordinary phase correlation (PC), BIPC can be computed efficiently for large images as a whole and the images do not need to overlap fully. A shortcoming of this method is that it only deals with image translation. In this paper, we present a blur and similarity transform (rotation, scaling and translation) invariant registration () method for images. This method is an extension of the theory of BIPC and carries similar invariance to centrally symmetric blur. The invariance to rotation and scaling is achieved through the use of bispectrum and log-polar mapping similar to the image registration method that uses the magnitude of the Fourier-Mellin transform () [7]. As with BIPC, the images do not need to overlap completely. First, in Section 2, we show what are the conditions of blur invariance in the Fourier domain. In Section 3, we review shortly the method, since our method is constructed similarly, and then in Section 4, we present our method. In Section 5, we discuss the implementation of, in Section 6, we show numerical examples, and finally, Section 7 concludes the paper. 2. BLUR INVARIANCE IN THE FOURIER DOMAIN In this section, we present the conditions under which it is possible to obtain blur invariance in the Fourier domain. If noise n(x) is neglected, () can be expressed in the Fourier domain using the convolution theorem by G(u) = F(u) H(u), where G, F, and H are Fourier transforms of images g and f, 27 EURASIP 755

2 and PSF h, respectively, and where u is a vector in the 2-D frequency space. In the phasor form, the equation becomes G(u) = G(u) e iφ g(u). If the Fourier transform G(u) is normalized by its magnitude, only the complex exponential containing the phase remains, namely G(u) G(u) = e iφ g(u) = e i[φ f (u)+φ h (u)], (2) where φ f (u) is the phase of the original image f(x) and φ h (u) the phase of the blur PSF h(x). Since h(x) is assumed to be centrally symmetric, its Fourier transform H(u) is real and its phase φ h (u) has only two possible values φ h (u) = φ h (u) = π. It follows from this and from the periodicity of the complex argument that the equality [e iφ g(u) ] 2n = e i2nφ g(u) = e i2nφ f (u) e i2nφ h(u) = [e iφ f (u) ] 2n holds for any integer n. Thus, any even power of the normalized Fourier transform, i.e. e i2nφ(u), of the observed image is invariant to the convolution of the original image with any centrally symmetric PSF. 3. IMAGE REGISTRATION USING THE FOURIER-MELLIN TRANSFORM It is well known that by correlating the magnitudes of the Fourier-Mellin transform it is possible to obtain an image registration method () invariant to translation, rotation and scaling [7, 8]. This method achieves first the invariance to translation by using only the shift invariant magnitude of the spectra. The remaining scale and rotation parameters are derived by the following properties of the Fourier transform: (3) f(ax,by) ab F(u a, v b ) (4) f(r,θ + α) F(ω,φ + α), (5) where f(x,y) is an image function, F(u,v) a 2-D DFT of f(x,y), and f(r,θ) and F(ω,φ) the corresponding representations in polar coordinates, and a, b and α are some constants. Property (4) makes it possible to estimate the scale differences of the images directly from the displacement of their logarithmically sampled amplitude spectra. Property (5) converts the problem of estimating the rotation angle into determining the displacement along the φ-axis in the polar coordinates. As a result, both scale and orientation can be solved simultaneously, simply by determining the shift between two log-polar sampled amplitude spectra. After the rotation and scaling parameters are solved, one of the images is rotated and scaled accordingly, and phase correlation is used to estimate the remaining unsolved translation parameters. 4. BLUR AND SIMILARITY TRANSFORM INVARIANT REGISTRATION OF IMAGES In [4], the blur invariance properties of the Fourier transform presented in Section 2 are utilized to construct the BIPC method that can be used to register blurred and translated images. BIPC differs from PC basically by the fact that the terms of the normalized cross power spectrum of the blurred images g and g 2, which are similar to the left hand side of 2, are raised to the second power; namely S(u, v) = [ ] G2 (u,v) 2 [ G ] (u,v) 2 G 2 (u,v) G (u,v) = e i2φ g 2 (u,v) e i2φ g (u,v) = e i2(ux +vy ). The inverse Fourier transform of (6) is now δ(x 2x,y 2y ), which is the Dirac delta function centered at (2x,2y ) corresponding to the spatial shift between the original images multiplied by two. It is not possible to use the magnitude spectrum and Fourier-Mellin transform to extend the invariance to rotation and scaling, as in Section 3, and at the same time maintain the blur invariance, because the amplitude spectrum cannot be made invariant to blur. Instead, we need an image representation that is shift invariant, has properties similar to (4) and (5) and utilizes only the information contained in the phase spectrum, so that it can be made invariant to blur. In [9], the phase-only bispectrum containing the above mentioned properties is used successfully for registration of rotated, scaled and shifted images (We call this method bispectrum registration, ). The bispectrum is defined by Ψ(u,u 2 ) = F(u )F(u 2 )F (u + u 2 ), where u and u 2 are vectors in the 2-D frequency space. It can be easily shown that Ψ is shift invariant [], and beside this, it does not loose any essential information about the original image, containing both amplitude and phase information. The bispectrum is a function of two vector arguments, containing a total of four scalar variables. Assuming that F(u i ), i =,2, is an N-by-N discrete Fourier transform (DFT) of an image f(x), the bispectrum becomes a fourdimensional N-by-N-by-N-by-N matrix. However, it is not necessary to evaluate the whole bispectrum. It is possible to take only 2-D slices of the original bispectrum, as was done also in [9], which contain basically the same information. There are various ways of defining the slices [, 2]. We define the slices as S k (u) = Ψ(u,ku) k R. Bispectrum slices have the same scaling and rotation properties, (4) and (5), as the Fourier transform, except for the scale factor of (4) which is replaced by [9]. This ab ab 3 means that bispectrum slices can be used in the same way as amplitude spectrum in the case of the Fourier-Mellin transform to estimate the scale and rotation parameters between images. In order to achieve blur invariant registration, we have to extract a phase-only bispectrum slice; namely (6) 27 EURASIP 756

3 P k (u) = S k(u) S k (u) = e iφ(u) e iφ(ku) e iφ[(k+)u]. According to the reasoning of Section 2 and equality (3), the individual phase-only terms become invariant to centrally symmetric blur of the original image if they are raised to an even power; namely B kn (u) = e i2nφ(u) e i2nφ(ku) e i2nφ[(k+)u] (7) = e i2n[φ(u)+φ(ku) φ((k+)u)], where n Z and k R. This blur invariant phase-only bispectrum slice can be used to estimate the rotation and scale parameters between images similarly to amplitude spectrum in the case of. Translation parameters are estimated using BIPC. We call this composite method blur and similarity transform invariant registration (). The use of the phaseonly spectrum has also the additional advantage of being invariant to uniform illumination changes and robust to partial background clutter and occlusions [9]. 5. IMPLEMENTATION In practice, we used value n = in (7) resulting in phase-only bispectrum slices B k (u) = e i2[φ(u)+φ(ku) φ((k+)u)], (8) where k R. Let us assume that the slices B k (u) and B k (u) of the two input images f(x) and f (x) as well as the DFTs that are used to compute them are N-by-N matrices. Before computation of the DFTs, the images are windowed using a Kaiser window with the value β = 2 along each coordinate and zero padded to size N-by-N. As can be seen from (8), we need frequency samples in points u, but also at ku and (k + )u to compute an arbitrary slice. The samples in the two latter cases can be extracted from DFT by utilizing its conjugate symmetry and periodicity. After the blur invariant phase-only bispectrum slices were computed, the log-polar sampling followed using bilinear interpolation. The sampling points are selected in the same way as in [7]; namely u i = t r i/t cosθ i v i = t r i/t sinθ i, where t = N/2, and (r i,θ i ) are the points in the new uniform sampling lattice. The log-polar sampled bispectrum slices for the image pair are denoted by R k (r i,θ i ) and R k (r i,θ i ). The accuracy of the registration method depends on the resolution of the DFTs and interpolation. As we wanted to demonstrate the accuracy of our method, we computed the DFTs and the interpolation with high resolution. In the experiments, the horizontal and vertical DFT sizes were four times the corresponding image sizes, with values r i {,,...,t} and θ i {,.25,...,36}. Next, the displacement between R k (r i,θ i ) and R k (r i,θ i ) is estimated using 2-D cross correlation. The correlation is linear along the r-axis and along the θ-axis it is cyclic. The Figure : An aerial image used in the first experiment. correlation result along the r-axis can be improved by windowing and zero-padding R k (r i,θ i ) and R k (r i,θ i ) along this dimension in the same way as in [7, 9]. We used again a 2-D Kaiser window with the value β = 2 and performed the windowing after interpolation. We computed two correlation functions C and C 2 independently between slices R (r i,θ i ) and R (r i,θ i ), and R 2 (r i,θ i ) and R 2 (r i,θ i ) and added these as C = C +C 2 to suppress noise. The use of more slices would improve the results slightly. The coordinates of the maximum values of C denoted by (r max,θ max ) give now the estimates for the rotation and scale parameters. The rotation is obtained directly from θ max and the scale is derived using the nonlinear mapping s = t rmax/t. There is no ambiguity about the correct rotation parameter as is the case when amplitude information is used [7]. After the rotation and scaling parameters are solved, either f(x) or f (x) is rotated and rescaled accordingly. Translation parameters are then estimated by computing BIPC (6) between the images. Correct translation parameters correspond to the coordinates of maximum value of the phase correlation function. Subpixel accuracy is achieved by using the method proposed in [3]. The execution time of image registration using the method depends on the number of bispectrum slices used and the accuracy of the log-polar sampling. Compared to the method the computation of a bispectrum slice from the DFT needs some extra operations, but the estimation of the translation parameters is faster because needs to compute translation parameters for two different rotations and test which is correct. In practice, one slice is slightly faster and two slice slower than. 6. EXPERIMENTS We compared the registration accuracy of our method to the and methods. The and methods were implemented as they are proposed in [9] and [7], respectively. In the first experiment, we simulated the registration of motion blurred and noisy aerial images by generating degraded and geometrically transformed 3 3 versions of the aerial image of Figure, which were then registered with a 3 3 section of the original image. Both images were motion blurred in arbitrary direction. The transform parameters were non-integer numbers: rotation angle was in the 27 EURASIP 757

4 Faulty rotation [%] Faulty scaling [%] (a) Rotation (b) Scale Faulty translation [%] (c) Translation Figure 2: Percentage of registrations with (a) rotation angle deviating more than.25 (solid lines) and (dashed lines), (b) scale factor deviating more than.8 (solid lines) and.2 (dashed lines), and (c) resultant of translation parameters deviating more than.5 pixels (solid lines) and 2 pixels (dashed lines) from the correct parameters. range [, 36 ], scale factor in the range [.5, 2] and translation parameters in the range [-25, 25] pixels. The resolution of the original image corresponded to the resolution of scale 2 and the transformed images were created using bilinear interpolation. We increased the blur length from to 2 in steps of 4 pixels and performed the registration test 8 times for each blur length and for each of the three methods. We recorded the percentage of faulty registrations, which are shown in the diagrams in Figure 2 when smaller (solid lines) or larger (dashed lines) error is accepted. Figure 2(a) shows the percentage of registrations that produced a faulty rotation angle when the accepted error is.25 (solid line) or (dashed line). Figure 2(b) shows a similar diagram for the scale parameter when the accepted error is.8 or.2, and Figure 2(c) depicts the case for resultant translation parameters when the accepted error is.5 or 2 pixels. PSNR in all the tests was 35 db. It can be seen, from the diagrams of Figures 2(a) and 2(b), that the method clearly produces the most accurate rotation and scale parameters when the blur length is larger than four pixels. The same results of the method deviate from the correct already when the blur length is four pixels. performs better if the acceptable error is larger in contrast to. As one of the images is rotated and rescaled according to the estimated parameters before translation estimation, the translation accuracy depends also on the accuracy of the rotation and scale estimation. This explains the large translation parameter errors of the and methods even in the case of a small blur. However, estimates also the translation parameters accurately. In the second experiment, we tested the performance of the method in the case of real images, blurred due to out of focus. We captured images of a poster on a wall rotating, zooming, shifting and defocusing the camera between the shots. Figures 3(a) and 3(b) show a typical example of the original, and the transformed and heavily defocused image. The image of Figure 3(b) is aligned to the image in Figure 3(a) in Figure 3(c) using the registration method with estimated parameters for rotation, scale and x and y translation [2.,.886, -5.8, 6.2]. The same parameters for the method were [.75,.95, -3.7, -8.4] and for the clearly 27 EURASIP 758

5 (a) (b) (c) Figure 3: Images of the second experiment: (a) original, (b) rotated, scaled, shifted and blurred, (c) image (b) aligned to image (a) using the method. incorrect method [.,.,.7, -4.8]. Before defocusing, with a sharp focus, the same parameters for the method were [2.,.94, -6.8, 5.7]. Although the defocusing changes slightly at least the scale, it seems that the parameters of are close to the focused case. 7. CONCLUSIONS In this paper, we have proposed the method that can be used to register rotated, scaled and shifted images. The method is invariant to centrally symmetric blur including linear motion and out of focus blur. Blur invariance can be achieved by using even powers of the phase-only bispectrum slices. The method correlates these log-polar sampled spectral slices in order to resolve the rotation and scale parameters. Translation parameters are estimated using BIPC. A further advantage of using the phase-only bispectrum is the invariance to uniform illumination changes. The method was tested with simulated and real images, and it gave accurate estimates of the rotation, scale and translation parameters in the presence of noise and heavy motion and out of focus blur. was also compared to the and methods. In the case of blur, outperformed these other methods producing more accurate registration results. 8. ACKNOWLEDGMENTS The authors would like to thank the Academy of Finland (project no. 75) for funding of this research. REFERENCES [] B. Zitová and J. Flusser, Image registration methods: a survey, Image and Vision Computing, vol. 2, no., pp. 977, Oct. 23. [2] Lisa Gottesfeld Brown, A survey of image registration techniques, ACM Computing Surveys (CSUR), vol. 24, no. 4, pp , Dec [3] J. Flusser and T. Suk, Degraded image analysis: An invariant approach, IEEE Trans. Pattern Anal. Machine Intell., vol. 2, no. 6, pp , 998. [4] Ville Ojansivu and Janne Heikkilä, Image registration using blur invariant phase correlation, IEEE Signal Processing Lett., vol. 4, no. 7, July 27. [5] Jan Flusser, Barbara Zitová, and T. Suk, Invariantbased registration of rotated and blurred images, in Geoscience and Remote Sensing Symposium IGARSS99, Hamburg, Germany, July 999, pp [6] Y. Bentoutou, N. Taleb, M. Chikr El Mezouar, M. Taleb, and L. Jetto, An invariant approach for image registration in digital subtraction angiography, Pattern Recognition, vol. 35, no. 2, pp , 22. [7] Qin sheng Chen, Michel Defrise, and F. Deconinck, Symmetric phase-only matched filtering of fouriermellin transforms for image registration and recognition, IEEE Trans. Pattern Anal. Machine Intell., vol. 6, no. 2, pp , 994. [8] B. Srinivasa Reddy and B. N. Chatterji, An fft-based technique for translation, rotation, and scale-invariant image registration, IEEE Trans. Image Processing, vol. 5, no. 8, pp , 996. [9] Janne Heikkilä, Image scale and rotation from the phase-only bispectrum, in Proc. IEEE International Conference on Image Processing (ICIP 24), Singapore, Oct. 24, pp [] V. Chandran, B. Carswell, B. Boashash, and S. Elgar, Pattern recognition using invariants defined from higher order spectra: 2-d image inputs, IEEE Trans. Image Processing, vol. 6, no. 5, pp , 997. [] Soheil A. Dianat and Raghuveer M. Rao, Fast algorithms for phase and magnitude reconstruction from bispectra, Optical Engineering, vol. 29, no. 5, pp , 99. [2] A. P. Petropulu and H. Pozidis, Phase reconstruction from bispectrum slices, IEEE Trans. Signal Processing, vol. 46, no. 2, pp , 998. [3] Hassan Foroosh, Josiane Zerubia, and Marc Berthod, Extension of phase correlation to subpixel registration, IEEE Trans. Image Processing, vol., no. 3, pp. 88 2, EURASIP 759