Coherent Image Processing using Quaternion Wavelets

Transcription

1 Coherent Image Processing using Quaternion Wavelets Wai Lam Chan, Hyeokho Choi, Richard G. Baraniuk Dept. of Electrical and Computer Engineering Rice University Houston, TX 775 ABSTRACT We develop a quaternion wavelet transform (QWT) as a new multiscale analysis tool for geometric image features. The QWT is a near shift-invariant, tight frame representation whose coefficients sport a magnitude and three phase values, two of which are directly proportional to local image shifts. The QWT can be efficiently computed using a dual-tree filter bank and is based on a 2-D Hilbert transform. We demonstrate how the QWT s magnitude and phase can be used to accurately analyze local geometric structure in images. We also develop a multiscale flow/motion estimation algorithm that computes a disparity flow map between two images with respect to local object motion. Keywords: Wavelets, quaternions, edges, image registration, disparity estimation, dual-tree 1. INTRODUCTION Encoding and estimation of the relative locations of image features plays an important role in image processing applications, ranging from feature detection and target recognition to image compression. For example, in edge detection, the goal is to locate the boundary of an object in an image. In image denoising or compression, stateof-the-art techniques achieve significant performance improvements by exploiting information on the relative locations of large transform coefficients. 1 4 An efficient way to compute and represent relative location information in one-dimensional (1-D) signals is through the phase of the Fourier transform. The Fourier shift theorem provides a simple linear relationship between the Fourier phase and the signal shift. When only a local region of the signal is of interest, the short-time Fourier transform (STFT) provides a local Fourier phase for each windowed portion of the signal. The discrete wavelet transform (DWT) is more natural than the STFT for signals containing isolated singularities, such as piecewise smooth functions. The locality of the wavelet basis functions leads to a sparse representation of such signals that compacts the signal energy into a small number of coefficients. Wavelet coefficient sparsity is the key enabler of algorithms such as wavelet-based denoising by shrinkage. 5 While wavelets have been successfully applied to many signal and image processing applications, one of the major problems with conventional real-valued wavelets is their lack of shift-invariance. A small shift of the signal results in significant fluctuations of wavelet coefficient energy, making it difficult to extract or model signal information from the coefficient values. There is also no notion of phase to encode signal location information as in the Fourier case. Complex wavelet transforms (CWTs), such as the dual-tree CWT, 6 can remedy these problems. In 1-D, the dual-tree CWT uses complex wavelet basis functions whose real and imaginary parts form a 1-D Hilbert transform pair. The 1-D dual-tree CWT is slightly (two times) redundant, but the magnitudes of its coefficients are almost shift invariant. 6 The CWT phase also contains information on the locations of signal singularities. 7 An important precursor of the dual-tree approach is the complex Daubechies wavelet transform, 8 which is orthonormal but not shift-invariant. In 2-D images, a single complex phase is not enough to encode image shifts. For example, in conventional 2-D Fourier analysis, as the underlying images translates, the phase value is a linear combination of the horizontal {wailam, choi, richb}@rice.edu. Web: dsp.rice.edu. This work was supported by NSF grant CCF 43115, ONR grant N , AFOSR grant FA , AFRL grant FA , and the Texas Instruments Leadership University Program.

2 and vertical image shifts, and inference of the shifts from a single phase value is impossible. The same phase ambiguity problem exists in the 2-D CWT, which measures phase shifts only in directions perpendicular to its wavelet orientations. 7 To remedy this problem, in this paper we marry the quaternion Fourier transform (QFT) 9 with the dualtree CWT to develop a new quaternion wavelet transform (QWT) for images and other 2-D data. We leverage the notions of 2-D Hilbert transform, 2-D analytic signal, and quaternion algebra to construct the QWT. The 2-D Hilbert transforms of a usual 2-D DWT tensor product wavelet (that is, the 1-D Hilbert transform of the wavelet along either or both of the horizontal and vertical directions) plus the wavelet itself constitute the four components of each quaternion wavelet. The QWT frame, which can be efficiently generated from a dual-tree filter bank, 6 is a 4 redundant tight frame that is stably invertible. Analogous to the QFT, 9 the QWT has a quaternion magnitude-phase representation that encodes image shifts in an absolute x, y-coordinate system. Being a local QFT, the QWT provides a theoretical framework for analyzing the phase behavior of 2-D image shifts. We conduct a thorough analysis of the QWT phase around edge regions. To demonstrate the power of our new transform, we also develop a multiscale flow/motion estimation algorithm for image registration based on the QWT phase. This paper is organized as follows. In Section 2 we briefly review the DWT and the dual-tree CWT. Section 3 develops the QWT, and Section 4 discusses some important properties of the new transform, in particular, its phase response to singularities. We develop and demonstrate the QWT phase-based image registration in Section 5. Section 6 concludes the paper and discusses the potentials of the QWT in other areas of image processing. 2. REAL AND COMPLEX WAVELET TRANSFORMS The discrete wavelet transform (DWT) represents a 1-D real signal f(t) in terms of shifted versions of a scaling function φ(t) and shifted and scaled versions of a wavelet function ψ(t). 1 The functions φ L,p (t) = 2 L φ(2 L t p) and ψ l,p (t) = 2 l ψ(2 l t p), l L, p Z form an orthonormal basis, and we can represent any f(t) L 2 as f(t) = c L,p φ L,p (t) + d l,p ψ l,p (t), (1) p Z l L,p Z where c L,p = f(t)φ L,p (t)dt and d l,p = f(t)ψ l,p (t)dt are the scaling and wavelet coefficients. L sets the coarsest scale space that is spanned by φ L,p (t). Behind each wavelet transform is a filterbank based on lowpass and highpass filters; we will use the notation φ h (t), ψ h (t) to denote the scaling and wavelet functions where h specifies a particular set of wavelet filters. The standard 2-D DWT is obtained using tensor products of 1-D DWTs over each dimension: the scaling function φ(x)φ(y) and three subband wavelets ψ(x)ψ(y), φ(x)ψ(y), and ψ(x)φ(y) are oriented in the diagonal, horizontal, and vertical directions, respectively. 1 The DWT suffers from shift-variance in all dimensions, but its complex counterparts can greatly ameliorate this problem. The 1-D dual-tree CWT expands a real signal in terms of two sets of wavelet and scaling functions obtained from two independent filterbanks. 6 The wavelet functions ψ h (t) and ψ g (t) from the two trees play the role of the real and imaginary parts of a complex analytic wavelet ψ c (t) = ψ h (t)+jψ g (t). The imaginary wavelet is the 1-D Hilbert transform of the real wavelet. The combined system is a 2 redundant tight frame that, by virtue of the fact that ψ c (t) is non-oscillatory, is shift-invariant. It is useful to recall that the Fourier transform of the imaginary wavelet Ψ g (ω) equals jψ h (ω) when ω > and jψ h (ω) when ω <. Thus, the spectrum of the complex wavelet function Ψ h (ω) + jψ g (ω) = Ψ c (ω) has no energy in the negative frequency region, making it an analytic signal. 6 Generalization of the 1-D CWT to a near shift-invariant wavelet transform in 2-D uses the theory of the 2-D Hilbert transform and 2-D analytic signal. However, there exist several different definitions of 2-D Hilbert transform. One straightforward extension of the 1-D definition to 2-D cancels out frequencies on all but a halfplane of the Fourier space (2-D Fourier frequencies u+v >, for example) and uses standard complex algebra for In practice, in order to have finite-length wavelets, the Hilbert transform is only approximately satisfied, ψ c (t) is only 6, 11 approximately analytic, and the CWT is only approximately shift-invariant.

3 +1 j +1 (a) F(Re(ψ c (x, y))) +j (b) F(Im(ψ c (x, y))) Figure 1. Fourier-domain relationships between the real and imaginary parts of a 2-D diagonally oriented complex wavelet. F( ) denotes the 2-D Fourier transform. The square corresponds to the 2-D Fourier plane with the origin at the center. Note how the real and imaginary parts constitute a 1-D Hilbert transform pair along the 45 line. manipulation. This is precisely the notion behind the 2-D dual-tree CWT. 6, 11 The 2-D CWT basis functions can be obtained by the tensor product of two 1-D complex wavelets, ψ c (x)ψ c (y), on the 2-D plane. 11 For instance, the complex wavelet oriented at 45 is given by (ψ h (x)ψ h (y) ψ g (x)ψ g (y)) + j(ψ h (x)ψ g (y) + ψ g (x)ψ h (y)). Figure 1 illustrates the spectral support of these 2-D complex wavelets. As a result, this construction yields a near shift-invariant 4 redundant wavelet frame, with 2-D CWT basis functions oriented along six directions. If we represent the complex wavelet ψ c (t) in terms of its magnitude and phase, then the CWT phase encodes signal shifts along a 1-D direction; when applied in 2-D, it suffers the same ambiguity problem as in the conventional 2-D Fourier transform. That is, a single phase component is not sufficient to encode image shifts in both horizontal and vertical directions on the 2-D plane. In the following we propose a new wavelet theory, with a quaternion phase representation, to tackle this phase ambiguity problem. 3. QUATERNION WAVELET TRANSFORM The QWT is based on an alternative definition of the 2-D Hilbert transform that zeros out not a half plane in the frequency domain but all but one quadrant (u, v >, for example). The resulting analytic signal has a real part and three additional imaginary components that are manipulated using quaternion algebra. 9 By organizing the four quadrature components as a quaternion, we obtain a 2-D analytic wavelet and its associated quaternion wavelet transform (QWT). Each quaternion wavelet consists of a standard DWT tensor wavelet plus three additional real wavelets obtained by 1-D Hilbert transforms along either or both coordinates. More specifically, if we denote the 1-D Hilbert transform operators along the x and y coordinates by H x and H y, respectively, then given the usual real tensor product wavelet ψ h (x)ψ h (y) from Section 2 (for the diagonal subband in this case), we complement it with H x {ψ h (x)ψ h (y)} = ψ g (x)ψ h (y), (2) H y {ψ h (x)ψ h (y)} = ψ h (x)ψ g (y), (3) H y H x {ψ h (x)ψ h (y)} = ψ g (x)ψ g (y). (4) Conveniently, each component can be computed as a combination of 1-D dual-tree complex wavelets. These three components resemble ψ h (x)ψ h (y) but are phase-shifted by 9 in the horizontal, vertical, and both directions, respectively. We can interpret these relationships in the Fourier domain as multiplying the quadrants of the Fourier transform of ψ h (x)ψ h (y) by ±j (where j = 1) and ±1 as shown in Figure 2. Using quaternion algebra, we can organize the four wavelet components to obtain a quaternion wavelet ψ q (x, y) = ψ h (x)ψ h (y) j 1 ψ g (x)ψ h (y) j 2 ψ h (x)ψ g (y) + j 3 ψ g (x)ψ g (y). Figure 3(a) shows the four components of a quaternion wavelet and its quaternion magnitude for the diagonal subband. Note the 9 phase shift of The set of quaternions H = {a + j 1b + j 2c + j 3d a, b, c, d R} has multiplication rules j 1j 2 = j 2j 1 = j 3 and j 2 1 = j 2 2 = 1. 12

4 j j j j (a) Ψ h (u)ψ h (v) +j j (b) Ψ g (u)ψ h (v) +j +j (c) Ψ h (u)ψ g (v) 1 +1 (d) Ψ g (u)ψ g (v) Figure 2. Fourier-domain relationships among the four components of a quaternion wavelet Ψ(u, v) in the diagonal subband. the components relative to each other. The magnitude of the quaternion wavelet ψ q (x, y) (square root of the sum-of-squares of all four components) is non-oscillatory, indicating that the transform is approximately shiftinvariant. The construction and properties are similar for the other two subband quaternion wavelets based on φ h (x)ψ h (y) and ψ h (x)φ h (y) (see Figure 3(b) and (c)). In summary, in contrast to the six complex pairs of CWT wavelets (12 functions in total), the QWT sports three quaternion sets of four QWT wavelets (12 functions in total). The quaternion wavelet transform is approximately a localized quaternion Fourier transform (QFT), 9 with the envelope of the wavelet as the space-domain window. The 2-D quaternion wavelets, like the 2-D analytic signal, 9 have spectral support only in a single (upper-right) quadrant in the QFT domain. The QFT of a real 2-D signal f is given by F q (u) = e j12πux f(x)e j22πvy dx, (5) R 2 where u = (u, v) indexes the frequency domain, x = (x, y) indexes the space domain, and the quaternion exponential e j12πux e j22πvy is the QFT basis function. The basis functions for the QWT are the scales and shifts of the quaternion wavelet (ψ h (x) j 1 ψ g (x))(ψ h (y) j 2 ψ g (y)) plus the wavelets for the other two subbands. 4. QWT PROPERTIES Since the QWT is based on combining 1-D CWT functions, it preserves the many attractive properties of the CWT. Furthermore, the quaternion organization and manipulation provides new features that are not present in either the 2-D DWT or CWT. In this section, we discuss some of the key properties of the QWT with special emphasis on the QWT phase Tight frame The QWT contains four orthonormal basis sets and thus forms a 4 redundant tight frame. The QWT wavelets can be written in a matrix form ψ h (x)ψ h (y) ψ h (x)φ h (y) φ h (x)ψ h (y) G = ψ g (x)ψ h (y) ψ g (x)φ h (y) φ g (x)ψ h (y) ψ h (x)ψ g (y) ψ h (x)φ g (y) φ h (x)ψ g (y). (6) ψ g (x)ψ g (y) ψ g (x)φ g (y) φ g (x)ψ g (y) Each column of the matrix G in (6) contains the 4 components of the quaternion wavelet corresponding to a subband of the QWT. For example, the first column contains the quaternion wavelet components in Figure 3(a), that is, the tensor product wavelet ψ h (x)ψ h (y) and its 2-D Hilbert transforms in equations (2) (4) in Section 3. Each row of G contains the wavelet functions necessary to form one orthonormal basis set. Since G has four rows, the quaternion wavelets form a 4 redundant tight frame. An important consequence is that the QWT is stably invertible. All wavelet bases in G can be generated using a 2-D dual-tree filter bank with O(N) efficiency.

5 (a) (b) (c) Figure 3. Each quaternion wavelet basis contains four components that are 9 phase shifts of each other in the vertical, horizontal, and both directions. (a) The set from the diagonal subband, from left to right: ψ h (x)ψ h (y) (a usual, real-valued tensor wavelet), ψ h (x)ψ g(y), ψ g(x)ψ h (y), ψ g(x)ψ g(y). The image on the far right is the quaternion wavelet magnitude ψ q (x, y), a non-oscillatory function, which implies the shift-invariance of the QWT. 6 (b) The set from the horizontal subband, from left to right: φ h (x)ψ h (y) (a usual, real-valued tensor wavelet), φ h (x)ψ g(y), φ g(x)ψ h (y), φ g(x)ψ g(y). (c) The set from the vertical subband, from left to right: ψ h (x)φ h (y) (a usual, real-valued tensor wavelet), ψ h (x)φ g(y), ψ g(x)φ h (y), ψ g(x)φ g(y) Relationship to 2-D CWT There exists a unitary transformation between the QWT coefficicents and the 2-D CWT coefficients. The complex wavelet for the 45 subband of the 2-D CWT is (ψ h (x) jψ g (x))(ψ h (y) jψ g (y)), similarly for the other subbands. We can obtain the CWT wavelets by multiplying the matrix G in (6) by the unitary matrix A = (7) 1 1 Therefore, both the CWT and the QWT are tight frames with the same redundancy. As we will see in the next section, both the CWT and the QWT phase encode 2-D signal shifts, but there exists no straightforward relationship between the phase angles of the QWT and CWT coefficients QWT magnitude-phase representation Using quaternion algebra, we can express each quaternion wavelet coefficient as q = q e j1θ1 e j3θ3 e j2θ2, 9 where q is the modulus of q and (θ 1, θ 2, θ 3 ) are the quaternion phase angles, with each angle uniquely defined within the range (θ 1, θ 2, θ 3 ) [ π, π) [ π 2, π 2 ) [ π 4, π 4 ]. For an image block with QWT coefficient q, q is almost invariant to signal shift while (θ 1, θ 2 ) encode the shift. The shift theorem for the QFT 9 approximately holds for the QWT since the QWT performs a local QFT analysis. The theorem states that when an image shift occurs (f(x) to f(x d)), the QFT phase undergoes the following changes: (θ 1 (u), θ 2 (u), θ 3 (u)) (θ 1 (u) 2πud 1, θ 2 (u) 2πvd 2, θ 3 (u)) where u = (u, v) are the

6 (a) (b) Figure 4. (a) The CWT has one phase angle that is linear in image shift d orthogonal to the wavelet s orientation. (b) The QWT has two phase angles to encode shifts, each linear in image shift (d 1, d 2) in absolute x, y-coordinates. v main quadrant d ( u, v ) (a) leakage quadrant (b) u leakage Figure 5. Illustration of a single edge in (a) space and (b) QFT domain. (a) Parameterization of the edge in a dyadic block. (b) QFT spectrum of the edge; shaded squares represent the QWT basis in the vertical, horizontal, and diagonal subbands. Spectrum of edge in (a), shown as 2 dark lines crossing at the origin, is captured by the horizontal subband, with spectral center (bu, bv). Region bounded by the dashed line demonstrates spectral support of QWT basis leaking into neighboring quadrant. axes of the 2-D QFT domain and d = (d 1, d 2 ). In the context of the QWT, (u, v) is the effective center of the signal spectrum within the frequency tile of the corresponding QWT basis for each coefficient. Therefore, (u, v) depends on the subband and scale of QWT basis and the actual spectral content of the image. Figure 4 contrasts the phase-shift property of the QWT with that of the 2-D CWT. The image blocks in Figure 4(a) and (b) undergo the same amount of shift toward the top left corner. Figure 4(a) illustrates how each CWT coefficient encodes the signal shift d along the direction perpendicular to the corresponding wavelet orientation (the 45 wavelet is shown), since it has only a single phase angle. On the other hand, the QWT encodes the signal shift (d 1,d 2 ) in x, y-coordinates using the two phase angles (θ 1,θ 2 ) as in Figure 4(b). Based on the relation between the QWT phase and image shifts, we can estimate the shift (d 1, d 2 ) from the phase change ( θ 1, θ 2 ) relative to a reference image. Conversely, we can estimate the phase shift once we know the signal shift. However, we need to first estimate the effective spectral center (û, v) for each QWT coefficient. For images having a smooth spectrum over the local spectral tile corresponding to each QWT basis, the center of the tile can be used as (û, v). Note that (û, v) should always lie in the first quadrant of the QFT domain (that is, û, v > ) QWT phase response for a single edge Since edges are important features in many image processing applications, it is interesting to see how we can use the QWT phase to analyze the location of edge singularities in images. When an image block contains a single edge, as illustrated in Figure 5(a), the spectrum is not smooth and we cannot choose (û, v) to be the center of

7 the local spectral tile. In fact, the QFT spectrum of a single edge at angle lies on two lines through the origin having orientations 9 and 9 in the QFT domain 9 as shown in Figure 5(b). Therefore, the effective spectral center can be expressed as (û, v) = c( cos, sin ), where c is a positive constant that depends on and the location of the spectral tile corresponding to each QWT basis. For an edge oriented at angle, any shift (d 1, d 2 ) in the (x, y) directions satisfying the constraint d 1 cos + d 2 sin = d (8) corresponds to the same shift as shifting the edge by d perpendicularly. Using the relations (û, v) = c( cos, sin ) in (8) and noting that 2πûd 1 = θ 1 and 2π vd 2 = θ 2, we obtain the expression where we choose θ 1 + θ 2 when tan >, and θ 1 θ 2 when tan <. d = θ 1 ± θ 2, (9) 2πc To verify this relationship, we apply the QWT to the edge image in Figure 5(a) and analyze the QWT magnitudes and phases corresponding to a sub-block. Figs. 6(a) and (b) verify that the coefficient magnitudes are almost invariant to signal shift d. Note that the magnitude is maximum when the edge orientation matches the orientation of the basis function ( for the φ(x)ψ(y) subband, 45 for the ψ(x)ψ(y) subband, and for the ψ(x)φ(y) subband). We experimentally study the linear relationships between θ 1 ± θ 2 and d for 1 various in all subbands to obtain the slope estimates 2πc in (9). For algorithmic simplicity, we obtain an estimate of c.7 for the ψ(x)φ(y) subband using the vertical edge ( = ) which gives the largest QWT magnitude for this subband; similarly for the ψ(x)φ(y) subband. Using the 45 edge, we obtain an estimate of c 1 for the ψ(x)ψ(y) subband. This simplification of using the same c in each subband to estimate d is valid because c is sensitive to the location of the spectral tile of the corresponding QWT basis but not to. Now, we can use equation (9) to estimate the edge offset d from the QWT phase using d = as the reference image. Figures 6(c) and (d) show the estimation accuracy for 2 subbands across various and d values. In small-magnitude regions, the phase angles are very sensitive to noise, and thus estimation errors are large. Using only the phase change of the QWT subband with large magnitude, we can estimate d with a maximum error of approximately.2, relative to the normalized unit edge length of the dyadic block under analysis. We also analyze the behavior of the third phase angle θ 3 for the same edge block in Figure 5(a). Theoretically, the QWT phase of an edge block should always give θ 3 = ± π 4.9 This corresponds to the singular case in the quaternion phase calculation, under which we can uniquely define only the sum or the difference of θ 1 and θ 2. 9 However, from our synthetic experiment, θ 3 gradually deviates from ± π 4 when the edge orientation changes from the diagonal to the vertical or horizontal directions. One possible explanation for this observation is leakage due to the imperfect 2-D Hilbert transform in the practical implementation of the QWT, which is more severe for the horizontal and vertical subbands. We discuss the proof for the QWT phase of an edge block here briefly and leave the details for a future paper. We can express the QWT coefficients for an edge block as the inner product between the QWT basis (window) and the edge signal in the QFT domain. This inner product is simply an integral over the main quadrant of the QFT domain (u, v > ) (see Figure 5(b)). This expression has the form A 1 + j 1 B 1 + j 2 B 1 j 3 A 1, where A 1 and B 1 are the quaternion components of the inner product integral involving the appropriate window and edge parameters. We can see that the phase angles of this quaternion expression obey the singular case, that is, θ 3 = ± π 4 and thus (θ 1,θ 2 ) follows the relationship in (9). When there is leakage, we can assume that the QWT basis has a small amount of overlap in the neighboring quadrant, shown by the region bounded by the dashed line in Figure 5(b). Let A 2 and B 2 be the quaternion components in the inner product integral between the QWT basis (window) and the edge signal in the leakage quadrant of the QFT domain. Then, we can modify the previous expression to (A 1 + A 2 ) + j 1 (B 1 + B 2 ) + j 2 (B 1 B 2 ) + j 3 (A 2 A 1 ). From Figure 5(b), we see less overlap between the edge signal and the leakage region when the edge orientation moves close to 45 (diagonal). We can simulate this effect by finding θ 3 for the modified expression (for the leakage case) θ 3 = 1 ( ) 1 ɛ 2 arcsin, (1) 1 + ɛ

8 6 6 Magnitude Magnitude.5 d (a) Magnitudes.5 d (b) Magnitudes Estimation error d 45 9 (c) Estimation errors Estimation error d (d) Estimation errors Figure 6. Magnitude of a QWT coefficient as a function of edge orientation and offset d for the (a) horizontal φ(x)ψ(y) subband and (b) diagonal ψ(x)ψ(y) subband. (c), (d) Absolute error for estimating d by the phase angles (θ 1,θ 2) from the same two subbands, relative to the normalized edge length of the dyadic block (= 1 unit). Estimates are accurate in the regions where the coefficient magnitudes are large. (a) (b) (c) (d) (e) Figure 7. Local edge detection using the QWT. (a) Highlights of different regions in the cameraman image are shown on the left; (b) (e) on the right are edge estimates from the corresponding quaternion wavelet coefficients. The upper row shows the original image region, the lower row shows a wedgelet (see Figure 5(a)) having the edge parameter estimates (, d). where ɛ = A2 2 +B2 2 is a rough measure of the ratio of basis or signal energy in the main quadrant to energy in the A 2 1 +B2 1 leakage quadrant. Moreover, we can show that the sum (or difference) between θ 1 and θ 2 are the same as in the case without leakage. These derivations are consistent with our experimental results. Based on our experimental analysis of the variation between θ 3 and edge orientation, we have designed a hybrid algorithm combining our knowledge of how θ 3 and the magnitude ratio between subbands change according to. Given the 3 subband QWT coefficients for a single-edge image block, we are able to estimate the edge orientation with a maximum error of ±3. Combined with edge offset estimation, we can perform local edge estimation from the QWT coefficients as shown in Figure 7.

9 (a) Middle frame (b) Disparity estimates Figure 8. Multiscale QWT phase-based disparity estimation results. (a) Middle frame of the Rubik s cube image sequence. 9 (b) Disparity estimates between two images in the sequence, shown as arrows overlaid on top of the reference image. 5. QWT-BASED IMAGE FLOW ESTIMATION The relative location information encoded in the phase angles of the QWT is particularly useful for image registration. 13 Given two images, we estimate the local shift of the target image compared to the reference image. We can then use this disparity information to build a warping function to align the two images and then super-resolve. A previous phase-based method for this problem uses Gabor filters and requires computation of the local phase derivative function. 9 Our method employs a more efficient multiscale scheme based on the QWT phase which first estimates shifts at a coarse scale and then refines the coarse-scale estimates using finer scale information. Recall that the QWT shift theorem states that a shift (d 1, d 2 ) in an image changes the QWT phase from (θ 1 (u), θ 2 (u), θ 3 (u)) to (θ 1 (u) 2πud 1, θ 2 (u) 2πvd 2, θ 3 (u)). By manually translating the reference image by one pixel both horizontally and vertically, we can first obtain estimates for the effective spectral center (u, v) for each dyadic block across all scales. The range of QWT phase angles limits our estimates (u, v) to [ 1 4D, 1 4D 2D, 1 2D ) and [ 1 ) for horizontal and vertical shifts respectively. Based on the linear relationship between shift and phase, we can then estimate the horizontal and vertical shifts independently based on the difference between the QWT phase corresponding to the same local blocks in the source and reference images. The main challenge in this phase-based approach is the phase wrap-around due to the limited range of phase angles. Due to the phase wrap around, each observed phase difference can be mapped to more than one disparity estimate. We tackle this problem with a multiscale approach. An initial estimate in the coarsest scale L, which is more robust to phase wrap-around, gives a rough estimate of the displacement of each region. Using bilinear interpolation of this coarse scale estimate, we obtain a prediction for the disparity at scale L 1. We unwrap the disparity estimate at scale L 1 towards the predicted estimates from scale L. Similar correction steps for the estimates in the finer scales yield a disparity estimate for each scale. Finally, we average the estimates from all scales to give a robust estimate of the disparity map. Figure 8 shows the result of our QWT phase-based disparity estimation scheme for a rotating Rubik s cube video sequence. 9 The arrows indicate both the directions and magnitudes of the local shifts, with the magnitudes stretched for better visibility. We can clearly see the rotating motion of the Rubik s cube on the circular stage. Our algorithm is entirely based on the dual-tree QWT and its shift theorem without the need to compute any phase derivative function in advance. Thus, compared to previous phase-based approaches of disparity estimation, 9 our approach is more computationally efficient (order O(N) as opposed to O(N log N)). We use a similar reliability measure as in the confidence mask 9 to threshold unreliable phase and offset estimates. However, provided a continuous underlying disparity flow, our multiscale scheme yields a denser disparity map, even for smooth regions within an image. In our current algorithm, we use the QWT phase in only the diagonal subband to obtain the estimates; we expect more accurate performance when we merge estimates from all three subbands.

10 6. CONCLUSIONS In this paper, we have extended the 2-D CWT to a new QWT using an alternative definition of the 2-D Hilbert transform, 2-D analytic signal, and quaternion algebra. The resulting quaternion wavelets have three phase angles; two of them encode phase shifts in an absolute x, y-coordinate system. The QWT s approximate shift theorem enables efficient and easy-to-use analysis of the phase behavior around edge regions. As a proof-ofconcept, we have developed a new multiscale phase-based image registration scheme. Beyond 2-D, the generalization of the Hilbert transform to n-d signals using hypercomplex numbers can be used to develop wavelet transforms in higher dimensions suitable for signals containing more complicated lowdimensional manifold structures. 14 The QWT developed here plays a central role in the analysis of (n 2)-D manifold singularities in n-d space. Finally, we note that there are interesting connections between our QWT and the (non-redundant) quaternion wavelet representations of Ates and Orchard 15 and Gang and Orchard. 16 ACKNOWLEDGMENTS We thank T. Gautama et al. for providing their image sequences and J. Barron et al. for making their image sequences publicly available. This material is based on research partially sponsored by AFRL/SNAT under agreement number AFRL grant FA (BAA 4 3 SNK Call 9). The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of AFRL/SNAT or the U.S. Government. REFERENCES 1. J. Shapiro, Embedded Image Coding Using Zerotrees of Wavelet Coefficients, IEEE Trans. Signal Processing 41, April M. Crouse, R. Nowak, and R. Baraniuk, Wavelet-Based Statistical Signal Processing Using Hidden Markov Models, IEEE Trans. Signal Processing 46, April H. Choi, J. Romberg, R. Baraniuk, and N. Kingsbury, Hidden Markov Tree Modeling of Complex Wavelet Transforms, IEEE Intl. Conf. on Acoustics, Speech, and Signal Processing, June L. Sendur and I. Selesnick, Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency, IEEE Trans. Signal Processing 5, pp , Nov D. L. Donoho, De-noising by soft-thresholding, IEEE Trans. Info. Theory 41(3), pp , N. G. Kingsbury, Complex wavelets for shift invariant analysis and filtering of signals, J. Applied and Computational Harmonic Analysis 1, pp , May J. Romberg, H. Choi, and R. Baraniuk, Multiscale edge grammars for complex wavelet transforms, in Proc. of Intl. Conf. on Image Processing, (Thessaloniki, Greece), Oct J. M. Lina and M. Mayrand, Complex Daubechies wavelets, J. Applied and Computational Harmonic Analysis 2, pp , T. Bülow, Hypercomplex Spectral Signal Representations for the Processing and Analysis of Images. PhD thesis, Christian Albrechts University, Kiel, Germany, M. Vetterli and J. Kovacevic, Wavelets and Subband Coding, Prentice Hall, Englewood Cliffs, NJ, I. W. Selensnick and K. Y. Li, Video denoising using 2-D and 3-D dual-tree complex wavelet transforms, in Proc. of SPIE Wavelets X, 76, (San Diego, CA), August I. L. Kantor and A. S. Solodovnikov, Hypercomplex Numbers, Springer-Verlag, B. Zitova and J. Flusser, Image registration methods: A survey, Image and Vision Computing 21, pp , W. L. Chan, H. Choi, and R. Baraniuk, Directional Hypercomplex Wavelets for Multidimensional signal analysis and processing, IEEE Intl. Conf. on Acoustics, Speech, and Signal Processing, May H. Ates, Modeling location information for wavelet image coding. PhD thesis, Princeton University, Princeton, New Jersey, G. Hua, Noncoherent image denoising, Master s thesis, Rice University, Houston, Texas, 25.