SIGNAL DECOMPOSITION METHODS FOR REDUCING DRAWBACKS OF THE DWT

Engineering Review Vol. 32, Issue 2, 70-77, 2012. 70 SIGNAL DECOMPOSITION METHODS FOR REDUCING DRAWBACKS OF THE DWT Ana SOVIĆ Damir SERŠIĆ Abstract: Besides many advantages of wavelet transform, it has several drawbacks, e.g. ringing, shift variance, aliasing and lack of directionality. Some of them can be eliminated by using wavelet packet transform, stationary wavelet transform, complex wavelet transform, adaptive directional lifting-based wavelet transform, or adaptive wavelet filter banks that use either L 2 or L 1 norm. This paper contains an overview of these methods. Keywords: wavelet transform filter bank lifting scheme complex wavelet transform adaptive wavelets 1. INTRODUCTION Sparse representation of signals (images, movies or other multidimensional signals) is very important for a wide range of applications, such as compression, denoising, feature extraction, estimation, superresolution, compressive sensing [1], blind separation of mixtures of dependent signals [2] and many others. In most cases, linear transformations or filter banks are used for obtaining the sparse representation, e.g. discrete cosine transform (DCT) or DCT filter bank; discrete wavelet transform (DWT) or DWT filter bank. Hence, some variants of these transformations are involved in modern methods for the compression of signals (MP3 ), images (JPEG, JPEG2000 ), as well as in many other implementations. An important advantage of the DWT is low computational complexity when it is implemented using wavelet filter banks. Usually, finite support filters are used for decomposition and reconstruction. Wavelet transform provides sparse representation for a large class of signals. However, DWT filter banks have several drawbacks: ringing near discontinuities, shift variance, the lack of directionality of decomposition functions and some others. Recently, a lot of research and publications are focused on solving these problems. One of the methods for obtaining sparser representation and lower dissipation in wavelet domain is the use of adaptive wavelet filter banks [3][4]. Lower ringing is achieved when L 2 or L 1 norm is minimized in some neighborhood of each data sample by adapting wavelet functions [5][6]. Problems with directionality and shift variance in 2D or n-dimensional wavelets can be reduced using the dual tree complex wavelet transform [7]. Moreover, adaptive directional lifting-based wavelet transform adjusts directions in dependence of the image orientation in the neighborhood on every pixel. It achieves directionality and sparse representation [8]. In all implementations, improvement can be reached using multiple representations. More statistical estimators with known properties for the same estimated value will give better quality of the estimation. One possible solution for the realization of multiple representations in wavelet domain for denoising is given in [9]. In this paper we give an overview of some of the mentioned methods. 2. WAVELET TRANSFORM Every analog signal with finite energy can be decomposed into a sum of shifted and dilated

71 A. Sović, D. Seršić: Signal Decomposition Methods wavelet functions and shifted scale functions : t k, 22, (1) where c(k) are scale coefficients and d(j,k) wavelet coefficients. This is a dyadic variant of the discrete wavelet transform (DWT). Scale and wavelet coefficients are calculated using scalar products:, 2, (2) 2. (3) Hence, filter banks with perfect reconstruction property can be used as a simple realization of the DWT (Figure 1). Filter is low-pass and associated to the scale function. The filter is high-pass and linked to the wavelet function [10]. Lifting scheme is another efficient way of realization of the DWT. Filter S(z) is the prediction stage of the lifting step and filter T(z) is the update stage. The result of the low-pass filtering is approximation A, and the result of the high-pass filtering is detail D. Alternating appropriate lifting steps can provide for desired frequency response of the filters (Figure 2) [11]. 3. 2D WAVELET TRANSFORM Separable 2D discrete wavelet transform is the simplest form of the two-dimensional wavelet generalization. It consists of a standard 1D DWT applied to each row and then to each column. If an image has N 1 rows and N 2 columns, decomposition results in four quarter size images (N 1 /2 x N 2 /2): details (AD, DA, DD) and approximation AA. Approximation AA is product of two low-pass filters and provides for an input to the next decomposition level (Figure 3). The reconstruction is performed in the opposite way: first on columns, then on rows. Separable 2D DWT has three wavelet functions (m and n are coordinates of the input image):, LH wavelet,, HL wavelet, (4), HH wavelet, and one scale function, associated to the approximation AA [12]. HH wavelet is the output of the cascade of the highpass filters which produce the function from the rows (dimension m) and columns (dimension n). Its frequency response has 45 and 45 orientations. This is called a checkerboard artifact. The separable 2D DWT always gives quadrant symmetric frequency responses. So, if the direction is important, it always fails in providing a sparse representation. In an analogous way, separable 2D DWT can be efficiently realized using 1D lifting steps. Figure 1. Wavelet filter bank Figure 3. Separable 2D wavelet transform 4. DRAWBACKS OF THE DWT Figure 2. Lifting scheme Although wavelet transform has a lot of benefits compared to the Fourier transform, there still exist some drawbacks [5]: Problem 1. Oscillations. Wavelet coefficients oscillate with positive and negative values around

Engineering Review Vol. 32, Issue 2, 70-77, 2012. 72 the singularities, which complicate their detection and modeling. Problem 2. Shift variance. If input signal is shifted in time or space, wavelet coefficients of the decimated DWT will be changed. Problem 3. Aliasing. Wavelet coefficients are calculated using iterative time discrete operations with the non-ideal high and low pass filters. Therefore, aliasing can appear. Inverse DWT cancels aliasing, but only if the wavelet coefficients were not processed. Problem 4. Lack of directionality. Separable 2D DWT efficiently detects horizontal and vertical edges. But, if the edges are under an acute angle, unwanted checkerboard artifacts appear. One of the possible solutions for mentioned problems is the wavelet packet transform. 5. WAVELET PACKET TRANSFORM The wavelet packet transform is based on wavelet filter bank (Figure 1), although the high-pass channel is also decomposed in similar manner and iterated as the low-pass channel (Figure 4). The complete tree is achieved. It offers high degree of freedom but with complex data-structure algorithms. The selection of best filters reduces shift sensitivity. However, a general representation of wavelet packet is not shift invariant. Wavelet packets perform better in terms of fidelity of direction but not in terms of improved directionality. The high-pass coefficients will oscillate around singularities of the signal [13][14]. 6. STATIONARY WAVELET TRANSFORM The decimation step after filtering makes the standard DWT shift variant. Therefore, stationary wavelet transform has a similar tree structure without any decimation. The balance for perfect reconstruction is preserved through upsample the filters (Figure 5). Wavelet coefficients at each level are of equal length and the method is shift-invariant. A drawback is a very large redundancy and increased computational complexity. The lack of directionality and oscillating persist because the stationary wavelet transform is based on a filterbank structure [13][15]. The complex wavelet transform (WT) solves the checkerboard problem. It is based on the Hilbert transform. Figure 5. Stationary wavelet transform 7. THE HILBERT TRANSFORM Real sine signal sin in Fourier domain consists of two frequencies, positive and negative. Complex exponential signal contains only one frequency. If the signal does not have negative frequencies, being causal in the frequency domain, it is called the analytic signal. Hilbert transform filter can be applied to real signals to produce complex part of the analytic signal. The impulse response of the Hilbert transform filter is 1. Its Fourier transform is 1 sign. The response of the Hilbert transform filter for input signal and for its frequency response is given by convolution Figure 4. Wavelet packet transform 1 (5)

73 A. Sović, D. Seršić: Signal Decomposition Methods in the time domain or by multiplication in the frequency domain sgn (6) Analytic signal is the sum of a signal and its Hilbert transform multiplied by j: or 2, 0,, 0, (7) 0, 0. Fourier transform of analytic signals equals zero for negative frequencies and is generally non-zero for positive frequencies [16][17]. 8. COMPLEX WAVELET TRANSFORM The wavelet function is often real, so it is not analytic. Using the Hilbert transform, every real non analytic wavelet function can be converted into an analytic one. Let us denote complex wavelet function as where is real and is imaginary part. Moreover, let and make the Hilbert transform pair. The same goes for the complex scale function:, where is real and is imaginary part and they form a Hilbert transform pair, too [18]. The WT can be used for analyzing either real or complex signals [7]: it solves most of the problems from Chapter 4. The most common approach for realization of the WT with the Hilbert transform is the dual-tree approach (Figure 6) [19]. The dual tree approach uses two real DWT-s: one for acquiring the real part of the transform, and the other for the imaginary part. Real wavelet is associated with the upper tree, and imaginary wavelet is associated with the lower tree. Each tree uses different sets of filters that satisfy perfect reconstruction conditions. is a low-pass and is a high-pass filter for the upper filter bank, and and are the low and the high-pass filters for the lower filter bank. Low-pass filters and must be designed so their wavelet functions are approximate Hilbert transform pairs. It appears that the connection between them is a half-sample delay: 0.5 (8) Figure 6. Realization of the dual tree complex wavelet transform Since these filters are discrete, the half-sample delay cannot be exactly realized. Therefore, the term is expounded into the magnitude and the phase functions of the Fourier transform:, (9) 0.5. (10) Impulse response of filter is of infinite length, and its transfer function is not rational. Even if it were of finite length, would not be. Thus, given conditions will be used only approximately and the complex wavelets will be approximately analytic. Some methods of the filter design are described in [7]. In practice, it turns out, that the same filters in every level of the dual tree decomposition do not get a good approximation of the analytic wavelets. One possible solution for this problem is the use of different filters for the first stage, e.g. the same short length filters in both trees, but shifted for one sample. On every other level, the filters need to satisfy the half sample delay, as already mentioned. Another variant is an implementation that utilized swapping. The filters of the upper and the lower tree are alternated in each decomposition stage: filters and are in the upper tree at the even levels, while filters and are there at the odd levels [7]. Separable 2D WT generalization is obtained in analogous way as for the real wavelets. 1D WT is applied first on each row, and then on each column. Since the WT is implemented using the dual tree approach, the resulting 2D WT will have four trees. Two of them provide for the real parts and two of them provide for the imaginary parts of the wavelet coefficients.

Engineering Review Vol. 32, Issue 2, 70-77, 2012. 74 After the transform is applied to the rows, the wavelet function is and the scale function is. Then, the transform is applied to the columns, and the wavelet function is denoted as, while the scale function is. All functions are analytic and oriented. To get all orientations of the complex 2D wavelets, the preceding functions must be multiplied as follows:,,,, (11),,,, (12),,, (13),,, (14),,, (15),,, (16) where new six wavelets have real and imaginary parts, 1 2,,,,, (17), 1 2,,,,, (18), 1 2,,,,, (19), 1 2,, for 1,2,3 and,,.,, for the LH wavelet; (20) (21),, for the HL wavelet; and,, (22) (23) for the HH wavelet. Factor 1 2 is introduced for orthonormality. The 2D WT is four times more computationally complex than the standard 2D wavelet transform. But, the real and the imaginary parts are oriented equally and problems with checkerboard effects are minimized. This transform is approximately analytic and therefore approximately shift invariant [7][20][21]. Another, completely different, approach for solving the problem of the checkerboard effects is using the adaptive directional lifting-based wavelet transform. 9. ADAPTIVE DIRECTIONAL LIFTING- BASED WAVELET TRANSFORM Unlike traditional 2D DWT, in the adaptive directional lifting-based wavelet transform (ADL) each lifting step depends on a local orientation of an image. The orientation does not necessarily have to be horizontal or vertical [8]. A typical lifting scheme begins with splitting the samples of the input image on even, and odd subset, at some index n. In the prediction step, the odd samples are predicted from the neighboring even samples. Here, the neighboring samples do not have to be from the horizontal or the vertical direction only. They can be positioned at some angle, as well. Prediction of the odd samples is calculated using:, sign 1 tg,, (24) where are weights given by the filter coefficients. An optimal angle can result in that the samples sign 1 tg, are not at the integer position on the image. Hence, interpolation

75 A. Sović, D. Seršić: Signal Decomposition Methods filters parameters a k can be found using some interpolation technique, like sinc function or by some other interpolation filter. One good method is minimizing the least-square error: min,,,,,,. (25) Although the prediction angle is a continuous variable, it has been found that nine uniformly quantized discrete angles, 0, 1, 2, 3, 4 are good enough in many applications of the ADL. High-pass wavelet coefficients are given as a difference of the predicted and real values:,,,. (26) In the update stage, even samples are found by:, sign tg,. (27) where are weights given by the filter coefficients. Angle does not have to be equal to the angle in the predict stage, but most often, it is. As before, sign tg, does not necessarily have integer values, so it needs to be interpolated between pixels. The low-pass coefficients are given by adding,,,. (28) After the 1D ADL is performed on each index n (the generalized vertical transformation), the generalized horizontal transformation is performed on each index m in the same way as before. Optimal predicted angle of the generalized horizontal transformation does not have to be necessarily perpendicular to the vertical angle [8]. As usual, more decomposition levels can be used. Hence, the ADL is well adapted to directional properties of images [8][22]. But, for reducing ringing near the edges, another adaptive approach should be used. 10. ADAPTIVE WAVELET FILTER BANK The realization of an adaptive wavelet filter bank is given in Figure 7. The parameters, and are found in such a way that detail D(z) is minimal. Plenty of the adaptation methods can be used for the minimization: the minimum of least squares errors on a window, weighted least squares with forgetting factor or an iteratively reweighted least square which enables minimization of the arbitrary norm, e.g. minimum least absolute values [3][5][6]. For a sliding window of the length N, we obtain, where is the output, are the estimated parameters, and 1 1 1 (29) is the input matrix. The estimation error is θ. (30) The well-known solution for the minimum of the least square error (L 2 norm) is, (31). (32) If W is a weighted matrix, the cost function is and the solution is (33) W W. (34) If diagonal elements of the weighted matrix are θ, (35) where y is the i-th element of the vector y, and U is the i-th row of the matrix U, the cost function is Fθ. (36)

Engineering Review Vol. 32, Issue 2, 70-77, 2012. 76 This is exactly the sum of the absolute values or L1 error norm. Since θ is not known in advance, W is also not known: they must be found iteratively. The minimum of the least absolute values gives sparser representation than the minimum of the least squares (Figure 8). Parameters of the update stage are optimized analogously. Separable or nonseparable solutions can be implemented for the 2D signals, as well [5][6]. X Figure 7. First stage of the lifting scheme with adaptive filters detail D parameter b 1 2 1 0 2 1 z 1 X e X o 1/2 1 + z 1 z 1 (1 z ) 2 U1 1/8 z 1 (1 z ) 2 U2 50 100 150 200 250 300 350 400 450 n L 2 norm 50 100 150 200 250 300 350 400 450 500 n Figure 8. Comparison of the minimum least square error and the least absolute error 11. CONCLUSION Prediction stage S(z) 3/16 z 1 (1 z ) 2 U3 L 1 norm In this paper, several decomposition methods for 1D and 2D signals are described. One method was developed by the authors of this paper; the others are a selection of the state-of-the-art signal and image transform techniques. The dual tree complex wavelet transform (WT) is presented and the design of the filters for its realization is described. The main advantage of the WT is its approximate shift invariance and lesser directional sensitivity. If the image contains textures that are under certain angle, the adaptive directional lifting based wavelet transform (ADL) can be used. It adapts to the directions of the decomposed signal. Finally, the 5/24 b 1 b 2 b 3 1+ z Y d D Y a c 1 c 2 c 3 V1 V2 V3 z 1 (1 z) 2 1/8 z 1 (1 z ) 2 1/4 z 1 (1 z ) 2 3/16 T(z) 5/24 Update stage A adaptive wavelet lifting filter bank developed by the authors of this paper provides less ringing near the singularities and thus sparser representation of signals in the transform domain. This paper gives an overview of signal and image transform methods that provides for sparser representation, which is a very important property for applications such as denoising, estimation, compression, compressive sensing, blind separation of statistically dependent sources and many others. 12. LIST OF SYMBOLS 1D and 2D wavelet function,, Wavelet coefficient, 1D and 2D scale function,, Scale coefficient Complex wavelet and scale func., Real wavelet and scale func., Imaginary wavelet and scale func., Low-pass filters h 0 (n),g 0 (n),s(z) High-pass filters h 1 (n),g 1 (n),t(z) Approximations A, AA Details D,AD,DA,DD Input signal, even and odd samples x, x e, x o Filtering parameters,, High and low pass coefficients h, l Horizontal and vertical angle, Prediction of even and odd samples u h, p 0 Sliding window N Inputs, outputs for optimization U, u 1,, Y, y d Parameters for optimization, b 1, b 2, b 3 REFERENCES [1] Baraniuk, R. G.: Compressive sensing. IEEE Signal Processing Mag., Vol. 24 (2007), No. 4, pp. 118 120, 124 [2] Li, Y., Amari, S., Cichocki, A., Guan, C.: Underdetermined blind source separation based on sparse representation, IEEE Trans. Inf. Theory, Vol. 52 (2006), No. 2, pp. 3139 3152 [3] Seršić, D.: A realization of wavelet filter bank with adaptive filter parameters, In Proc. EUSPICO 2000. Tampere, 2000, vol. 3, pp. 1733-1736 [4] Vrankić, M., Seršić, D., Sučić, V.: Adaptive 2-D wavelet transform based on the lifting scheme with preserved vanishing moments. IEEE Transactions on Image Processing. Vol. 19 (2010), No. 8; pp. 1987-2004,.

77 A. Sović, D. Seršić: Signal Decomposition Methods [5] Sović, A., Seršić, D.: Robustly adaptive wavelet filter bank using L1 norm, In Proc. of the IWSSIP, Bosnia and Herzegovina, 2011, pp. 9-12 [6] Sović, A., Seršić, D.: Adaptive wavelet image decomposition using LAD criterion, In Proc. of the Eusipco, Spain, 2011, pp. 594-598 [7] Selesnick, I. W., Baraniuk, R. G., Kingsbury, N. G.: The Dual-Tree Complex Wavelet Transform. IEEE Signal Processing Magazine, vol. 22 (2005), No. 6, pp. 123-151 [8] Ding, W., Wu, F., Wu, X., Li, S., Li, H.: Adaptive directional lifting-based wavelet transform for image coding. IEEE Transaction on Image Processing, (2007), pp. 416-427 [9] Choi, H., Baraniuk, R.: Multiple Wavelet Basis Image Denoising Using Besov Ball Projections. IEEE Signal Processing Letters, Vol. 11 (2004), No. 9 [10] Mallat, S.: A wavelet tour of signal processing, Academic Press, London, 1999 [11] Sweldens, W., Daubechies, I.: Factoring Wavelet Transforms into Lifting Steps. J. Fourier Analysis and Applications, Vol. 29 (1997), pp. 511-546 [12] Antonini, M., Barlaud, M., Mathieu, P., Daubechies, I.: Image coding using wavelet transform. IEEE Trans. Image Processing, Vol. 1 (1992), No. 2, pp. 205-220 [13] Weickert, T., Benjaminsen, C., Kiencke, U.: Analytic wavelet packets - combining the dual-tree approach with wavelet packets for signal analysis and filtering, IEEE Transaction on signal processing, Vol. 57 (2009), No. 2, pp. 493-502 [14] Ramchandran, K., Vetterli, M.: Best Wavelet Packet Bases in a Rate-Distorsion Received: 09.02.2012. Sense, IEEE Trans. Image processing, Vol. 2 (1993), No. 2, pp. 160-175 [15] Shensa, M. J.: The discrete wavelet transform: wedding the a trous and Mallat algorithms, IEEE Transaction on Signal Processing, Vol 40, (1992), No 10, pp. 2464-2482 [16] King, F. W.: Hilbert Transforms, Cambridge University Press, Cambridge, 2009 [17] Duoandikoetxea, J.: Fourier Analysis, American Mathematical Society, Providence, Rhode Island, 2000 [18] Selesnick, I. W.: The design of approximate Hilbert transform pairs of wavelet bases, IEEE Trans. On Signal Processing Vol. 50 (2002), pp. 1144 1152 [19] Yang, Y.: Image coding using Dual-Tree discrete wavelet transform, IEEE Trans. on Image Processing, Vol. 17 (2008), No 9, pp. 1555-1569 [20] Kingsbury, N. G.: Shift Invariant Properties of the Dual-Tree Complex Wavelet Transform, Proc. IEEE Conf. on Acoustics, Speech and Signal Processing, Phoenix, (1999), pp. 16-19 [21] Kingsbury, N. G., Complex Wavelets for Shift Invariant Analysis and Filtering of Signals, Journal of Applied and Computational Harmonic Analysis, Vol. 10 (2001), No. 3, pp. 234-253 [22] Liu, Y., Ngan, K.N.: Weighted adaptive lifting-based wavelet transform for image coding, IEEE Trans Image Process. Vol. 17 (2008) No. 4, pp. 500-11 Accepted: 14.05.2012. Author s address: Ana Sović Damir Seršić Faculty of Electrical Engineering and Computing, University of Zagreb Unska 3, 10000 Zagreb ana.sovic@fer.hr damir.sersic@fer.hr