The Nonsubsampled Contourlet Transform: Theory, Design, and Applications

Transcription

1 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 The Nonsubsampled Contourlet Transform: Theory, Design, and Applications Arthur L. Cunha, Student Member, IEEE, Jianping Zhou, Student Member, IEEE, and Minh N. Do, Member, IEEE, Abstract The contourlet transform was proposed to address the lack of geometrical structure in the separable two-dimensional wavelet transform. Because of its filter bank structure the contourlet transform is not shift-invariant. In this paper we propose the nonsubsampled contourlet transform (NSCT) and study its applications. The construction proposed in this paper is based on a nonsubsampled pyramid structure and nonsubsampled directional filter banks. The result is a flexible multi-scale, multi-direction, and shift-invariant image decomposition that can be efficiently implemented via the à trous algorithm. At the core of the proposed scheme is the nonseparable two-channel nonsubsampled filter bank. We study the filter design problem and propose a design framework based on the mapping approach. We exploit the less stringent design condition of the nonsubsampled filter bank to design filters that lead to a NSCT with better frequency selectivity and regularity when compared to the contourlet transform. The proposed design allows for a fast implementation based on a lifting or ladder structure that only uses one-dimensional filtering in some cases. In addition, our design ensures that the corresponding frame elements are regular, symmetric, and the frame is close to a tight one. We assess the performance of the NSCT in image denoising and enhancement applications. In both applications the NSCT compares favorably to other existing methods in the literature. Index Terms Multidimensional Filter Banks, Nonsubsampled Filter Banks, Frames, Contourlet transform, Image Denoising, Image Enhancement. A. L. Cunha and J. Zhou are with the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana IL 68 ( {cunhada,jzhou}@uiuc.edu) M. N. Do is with the Department of Electrical and Computer Engineering, the Coordinated Science Laboratory, and the Beckman Institute, University of Illinois at Urbana-Champaign, Urbana IL 68 ( minhdo@uiuc.edu). This work was supported by a CAPES fellowship, Brazil, and the U.S. National Science Foundation under grant CCR (CAREER).

2 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 I. INTRODUCTION A number of image processing tasks are efficiently carried out in a domain other than the pixel domain, often by means of an invertible linear transformation. For example, image compression and denoising are efficiently done in the wavelet transform domain [], []. A good transform or representation would capture the essence of a given signal or class of signals with few basis elements. The set of basis functions completely characterizes the transform and this set can be redundant or not, depending on whether the basis functions are linear dependent. By allowing redundancy, it is possible to enrich the set of basis functions so that the representation is more efficient in capturing some signal behavior. Applications such as edge detection, contour detection, denoising, and image restoration can greatly benefit from redundant representations. In the context of multi-scale expansions implemented with filter banks, dropping the linear independence requirement offers the possibility of a shift-invariant expansion, a crucial property in some applications [3]. For instance, in image denoising by thresholding in the wavelet domain, the lack of shift-invariance causes pseudo-gibbs phenomena around singularities [4]. Not surprisingly, most stateof-the-art wavelet denoising routines (see for example [5], [6], [7]) use an expansion with less shift sensitivity than the standard maximally decimated wavelet decomposition the most common being the nonsubsampled wavelet transform (NSWT) computed with the à trous algorithm [8]. In addition to shift-invariance, it has been recognized that an efficient image representation has to account for the geometrical structure pervasive in natural scenes. In this direction, several representation schemes have recently been proposed. These include adaptive schemes such as wedgelets [9], [] and bandelets [], and non-adaptive ones such as curvelets [] and contourlets [3]. The contourlet transform is a multi-directional and multi-scale transform that is constructed by combining the Laplacian pyramid [4], [5] and the directional filter bank (DFB) [6]. Due to downsamplers and upsamplers present in both the Laplacian pyramid and the DFB, the contourlet transform is not shift-invariant. In this paper we propose an overcomplete transform that we call the nonsubsampled contourlet transform (NSCT). The NSCT is a fully shift-invariant, multi-scale, and multi-direction expansion that has better directional frequency localization and a fast implementation. In essence, the proposed structure achieves a similar subband decomposition as that of contourlets, but without downsamplers and upsamplers in it. Because of its redundancy, the filter design problem of the NSCT is much less constrained than Denoising by thresholding in the NSWT domain can also be realized by denoising multiple circular shifts of the signal with a critically sampled wavelet transform and then averaging the results. This has been termed cycle spinning after [4].

3 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 3 that of contourlets. This enables us to design filters with better frequency selectivity thereby achieving a better subband decomposition. Using the mapping approach we provide a framework for filter design that ensures good frequency selectivity in addition to having a fast implementation through ladders steps. The NSCT has proven to be very efficient in image noise removal and image enhancement as we show in this paper. The paper is structured as follows. In Section II we describe the NSCT and its building blocks. We introduce a pyramid structure that ensures the multi-scale feature of the NSCT and the directional filtering structure based on the DFB. The basic unit in our construction is the nonsubsampled filter bank (NSFB) which is discussed in Section II. In Section III we study the issues associated with the NSFB design and implementation problems. We propose a design methodology based on one-dimensional (-D) prototype filters that allows fast implementation through ladder steps. Several examples are presented. Applications of the NSCT in image denoising and enhancement are discussed in Section IV. Concluding remarks are drawn in Section V. Notation: Throughout the paper, a two-dimensional (-D) filter is denoted by H(z) where z = [z, z ] T. If m = [m, m ] T is a -D vector, then z m = z m zm whereas if M is a matrix, then z M = [z m,z m ] with m,m the columns of M. In this paper we often deal with zero-phase -D filters. Such filters can be written as polynomials in cos ω = (cos ω, cos ω ). We thus write F(x, x ) for a zero-phase filter in which x and x denote cos ω and cos ω respectively. II. NONSUBSAMPLED CONTOURLETS AND FILTER BANKS A. The Nonsubsampled Contourlet Transform Figure (a) displays a high level view of the proposed NSCT. The structure consists in a bank of filters that splits the -D frequency plane in the subbands illustrated in Figure (b). Our proposed transform can thus be divided into two parts that are both shift-invariant: () A nonsubsampled pyramid structure that ensures the multi-scale property and () A nonsubsampled DFB structure that gives directionality. ) The Nonsubsampled Pyramid (NSP): What gives the multi-scale property of the NSCT is a shiftinvariant filtering structure that achieves a subband decomposition similar to that of the Laplacian pyramid. Our solution is obtained by using two-channel nonsubsampled -D filter banks. Figure illustrates the proposed nonsubsampled pyramid (NSP) decomposition with J = 3 stages. Such expansion is conceptually similar to the -D nonsubsampled wavelet transform computed with the à trous algorithm [8] and has J + redundancy, where J denotes the number of decomposition stages. The ideal passband support of the lowpass filter at the j-th stage is the region [ π, π j ]. Accordingly, the ideal support of j

4 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 4 ω (π, π) Image Bandpass directional subbands ω Bandpass directional subbands ( π, π) (a) (b) Fig.. The nonsubsampled contourlet transform. (a) Nonsubsampled filter bank structure that implements the NSCT. (b) The idealized frequency partitioning obtained with the proposed structure. the equivalent highpass filter is the complement of the lowpass, i.e., the region [ π j, π j ] \[ π j, π j ]. The filters for subsequent stages are obtained by upsampling the filters of the first stage. This gives the multi-scale property without the need for additional filter design. The proposed structure is thus different from the separable nonsubsampled wavelet transform (NSWT). In particular, one bandpass image is produced at each stage resulting in J + redundancy. By contrast, the separable NSWT produces three directional images at each stage resulting in 3J + redundancy. H (z 4I ) H (z) H (z I ) H (z 4I ) y ω (π, π) x H (z) H (z I ) y y 3 ω y 3 ( π, π) (a) (b) Fig.. The proposed nonsubsampled pyramid is a -D multi-resolution expansion similar to the -D nonsubsampled wavelet transform. (a) A three-stage pyramid decomposition. The lighter gray regions denote the aliasing caused by upsampling. (b) The subbands on the -D frequency plane. The proposed pyramid is not the only -D pyramid with J + redundancy. The -D pyramid proposed

5 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 5 in [7] is obtained with a similar structure. Specifically, the NSFB of [7] is built from a given lowpass filter H (z). One then sets H (z) = H (z), and G (z) = G (z) =. This perfect reconstruction system can be seen as a particular case of our more general structure. The advantage of our construction is that it is less restrictive and as a result, better filters can be obtained. ) The Nonsubsampled Directional Filter Bank (NSDFB): The directional filter bank of Bamberger and Smith [6] is constructed by combining critically-sampled two-channel fan filter banks and resampling operations. The result is a tree-structured filter bank that splits the frequency plane in the directional wedges shown in Figure 3 (a). 7 ω 3 4 (π, π) U eq (z) S y S V eq (z) ω x U eq (z) S y S V eq (z) + ˆx 4 7 ( π, π) 3 U eq L (z) S L y L S L V eq L (z) (a) (b) Fig. 3. The directional filter bank. (a) Ideal partitioning of the -D frequency plane into 3 = 8 wedges. (b) Equivalent multi-channel filter bank structure. The number of channels is L = l, where l is the number of stages in the tree structure. Using multirate identities [8], the tree-structured DFB can be put into the equivalent form shown in Figure 3 (b), where the downsampling/upsampling matrices S k for k l are given by diag ( l, ), for k l ; S k = diag (, l ), for l k l, with l denoting the number of stages in the tree structure. It is clear from the above that the DFB is not shift-invariant. A shift-invariant directional expansion is obtained with a nonsubsampled DFB (NSDFB). The NSDFB is constructed by eliminating the downsamplers and upsamplers in Figure 3 (b). This is equivalent to switching off the downsamplers in each two-channel filter bank in the DFB tree structure and upsampling the filters accordingly. This results in a tree composed of two-channel nonsubsampled filter banks. The equivalent filter in the k-th channel of the analysis NSDFB tree is the filter U eq k (z) in Figure 3 (b). Thus, each filter U eq k (z) is a product of l simpler filters. Just like the DFB, all NSFB s in the NSDFB tree structure are obtained from a single NSFB with fan filters (see Figure 6 (b)). Figure 4 illustrates a ()

6 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 6 four channel decomposition. Note that in the second level, the upsampled fan filters U i (z Q ), i =, have checker-board frequency support, and when combined with the filters in the first level give the four directional frequency decomposition shown in Figure 4. The synthesis filter bank is obtained similarly. Just like the NSP case, each filter bank in the NSDFB tree has the same computational complexity as that of the prototype NSFB. U (z Q ) U (z) y U (z Q ) ω (π, π) x y U (z Q ) 3 U (z) U (z Q ) y ω y 3 ( π, π) (a) (b) Fig. 4. A four-channel nonsubsampled directional filter bank constructed with two-channel fan filter banks. (a) Filtering structure. The equivalent filter in each channel is given by U eq k (z) = Ui(z)Uj(zQ ). (b) Corresponding frequency decomposition. 3) Combining the NSP and NSDFB in the NSCT: The NSCT is constructed by combining the NSP and the NSDFB as shown in Figure (a). In constructing the nonsubsampled contourlet transform, care must be taken when applying the directional filters to the coarser scales of the pyramid. Due to the tree-structure nature of the NSDFB, the directional response at the lower and upper frequencies suffers from aliasing which can be a problem in the upper stages of the pyramid (see Figure 9). This is illustrated in Figure 5 (a), where the passband region of the directional filter is labelled as Good or Bad. Thus we see that for coarser scales, the highpass channel in effect is filtered with the bad portion of the directional filter passband. This results in severe aliasing and in some observed cases a considerable loss of directional resolution. We remedy this by judiciously upsampling the NSDFB filters. Denote the k-th directional filter by U k (z). Then for higher scales, we substitute U k (z mi ) for U k (z) where m is chosen to ensure that the good part of the response overlaps with the pyramid passband. Figure 5 (b) illustrates a typical example. Note that this modification preserves perfect reconstruction. In a typical 5-scale decomposition, we upsample

7 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 7 replacements (π, π) ω ( π, π) ω Bad Good ω ( π, π) (π, π) ω (a) (b) Fig. 5. The need for upsampling in the NSCT. (a) With no upsampling, the highpass at higher scales will be filtered by the portion of the directional filter that has bad response. (b) Upsampling ensures that filtering is done in the good region. by I the NSDFB filters of the last two stages. Filtering with the upsampled filters does not increase computational complexity. Specifically, for a given sampling matrix S and a -D filter H(z), to obtain the output y[n] resulting from filtering x[n] with H(z S ), we use the convolution formula y[n] = k supp (h) h[k]x[n Sk]. () Therefore, each filter in the NSDFB tree has the same complexity as that of the building-block fan NSFB. Likewise, each filtering stage of the NSP has the same complexity as that incurred by the first stage. Thus, the complexity of the NSCT is dictated by the complexity of the building-block NSFB s. If each NSFB in both NSP and NSDFB requires L operations per output sample, then for an image of N pixels the NSCT requires about BN L operations where B denotes the number of subbands. For instance, if L = 3, a typical decomposition with 4 pyramid levels, 6 directions in the two finer scales and 8 directions in the two coarser scales would require a total of 536 operations per image pixel. If the building block -channel NSFB s in the NSP and NSDFB are invertible, then clearly the NSCT is invertible and thus it implements a frame expansion (see Section II-C). The frame elements are localized in space and oriented along a discrete set of directions. The NSCT is flexible in that it allows any number of l directions in each scale. In particular, it can satisfy the anisotropic scaling law a key property in establishing the expansion nonlinear approximation behavior [], [3]. This property is ensured by doubling the number of directions in the NSDFB expansion at every other scale. The NSCT has redundancy given by + J j= lj, where l j denotes the number of levels in the NSDFB at the j-th

8 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 8 scale. B. Nonsubsampled Filter Banks At the core of the proposed NSCT structure is the -D two-channel nonsubsampled filter bank. Shown in Figure 6 are the NSFB s needed to construct the NSCT. In this paper we focus exclusively on the FIR case simply because it is easier to implement in multiple dimensions. For a general FIR two-channel NSFB, perfect reconstruction is achieved provided the filters satisfy the Bezout identity: H (z)g (z) + H (z)g (z) =. (3) The Bezout identity puts no constraint on the frequency response of the filters involved. Therefore, to obtain good solutions one has to impose additional conditions on the filters. H (z) G (z) U (z) V (z) y y x + ˆx x + ˆx y y H (z) G (z) U (z) V (z) (a) (b) Fig. 6. The two-channel nonsubsampled filter banks used in the NSCT. The system is two times redundant and the reconstruction is error free when the filters satisfy Bezout s identity. (a) Pyramid NSFB. (b) Fan NSFB. C. Frame Analysis of the NSCT The nonsubsampled filter bank can be interpreted in terms of analysis/synthesis operators of frame systems. A family of vectors {φ n } n Γ constitute a frame for a Hilbert space H if there exist two positive constants A, B such that for each x H we have A x x, φ n B x. (4) n Γ In the event that A = B the frame is said to be tight. The frame bounds are the tightest positive constants satisfying (4). Consider the NSFB of Figure 6 (a). The family {h [ n], h [ n]} n Z is a frame for l (Z ) if and only if there exist constants < A B < such that [9]

9 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 9 Thus, the frame bounds of an NSFB can be computed by A H (e jω ) + H (e jω ) B. (5) }{{} t(e jω ) A = ess. inf ω [ π,π] t(e jω ), B = ess.sup ω [ π,π] t(e jω ), (6) where ess. inf, and ess. sup denote the essential infimum and essential supremum respectively. From (5), we see that the frame is tight whenever t(e jω ) is almost everywhere constant. As a result, FIR filters corresponding to tight frames cannot be linear-phase, except for Haar-type filters (see [8] pp ). Because the NSFB is redundant, an infinite number of inverses exist. Among them, the pseudo-inverse is the best inverse in a number of senses []. Given a frame of analysis filters, the synthesis filters corresponding to the frame pseudo-inverse are given by G i (z) = H i (z)/t(z) for i =, [9]. In this case, the synthesis filters form the dual frame with lower and upper frame bounds given by B and A respectively. When the analysis filters are FIR, then unless the frame is tight, the synthesis filters of the pseudo-inverse will be IIR. From the above discussion we gather two important points: () linear phase filters and tight frames are mutually exclusive; () the pseudo-inverse is desirable, but is IIR if the frame is not tight. Consequently, an FIR NSFB system with linear phase filters and with synthesis filters corresponding to the pseudoinverse is not possible. However, we can approximate the pseudo-inverse with FIR filters. For a given number of filter taps, the closer the frame is to being tight, the better an FIR approximation of the pseudo-inverse can be []. Thus, in the designs we seek linear phase filters that underly a frame that is as close to a tight one as possible. In a general FIR perfect reconstruction NSFB system both analysis and synthesis filters form a frame. If we denote the analysis and synthesis frame bounds by A a, B a and A s, B s respectively, the frames will be close to tight provided r a := A a /B a, and r s := A s /B s. We always assume that the filters are normalized so that we have A a B a. In case the pseudoinverse is used, then we also have A s B s. The following result shows the NSCT is a frame operator for l (Z ) whenever the constituent NSFB s each forms a frame. Proposition : In the nonsubsampled contourlet transform if the pyramid filter bank constitute a frame with frame bounds A p and B p, and the fan filters constitute a frame with frame bounds A q and B q, then

10 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 J A actual A estimated B actual B estimated TABLE I FRAME BOUNDS EVOLVING WITH SCALE FOR THE PYRAMID FILTERS GIVEN IN EXAMPLE IN SECTION III the NSCT is a frame with bounds A and B satisfying Proof: See Appendix A. A J min {lj} paq A B Bp J max {lj} Bq Remark : When both the pyramid and fan filter banks form tight frames with bound, then A p = A q = B p = B q =, and from the above proposition, the nonsubsampled contourlet transform is also a tight frame with bound. The above estimates on A and B can be accurate in some cases, especially when the frame is close to a tight one and the number of levels is small (e.g., J = 4). In general however they are not accurate estimates. Their purpose is more of giving an interval for the frame bounds rather than the actual values. Table I shows estimates for different number of scales. The actual frame bound is computed from (5-6) whereas the estimates are the ones given according to Proposition. III. FILTER DESIGN AND IMPLEMENTATION The filter design problem of the NSCT comprises the two basic NSFB s displayed in Figure 6. The goal is to design the filters imposing the Bezout identity (i.e. perfect reconstruction) and enforcing other properties such as sharp frequency response, easy implementation, regularity of the frame elements, and tightness of the corresponding frames. It is also desirable that the filters are linear-phase. Two-channel -D NSFB s that underly tight frames are designed in [9]. However, the design methodology of [9] is not easy to extend to -D designs since it relies on spectral factorization which is hard in -D. If we relax the tightness constraint then the design becomes more flexible. In addition, as we alluded to earlier, non-tight filters can be linear phase. An effective and simple way to design -D filters is the mapping approach first proposed by McClellan [] in the context of digital filters and then used by several authors [], [3], [4], [5] in the context

11 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 of filter banks. In such approach, the -D filters are obtained from -D ones. In the context of NSFB s, a set of perfect reconstruction -D filters is obtained in the following way: Step. Construct a set of -D polynomials {H (D) i Step. Given a -D FIR filter f(z), then {H (D) i the Bezout identity. (x), G (D) i (x)} i=, that satisfies the Bezout identity. (f(z)), G (D) (f(z))} i=, are -D filters satisfying Thus, one has to design the set of -D filters and the mapping function f(z) so that the ideal responses are well approximated with a small number of filter coefficients. One advantage of the mapping approach is that one can control the frequency and phase responses through the mapping function. Moreover, if the mapping function is zero-phase, then f(z) = f(z ) and it follows that the mapped filters are also zero-phase. Therefore, we restrict our attention to zero-phase designs only. In this case, on the unit sphere, the mapping function is a -D polynomial in (cos ω, cos ω ). We thus denote it by F(x, x ), where it is implicit that f(e jω ) = F(cos ω, cos ω ). i + + x P (D) (f(z)) Q (D) (f(z)) Q (D) (f(z)) P (D) (f(z)) + x + + Fig. 7. Lifting structure for the nonsubsampled filter bank designed with the mapping approach. The -D prototype is factored with the Euclidean algorithm. The -D filters are obtained by replacing x f(z). A. Implementation Through Lifting Filters designed with the mapping approach can be efficiently factored into a ladder [6] or lifting [7] structure that simplifies computations. To see this, assume without loss of generality that the degree of the highpass prototype polynomial H (D) (x) is smaller than that of H (D) (x). Suppose also that there are synthesis filters G (D) (x) and G (D) (x) such that the Bezout identity is satisfied. In this case it follows that gcd{h (D), H (D) } =. The Euclidean algorithm then enables us to factor the filters in the following way [8], [6], [7]: H(D) (x) H (D) (x) = N i= P (D) i (x) Q(D) i (x). (7)

12 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 As a result, we can obtain a -D factorization by replacing x with f(z). This factorization characterizes every -D NSFB derived from -D through the mapping method. Figure 7 illustrates the ladder structure with one stage. In general, the lifting implementation at least halves the number of multiplications and additions of the direct form [7]. The complexity can be reduced further if the lifting steps in the -D prototype are monomials and the mapping filter f(z) has the form f(z) = f (z p zp )f (z q zq ), (8) for suitable f (z), f (z), and integers p, p, q, q. Note that if f(z) is a -D filter, then the -D filter f(z p zq ) for integers p and q has the same complexity as that of f(z). Therefore, filters of the form in (8) have the same complexity as that of separable filters (i.e., filters of the form f(z) = f(z )f(z ) ) which amounts to two -D filtering operations. Notice that if f(z) is as in (8), then for an arbitrary sampling matrix S, f(z S ) also has the form in (7). Consequently, all NSFB s in the NSDFB tree structure can be implemented with -D operations whenever the prototype fan NSFB can be implemented with -D filtering operations. The same reasoning applies to the NSFB s of the NSP. B. Pyramid Filter Design In the pyramid case, we impose line zeros at ω = (π, ω ) and ω = (ω, π). Notice that an N-th order line zero at ω = (π, ω ), for example, amounts to a ( + e jω ) N factor in the low pass filter. Such zeros are useful to obtain good approximation of the ideal frequency response of Figure 6, in addition to imposing regularity of the scaling function. We point out that for the approximation of polynomial images, point zeros at (±π, ±π) would suffice [9]. However, our experience shows that point zeros alone do not guarantee a reasonable frequency response of the pyramid filters. The following proposition characterizes the mapping function that generates zeros in the resulting -D filters. Proposition : Let G (D) (z) be a polynomial with roots {z i } n i= where each z i has multiplicity n i. Suppose we want a mapping function F(x, x ) such that G (D) (F(x, x )) = (x c) N (x d) N L(x, x ), (9) where L(x, x ) is a bivariate polynomial. Then G (D) (F(x, x )) has the form in (9) if and only if F(x, x ) takes the form F(x, x ) = z j + (x c) N (x d) N L F (x, x ), ()

13 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 3 for some root z j {z i } n i=, where L F(x, x ) is a bivariate polynomial, and N, N N n i N and N n j N. Proof: See Appendix B. are such that The above result holds in general, even for point zeros (as opposed to line zeros as in Proposition ). We will explore this extensively in the designs that follow. Suppose the prototype filters H (D) (x), G (D) (x) each has zeros at x =. Then, in order to produce a suitable zero-phase mapping function for the pyramid NSFB we consider the class of maximally-flat filters given by the polynomials [3] ( ) + x N L N P N,L (x) := l= ( ) ( ) N + l x l, () l where N controls the degree of flatness at x = and L controls the degree flatness at x =. Following Proposition, we can construct a family of mapping functions as F(x, x ) = + P N,L (x )P N,L (x ) () so that zero moments at x = and x = are guaranteed. Note that F(x, x ) can be implemented with -D filtering operations only. C. Fan Filter Design To design the fan filters idealized in Figure 6(b) we use the same methodology as in the pyramid case. The distinction occurs in the mapping function. The fan-shaped response can be obtained from a diamond-shaped response by simple modulation in one of the frequency variables. This modulation preserves the perfect reconstruction property. A useful family of mapping functions for the diamondshaped response is obtained by imposing flatness around ω = (±π, ±π) and ω = (, ) in addition to point zeros at (±π, ±π). If the mapping function is zero-phase, this amounts to imposing flatness in a polynomial at (x, x ) = (, ) and (x, x ) = (, ), and zeros at (x, x ) = (, ). Mapping functions satisfying these desiderata with minimum number of polynomial coefficients are given by F N (x, x ) = + Q N (x, x ), (3) where the polynomials Q N (x, x ) give the class of maximally-flat half-band filters with diamond support. A closed-form expression for Q N (x, x ) is given in [3]. Table II displays the first six mapping functions F N (x, x ) for the diamond filter bank. We point out that diamond maximally-flat mapping polynomials can be generated from -D ones by a separable product and then an appropriate change of variables. In this case the mapping function is

14 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 4 N F N(x, x ) (x + x) (x + x)(3 xx) 4 6 (x + x)(5 x 8x x x + 3x x ) 3 (x + x)(35 5x 5x x + 3x 3 x 5x + 5x x + 3x x 3 5x 3 x 3 ) 56 (x + x) 35 7x + 3x 4 8x x + 7x 3 x 7x + 8x x 3x 4 x + 7x x 3 x 3 x 3 + 3x 4 3x x x 4 x 4 5 (x + x) 693 x + x 4 735x x + 94x 3 x 5x 5 x x + 756x x x 4 x +94x x 3 54x 3 x 3 + 7x 5 x 3 + x 4 x x x 4 x 4 5x x 5 + 7x 3 x 5 63x 5 x 5 TABLE II MAXIMALLY FLAT MAPPING POLYNOMIALS USED IN THE DESIGN OF THE NONSUBSAMPLED FAN FILTER BANK. separable and has the form f(z) = f (z z )f (z z ). This has been done for instance in [3]. However, the nonseparable solution is generally shorter (roughly by a factor of ) while yielding the same number of zeros at the aliasing frequencies and a similar frequency response. For faster implementation, one may choose the longer mapping filters which can be implemented with -D filtering operations. D. Design Examples The design through mapping is based on a set of -D polynomials that satisfies the Bezout identity, that is, H (D) (x)g (D) (x) + H (D) (x)g (D) (x) =. The design can be simplified if we impose the restriction that H (D) (x) = G (D) ( x) and G (D) (x) = H (D) ( x). One advantage of this choice is that the frequency response of the filters can be controlled by the lowpass branch the highpass will automatically have the complementary response. Another advantage is that, under an additional condition, the frame bounds of the analysis and synthesis frames are the same and can be computed from the -D prototypes. To see this, suppose that f is the mapping function and that Ran(f) = Ran( f) with Ran(f) denoting the range of the mapping function f. Then we have that A a = inf ω [π,π] H (D) (f(e jω )) + G (D) ( f(e jω )) = inf x Ran(f) H(D) (x) + G (D) ( x) = inf x Ran( f) H(D) ( x) + G (D) (x) = inf x Ran(f) H(D) ( x) + G (D) (x) = A s. (4)

15 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 5 A similar argument shows that B a = B s = sup H (D) ( x) + G (D) (x). (5) x Ran(f) Example ( Pyramid filters very close to tight ones ): In order to get filters that are almost tight we design the prototypes H (D) (x) and G (D) (x) to be very close to each other. If we let H (D) (x) = G (D) ( x) and G (D) (x) = H (D) ( x), then the following filters can be checked to satisfy the Bezout identity: H (D) (x) = K ( + k x + k k 3 x ), G (D) (x) = K ( k x k 3 x + k k x k k k 3 x 3). To obtain the prototype we choose the constants K, k, k, and k 3 such that each filter has a zero at z = and in addition that H (D) ( ) = G (D) ( ) =. We obtain H (D) (x) = ( (x + ) ) + ( )x, G (D) (x) = (x + ) ( + (4 3 )x + ( 3)x ). The lifting factorization of the prototype filters is given by H(D) (x) = αx γx K (6) H (D) (x) βx K with α = γ =, β =, K = K =. Notice that this implementation has 5 multiplies/sample whereas the direct form has multiplies/sample. In this example we set F(x, x ) = + P,4 (x)p,4 (y) so that each of the filters has a 4-th order zero at ω = ±π and at ω = ±π. Since the ladder steps are monomials, the NSFB can be implemented with -D filtering operations. The frame bounds are computed using (4-5): A a = A s =.96, B a = B s =.4. Thus we have r a = r s =.83 and the frame is almost tight. The support size of H (z) is 3 3 whereas G (z) has support size 9 9. Figure 8 shows the response of the resulting filters. Example (Maximally-flat fan filters): Using the prototype filters of Example, we modulate a maximallyflat diamond mapping function to obtain the maximally-flat fan mapping function. Thus we choose

16 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY (a) H (e jω ) (b) G (e jω ) (c) H (e jω ) (d) G (e jω ) Fig. 8. Magnitude response of the filters designed in Example with maximally-flat filters. The nonsubsampled pyramid filter bank underlies almost tight analysis and synthesis frames. F N (x, x ) in Table II with N = 3. Figure 9 shows the magnitude response of the filters. The support size of U (z) is whereas the support size of V (z) is 3 3. E. Regularity of the NSCT basis functions The multi-scale property of the NSCT is obtained through the nonsubsampled pyramid implemented with a -D à trous algorithm. Denoting by H (ω) the scaling lowpass filter used in the pyramid (we write H (ω) instead of H (e jω ) for convenience), we have the associated scaling function Φ(ω) := H ( j ω), j= where convergence is in the weak sense. In our proposed design the filter H (ω) can be factored as ( + e jω ) N ( + e jω ) N H (ω) = R H(ω). (7)

17 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY (a) U (e jω ) (b) V (e jω ) (c) U (e jω ) (d) V (e jω ) Fig. 9. Fan filters designed with prototype filters of Example and diamond maximally-flat mapping filters. Notice that the remainder filter R H (ω) is not separable. Therefore, one cannot separate the regularity estimation as two -D problems. Nonetheless, a similar argument can be developed and an estimate of the -D regularity of the scaling filter is obtained. Let Proposition 3: Let H (ω) be a scaling filter as in (7) with the corresponding scaling function Φ(ω). B = sup R H (ω). ω [π,π] Then ( ) N log B ( Φ(ω) C + ω + ω Proof: See Appendix C. ) N log B As an example, consider the prototype filter in Example and the mapping F(x, x ) = + P, (x)p, (y). The resulting filter has second-order zeros at ω = ±π and at ω = ±π. It can be verified.

18 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 8 that R H (ω).83 and R G (ω).49 so that the regularity exponent is at least log.83.3 for H (ω) and log for G (ω). Thus the corresponding scaling functions and wavelets are at least continuous. We point out that better estimates are possible applying similar -D techniques. For instance, one could prove a result similar to Lemma 7.. in [3], pp. 7, as a consequence of Proposition 3. Figure shows the basis functions of the NSCT obtained with the filters designed via mapping. As the picture shows, the functions have a good degree of regularity. A. Image Denoising IV. APPLICATIONS In order to illustrate the potential of the NSCT designed using the techniques previously discussed we study additive white Gaussian noise (AWGN) removal from images by means of thresholding estimators. We test the NSCT under two denoising schemes described next. ) Hard Threshold: For the hard threshold estimator we choose a global threshold T i,j = Kσ nij for each directional subband. This has been termed K-sigma thresholding in [3]. We set K = 4 for the finer scale and K = 3 for the remaining ones. We refer to this method as NSWT-HT when applied to NSWT coefficients and NSCT-HT when applied to NSCT coefficients. We use five scales of decomposition for both NSCT and NSWT. For the NSCT we use 4,8,8,6,6 directions in the scales from coarser to finer respectively. ) Local adaptive shrinkage: We perform soft thresholding (shrinkage) independently in each subband. Following the work in [5] we choose the threshold T i,j = σ N ij σ i,j,n, where σ i,j,n denotes the variance of the n-th coefficient at the i-th directional subband of the j-th scale, and σ N ij is the noise variance at scale j and direction i. It is shown in [5] that shrinkage estimation with T = σ σ X, and assuming X generalized Gaussian distributed yields a risk within 5% of the optimal Bayes risk. As studied in [33], contourlet coefficients are well modeled by generalized Gaussian distributions. The signal variances are estimated locally using the neighboring coefficients contained in a square window within each subband and a maximum likelihood estimator. The noise variance in each subband is inferred using a Monte-Carlo technique where the variances are computed for a few normalized noise images The regularity exponent of a scaling function φ(t) is the largest number α such that Φ(ω) decays as fast as (+ ω + ω ) α.

19 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 9 Basis functions, Level (a) Basis functions, Level 3 Basis functions, Level 4 (b) Fig.. Basis functions of the nonsubsampled contourlet transform. (a) Basis functions of the second stage of the pyramid, (b) Basis functions of third (top 8) and fourth (bottom 8) stages of the pyramid. and then averaged to stabilize the results. We refer to this method as local adaptive shrinkage (LAS). Effectively, our LAS method is a simplified version of the denoising method proposed in [34] that works in the NSCT or NSWT domain.

20 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 To benchmark the performance of the NSCT we have used some of the best denoising schemes reported in the literature: ) Hard thresholding with the NSWT. ) Hard thresholding in the curvelet transform domain [3]. 3) Bivariate shrinkage with local variance estimation [6]. 4) Bayes least-squares with a Gaussian scale-mixture model (BLS-GSM) proposed in [7]. For the LAS estimator we use four scales for both the NSCT and NSWT. For the NSCT we use 3,3,4,4 directions in the scales from coarser to finer respectively. We use three 5 5 standard images in our test suit: Lena, Barbara, and Peppers. Table III shows the PSNR results for the various methods used in our study 3. The results show that for hard thresholding, the NSCT is consistently superior to curvelets and NSWT in PSNR measure. For the Barbara image the NSCT-HT yields improvements in excess of.9 db in PSNR over the NSWT-HT. Figure displays the reconstructed images using the the NSWT-HT, curvelets and NSCT-HT. As the figure shows, both the NSCT and the curvelet transform offers a better recovery of edge information relative to the NSWT. But improvements can be seen in the NSCT, particularly around the eye. The NSCT coupled with the local adaptive shrinkage estimator (NSCT-LAS) produced very satisfactory results as well. In particular, among the methods studied, the NSCT-LAS yields the best results for the Barbara image, being surpassed by the BLS-GSM method for the other images. Despite its slight loss in performance relative to BLS-GSM, we believe the NSCT has potential for better results. This is because by comparison, the BLS-GSM is a considerably richer and more sophisticated estimation method than our simple local thresholding estimator. However, studying more complex denoising methods in the NSCT domain is beyond the scope of the present paper. Figure displays the denoised images with both BLS- GLM and NSCT-LAS methods. As the pictures show, the NSCT offers a slightly better reconstruction. In particular, the table cloth texture is better recovered in the NSCT-LAS scheme. We briefly mention that in denoising applications, one can reduce the redundancy of the NSCT by using critically sampled directional filter banks over the nonsubsampled pyramid. This results in a transform with J + redundancy which is considerably faster. There is however a loss in performance as Table IV shows. Nonetheless, in some applications, the small performance loss might be a good price to pay given the reduced redundancy of this alternative construction. 3 The PSNR values of the BivShrink method were obtained from the tables in [6]. In the results shown in [6] the authors do not use the Peppers image as a test image, hence we do not have a BivShrink column for Peppers.

21 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 Lena image, PSNR (db) σ Noisy NSWT-HT Curvelets [3] NSCT-HT NSWT-LAS BivShrink [6] BLS-GSM [7] NSCT-LAS Barbara image, PSNR (db) σ Noisy NSWT-HT Curvelets [3] NSCT-HT NSWT-LAS BivShrink [6] BLS-GSM [7] NSCT-LAS Peppers image, PSNR (db) σ Noisy NSWT-HT Curvelets [3] NSCT-HT NSWT-LAS BLS-GSM [7] NSCT-LAS TABLE III DENOISING PERFORMANCE OF THE NSCT. THE LEFT-MOST COLUMNS ARE HARD THRESHOLDING AND THE RIGHT-MOST ONES SOFT ESTIMATORS. FOR HARD THRESHOLDING, THE NSCT CONSISTENTLY OUTPERFORMS CURVELETS AND THE NSWT. THE NSCT-LAS PERFORMS ON A PAR WITH THE MORE SOPHISTICATED ESTIMATOR BLS-GSM. σ Lena Barbara Peppers TABLE IV RELATIVE LOSS IN PSNR PERFORMANCE (DB) WHEN USING THE NSP WITH A CRITICALLY SAMPLED DFB AND THE LAS ESTIMATOR WITH RESPECT TO THE NSCT-LAS METHOD.

22 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 (a) Original (b) NSWT-HT (c) Curvelets-HT (d) NSCT-HT Fig.. Image denoising with the NSCT and hard thresholding. The noisy intensity is. (a) Original Lena image. (b) Denoised with the NSWT-HT, PSNR = 3.4dB. (c) Denoised with the curvelet transform and hard thresholding, PSNR = 3.5dB. (d) Denoised with the NSCT-HT, PSNR = 3.3dB. B. Image Enhancement Existing image enhancement methods amplify noise when they amplify weak edges since they cannot distinguish noise from weak edges. In the frequency domain, both weak edges and noise produce low-

23 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 3 (a) Original (b) BLS-GSM (c) NSCT-LAS Fig.. Comparison between the NSCT-LAS and BLS-GSM denoising methods. The noise intensity is. (a) Original Barbara image. (b) Denoised with the BLS-GSM method of [7], PSNR = 3.8dB. (c) Denoised with NSCT-LAS, PSNR = 3.6dB. magnitude coefficients. The NSCT provides not only multiresolution analysis, but also geometrical and directional representation. Since weak edges are geometric structures and noise is not, we can use this geometric representation to distinguish them. The NSCT is shift-invariant so that each pixel of the transform subbands corresponds to that of the original image in the same location. Therefore, we gather the geometrical information pixel by pixel from the NSCT coefficients. We observe that there are three classes of pixels: strong edges, weak edges, and noise. First, the strong edges correspond to those pixels with large magnitude coefficients in all subbands. Second, the weak edges correspond to those pixels with large magnitude coefficients in some directional subbands but small magnitude coefficients in other directional subbands within the same scale. Finally, the noise corresponds to those pixels with small magnitude coefficients in all subbands. Based on this observation, we can classify pixels into three categories by analyzing the distribution of their coefficients in different subbands. One simple way is to compute the mean (denoted by mean) and the maximum (denoted by max) magnitude of the coefficients for each pixel, and then classify it by strong edge if mean cσ weak edge if mean < cσ, max cσ, (8) noise if mean < cσ, max < cσ where c is a parameter ranging from to 5, and σ is the noise standard deviation of the subbands at a specific level. We first estimate the noise variance of the input image with the robust median operator [34] and then compute the noise variance of each subband [3].

24 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 4 The goal of image enhancement is to amplify weak edges and to suppress noise. To this end, we modify the NSCT coefficients according to the category of each pixel by a nonlinear mapping function (similar to [35]) y(x) = x strong edge pixels max(( cσ x )p, )x, weak edge pixels noise pixels, (9) where the input x is the original coefficient, and < p < is the amplifying ratio. This function keeps the coefficients of strong edges, amplifies the coefficients of weak edges, and zeros the noise coefficients. We summarize our enhancement method using the NSCT in the following algorithm: ) Compute the NSCT of the input image for N levels. ) Estimate the noise standard deviation of the input image. 3) For each level DFB, a) Estimate the noise variance. b) Compute the threshold and the amplifying ratio. c) At each pixel, compute the mean and the maximum magnitude of all directional subbands at this level, and classify it by (8) into strong edges, weak edges, or noise. d) For each directional subband, use the nonlinear mapping function given in (9) to modify the NSCT coefficients according to the classification. 4) Reconstruct the enhanced image from the modified NSCT coefficients. We compare the enhancement results by the proposed algorithm with those by the NSWT. In the experiments, we choose c = 4 and p =.3. Figure 3 shows the results obtained for the Barbara image. We observe that our proposed algorithm does a better job in enhancing the weak edges in the textures. V. CONCLUSION We have proposed a fully shift-invariant version of the contourlet transform, the nonsubsampled contourlet transform. The proposed construction reduces the filter design problem to that of a nonsubsampled pyramid filter bank and a nonsubsampled fan filter bank. We have addressed the -D design problem using the mapping approach, thus overcoming the need of factorization. We also developed a lifting/ladder structure for the -D NSFB. This structure, when coupled with the filters designed via mapping, provides a very efficient implementation that under some additional conditions can be reduced to -D filtering

25 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 5 (a) (b) (c) Fig. 3. NSCT. Image enhancement with the NSCT. (a) Original Barbara image. (b) Enhanced by the NSWT. (c) Enhanced by the operations. Applications of our proposed transform in image denoising and enhancement were studied. In denoising, we studied the performance of the NSCT when coupled with a hard thresholding estimator and a local adaptive shrinkage. For hard thresholding, our results indicate that the NSCT provides better performance than competing transform such as the NSWT and curvelets. Concurrently, our local adaptive shrinkage results are competitive to other denoising methods. In particular, our results show that a fairly simple estimator in the NSCT domain yields comparable performance to state-of-the-art denoising

26 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 6 methods that are more sophisticated and complex. In image enhancement, the results obtained with the NSCT are visually superior to those of the NSWT. A. Proof of Proposition APPENDIX Consider the pyramid shown in Figure (a). If J =, then we have that y + y B p x. Now, suppose we have J levels and assume J j= y i B J p x. Then if we further split y J into y J and y J+, noting that B p we have that J y j + y J + y J+ j= J y j + B p y J j= B p J y j + y J j= B J+ p x. Thus, by induction we conclude that J j= y i B J p x for any J. A similar argument shows that in the NSDFB with l j stages, one has that l j k= y j,k B lj q y j so that J y J + j= l j k= J y j,k y J + j= y J + B B lj q y j J max {lj} q j= B J max {lj} Bq x. The bound for A is proved similarly, by just reversing the inequalities. y j B. Proof of Proposition We prove the claim for the case in which the zeros of G (D) (z) are distinct. The proof for the case of repeated roots can be handled similarly. Denote n G (D) (z) = g (z z i ), g C. () Then sufficiency follows by direct substitution of () in (). i=

27 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 7 We prove necessity by induction. Suppose G (D) (F(x, x )) = (x c) N L(x, x ) for some polynomial L(x, x ). Note that G (D) (F(c, x )) = for all x if and only if F(c, x ) = z j for all x and some zero z j of G (D) (z). So, it follows that (F(x, x ) z j ) x=c = for all x, () which implies that F(x, x ) = z j + (x c)l (x, x ) with L (x, x ) a polynomial. Suppose F(x, x ) = z j + (x c) k L k (x, x ), where k N. By successively applying the chain rule for differentiation we get that = k G (D) (F(x, x )) x k x=c = G (D) (F(c, x )) k F(x, x ) x k x=c Because F(c, x ) = z j and the z j s are distinct, we have that G (D) (F(c, x )) and then k F(x, x ) x k x=c k F(x, x ) x k = = (x c)l k (x, x ) F(x, x ) x = (x c) k L k (x, x ). () Combining () with () we obtain that F(x, x ) = z j + (x c) k L k (x, x ). By induction we conclude that F(x, x ) = z j + (x c) N L N (x, x ). If G (D) (F(x, x )) = (x d) N L(x, x ) for some polynomial L(x, x ) then a similar argument shows that where z i is a zero of G (D) (z). Thus, and hence we have z j = z i. Therefore, and so F(x, x ) = z i + (x d) N L N (x, x ), z i z j = (x c) N L N (x, x ) (x d) N L N (x, x ), (x c) N L N (x, x ) = (x d) N L N (x, x ), L N (x, x ) = (x d) N L F (x, x ), and () is established with N = N and N = N..

28 IEEE TRANSACTIONS ON IMAGE PROCESSING, MAY 5 8 C. Proof of Proposition 3 The proof follows the same lines as for the -D case (see e.g., [] pp. 45). Using the identity ( ) N + e j ( ) k ω + e jω N = ω we have that k= ( + e jω ) N ( + e jω ) N Φ(ω) = R ( j ω). ω Because H is continuously differentiable at ω =, the same also holds for R. Now write R in polar coordinates (r, ϕ) where r = ω + ω. Since R is continuously differentiable, for each ϕ, R (r, ϕ) is a continuously differentiable function of r. Since R () =, from the mean value theorem we have that for ǫ >, and r < ǫ, ω j= R(r, ϕ) + r R(ρ, ϕ) r + Kr where K := sup{ R(ρ, ϕ) : ρ < ǫ, ϕ [, π]} <. This gives r < ǫ = R ( j ω) ( + K j r) e Kǫ, j= where we have used the inequality + r e r. Now choose J so that J ǫ r J ǫ. We then obtain, for r > ǫ, R ( j ω) = j= j= j= J R ( j ω) R ( j J ω) B J e Kǫ C r log B e Kǫ j= so that for each ω R, ( R ( j ω) e Kǫ + C r log B) j= Now, putting all together we obtain which completes the proof. = e Kǫ ( + C ( ω + ω ) log B). ( Φ(ω) C + C ( ω + ω ) log B) ω N ω N ( ) N log B ( ) N log B C 3, + ω + ω