2-D Wavelet Packet Decomposition on Multicomputers

Transcription

1 -D Wavelet Packet Decomposition on Multicomputers Manfred Feil Andreas Uhl RIST++ & Department of Scientific Computing University of Salzburg, AUSTRIA Abstract In this work we describe and analyze algorithms for -D wavelet packet decomposition for MIMD distributed memory architectures. The main goal is the generalization of former parallel WP algorithms which are constrained to a number of processor elements equal to a power of. We discuss several optimizations and generalizations of data parallel message passing algorithms and finally compare the results obtained on a Cray T3D. 1. Introduction Wavelet packets [] represent a generalization of the method of multiresolution decomposition and comprise the entire family of subband coded (tree) decompositions. Whereas in the wavelet case the decomposition is applied recursively to the coarse scale approximations (leading to the well known (pyramidal) wavelet decomposition tree), in the wavelet packet decomposition the recursive procedure is applied to all the coarse scale approximations and detail signals, which leads to a complete wavelet packet tree (i.e. binary tree and quadtree in the 1D and D case, respectively) and more flexibility in frequency resolution. There are several possibilities how to determine the frequency subbands suited well for an application (the meaning of suitable depends on the type of application, e.g. signal/image compression [], classification algorithms [18], etc.). The wavelet packet best basis algorithm [3] performs an adaptive optimization of the frequency resolution of a complete wavelet packet decomposition tree by selecting the most suitable frequency subbands for signal compression. The same algorithm employed with nonadditive cost function is denoted near-best basis algorithm [1], if the subband structure is restricted to uniform timefrequency resolution the corresponding algorithm is denoted best level selection []. In the context of image compression a more advanced technique is to use a framework that includes both rate and distortion, where the best basis subtree which minimizes the global distortion for a given coding budget is searched [16]. Other methods use fixed bases of subbands for similar signals (e.g. fingerprints[11]) or search for good representations with genetic optimization methods [, 1]. Recently, wavelet packet based compression methods have been developed [7, 1] which outperform the most advanced wavelet coders (e.g. SPIHT [17]) significantly for textured images in terms of rate-distortion performance. Therefore, wavelet packet decomposition currently attracts much attention and a thorough examination of parallelization possibilities is desirable. Whereas a significant amount of work has been already done concerning parallel algorithms for the fast wavelet transform, only few papers have been devoted to parallel wavelet packet decomposition and its specific features and demands (e.g. subband based approaches for performing the best basis algorithm and the irregular decomposition into such a basis on parallel MIMD [8, 3,, 6] and SIMD [] architectures, parallel wavelet packet decomposition in numerics [] and corresponding applications [13, 1, 1, 1, 9]). In this work we describe and analyze subband based algorithms for -D wavelet packet decomposition. It should be noted that the successive computation and availability of all frequency subbands of all decomposition levels is a must for all kinds of adaptive image processing applications. We base our implementations on algorithms described in [8, 6, 7] and focus on the generalization of these algorithms with regard to the number of used processor elements (PEs). Due to the subband structure inherent in -D WP decomposition (leading to j frequency subbands at decomposition level j) former subband based algorithms are restricted to a number of PEs equal to a power of. We present first a more general version for processor fields of the size of a power of and finally an algorithm for an arbitrary number of PEs. In the last section we analyze the results obtained on a Cray T3D in consideration of the used method of boundary treatment (zero padding vs. N-point symmetry).

2 . The sequential WP algorithm In this and the following sections we use a notation introduced in [8]. Suppose we have a two-dimensional signal S = (n k;l ) with size k < Xmax and l < Y max. We begin our calculation with the initial subband C k;l = n k;l and filter coefficients F and define every other subband recursively by the formula: fj +d C j+1 kj+1;lj+1 = P nj;mj F (d; f j) nj?k j+1;m j?l j+1 with the indices in the range: fj Cj nj ;mj j < jmax = min(xmax; Y max) : : : last calculated level f j < j : : : index of subband in use d < : : : index of filter in use n; m < filter length : : : indices of filter coefficient k j+1 < Xmax?(j+1) : : : coordinate of data-point in x-direction l j+1 < Y max?(j+1) : : : coordinate of data-point in y-direction From a one-dimensional wavelet (defined by its filter g n ) and its smoothing function (defined by its filter h n ) we obtain four two-dimensional filters by tensor products: F () n;m = h n h m, F (1) n;m = h n g m, F () n;m = g n h m, and F (3) n;m = g n g m. Simplifying, we may say that convolving the coefficients of a subband C j at level j with different filters F () : : : F (3) we obtain subbands C j+1 at level j + 1. Fig.1 shows the data arrangement for levels j = : : : : at level j we get j subbands, each with Xmax?j by Y max?j coefficients. Ymax level : level 1: level : f= Ymax-1 f= Xmax Xmax-1 f=1 f= f=3 Figure 1. Data arrangement and labeling for a D wavelet packet decomposition Further we have p PEs, on which we want to compute the two-dimensional wavelet packet decomposition of signal S in parallel WP Algorithms for MIMD architectures In this section we discuss parallel algorithms for the twodimensional periodic wavelet packet transform. The proposed algorithms assume that the size of PE memory is sufficient to keep the wavelet packet coefficients of all levels. For machines or applications which do not meet this condition a specific technique has been derived for parallel decomposition into any given basis []. The main difficulty of the development of a parallel WP implementation is the choice of an appropriate data distribution strategy that ensures a well balanced dispersion of the computational work. In the following subsections we show the most efficient strategies for architecture settings with: a number of PEs equal to a power of. a number of PEs equal to a power of. an arbitrary number of PEs. For all of the following algorithms, two methods of handling the border data (f ilter length coefficients of the neighbour PE are required to compute a single output coefficient at the border) can be used[6, ]: data swapping: each PE computes only non-redundant data and exchanges these results with the appropriate neighbour PEs in order to get the necessary data for the next calculation step (i.e. the next decomposition level). redundant data calculation: in the initialisation step we do not only provide its share of the original signal to a PE but provide the entire data set to each PE which is required to compute also redundant data in order to avoid additional communication with neighbour PEs to obtain the border data. In first experiments on the Cray T3D we found the redundant data calculation approach to be not competitive at all for this specific machine which is true also for different architectures[6], so we carried on our developments using the data swapping method. However, all the following algorithms are easily adapted to the second border treatment method if the architecture in question should require it WP algorithm for p = jp The most natural way to distribute the computational work of a WP transform can be found on an MIMD architecture with a number of PEs of a power of : we separate each level into jp parts of equal size Xmax?jp by Y max?jp and use the decomposition formula in two different ways depending upon whether level j is smaller or larger then jp. If j < jp, a subband at frequency f is not

3 assigned to a single PE but is shared by PEs p = jp?j f to p = jp?j (f + 1)? 1. Therefore, in the initialisation step those jp?j PEs will exchange their data in order to have the entire shared subband on each of them. Then, in the second step, they will calculate their own part of the subband they share at level j + 1. If j jp each PE has j?jp subbands and holds also the four children of each of them. Thus, no communication among PEs is needed and we will use the sequential algorithm on the subset of subbands each PE carries at level j. The overall communication amount is OP a?1 j=1 Xmax?a Y max?a ( a?j? 1) a?j. j In fig.(a) we show the distribution of level 1 among PEs. Now the entire output-subband resides on each PE; therefore, for the computation of the following levels no more communication is needed. we get a rectangular partitioning of data ( Xmax?d jp e by Y max?b jp c ) and an unequal number of communications in horizontal and vertical direction. The overall communication amount is dxa e?1 O j=1 Xmax?b a c Y max?d a e ( a?j? 1) a?j j Fig.(b) shows the data distribution of level 1 among 8 PEs. For the computation of the next level, communication only between two vertical neighbours (arrows in the figure) is required WP algorithm for arbitrary p PE PE PE PE PE PE6 PE1 PE PE1 PE PE PE3 PE PE3 PE6 PE7 (a) on PEs (b) on 8 PEs PE8 PE1 PE1 output subband PE9 host PE PE PE3 node PE PE7 data swapping assignment of node work (c) on 11 PEs Figure. Distribution of level 1 The main contribution of this work is the development of a parallel WP algorithm which uses an arbitrary number of PEs. To deal with the problem of how to integrate the PEs i < p into the former setting of PEs i.e. to avoid the need of handling with fractions of subbands (which would introduce a huge amount of additional communication) we employ a variant of the well known host-node principle: PEs < i, from now on denoted the hosts, compute their part of the WP transform as described in the last subsection. Each other PE ( i < p), the node, is assigned to a host and both share the computational work of one level: the node receives the input subband from its host, computes one half of the required subbands and sends the output back (see fig.(c)). The most important task is now to find a node assignment strategy which takes into account several points. These conditions are important to keep communication demand as small as possible: to avoid splitting of subbands or decomposition levels the nodes should change to a new host only at the beginning of a new level, and one node should be assigned to one host only and vice versa. 3.. WP algorithm for p = jp The first generalization of the former algorithm deals with architectures where the number of PEs is a power of. For this case the algorithm does not need major changes, because without loss of generality we can assume that the computational work of two neighbouring PEs in a p = jp (jp = n; n N) setting is merged into one if p = jp?1 PEs are used. For this number of processors the nodes should change their hosts as often as possible in order to distribute the computational work uniformly. the nodes should change their hosts as rare as possible in order to avoid exaggerating communication for sending the input data to the nodes resp. the output data back to the hosts. After testing different strategies we found the following assignment scheme the most suitable one:

4 Let p be the number of processors. We divide now the processor pool in h = hosts and n = p? h nodes and compute for host h i ( i < h) the node assignment scheme for level j ( j < jmax = min(xmax; Y max)) by the following algorithm (n k ( k < n) is the node assigned to this host): j=1; k=i while (j<jmax) loop if (mod(k,m)<s) then assign node n_k to host h_i for THIS and the NEXT level j=j+; k=k+1 else no node for host for THIS level j=j+1; k=k+1 end if end loop To show this concept clearer we demonstrate as an example a 9 level WP transform (full decomposition of a 1*1 pixel image) on 7 PEs ( hosts, 3 nodes): levels 1 3 h n n n 1 n 1 h 1 n 1 n 1 n n hosts h n n - n h 3 - n n n 1 levels n n - n n - n n n 1 n 1 n n 1 n 1 n n nodes n 1 n n - n Note that the duplicate assignment of one node to more hosts (e.g. n in level ) is no contradiction to the demand of non-partitioning of subbands or levels: h finishes level 1 and because of the help of n in the same time as h 3 needs just for the first level. The node n adopts partial work of h in the first levels, then switches to h 3 and continues with level. Therefore the amount of idle time is reduced to a minimum for each PE. The communication amount is given by the formula for the former algorithm plus the transfers for each additional node PE: O d X e?1 Xmax?b c Y max?d e j=1 (?j? 1)?j j?1 Xmax?b + c Y max?d e jmax (p? ). Experimental Results Our practical experiments were run on a 1-PE Cray T3D situated at the Edinburgh Parallel Computing Centre (EPCC). Each PE consists of a DECchip 16 Alpha processor (1MHz) with 6Mb of memory, and the system has a peak performance of 76.8 GFlops. As message passing interface we used CRI/EPCC MPI[19], a native implementation of the MPI 1.1 Standard for the Cray T3D. First we want to analyze if our scheduling algorithm behaves in reality as predicted by the theory. We decompose an image of 1 by 1 pixels into 9 levels and illustrate the required time for each level and PE in fig.3 (the figure shows just the host PEs; hosts supported by nodes are shaded). We use the Daubechies W filter ( filter coefficients) and run the algorithm on 7 PEs ( hosts and 3 nodes). To investigate the effect of different boundary treatments we employed two classical methods, zero padding (the input signal is extended with zeros beyond the border) and N-point symmetry (the input signal is repeated with a period of N)[]. This is important because the success of our scheduling algorithm is based on the assumption that the computation of each level needs roughly the same time in order not to induce a node to wait for its next host. Fig.3(a) shows the scheduling results for the zero padding and confirms exactly this problematic behaviour: the compiler recognizes the large amount of multiplications with zero in higher levels and optimizes this calculations which leads to smaller time consumption and less efficiency for node supported hosts. The N-point symmetry method (fig.3(b)) assures the same computational demand for every level. Therefore, the synchronization between hosts and nodes works as expected (hosts with support of a node need roughly half the time as unsupported ones) and the algorithm shows better scalability although the absolute computation time is significantly higher as compared to the zero padding algorithm. It should be noted that the zero padding approach leads to severe border artifacts in image processing applications whereas the N-point symmetry assures constant quality across the entire image. In fig. we show speedup results for the parallel algorithms executed on 1 to 18 PEs. As mentioned before, the redundant data calculation approach does not reach any considerable performance and was therefore neglected in further developments. The zero padding and the N-point symmetry show a nearly linear speedup in the first PEs but break out in irregularity later because of the difficulty to integrate each additional node in an equally efficient assignment pattern. Nevertheless, the N-point symmetry features the better scalability. Note the jumps in speedup at power of (denoted in the plot as and ) which originate from the absence of node PEs and the corresponding reduced communication. At 18

5 speedup WP (torodial wrap, data swapping) WP (zero padding, data swapping) WP (zero padding, redundant data calculation) sec 3 level #PE host host1 host host3 Figure. Speedup results on (1*1 pixels, 9 levels, W filter) (a) using zero padding 9 ciple. Further we pointed out the importance of the choice of the boundary treatment method Acknowledgements sec 1 host host1 host host level The first author was partially supported by the Austrian Science Fund FWF, project no. P11-ÖMA. Most of this work was accomplished in the course of a TRACS (Training and Research on Advanced Computing Systems) research visit at the Edinburgh Parallel Computing Centre (EPCC). References (b) using N-point symmetry Figure 3. WP transform on Cray T3D (1*1 pixels, 9 levels) (= 7 ) the expected jump is not exhibited in the plot because at deeper decomposition levels the advantage of exact subband-pe assignments is diminished by the higher communication demand. The obvious non-determinism in the plot for more than 6 PEs is a result of non-determinism in the operating system (e.g. different initialisation speed on different PEs).. Conclusion In this study we presented a generalized parallel WP algorithm for MIMD architectures with an arbitrary number of processing elements. We showed that a satisfying performance can be obtained by employing the host-node prin- [1] A. Bruckmann, T. Schell, and A. Uhl. Evolving subband structures for wavelet packet based image compression using genetic algorithms with non-additive cost functions. In Proceedings of the International Conference Wavelets and Multiscale Methods (IWC 98), Tangier, INRIA, Rocquencourt, Apr pages. [] C. Chu. Genetic algorithm search of multiresolution tree with applications in data compression. In H. Szu, editor, Wavelet Applications, volume of SPIE Proceedings, pages 9 98, 199. [3] R. Coifman and M. Wickerhauser. Entropy based methods for best basis selection. IEEE Transactions on Information Theory, 38():719 76, 199. [] S. Corsaro, L. D Amore, and A. Murli. On the parallel implementation of the fast wavelet packet transform on MIMD distributed memory environments. In P. Zinterhof, M. Vajtersic, and A. Uhl, editors, Parallel Computation. Proceedings of ACPC 99, volume 17 of Lecture Notes on Computer Science, pages Springer-Verlag, [] M. Feil and A. Uhl. Wavelet packet decomposition and best basis selection on massively parallel SIMD arrays. In Proceedings of the International Conference Wavelets and Multiscale Methods (IWC 98), Tangier, INRIA, Rocquencourt, Apr pages.

6 [6] M. Feil and A. Uhl. Algorithms and programming paradigms for -D wavelet packet decomposition on multicomputers and multiprocessors. In P. Zinterhof, M. Vajtersic, and A. Uhl, editors, Parallel Computation. Proceedings of ACPC 99, volume 17 of Lecture Notes on Computer Science, pages Springer-Verlag, [7] M. Feil and A. Uhl. Real-time image analysis using wavelets: the à trous algorithm on MIMD architectures. In D. Sinha, editor, Real-Time Imaging IV, volume 36 of SPIE Proceedings, pages 6 6, [8] E. Goirand, M. Wickerhauser, and M. Farge. A parallel two-dimensional wavelet packet transform and some applications in computing and compression analysis. In R. Motard and B. Joseph, editors, Applications of Wavelet Transforms in Chemical Engineering, pages Kluwer Academic Publishers Group, 199. [9] C. Guerrini and D. Lazzaro. Parallel deconvolution and signal compression using adapted wavelet packet bases. In E. Hollander, G. Joubert, F. Peters, and D. Trystram, editors, Parallel Computing: State of the Art and Perspectives, volume 11, pages [1] C. Guerrini and M. Piraccini. Parallel wavelet-galerkin methods using adapted wavelet packet bases. In C. Chui and L. Schumaker, editors, Approximation Theory VIII: Wavelets and Multilevel Approxiamtion, pages [11] T. Hopper. Compression of gray-scale fingerprint images. In H. Szu, editor, Wavelet Applications, volume of SPIE Proceedings, pages , 199. [1] F. Meyer, A. Averbuch, J. Strömberg, and R. Coifman. Fast wavelet packet image compression. In Proceedings Data Compression Conference (DCC 98), page 63. IEEE Computer Society Press, Mar [13] L. B. Montefusco. Parallel numerical algorithms with orthonormal wavelet packet bases. In C. Chui, L. Montefusco, and L. Puccio, editors, Wavelets: Theory, Algorithms and Applications, pages 9 9. Academic Press, San Diego, 199. [1] L. B. Montefusco. Semi-orthogonal wavelet packet bases for parallel least-squares approximation. Journal of Computational and Applied Mathematics, 73:191 8, [1] L. B. Montefusco. Wavelet decomposition: a new tool for the construction of parallel algorithms. In E. Hollander, G. Joubert, and F. Peters, editors, Parallel Computing: State-of-Art and Perspectives, pages Elsevier Science Publishers B.V., [16] K. Ramchandran and M. Vetterli. Best wavelet packet bases in a rate-distortion sense. IEEE Trans. on Image Process., ():16 17, [17] A. Said and W. Pearlman. A new, fast, and efficient image codec based on set partitioning in hierarchical trees. IEEE Transactions on Circuits and Systems for Video Technology, 6(3):3 9, [18] N. Saito and R. Coifman. Local discriminant bases. In A. Laine and M. Unser, editors, Wavelet Applications in Signal and Image Processing II, volume 33 of SPIE Proceedings, pages 1, 199. [19] T. M. S. G. Smith. CRI/EPCC MPI for Cray T3D. EPCC, May [] S. Sullivan. Vector and parallel implementations of the wavelet transform. Technical report, Center for Supercomputing Research and Development, University of Illinois, Urbana, [1] C. Taswell. Satisficing search algorithms for selecting nearbest bases in adaptive tree-structured wavelet transforms. IEEE Transactions on Signal Processing, (1):3 38, [] C. Taswell and K. McGill. Wavelet transform algorithms for finite-duration discrete-time signals. ACM Transactions on Mathematical Software, (3):398 1, 199. [3] A. Uhl. Adapted wavelet analysis an moderate parallel distributed memory MIMD architectures. In A. Ferreira and J. Rolim, editors, Parallel Algorithms for Irregulary Structured Problems, volume 98 of Lecture Notes in Computer Science, pages 7 8. Springer, 199. [] A. Uhl. Wavelet packet best basis selection on moderate parallel MIMD architectures. Parallel Computing, (1):19 18, [] M. Wickerhauser. Adapted wavelet analysis from theory to software. A.K. Peters, Wellesley, Mass., 199. [6] M.-L. Woo. Parallel discrete wavelet transform on the Paragon MIMD machine. In R. S. et al., editor, Proceedings of the seventh SIAM conference on parallel processing for scientific computing, pages 3 8, 199. [7] Z. Xiong, K. Ramchandran, and M. Orchard. Wavelet packet image coding using space-frequency quantization. IEEE Transactions on Image Processing, 7(6):89 898, June 1998.