Customized Evolutionary Optimization of Subband Structures for Wavelet Packet Image Compression THOMAS SCHELL AND ANDREAS UHL Department of Scientific Computing University of Salzburg Jakob-Haringer-Str. 2 AUSTRIA ftschell,uhlg@cosy.sbg.ac.at Abstract: In this work we describe the optimization of wavelet packet subband structures for still image compression. In recent work we have demonstrated that non-additive information cost functions may be optimized efficiently using genetic methods. In order to gain more insight into the evolution process a new crossover operator is introduced in this paper. Specifically, the crossover operator is modified such that the tree structure of the subbands are less disrupted than with conventional crossover operators. Key-Words: Wavelet-Packet-Compression, Genetic Algorithms, Crossover-Operator Introduction Image compression methods that use wavelet transforms [35] (which are based on multiresolution analysis MRA) have been successful in providing high rates of compression while maintaining good image quality, and have proven to be serious competitors to discrete cosine transform- (DCT) [37, 28] or fractal-[3, 23] based compression schemes. A wide variety of techniques to process the wavelet transform domain further for compression purposes have been reported in the literature [2, 22], ranging from simple entropy coding to more complex techniques such as vector quantization [, 0], adaptive transforms [, 36], zero-tree encoding [32], and edge-based coding [4]. The latest compression algorithms are based on set partitioning in hierarchical trees [30] and some improvements in arithmetic coding [39]. In several papers (see e.g. [27, 2]) the suitability of different lossy compression schemes with respect to medical images have been investigated thoroughly. In accordance to the results for general image types it has been found that advanced wavelet based compression schemes offer the best rate-distortion performance of all coders considered. For these reasons, the upcoming standard for still image compression, JPEG2000 [6], will be wavelet based. In this paper we investigate a specific optimization technique when employing adaptive wavelet algorithms instead of the classical fixed pyramidal wavelet decomposition. The idea of using adaptive algorithms can be used in two ways: on the one hand, the adaptive process can be performed for each new image thereby causing computational and coding overhead as compared to the non-adaptive case, on the other hand the same (adaptively generated) setting may be employed for classes of similar images (this has been done in the context of the FBI fingerprint compression standard [8]). In particular, we target onto the transform step of the wavelet compression scheme and investigate adaptively generated subband structures ( wavelet packets") with respect to their performance in compressing still images. In section 2 we introduce in brief the concept of wavelet packet decomposition. Then we present the basic notions of a genetic algorithm (section 3) and discuss a crossover operator which is specifically modified in order to be ideally suited for optimizing tree structures arising from wavelet packet subband tree decompositions. In the experimental section we compare this crossover operator to classical two-point-crossover in the context of different non-additive cost functions used for subband optimization. 2 Wavelet Packet Decomposition and Basis Choice Wavelet packets [38] represent a generalization of the method of multiresolution decomposition and comprise the entire family of subband coded (tree) decompositions. Whereas in the wavelet case the decomposition is applied recursively to the coarse scale approximations only (leading to the well known (pyramidal) wavelet decomposition tree), in the wavelet packet (WP) decomposition the recursive procedure is applied to all the coarse scale approximations and detail signals, which leads to a complete WP tree (i.e. binary tree and quadtree in the D and 2D case, respectively see figure ) and more flexibility in frequency resolution. There are several possibilities how to determine the frequency subbands suited well for an application (the meaning of suitable depends on the type of application, e.g. signal/image compression [38], feature extraction [3], classification
Figure : D wavelet packet and pyramidal wavelet decomposition tree algorithms [9, 20], telecommunication applications [2], numerical mathematics [26], and many more). The WP best basis algorithm" [9] performs an adaptive optimization of the frequency resolution of a complete WP decomposition tree by selecting the most suitable frequency subbands for signal compression. This is done by optimizing additive information cost functions. The same algorithm employed with non-additive cost function is denoted near-best basis algorithm" [34], if the subband structure is restricted to uniform timefrequency resolution the corresponding algorithm is denoted best level selection" [38]. In the context of image compression a more advanced technique is to use a framework that includes both rate and distortion, where the best basis subtree which minimizes the global distortion for a given coding budget is searched [29]. Other methods use fixed bases of subbands for similar signals (e.g. fingerprints [8]) or search for good representations with a genetic algorithm [7, 5]. Recently, WP based compression methods have been developed [40, 25] which outperform the most advanced wavelet coders (e.g. SPIHT [30]) significantly for textured images in terms of rate-distortion performance. However, the class of images for which excellent results have been achieved with classical (additive) cost-function based wavelet packet methods is somewhat limited. Usually, special frequency characteristics are required to guarantee a better behaviour as compared to the wavelet case: this has been shown for fingerprints [8] (and was verified with our software as well [5] in that case) and for the testimage Barbara" [24, 40, 25]. For mammograms [4] no advantages over pyramidal wavelet coding could be achieved using wavelet packet methods. Classical non-additive information cost functions (as given in [33, 34]) include the (non-additive) Shannon entropy, the weak l p -norm, and the compression area. Almost all of these functions require a sorted list of transform coefficients to be evaluated and are therefore very expensive from a computational point of view. Additionally, in contrast to the best basis algorithm the near-best basis algorithm does not guarantee to find the optimal subband structure (in terms of optimality of the value of the cost function) due to the non-additivity of the cost-functions involved. For these reasons we have focused in recent work on developing new non-additive cost-functions (of a type where no sorting procedure is required) [5]. Furthermore, evolutionary optimization is investigated for its applicability in the context of optimizing WP subband structures with non-additive cost-functions [4]. It has turned out that ffl evolutionary optimization clearly outperforms the near-best basis algorithm in terms of cost function values and ffl newly developed non-additive cost-functions with significantly lower computational demand give at least equal rate-distortion performance as compared to those given in the literature so far. See figure 2 for a comparison of pyramidal wavelet decomposition, the near-best basis algorithm and a genetic algorithm if the values of the compression area cost-function are considered. It is clearly visible that after a relatively small amount of time the GA outperforms both other techniques significantly. Costs 0.00035 0.0003 0.00025 0.0002 0.0005 0.000 5e-05 Compression Area Genetic Method Near Best Basis Classic Wavelet Method 0 500 000 500 2000 Time Figure 2: Optimization of the compression area, applied to a mammogram. Figure 3 shows the resulting final WP subband structures computed with different optimization algorithms corresponding to figure 2. Figure 3: Quadtree structures: pyramidal and optimized with near-best basis algorithm and GA. Summarizing, we have found that evolutionary optimization is a promising technique for optimizing WP subband structures with respect to all types of non-additive cost-functions. Note that this technique may be also employed using PSNR as a rate-distortion based cost function provided an embedded WP codec is available (since a fixed bitrate must be guaranteed during the optimization process for different subband structures in this case).
Here, we will investigate the crossover operator within the GA in more detail. Specifically, when applying this operator onto the binary string representing the WP subband structure in the classical way, already optimized parts of these trees may be entirely destroyed again. For this reason we introduce and experimentally evaluate a modified crossover operator which takes specific WP subband structure properties into account. 3 Genetic optimization of wavelet packet subband structures Genetic algorithms (GAs) [5] are evolution-based search algorithms especially designed for parameter optimization problems with vast search spaces (first proposed by Holland [7]). It is assumed that there is a given problem with a function which is to be optimized (objective function). Depending on the underlying problem, a coding or mapping function has to be defined to transform various solutions of different quality into genotypes, which are coded solutions for the optimization problem. The genotype itself is a string of finite length within a finite alphabet. The quality (fitness) of a solution and its corresponding genotype is determined by a fitness function which is derived from the objective function. The combination of genotype and its fitness is called individual. The fitness value of an individual and the applied selection scheme determine which genotypes are used for reproduction. The selection scheme mimics nature's principle of the survival of the fittest. The selected genotypes are recombined by a crossover operator. Two genotypes selected for reproduction are cut at a position which is chosen at random. One of the two genotype's parts is exchanged with the corresponding part of the second genotype. The new genotypes are inserted into an intermediate population. Afterwards, a mutation operator is applied to all genotypes. The mutation operator arbitrarily changes some of the alleles (values at certain positions (loci) of the genotype) with a small probability in order to ensure, that alleles which might have vanished from the population have a chance to reenter. After applying mutation, the intermediate population has turned into a new one (second generation) replacing the former. In our concrete application the genotypes b ~ i ; i 2 f;:::;ng are strings of finite length L over a binary alphabet f0; g. All genotypes can be interpreted as encoded wavelet packet subband structures. An allele on the genotype at index (locus) k; k 2 f;:::;lg provides the basis for determining if the corresponding subband is further decomposed. ρ b k = decompose 0 stop () If the allele at index k equals to (b k = ) the indices of the resulting 4 subbands are derived by k 0 m = 4 k + m; m 2 f;:::;4g: (2) 3. Tree-Crossover Standard crossover operators (eg. one-point or two-point crossover) have a considerably disruptive effect on the tree structure of subbands which is encoded into a binary string. With the encoding discussed above a one- or two-point-crossover results in two new individuals with tree-structures which are almost unrelated to the tree-structures of their parents. This obviously contradicts the basic idea of a GA i.e. the GA is expected to evolve better individuals from good parents. To demonstrate the effect of standard one-pointcrossover we present a simple example. The chomosomes of the parent individuals A and B are listed in table and the according binary trees are shown 2 3 4 5 6 7 8 9 0 ::: A 0 0 0 0 0 0 0 0 ::: B 0 0 0 0 0 ::: Table : Chromosomes of two individuals in figure 4. 2 3 4 5 6 7 8 9 0 2 3 4 5 As cut-point for the crossover we 4 5 6 7 8 9 2 0 2 3 4 5 Figure 4: Parent individuals before crossover choose the gap between gene 6 and 7. The chromosome parts from locus 7 to the right end of the chromosome are exchanged between individual A and B. This results in two new trees which are displayed in figure 5. Evidently, the new generation 2 3 4 5 6 7 8 9 0 2 3 4 5 4 5 6 7 8 9 0 2 3 4 5 Figure 5: Indivduals created by conventional one-point crossover of trees differ considerably from their parents. Therefore, the notion is to introduce a probleminspired crossover such that the overall tree structure is preserved while only local parts of the subband trees are altered. Specifically, one node in each individual (i.e. subband tree) is chosen at random, then the according subtrees are exchanged between the individuals. In our example, the canditate nodes for the crossover are node 2 in indvidual A and node 0 in individual B. The result of the tree-crossover is displayed in figure 2 3 3
2 3 2 3 6.48e-05 Compression Area 4 5 6 7 4 5 6 7 6.46e-05 8 9 0 2 3 4 5 8 9 0 2 3 4 5 6.44e-05 Figure 6: Individuals created by tree-crossover Fitness 6.42e-05 6. Compared to the standard crossover operator, tree crossover moderately alters the structure of the parent individuals and generates new ones. 4 Experiments For our experiments we have implemented a standard GA. The population size was set to 30 and the GA was stopped after 200 generations. After each reproductive cycle the whole population is replaced by a new one while the all-time-best individual is stored aside the evolution process (i.e. no elitism). Generally, the choice of the selection method is problem-dependent. Therefore, we have applied a variety of different selection operators. Specifically, we have used roulette wheel selection with windowing (RWwW), tournament selection with various tournament sizes (TS t 2 f2; 3; 4g), linear (RK L = 0:4) and exponential (RK E c = 0:968) ranking selection. For an exhausting (Hi Karli) overview concerning selection methods see [3, 6]. As crossover operator we applied a standard two-point-crossover at a rate of 0.6 on the one hand and our tree-crossover at the same rate on the other. Furthermore, we used mutation which arbitrarily changes genes at a rate of 0.00. The results of random search (Rd) have been included in all figures to provide a baseline for performance comparisons. As test image, we used the standard 8bpp 52 * 52 pixels image Lena. In all figures it is clearly visible that TS with t = 3 and t = 4 are the top performing selection methods (which are both techniques with high selection pressure). Figures 7 and 8 display results concerning the cost function compression area. After 200 generations, the final cost function value is superior when using the tree-crossover operator for all selection techniques considered (except for RWwW). Additionally, the shape of the resulting costfunction value curves attains higher values in the tree-crossover case in earlier stages of the optimization process for all techniques but RWwW and RK L. Note, that only TS with t = 3 and t = 4 perform better than Rd. Figures 9 and 0 show results obtained when employing the non-additive Shannon entropy as cost function. Note that the maximum of this cost-function is attained if a complete decomposition is performed. In terms of the binary string to be optimized this is similar to the onemaxproblem (this is why Rd performs very poorly in 6.4e-05 Rd 0.0 RWwW 0.0 TS t = 2 0.0 6.38e-05 TS t = 3 0.0 TS t = 4 0.0 RK L eta 0.4 0.0 RK E c 0.968 0.0 6.36e-05 0 20 40 60 80 00 20 40 60 80 200 Figure 7: Evolutionary optimization of compression area. Fitness 6.48e-05 6.46e-05 6.44e-05 6.42e-05 6.4e-05 Compression Area with tree x-over 6.38e-05 Rd 0.0 RWwW 0.0 TS t = 2 0.0 6.36e-05 TS t = 3 0.0 TS t = 4 0.0 RK L eta 0.4 0.0 RK E c 0.968 0.0 6.34e-05 0 20 40 60 80 00 20 40 60 80 200 Figure 8: Evolutionary optimization of compression area using tree-crossover. this case). Besides the the final result, the evolution in time is quantified by displaying the value of the integral under the curves in the legend. Four out of six selection methods show superior integral values in the tree-crossover case (i.e., RWwW, TS t=2, TS t=4, and RK L) but only three methods lead to a superior final result (i.e., TS t=2, TS t=4, and RK L). It is interesting to notice that the by far best performing method in the classical" case (which is TS t=3) drops significantly in the tree-crossover case thereby reducing the overall maximum by 20%. 5 Conclusion In this work we focus on wavelet packet decomposition for still image compression using a GA to optimize the subband tree-structure. We found that a standard GA applying two-point-crossover combined with the tree-encoding-scheme provided in section 3 causes considerable disruptions among the parent and children individuals involved. Therefore, we have introduced a problem-inspired tree-
Fitness 0.6 0.5 0.4 0.3 0.2 0. Rd.99 RWwW 6. TS t = 2 5.29 TS t = 3 42.88 TS t = 4 24.65 RK L eta 0.4.8 RK E c 0.968 0.5 Shannon Entropy 0 0 20 40 60 80 00 20 40 60 80 200 Figure 9: Evolutionary optimization of non-additive Shannon entropy. Fitness 0.45 0.4 0.35 0.3 0.25 0.2 0.5 0. 0.05 Rd.72 RWwW 7.9 TS t = 2 20.32 TS t = 3 3.92 TS t = 4 28.59 RK L eta 0.4 2.43 RK E c 0.968 7.22 Shannon Entropy with tree-x-over 0 0 20 40 60 80 00 20 40 60 80 200 Figure 0: Evolutionary optimization of non-additive Shannon entropy using tree-crossover. crossover-operator in order to lessen the disruptive effect of crossover. From our experiments in section 4, we found some evidence that the GA employing tree-crossover improves the performance of the GA. References [] M. Antonini, M. Barlaud, P. Mathieu, and I. Daubechies. Image coding using wavelet transform. IEEE Transactions on Image Processing, (2):205 220, 992. [2] A. Averbuch, D. Lazar, and M. Israeli. Image compression using wavelet transform and multiresolution decomposition. IEEE Trans. on Image Process., 5():4 5, 996. [3] T. Blickle and L. Thiele. A comparison of selection schemes used in evolutionary algorithms. Evolutionary Computation, 4(4):36 394, 997. [4] A. Bruckmann, H. Flatscher, Th. Schell, and A. Uhl. Adaptive wavelet techniques for compressing digital mammograms (Invited Paper). TASK Quarterly (Special Issue on Informatics in Medicine), 3(4):38 396, 999. [5] A. Bruckmann, Th. Schell, and A. Uhl. Evolving subband structures for wavelet packet based image compression using genetic algorithms with non-additive cost functions. In Proceedings of the International Conference Wavelets and Multiscale Methods" (IWC'98), Tangier, 998. INRIA, Rocquencourt, April 998. 4 pages. [6] M. Charriera, D. Santa Cruz, and M. Larsson. JPEG2000, the next millenium compression standard for still images. In Proceedings of the IEEE ICMCS'99, June 999. [7] C.H. Chu. Genetic algorithm search of multiresolution tree with applications in data compression. In H.H. Szu, editor, Wavelet Applications, volume 2242 of SPIE Proceedings, pages 950 958, 994. [8] C.K. Chui, editor. Wavelets: A Tutorial in Theory and Applications. Academic Press, San Diego, 992. [9] R.R. Coifman and M.V. Wickerhauser. Entropy based methods for best basis selection. IEEE Transactions on Information Theory, 38(2):79 746, 992. [0] P.C. Cosman, R.M. Gray, and M. Vetterli. Vector quantization of image subbands: A review. IEEE Transactions on Image Processing, 5(2):202 225, 996. [] P. Desarte, B. Macq, and D.T.M. Slock. Signal-adapted multiresolution transform for image coding. IEEE Transactions on Information Theory, 38(2):897 904, 992. [2] B.J. Erickson, A. Manduca, et al. Wavelet compression of medical images. Radiology, 206(3):599 607, 998. [3] Y. Fisher, editor. Fractal Image Encoding and Analysis, volume 59 of NATO ASI Series. Series F: Computer and Systems Sciences. Springer-Verlag, Berlin, Heidelberg, New York, Tokyo, 998. [4] J. Froment and S. Mallat. Second generation compact image coding. In [8], pages 655 678. Academic Press, 992. [5] D. E. Goldberg. Genetic Algorithms in Search, Optimization & Machine Learning. Addison-Wesley, Reading, Massachusetts, 989. [6] P. J. B. Hancock. Evolutionary Computation, chapter An Empirical Comparison of Selection Methods in Evolutionary Algorithms, pages 80 94. Springer-Verlag, Berlin, D, 994.
[7] J. H. Holland. Adaptation in Natural And Artificial Systems. MIT Press, Ann Arbor, MI, 975. [8] T. Hopper. Compression of gray-scale fingerprint images. In H.H. Szu, editor, Wavelet Applications, volume 2242 of SPIE Proceedings, pages 80 87, 994. [9] A. Laine and J. Fan. Texture classification by wavelet packet signatures. IEEE Transactions on Pattern Analysis and Machine Intelligence, (5):86 9, 993. [20] R.E. Learned and A.S. Willsky. A wavelet packet approach to transient signal classification. Applied and Computational Harmonic Analysis, 2:265 278, 995. [2] A.R. Lindsey and J.C. Dill. Digital transceiver implementation for wavelet packet modulation. In H.H. Szu, editor, Wavelet Applications V, volume 339 of SPIE Proceedings, pages 255 264. SPIE, 998. [22] J. Lu, V.R. Algazi, and R.R. Estes. Comparative study of wavelet image coders. Optical Engineering, 35(9):2605 269, 996. [23] N. Lu, editor. Fractal Imaging. Academic Press, San Diego, CA, 997. [24] D. Marpe, H.L. Cycon, and W. Li. Complexity constrained best-basis wavelet packet algorithm for image compression. IEE Proceedings Vision, Image, and Signal Processing, 45(6):39 398, 998. [25] F.G. Meyer, A.Z. Averbuch, J.O. Strömberg, and R.R. Coifman. Fast wavelet packet image compression. In J.A. Storer and M. Cohn, editors, Proceedings of the Data Compression Conference DCC'98, page 563. IEEE Press, March 998. [26] L. Bacchelli Montefusco. Semi-orthogonal wavelet packet bases for parallel least-squares approximation. Journal of Computational and Applied Mathematics, 73:9 208, 996. [27] L.P. Panych. Theoretical comparison of Fourier and Wavelet encoding in Magnetic Resonance Imaging. IEEE Trans. on Medical Imaging, 5(2):4 53, 997. [28] W.B. Pennebaker and J.L. Mitchell. JPEG Still image compression standard. Van Nostrand Reinhold, 993. [29] K. Ramchandran and M. Vetterli. Best wavelet packet bases in a rate-distortion sense. IEEE Trans. on Image Process., 2(2):60 75, 993. [30] A. Said and W.A. Pearlman. A new, fast, and efficient image codec based on set partitioning in hierarchical trees. IEEE Transactions on Circuits and Systems for Video Technology, 6(3):243 249, 996. [3] N. Saito and R.R. Coifman. Local discriminant bases. In A.F. Laine and M.A. Unser, editors, Wavelet Applications in Signal and Image Processing II, volume 2303 of SPIE Proceedings, pages 2 4, 994. [32] J.M. Shapiro. Embedded image coding using zerotrees of wavelet coefficients. IEEE Trans. on Signal Process., 4(2):3445 3462, 993. [33] C. Taswell. Near-best basis selection algorithms with non-additive information cost functions. In Moeness Amin, editor, Proceedings of the IEEE International Symposium on Time-Frequency and Time-Scale Analysis, pages 3 6. IEEE Press, 994. [34] C. Taswell. Satisficing search algorithms for selecting near-best bases in adaptive treestructured wavelet transforms. IEEE Transactions on Signal Processing, 44(0):2423 2438, 996. [35] P.N. Topiwala, editor. Wavelet Image and Video Compression. Kluwer Academic Publishers Group, Boston, 998. [36] A. Uhl. Image compression using nonstationary and inhomogeneous multiresolution analyses. Image and Vision Computing, 4(5):365 37, 996. [37] G.K. Wallace. The JPEG still picture compression standard. Communications of the ACM, 34(4):30 44, 99. [38] M.V. Wickerhauser. Adapted wavelet analysis from theory to software. A.K. Peters, Wellesley, Mass., 994. [39] Z. Xiong, K. Ramchandran, and M.T. Orchard. Efficient arithmetic coding for wavelet image compression. In J. Biemond and E.J. Delp, editors, Visual Communications and Image Processing '97, volume 3024 of SPIE Proceedings, pages 3 24, San Jose, February 997. [40] Z. Xiong, K. Ramchandran, and M.T. Orchard. Wavelet packet image coding using space-frequency quantization. IEEE Transactions on Image Processing, 7(6):892 898, June 998.