Vector quantization based on genetic simulated annealing

Transcription

1 Signal Processing 81 (2001) 1513}1523 Vector quantization based on genetic simulated annealing Hsiang-Cheh Huang, Jeng-Shyang Pan*, Zhe-Ming Lu, Sheng-He Sun, Hsueh-Ming Hang Department of Electronic Engineering, National Chiao-Tung University, Taiwan Department of Electronic Engineering, National Kaohsiung University of Applied Sciences, 415 Chien-Kung Road, Kaohsiung, Taiwan Department of Automatic Test and Control, Harbin Institute of Technology, Harbin, China Received 18 November 1999; received in revised form 25 February 2001 Abstract Genetic algorithm (GA) has been successfully applied to codebook design for vector quantization (VQ). However, most conventional GA-based codebook design methods need long runtime because candidate solutions must be "ne tuned by LBG. In this paper, a partition-based GA is applied to codebook design, which is referred to as genetic vector quantization (GVQ). In addition, simulated annealing (SA) algorithm is also used in GVQ to get more promising results and the corresponding method is referred to as GSAVQ. Both GVQ and GSAVQ use the linear scaling technique during the calculation of objective functions and use special crossover and mutation operations in order to obtain better codebooks in much shorter CPU time. Experimental results show that both of them save more than 71}87% CPU time compared to LBG. For di!erent codebook sizes, GVQ outperforms LBG by 1.1}2.1 db in PSNR, and GSAVQ outperforms LBG by 1.2}2.2 db in PSNR. In addition, GVQ and GSAVQ need a little longer CPU time than, the maximum decent (MD) algorithm, but they outperform MD by 0.2}0.5 db in PSNR Elsevier Science B.V. All rights reserved. Keywords: Vector quantization; Codebook design; Genetic algorithm; Simulated annealing 1. Introduction As an e!ective method for data compression, VQ has been successfully used in speech coding and image compression. The k-dimensional, N-level vector quantizer is de"ned as a mapping from a k-dimensional Euclidean space R into a certain "nite set C"c, c,2, c. The quantizer is completely described by the codebook C together with * Corresponding author. Tel.: # ext. 5636; fax: # address: jspan@cc.kuas.edu.tw (J.-S. Pan). the partitioned set consisting of subspaces of R, S"s, s,2, s, and the mapping function Q(z): Q(X)"c, ifx3s. (1) The elements of the partitioned set S satisfy s "R and s s "Φ, ifioj. If the squared Euclidean distortion measure is used, the output of the vector quantizer is the index i of the codeword c which satis"es i"arg min (x!c ). (2) Codebook design is the key problem of VQ and the generated codebook has more e!ect on the /01/$ - see front matter 2001 Elsevier Science B.V. All rights reserved. PII: S ( 0 1 )

2 1514 H.-C. Huang et al. / Signal Processing 81 (2001) 1513}1523 Nomenclature VQ GA GVQ GSAVQ SA LBG Unit j P(t) A (t) A (t) n(t) vector quantization genetic algorithm genetic vector quantization genetic simulated annealing vector quantization simulated annealing conventional codebook design algorithm the jth partitioned set or the jth cluster population of the tth generation the lth individual for population P(t) the best individual that has minimal overall distortion in population P(t) the number of genes that belong to unit j for individual l in population P(t) a (t) X >(t) C (t) D (t) D (t) S (t) f (t) p (t) the ith gene of unit j for individual l in population P(t) the training vector whose label is a (t)in the chromosome the rth codeword for individual l in population P(t) the corresponding codebook of individual A (t) the overall distortion of individuala (t) the overall distortion of the best individuala (t) the inverse of the overall distortion D (t) the "tness function of individual A (t) the selected probability of individual A (t) compression performance. The traditional codebook design method*lbg algorithm [7] is a!ected by the initial codebook, often generates the local optimal codebook and needs intensive computation. Research e!orts in codebook design have been concentrated in two directions: to generate a better codebook that approaches the global optimal solution, and to reduce the computational complexity. To generate better codebooks, SA has been proposed in [13]. The SA method attempts to obtain a better codebook by shaking the codebook o! the local valley in the hope that it will converge to another valley that gives less error. The SA method makes the LBG algorithm converge again and again to obtain a lower overall error. Thus, the computation time of SA is much longer in comparision with LBG. The stochastic relaxation approach [14] is also proposed to improve the codebook design. The basic idea of stochastic relaxation approach is to add some values to the codewords de"nitely for each iteration. In addition, deterministic annealing [12] has been used for codebook design recently. However, these methods need a great deal of time in order to obtain better codebooks. To reduce the codebook generation time, many methods have appeared in the literatures. The subspace distortion method [11] attempts to reduce the computation time by reducing the dimension measure in the LBG algorithm. The pairwise nearest neighbor (PNN) algorithm [2] generates a codebook by merging nearest training vector clusters until the desired number of codewords is obtained. The codebooks generated by both methods are slightly degraded even though the computation time is reduced by several times. Maximum decent algorithm [8] was also proposed for VQ codebook design. The algorithm begins with treating the training vector set as a global cluster. The algorithm generates the required number of clusters one by one using the maximum criterion until the desired number of codewords are obtained. Compared with the LBG algorithm, the codebook performance is improved and the computation time is substantially reduced. However, this algorithm can hardly obtain a global optimal codebook. The aim of the codebook design is to "nd the best classi"cation of training vectors. Given the number of the training vectors M and the number of

3 H.-C. Huang et al. / Signal Processing 81 (2001) 1513} codewords N, codebook design problem is an NPhard problem to classify M training vectors into N clusters. For larger M and N, a traditional search method can hardly "nd the global optimal classi- "cation. GA [4}6,10] is an e$cient, parallel and near global optimum search method based on the ideas of nature selection and nature genetics. During the search processing, it can automatically achieve and accumulate the knowledge about the search space, and adaptively control the search process to approach the global optimal solution. However, the convergence speed of GA is a little slower because of its poor local optimum search ability. Genetic algorithm [1,9] has been used to generate better codebooks in recent years. Delport and Koschorreck proposed the partition-based GA codebook design algorithm [1] whose coding string is the codebook indices of the training data. The codebook vectors were also used as the coding string to design the codebook [9] which can be referred to as the codebook-based codebook design algorithm. However, previous GA-based codebook design algorithms have the same shortcoming, i.e., long runtime. This shortcoming is caused by the following two reasons: (1) all candidates are iterated by LBG, and (2) the crossover operation is not e$cient. In order to improve the crossover operation, genetic algorithm with deterministic crossover [3] has been used to improve the e!ectiveness of the crossover operation, but this algorithm still needs much more CPU time than LBG algorithm. SA is a near global optimum search method based on the idea of physical annealing, whose operation object is not a group of approximate solutions like GA but a single approximate solution. The convergence speed of SA is higher than GA, but the performance of SA is a!ected by a lot of factors and it is not so easy to approach the global optimal solution. Because of this, we can make full use of the virtues of GA and SA and present a better codebook design method referred to as genetic simulated annealing vector quantization (GSAVQ). In the next section, both GVQ and GSAVQ algorithms are intensively discussed. 2. GVQ and GSAVQ algorithms 2.1. Genetic vector quantization In this section, we will introduce a partitionbased GVQ codebook design algorithm. Suppose that the number of training vectors, the codebook size and the vector dimension are M, N, and K, respectively. The label i (i"1,2,2, M) of the training vector is viewed as a gene. A basic object for genetic operations is a classi"cation of training vectors. The basic object is made up of N units, and each unit is made up of several labels that belong to this unit. N codewords are used to represent N units; here, a codeword is the centroid of the vectors whose indices belong to a certain unit. Suppose a population is made up of individuals, then the population at the tth generation can be described as P(t)"A (t), A (t),2, A (t). Individual A (t)(l"1,2,2, ) of population P(t) can be illustrated as in Fig. 1. In Fig. 1, n(t)( j"1,2,2, N) is the number of genes in unit j, and n(t)"m.(a (t)i"1,2,2, n(t)) is the ith gene of unit j. If the training set can be described as S"(X, X,2, X ), then a (t) is the label of vector X. Suppose the corresponding codebook of individual A (t) can be described as C (t)" >(t), >(t),2, >(t), then codeword >(t)(r" 1,2,2, N) can be calculated by the following formula >(t)" X. (3) n(t) The main idea of GVQ is to "nd the training vector with maximum distance from the centroid (codeword) of a unit for one individual and then randomly move this training vector to the other unit or Fig. 1. Individual A (t).

4 1516 H.-C. Huang et al. / Signal Processing 81 (2001) 1513}1523 move this training vector to the other unit according to the condition of the second individual. The GVQ algorithm can be stated as follows: Step 1: Population initialization. GA is strongly robust; so, how to generate the initial population has little e!ect on the codebook performance but may have some e!ect on the runtime. The population is required to include enough number of individuals to ensure the great diversity of individuals in the population. In this paper, the initial population P(0) is made up of ( '1) individuals, which are randomly generated, namely, P(0)"A (0), A (0),2, A (0). Step 2: Fitness function computation. The overall distortion D (t) of individual A (t) can be expressed as D (t)" X!> (t), (4) where >(t) is the rth codeword in codebook C (t), X is the training vector whose label is a (t), and X!> (t) is the squared Euclidean distance between X and > (t), i.e., the distortion of the input vector X. Because the ultimate aim of VQ is to obtain the best classi"cation that has the least overall distortion, the objective function S (t) can be directly de"ned as the inverse of the overall distortion, namely, S (t)"1/d (t). Fitness function is usually used to evaluate the "tness of an individual for the environment. In this paper, "tness function is used to evaluate the codebook performance. The more "tness the individual has, the higher performance its corresponding codebook has. Thus, "tness function can be de"ned as the objective function S (t), as de"ned above. In order to restrain the probable misguided inclination of `super-individuala at the beginning of the search process and avoid converging too early, the linear scaling technique is used in this paper. Suppose the maximum objective function of Population P(t)isS (t), and the average objective function of population P(t)isS (t)" S (t)/, then the "tness function f (t)(l"1,2,2, ) of individual A (t) can be de"ned as f (t)" S (t) S (t)!s (t) (S (t)#s (t)!2s (t)). (5) Step 3: Selection. The individuals with more "tness ought to have more opportunity to be selected as the mother individuals of next generation. The selection probability p (t)(l"1,2,2, ) of individual A (t) can be de"ned as p (t)" [f (t)!f (t)] [f (t)!f (t)], (6) where f (t)"min(f (t), f (t),2, f (t)). Then L pairs of mother individuals are selected according to the probability p (t)(l"1,2,2, ). Step 4: Crossover. Each pair of mother individuals generates a new individual by the crossover operation, thus pairs generate a new population. As shown in Fig. 2, the crossover approach of mother individual 1 and mother individual 2 can be described as follows: (a) In individual 1, select a certain unit m randomly and "nd vector j in unit m which has maximum distortion. (b) In individual 2, "nd unit p including vector j and select another vector u in this unit randomly. (c) In individual 1, "nd unit n including vector u. (d) In individual 1, if mon, push vector j from units m to n with certain crossover probability p. (e) Repeat the above operations for several times. The "nally obtained individual 1 is the individual of child population. In this paper, the number of repeated times equals the population size. Step 5: Mutation. In order to avoid the local optimum problem, the mutation operation is essential. As shown in Fig. 3, for each individual of the child population which is obtained by crossover operations, select two units randomly, for example, units m and n, move vector j in unit m which has the maximum distortion in unit n with certain probability p. Step 6: Termination. Two termination criteria are used here. For the "rst one, the process is executed for "xed number of iterations and the best solution obtained is taken to be the optimal one. For the second one, the algorithm is terminated if no further improvement in the "tness value of the best solution is observed for "ve iterations, and the best solution is take as an optimal one. If one of the above termination criteria is satis"ed, then the algorithm is terminated. Otherwise, go to Step 2.

5 H.-C. Huang et al. / Signal Processing 81 (2001) 1513} Fig. 2. Crossover approach of GVQ. Fig. 3. Mutation approach of GVQ. From above, we know that the proposed GVQ is partition-based, i.e., each solution is not a codebook but a partition of the training vectors. Secondly, the genetic operations are di!erent from the conventional GA-based VQ. The conventional SA-based VQ and GA-based VQ often include LBG iterations in each iteration of SA or GA. However, in this paper, GLA iterations are not included in GA. Moreover, in the proposed semideterministic and semi-random crossover and mutation operations, the labels whose corresponding vectors have maximum distortion are moved in order to let the child solution approach the global optimal solution with more opportunities. In addition, no codeword search process is required in the calculation of the objective function. So the proposed GVQ is fast and e$cient Genetic simulated annealing algorithm In order to improve the local optimum search ability of GA and avoid the `premature phenomenoa of GA, SA is introduced in GVQ algorithm. SA is a global optimum search method based on the idea of physical annealing, whose operation object is not a group of approximate solutions like

6 1518 H.-C. Huang et al. / Signal Processing 81 (2001) 1513}1523 Fig. 4. Exchange approach of GSAVQ. GA but a single approximate solution. The convergence speed of SA is higher than GA, but the initial temperature and the decreasing speed of the annealing temperature a!ect the performance of SA. If the initial temperature is high enough and the annealing temperature decreases slowly enough, the obtained solution approaches the global solution with probability one but the convergence speed is very low. On the other hand, SA is easy to fall into local optimum if the annealing temperature decreases quickly. If SA is used in the environment of a group of approximate solutions and made some modi"cation, the performance of SA can be improved. The genetic simulated annealing vector quantization (GSAVQ) can be described as follows: Step 1. Initialize parameters ¹ and threshold ε (0(ε(1), set t"0, i"0, D (!1)"R. Step 2. Generate an intial population P(0)"A (0), A (0),2, A (0)( '1). Step 3. Calculate the overall distortion for each individual in P(t). Step 4. In P(t), "nd the best individual A (t) that has minimum overall distortion. If the overall distortion D (t) of A (t) satis"es (D (t)! D (t!1))/d (t)(ε or the iteration times t is larger than a given value, the algorithm is terminated with the corresponding codebook C (t)" > (t), > (t),2, > (t) whose codeword > (t)(r"1,2,2, N) can be calculated by > (t)" X, (7) n (t) otherwise continue. Step 5. Decrease the annealing temperature in a certain way, i.e. ¹ "¹! ¹ ( ¹ '0), set i"i#1. Step 6. Generate a new individual A (t) in the neighbourhood of individual A (t). As shown in Fig. 4, A (t) is generated as follows: (1) select two units randomly, for example, units i and j; (2) "nd the vector which has maximum distortion in each unit, for example, vector m in unit i and vector n in unit j; (3) exchange vector m with vector n; (4) repeat the above operations for several times. Here, the number of repeated times is set to be the same as the population size. Step 7. Compute the overall distortiond (t) for A (t) and D (t) for A (t), then calculate the di!erence D(t)"D (t)!d (t). If D(t)(0, accept A (t) as a new individual of P(t), else if D(t)*0, accept A (t) as a new individual of P(t) with probability exp(! D(t)/¹ ). Step 8. If A (t) is accepted, add it into P(t) as a new individual. Otherwise, randomly select an individual in P(t) and add it into P(t) as a new individual. Step 9. For the previous population P(t), generate a new population P(t#1) using genetic operations (steps 3}5 of GVQ). Remove the individual that has the worst performance from P(t#1). Set t"t#1, go to step 3. From above, we know that GSAVQ applies simulated annealing method to decide whether a training vector with maximum distance from the center (codeword) of a unit is suitable to be exchanged with the other training vector, and then combine with GVQ algorithm to get a better

7 H.-C. Huang et al. / Signal Processing 81 (2001) 1513} codebook. Indeed, the incorporation of SA into GVQ can be viewed as the additional mutation operation besides genetic mutation operations, which can further reduce the opportunities of the algorithm getting into the local minimum. So GSAVQ can obtain better performance than GVQ in theory. 3. A concrete example To explain the crossover and mutation operations of the proposed algorithms, we give a concrete example here. Suppose there are 16 two-dimensional training vectors, the aim is to generate four codewords. Thus M"16, N"4 and K"4. Here, we assume the 16 training vectors to be X "(13,14), X "(7,8), X "(0,3), X "(2,7), X "(3,6), X "(1,7), X "(15,14), X "(2,11), X "(6,14), X "(5,10), X "(10,4), X " (1,2), X "(5,8), X "(10,14), X "(11,9) and X "(15,12). First, we explain how an individual corresponds to a codebook. Assume a certain individual can be expressed as A "(16,9,4,7,5), (2,10), (13,6,14,1,3, 12),(8,11,15), with each element being the label of a training vector, e.g., the element &16' denotes the training vector X. The labels are classi"ed into four units, and the numbers of elements in four units are n"5, n"2, n"6 and n"3, respectively. The codeword of a unit is the centroid of the training vectors whose labels belong to this unit, thus the individual A corresponds to a codebook C ">, >, >, >, where > "(X #X # X #X #X )/5"(8,11), > "(X #X )/2" (6,9), > "(X #X #X #X #X # X )/6"(5,8), > "(X #X #X )/3"(8,8). Then we turn to explain the crossover operation used in GVQ and GSAVQ. Assume two individuals A "(16,9,4,7,5),(2,10),(13,6,14,1,3,12), (8,11,15) and A "(2,4,8,5),(3,15,7),(11,14,1,3,12), (6,10,13,9) are selected to perform the crossover operation. Firstly, in individual A, select a certain unit m randomly and "nd vector j in unit m which has maximum distortion. Assume unit 1 is selected, i.e., m"1, then vector X (j"7) has maximum distortion in unit (16, 9, 4, 7, 5), i.e., d(x, > )"maxd(x, > ), d(x, > ), d(x, > ), d(x, > ), d(x, > ). Secondly, in individual A, "nd unit p including vector j and select another vector u in this unit randomly. Based on the above assumption, i.e., j"7, then p"2. And assume we select vector X (u"3) in unit 2. Thirdly, in individual A, "nd unit n including vector u. Based on the above assumptions, we know that vector X is in unit 3 of individual A, i.e., n"3. Fourthly, In individual A,ifmOn, push vector j from unit m to unit n with certain crossover probability p. Based on the above assumptions, because m"1, n"3, mon, then we can push vector X from units 1 to 3 in individual A. After the above four steps, we can obtain new individual A "(16,9,4,5), (2,10), (13,6,14,1,7,3,12),(8,11,15). Repeat the above four steps for several times, then we can obtain the "nal child individual A. Finally, we turn to explain the mutation operation. Assume the individual A "(16,9,4,7,5), (2,10),(13,6,14,1,3,12),(8,11,15) is selected to perform the mutation operation. The operation is performed as follows: select two units randomly, for example, units 1 and 4, move vector X in unit 1 which has the maximum distortion into unit 4 with certain probability p. If the move is performed, then the individual A becomes another new individual A "(16,9,4,5),(2,10),(13,6,14,1,3, 12),(7,8,11,15). 4. Experimental results To evaluate the e$ciency of the proposed algorithms, GSAVQ and GVQ, accompanied with SA, MD and LBG, were coded in Visual C## language and run on Pentium II computer. Two images, Lena and Peppers, with resolution pixels, 8 bits per pixel, were used in this paper. The Lena image was used to generate the codebooks of di!erent sizes with dimension 16 (44), the Peppers image was used to test the performance of the codebook. For LBG, the initial codebook is generated randomly and no speeding up methods are used in LBG and ε" In the MD algorithm, the successive search method is used for searching the optimal partitioning hyperplane. For both GVQ and GSAVQ, the crossover probability p is 0.9 and the mutation probability p is 0.01, the population size is 40, the number of

8 1520 H.-C. Huang et al. / Signal Processing 81 (2001) 1513}1523 Table 1 The CPU time(s) comparisons of LBG, SA, MD, GVQ and GSAVQ for di!erent codebook sizes Codebook size LBG SA MD GVQ GSAVQ Table 2 Performance (PSNR) comparisons of LBG, SA, MD, GVQ and GSAVQ for the image inside the training set Codebook size LBG SA MD GVQ GSAVQ Table 3 Performance (PSNR) comparisons of LBG, SA, MD, GVQ and GSAVQ for the image outside the training set Codebook size LBG SA MD GVQ GSAVQ iterations is limited to 150. For GSAVQ, the initial temperature ¹ is 50 and the temperature is decreased by 0.6% after each iteration until the number of iterations reaches 150, ε" For SA [14], the initial temperature ¹ is 35 and the temperature is decreased by 1% after each iteration step until the number of iterations reaches 30. Experimental results are shown in Tables 1}4. As shown in Table 1, the "ve algorithms are compared in CPU time by generating di!erent codebook sizes. Table 2 compares the "ve algorithms in PSNR for di!erent codebook sizes using Lena as Table 4 Performance (MSE and time) comparisons of LBG, GVQ and GSAVQ for coding four-dimensional Guass}Markov sources with di!erent codebook sizes Codebook size Performance (s) LBG GVQ GSAVQ 8 Time MSE Time MSE Time MSE the test image. For the image outside the training set, Table 3 shows the performance of codebooks with di!erent sizes generated by di!erent algorithms. Fig. 5 shows the comparison of the test picture and pictures recovered by di!erent algorithms. Fig. 6 shows the relationship of PSNR and computation time for codebook sizes 128 and 256, and because the CPU time of SA is nearly 25}27 times that of LBG, the relationship of PSNR and computation time of SA is excluded in Fig. 6. Compared with LBG method for codebook sizes 64, 128 and 256, GVQ and GSAVQ both save more than 71%, 77% and 80% CPU time, respectively. For codebook size 512, GVQ saves more than 83% CPU time and GSAVQ saves more than 87% CPU time compared with LBG. For di!erent codebook sizes, the codebooks obtained by GVQ outperform those by LBG method by 1.1}2.1 db in PSNR, and the codebooks obtained by GSAVQ outperform those by LBG method by 1.2}2.2 db in PSNR. For di!erent codebook sizes, the codebooks obtained by GVQ and GSAVQ outperform those by the SA method by 0.2}0.5 db in PSNR. Thus the e!ectiveness of GVQ and GSAVQ is proven. Although MD algorithm can obtain the codebook in a much shorter time than GVQ and GSAVQ, its PSNR is lower than SA, GVQ and GSAVQ. In addition, we can see that the di!erence in the performance between GVQ and GSAVQ is slight after limited iteration times. However, if the iteration time is unlimited, the performance of GSAVQ will be much better than GVQ. In order to demonstrate the e!ectiveness of the proposed algorithms, we also made another

9 H.-C. Huang et al. / Signal Processing 81 (2001) 1513} Fig. 5. Comparisons of pictures recovered by di!erent algorithms for 256 codewords. (a) Original picture. (b) Picture recovered by LBG. (c) Picture recovered by SA. (d) Picture recovered by MD. (e) Picture recovered by GVQ. (f) Picture recovered by GSAVQ.

10 1522 H.-C. Huang et al. / Signal Processing 81 (2001) 1513}1523 Fig. 6. Relationship between PSNR and CPU time for di!erent codebook sizes for LBG, GVQ and GSAVQ algorithms (a) size 128 and (b) size 256. experiment. In this experiment, the proposed algorithms together with the LBG algorithm were used to design VQ codebooks using 1000 fourdimensional Guass}Markov vectors with unit variance and zero mean and σ"0.5 in each dimension. Each algorithm generates three codebooks with sizes 8, 16 and 32. The experimental results (MSE and time) are shown in Table 4. From Table 4, we can see that the proposed algorithms can obtain better performance in a shorter time compared to LBG. and GSAVQ also utilizes the virtues of simulated annealing (SA) in order to avoid the `premature phenomenona of GA and obtain a near global optimum codebook in a much shorter time. Experimental results show that GVQ and GSAVQ both have better performance and need shorter computer time than SA and LBG algorithms, and GSAVQ is superior to GVQ. In addition, although GVQ and GSAVQ need a little longer CPU time than maximum decent (MD) algorithm, they outperform MD in PSNR. 5. Summary and conclusion In this paper, two methods for codebook design are presented. These two algorithms are genetic vector quantization (GVQ) and genetic simulated annealing vector quantization (GSAVQ). For both algorithms, the label of the training vector is viewed as a gene and the basic object for genetic operations is a classi"cation of training vectors. The basic object is made up of a certain number (equals to codebook size) of units, and each unit is made up of several labels that belong to this unit. A certain number of codewords are used to represent these units; here a codeword is the centroid of a unit. In order to restrain the probably misguided inclination of `super-individuala at the beginning of the search process and avoid converging too early, the linear scaling technique is used in both algorithms. Both GVQ and GSAVQ make full use of the near global optimum ability of genetic algorithm (GA), References [1] V. Delport, M. Koschorreck, Genetic algorithm for codebook design in vector quantization, Electron. Lett. 31 (2) (1995) 84}85. [2] W.H. Equitz, A new vector quantization clustering algorithm, IEEE Trans. Acoust. Speech Signal Process 37 (10) (1989) 1568}1575. [3] P. Franti, Genetic algorithm with deterministic crossover for vector quantization, Pattern Recognition Lett. 21 (2000) 61}68. [4] M. Gen, R. Cheng, Genetic Algorithms and Engineering Design, Wiley, New York, [5] D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison Wesley Publishing Company, Reading, MA, [6] J.H. Holland, Adaptation in Natural and Arti"cial Systems, The University of Michigan Press, Ann Arbor, MI, [7] Y. Linde, A. Buzo, R.M. Gray, An algorithm for vector quantizer design, IEEE Trans. Commun. 28 (1) (1980) 84}95. [8] C.K. Ma, C.K. Chan, Maximum descent method for image vector quantization, Electron. Lett. 27 (12) (1991) 1772}1773.

11 H.-C. Huang et al. / Signal Processing 81 (2001) 1513} [9] J.S. Pan, F.R. McInnes, M.A. Jack, VQ codebook design using genetic algorithms, Electron. Lett. 31 (17) (1995) 1418}1419. [10] J.S. Pan, F.R. McInnes, M.A. Jack, Application of parallel genetic algorithm and property of multiple global optima to VQ codevector index assignment, Electron. Lett. 32 (4) (1996) 296}297. [11] L.M. Po, Novel subspace distortion measurement for e$cient implementation of image vector quantizer, Electron. Lett. 26 (29) (1990) 480}482. [12] K. Rose, Deterministic annealing for clustering, compression, classi"cation, regression and related optimization problems, Proc. IEEE 86 (1998) 2210}2239. [13] J. Vaisey, A. Gersho, Simulated annealing and codebook design, Proceedings ICASSP'88, 1988, pp. 1176}1179. [14] K. Zeger, A. Gersho, Stochastic relaxation algorithm for improved vector quantizer design, Electron. Lett. 25 (14) (1989) 896}898.