Vector Quantization Codebook Design and Application Based on the Clonal Selection Algorithm

Transcription

1 Sensors & Transducers 03 by IFSA Vector Quantzaton Codeboo Desgn and Applcaton Based on the Clonal Selecton Algorthm Menglng Zhao, Hongwe Lu School of Scence, Xdan Unversty, X'an Unversty of Scence and Technology, 70054, Chna A School of Scence, Xdan Unversty, 70054, Chna Tel.: , fax: E-mal: zhaomenglng@6.com Receved: November 03 /Accepted: 9 November 03 /Publshed: 30 November 03 Abstract: In the area of dgtal mage compresson, the vector quantzaton algorthm s a smple, effectve and attractve method. After the ntroducton of the basc prncple of the vector quantzaton and the classcal algorthm for vector quantzaton codeboo desgn, the paper, based on manfold dstance, presents a clonal selecton code boo desgn method, usng dsntegratng method to produce ntal code boo and then to obtan the fnal code boo through optmzaton wth the clonal selecton cluster method based on the manfold dstance. Through experment, based on manfold dstance, compared the clonal selecton codeboo desgn algorthm (MDCSA) wth the heredtary codeboo desgn algorthm and LBG algorthm. Accordng to the result of the experment, MDCSA s more sutable for the evoluton algorthm of the mage compresson. Copyrght 03 IFSA. Keywords: Vector quantzaton, Codeboo desgn, Clonal selecton, Manfold dstance, Image compresson.. Introducton Along wth the hgh-speed development of computer and dgtal communcaton technology, people need to transfer more and more mage nformaton whch contans a large amount of redundant nformaton. The purpose of mage compresson codng les n elmnatng all nds of redundancy wth the premse of ensurng the qualty of the mage and n representng and reconstructng mage by usng as few bts as possble n a gven dstorton condtons. Research on mage compresson codng s consdered to be one of the most actve felds n the nformaton technology. In 948, Olve proposed TV sgnal dgtzaton n the feld of mage compresson codng technology, and then a lot of new compresson methods and nternatonal standards were put forward successvely. In the early 960s and ts metaphase, there appeared the earlest theory on vector quantzaton, whch gradually became perfect n the 980s. Stenhaus, n 956, frst systematcally developed the problem about the best vector quantzaton. In 978, Buzo frst put forward the actual vector quantzer. Untl 980 Lnde, Buzo and Gray ntroduced clusterng algorthm nto the vector quantzer desgn and proposed a famous algorthm n vector quantzaton codeboo desgn - LBG algorthm [-], whch pushed the research and applcaton of the vector quantzaton technology to a clmax and whch became a mlestone n the development of ths feld. Although some achevements have been made n the research on the basc vector quantzaton technology, there stll exst many problems. Therefore, a long way s ahead n the research on vector quantzaton technology. Artcle number P_RP_005 45

2 . Relevant Bacground.. Theoretcal Bass of Vector Quantzaton Vector quantzaton s based on the theory of Shannon rate dstorton [3]. In 959, Shannon defned the rate-dstorton functon R (D) as the mnmum code rate codng system can acheve n a condton not beyond a gven dstorton D, and defned dstorton rate functon D (R) as the mnmum dstorton codng system can acheve n a condton not beyond a gven code rate R. Based on vector quantzaton, codng performance can be close to the rate-dstorton functon. Rate dstorton theory suggests that vector quantzaton s always superor to the scalar quantzaton because vector quantzaton can effectvely apply the mutual correlaton propertes of components of the vector to elmnate the redundancy of data. However, rate dstorton theory s a theory of exstence rather than a structural one for t does not specfy how to structure vector quantzer... The Defnton of Vector Quantzaton Vector quantzaton s to replace the value of a group of nput samples (nput vector) wth the bestmatch set of output value (code word) of the output group set (code boo) n the quanttatve process based on the relatonshp between the adjacent samplng. The advantage of vector quantzaton les n ts smple decodng and the hgh rato of compresson. The basc vector quantzer can be defned as a K mappng Q from European space R to a lmted K subset C : Q: R C. C= ( xˆ ; =,,, N) s called code boo, xˆ s called code word and N s code word sze. The mappng satsfes Q( x) = xˆ p, and meets: (, ˆ ) mn (, ˆ ) d x x = d x x () p N where d( x, xˆ ) s the measure for the smlarty between x and xˆ. The basc codng and decodng process of vector quantzaton s shown n Fg.. Accordng to certan measure of smlarty, vector quantzaton encoder wll search out the mnmum code word between dstortons of nput vector n the code boo and only transmt the ndex of ths code word n transmsson. Vector quantzaton decodng process s very smple. Frst, fnd out the code word n the code boo accordng to the receved code word ndex and then wll t as the reconstructon vector of nput vector. Fg.. Dagram of VQ encodng and decodng.3. Performance Indcator The dstorton between the nput sgnal and the reconstructed sgnal s often descrbed by usng mean square error (MSE), sgnal-to-nose rato (SNR) and pea sgnal-to-nose rato (PSNR) [4] to descrbe. MSE,SNR and PSNR are defned as follows: MSE M N j = = 0 = 0 ( x y ) j M N N ( xj yj ) = 0 j= 0 j M N () xj = 0 j= 0 SNR = 0 log 0 M (3) ( L ) PSNR = 0 log (4) 0 MSE where M N s the sze of gray mage, L s the grayscale, xj For the pxels of orgnal mage, yj s the pxels of reconstructon mage. 0 M, 0 j N. The PSNR s used n ths artcle..4. The Key Technology of Vector Quantzaton The ey technology of vector quantzaton ncludes three aspects: code boo desgn algorthm, code word search algorthm and dstrbuton algorthm of code word ndex. 46

3 .4.. Code Boo Desgn The frst problem of vector quantzaton s to desgn a code boo wth good performance because the performance of code boo drectly determnes the qualty of the reconstructed vector and the performance of vector quantzaton system. Code boo desgn process s actually seeng an optmal scheme to dvde M tranng vectors nto N classes n the condton of mnmum average dstorton. All nds of centrod vectors are the code words. The purpose of the code boo desgn s to see an effectve method to mae code boo close to or reach optmum and reduce the computatonal complexty..4.. Code Word Search For a gven nput vector, code word search algorthm of vector quantzaton can be used to search the code word wth mnmum dstorton among the nput vectors n the code boo already C= xˆ ; =,,, N, desgned. In the gven code boo ( ) xˆ s the code word and N s the sze of code boo. The code word wth mnmum dstorton meets d( x, xˆ ) = mn d( x, xˆ p ).There s a hgh complexty N of calculaton n searchng the nput vector to the end, so the code word search's man purpose s to see the rapd and effectve algorthm to reduce the computatonal complexty Code Word Index Dstrbuton In vector quantzaton codng system, channel nose often causes nconsstency between the code word from the sendng end and that from the recevng end. The dstrbuton algorthm of code word ndex can redstrbute the codeword ndex and reduce the dstorton. The local optmal search algorthm s often used now and to see the optmal or near global optmal algorthm has become a new branch n the feld of vector quantzaton technque. 3. The Algorthm of Clonal Selecton Code Boo Desgn Based on the Manfold Dstance Ths algorthm s based on the clonal selecton code boo desgn and each antbody s a code boo n a gven antbody populaton. LBG algorthm s more senstve to the ntal code boo, so clonal selecton algorthm [5-6] s ntroduced n ths paper. by whch splttng method wll be appled to generate the ntal code boo. That s, all of ntal antbody forms the ntal antbody populaton. The algorthm of clonal selecton code boo desgn based on the manfold dstance can effectvely search out the global optmal soluton the optmal code boo. 3.. The Formaton of Intal Antbody Populaton In ths algorthm, real number codng mode based on the clusterng center s adopted for the ntal antbody group and each antbody represents a code boo. If a tranng vector group X and the sze of the desgn code boo M are set, splttng method wll be used to ntalze code boo. Frst, the whole tranng sample wll be taen as a cell, whch wll be dvded nto two and two nto four va the optmal partton hyperplane untl t becomes M cells. They stand for the sze of the code boo M, whch s regarded as the clusterng number to generate an ntal code boo consstng of M code words. Accordng to the sze of the antbody populaton, the same method wll be used to generate Sze of the antbody populaton, every antbody s the ntal code boo as a need to optmze a fnal code boo. The specfc steps of the splttng method: ) Calculatng the center of mass of the tranng samples X and t s set as C the frst code word; 0 ) Choosng dsturbance vector δ, tang { C δ, C + δ} as the ntal code boo and applyng 0 0 LBG algorthm to desgn a code boo contanng only two code words; 3) Tang { C δ, C + δ, C δ, C + δ } as the ntal code boo and applyng LBG algorthm to desgn a code boo contanng only four code words. Through repeatng the above steps and desgnng M log tmes, the ntal code boo consstng of M code words requred can be obtaned. The calculaton method for dsturbance vector s: δ = aσ = ( aσ, aσ,, aσ ). a > 0 s the scale factor σ s standard varance of the th component among the tranng vectors; =,,, ; s the dmenson of tranng vectors. In ths algorthm, for a dversty of the antbody populaton, the frst code word wll be generated at random when the splttng method s appled to produce the antbody and the other steps are the same as those of the standard splttng method. 3.. The Algorthm of Clonal Selecton Code Boo Desgn Based on the Manfold Dstance 3... Clonal Selecton Theory [7-9] Due to the gene mutaton of genetc and mmune cell prolferaton, a dversty of mmune cells s formed, the prolferaton of whch wll become a clonng system. In 958, Burnet and other researchers proposed the theory of mmune clonal selecton, whch argues that lymphocytes, n addton to beng amplfed or dfferentated nto plasma cells, can also be dfferentated nto B memory cells wth longer 47

4 lfe. Meetng the correspondng antgen, memory cells wll be selected n advance by the mmune system, beng rapdly actvated, prolferated and dfferentated nto cells of antbody generaton and mplementng effcent and durable mmune functon. Clonal selecton s a dynamc process durng whch organsm mmune system tres to adapt tself to antgenc stmulaton. Inspred by the clonal selecton theory, De Castro and other researchers put forward a nd of clonal selecton algorthm (CSA), whch s a new approach for artfcal mmune system based on the bologcal characterstcs of antbody clonal selecton. Wth the ad of antbody clonal selecton mechansm of the mmune system, Clonal selecton algorthm structures clone operator sutable for artfcal ntellgence. In terms of clone operator, clonal selecton algorthm s a nd of group search strategy characterzed by parallelsm and randomness of the search change, whch helps t avod fallng nto local optmal value and whch can get the global optmal soluton wth the larger probablty. Du Hafeng and others put forward the algorthm of mmune clone strategy manly suggestng the clone operator consstng of three steps: clonng, clone varaton and clonal selecton. In the antbody group A() t = { A( t), A ( t),, A ( t)}, the state transton can N be expressed n a random process as followng: C A A A A clone ' mutaton " selecton : ( ) ( ) ( ) ( + ) 3... Producton of Antbody Codng and Intal Antbody Group (5) Real number codng mode on the bass of clusterng center s adopted n the Clonal selecton algorthm. In n data samples, dmenson d, every antbody A s composed of clusterng centers, whch can be as a real number codng wth the length l = d. An antbody can be shown below: ( ) A t = { a a a a a a a a } d d d c c c (6) In ths equaton, the correspondng codng of c, c,, c s the coordnate for the clusterng center n the sample space. The results of K-MEANS terated 5 tmes wll be appled to ntalze antbody group. Of course, usng random number to ntalze antbody group s also feasble but the accuracy and convergence speed are both nferor to the results of K-MEANS terated many tmes. More tmes of K - MEANS teraton cannot mprove the clusterng results, but taes more tme Clone Operator Clonng Operaton In order to control the optmal antbody prolferaton, frst antbody affnty and dmenson of antbody populaton wll be arranged accordng to the descendng order. The correspondng antbody populaton also arranged by the descendng order of antbody affnty degree sequence, and then the equaton (3) s used to calculate the number of each optmal antbody to be cloned. Because the ndvdual clonng number s relevant to antbody affnty and the values of t, the lower the antbody affnty and the value are, the smaller the number of clonng wll be. n where round ( ) β n n, b (7) b = round, s the nteger functon, β s the clonng coeffcent and s a constant, n s the total b number of ndvduals selected to be cloned Clone Varaton Operaton Accordng to mmunology, the mprovement of antbody affnty and the appearance of antbody dversty manly rely on hgh frequency varaton of antbody but not on cross or reorganzaton. Hence, dfferent from the general genetc algorthm suggestng that cross s a major operator and mutaton s bacground operator, clonal selecton algorthm puts more emphass on the role of varaton. Varaton mmcs gene mutaton n natural bologcal evoluton and t chooses several gene mutatons n one or several genes of the antbody wth a certan probablty, randomly changng the gene value so as to change the structure of the antbody. In ths paper the unform mutaton operator wll be adopted and the operaton process s: for each varaton pont, a random number wll be chosen from the value range of the correspondng gene to replace the orgnal gene value. That s, ' P mn ( U U ) = U + r (8) max r s the random number wthn the scope of (,) 0 ; U,U are the upper and lower lmts of the value max mn of ths gene. mn The Clonal Selecton Operaton. Dfferent from the optons n genetc algorthm, clonal selecton operaton chooses the preemnent ndvduals from the cloned progenes of each antbody to form a new populaton a neuter selecton process. 48

5 3..3. Ftness Functon Manfold dstance measure s adopted n the ftness functon of the algorthm and the specfc approach s descrbed as below [0-5]: Data clusterng s characterzed by local and global consstency. Local consstency refers to the hgh smlarty between the adjacent data ponts n the space poston; global consstency refers to the hgh smlarty between the data ponts n the same manfold. Smlarty metrc based on Eucldean dstance can only reflect the local consstency of clusterng results the hgh smlarty of the adjacent data n the space poston, but cannot reflect the clusterng global consstency the hgh smlarty located n the same manfold sample. A smple example can llustrate ths as s shown n Fg.. The smlarty between data pont a and data pont e s expected to be larger than the smlarty between data pont a and data pont f. So just usng Eucldean dstance as the smlarty measure wll serously affect the performance of the clusterng algorthm. Fg.. Illustraton of European measure unable beng to meet the globally consstency. In order to guarantee the performance of the clusterng algorthm, meetng the dstance of clusterng global consstency does not necessarly mean meetng the trangle nequalty of European measure. In other words, meetng the dstance of clusterng global consstency cannot defntely lead to the shortest drect path between two ponts. The path length n the same manfold connected wth the short sde must be made shorter than the dstance between two ponts drectly connected through the low densty area, as s shown n Fg..: ab + bc + cd + de < ae. To acheve ths objectve, frst a manfold lne length wll be defned. Defnton : Manfold Lne Length (, j) ( j) d( x, xj) L x, x = ρ (9) s the Eucldan dstance between x d x x and x ; ρ > s the expanson factor. j Defnton : Manfold Dstance Measure The data pont s taen as the culmnaton of l graph G= ( V, E) ; p V represents the path between the two connectons p and p p wth the length l= p n the fgure; the sde s ( p, p ) +, < p. P stands j for the set of all paths of connecton data ponts x and x j ; the manfold dstance between x and x j can be calculated va the followng equaton: p (, ) mn (, j + ) D x x = L p p (0) p P, j = Obvously, manfold dstance measure meets four condtons of measure, namely, symmetry: Dxx (, ) = Dx (, x) j j, non-negatvty: Dxx (, ) 0 j : trangle nequalty: for any x, x, x j, D( x, x ) D( x, x ) + D( x, x ) j j, self-reversty: (, ) 0 x = x. Manfold dstance Dxx =, f and only f j j measure can measure the shortest path along the manfold and ths maes the two ponts n same manfold can be connected by many short sdes. At the same tme, the two ponts located n dfferent manfolds should be connected by the relatvely longer sde, fnally achevng the purpose of lengthenng the dstance between data ponts n dfferent manfolds and shortenng the dstance between data ponts n the same manfold. Based on the manfold dstance, a ftness functon of calculaton smlarty can be desgned and the ftness functon of P (the th antbody): f P = = + Dev C +, ( ) D( c ) () C C C C s the center set of all clusterng; c s the class center of clusterngc ; Dc (, ) s the manfold dstance between data pont of clusterng C and the clusterngc The Process of the algorthm Of Clonal Selecton Code Boo Desgn Based on the Manfold Dstance Specfc process of the algorthm s lsted below: Step : ntalze antbody populaton. The scale of antbody group s set as N ; t = 0 ; splttng method s used to ntalze the code boo and each code boo represents an antbody group; a total of N ntal antbodes s produced to form an ntal antbody populaton C( t. ) 49

6 Step : Accordng to the clusterng center code of the antbody, each sample pont wll be grouped nto dfferent clusterng va manfold dstance measure based on the nearest neghbor prncple; antbody affnty and degree value s calculated. Step 3: Clonng operaton. After clonal varaton operaton and Clonal selecton operaton, a new generaton of antbodes Ct+ ( ) s generated. Step 4: Calculatng ftness of the updated antbody populaton. Step 5: t = t +. If the clusterng error rate s less than stop threshold e, or reaches to the upper lmt of teratons ( t max ), the progress wll termnate and output the optmal antbody-optmal code boo. Otherwse, the process wll turn to Step The Analyss of Expermental Results 4.. Expermental Setup generated by all nds of algorthms (based on the manfold dstance clonal selecton algorthm (MDCSA), based on the manfold dstance genetc algorthm (MDGA) and the tradtonal LBG algorthm) perform mage codng on the four pctures respectvely are compared. Accordng to Table 3, for the four mages tested, PSNR value of MDCSA algorthm s much hgher than that of MDGA and LBG algorthm when the code boo sze s 8; when the code boo sze s 56, PSNR value of MDCSA and MDGA s the same, but t s much hgher than that of LBG algorthms, when the code boo sze s 5, n addton to PSNR value of MDGA algorthm of frut mage s hgh, PSNR values of MDCSA algorthm of others s much hgher. Therefore, a concluson can be reached that the method of clonal selecton code boo desgn based on the manfold dstance s characterzed by better performance and effectveness. In the experment, the man parameters are shown n Table and Table below. Table. Parameters of the algorthm of genetc code boo desgn selecton matrx rules. Parameters Genetc Algorthm Maxmum teraton 00 Populaton sze 40 Termnaton condton ε = e 5 Crossover probablty p = 0. 8 Mutaton probablty p = 0. 5 c m (a) Boat (5 5) (b) Brd (48 3) Table. Parameters of the algorthm of clonal selecton code boo desgn. Parameters Clonal Selecton Algorthm Maxmum teraton 00 Populaton sze 40 Termnaton condton ε = e 5 Clonng coeffcent β = 3. 5 Numbers to be cloned n = 5 Mutaton probablty p = Analyss of the Results In order to valdate the performance and effectveness of the algorthm, n ths paper the four pctures of natural mage are compressed and reconstructed. These pctures and ther sze are shown n Fg. 3. In the experment, the three pctures of boat, brd, Barbara and buld wth eght bts per pxel produce code boos of all sorts of sze as the tranng mages, wth ther respectve vector dmenson 6 (4x4). In Fg. 3, pea sgnal-to-nose ratos (PSNR) produced when the code boos of dfferent szes b m (c) Barbara (5 5) (d) Buld (48 3) Images Boat Brd Barbara Buld Fg. 3. Images tested. Table 3. Image tested PSNR (db). Code boo sze MDCSA MDGA LBG

7 5. Concluson Ths paper s manly about the mage compresson problem of mage clusterng, n whch the basc prncples of vector quantzaton and classc LBG algorthm are ntroduced. Besdes, a method of clonal selecton code boo desgn based on manfold dstance s put forward. Durng the process, splttng method s adopted to produce ntal code boo and clonal selecton clusterng approach based on manfold dstance s appled to generate the fnal code boo. In the experment, algorthm of clonal selecton code boo desgn, algorthm of genetc code boo desgn and the tradtonal LBG algorthm are compared. The results show that for code boos of dfferent szes relatvely hgh PSNR value can be acheved va MDCSA, whch proves that ths algorthm has good performance and effectveness. Acnowledgements Ths wor s supported partly by the Natonal Natural Scence Foundaton of Chna under Gran No and No References []. B Ramamurth, A Gersho, Classfed Vector Quantzaton of Images, IEEE Trans. Communcaton, Vol. 34, No., 986, pp []. K. W. Chan, K. L. Chan, Subband VPIC wth Classfed Jont Vector Quantzaton, Sgnal Processng: Image Communcaton, Vol. 3, No. 3, 998, pp [3]. Dong-L, Chang Sougul Ann, Choong Woong Lee, A Classfed Splt Vector Quantzaton of LFS Parameters, Sgnal Processng, Vol. 59, No. 3, 997, pp [4]. Rafael C. Gonzalez, Rchard E. Woods, Steven L. Eddns, Dgtal Image Processng, Publshng House of Electroncs Industry, Chna, 008. [5]. Yngchun Zhang, Juan Cao, Bohong Su, Image Regstraton Based on Genetc Algorthm, JCIT: Journal of Convergence Informaton Technology, Vol. 8, No. 8, 03, pp [6]. Zhou Yang, Ma L, Ba Ln, Clusterng Method wth Evolutonary Immune Networ Based on Polyclonal Algorthm, Computer Engneerng and Applcatons, Vol. 45, No. 7, 009, pp [7]. Zhu Feng, Zhang Qun, Ba You-Qng, et al., A Novel Reconstructon Method Based on Genetc Algorthm n CS Theory and Its Applcaton n SAR HRRP reconstructon, Control and Decson, Vol. 7, No., 0, pp [8]. Lu Ruo-Chen, Immune Clonal Strategy Algorthms and Ther Applgcaton, Xdan Unversty, 007. [9]. Wang Juan, L Fe, A Quantum-nspred Immune Clonal Algorthm Based on Real-Encodng, Computer Engneerng, Vol. 38, No. 8, 0, pp [0]. Dong Hong-Bn, Yang Bao-D, Lu Ja-Yuan, et al., A Co-Evolutonary Algorthm for Encodng, Paghtedttern Recognton and Artfcal Intellgence, Vol. 5, No. 04, 0, pp []. Zhang Yng-Je, Fan Chao-dong, Improved Clonal Immune Algorthm for All Solutons of SAT Based on Mult-Populaton, Informaton and Control, Vol. 40, No. 0, 0, pp []. Yang Ru-Ru, Nu Jan-Qang, Meng Hong-Fe, Pavement Crac Extracton Usng Iteratve Clusterng Algorthm Based on Manfold Dstance, Computer Engneerng, Vol. 37, No., 0, pp. -4. [3]. Kelvn Lo Yr Sang, Sa Wang Khor, Path Clusterng usng a Mxture of Dynamc Tme Warpng Technque and Cluster Searchng Index Method, IJIIP: Internatonal Journal of Intellgent Informaton Processng, Vol. 3, No. 4, 0, pp [4]. Yang Jn-Fu, Song Mn, L Mng-A, A Novel Reducton K-Nearest Neghbor Classfcaton Method Based on Weghted Dstance, Journal of Electroncs & Informaton Technology, Vol. 33, No. 0, 0, pp [5]. Song Dan, La Xu-Zh, Wu Mn, Clonal Selecton Algorthm Based on Grade Varaton, Paghtedttern Recognton and Artfcal Intellgence, Vol. 4, No. 03, 0, pp Copyrght, Internatonal Frequency Sensor Assocaton (IFSA). All rghts reserved. ( 4