Speech Recognition Using Vector Quantization through Modified K-meansLBG Algorithm

Transcription

1 Computer Engneerng and Intellgent Systems ISSN (Paper) ISSN (Onlne) Speech Recognton Usng Vector Quantzaton through Modfed K-meansLBG Algorthm Balwant A. Sonkamble 1* D. D. Doye 2 1. Pune Insttute of Computer Technology, Dhankawad, Pune, Inda 2. Proefessor, Electroncs and Telecommuncaton Department, SGGSIE&T, Nanded, Inda *sonbalwant@yahoo.com Abstract In the Vector Quantzaton, the man task s to generate a good codebook. The dstorton measure between the orgnal pattern and the reconstructed pattern should be mnmum. In ths paper, a proposed algorthm called Modfed K-meansLBG algorthm used to obtan a good codebook. The system has shown good performance on lmted vocabulary tasks. Keywords: K-means algorthm, LBG algorthm, Vector Quantzaton, Speech Recognton 1. Introducton The natural way of communcaton among human bengs s through speech. Many human bengs are exchangng the nformaton through moble phones as well as other communcaton tools n a real manner [L. R. Rabner et al., 1993]. The Vector Quantzaton (VQ) s the fundamental and most successful technque used n speech codng, mage codng, speech recognton, and speech synthess and speaker recognton [S. Furu, 1986]. These technques are appled frstly n the analyss of speech where the mappng of large vector space nto a fnte number of regons n that space. The VQ technques are commonly appled to develop dscrete or sem-contnuous HMM based speech recognton system. In VQ, an ordered set of sgnal samples or parameters can be effcently coded by matchng the nput vector to a smlar pattern or codevector (codeword) n a predefned codebook [[Tzu-Chuen Lu et al., 2010]. The VQ technques are also known as data clusterng methods n varous dscplnes. It s an unsupervsed learnng procedure wdely used n many applcatons. The data clusterng methods are classfed as hard and soft clusterng methods. These are centrod-based parametrc clusterng technques based on a large class of dstorton functons known as Bregman dvergences [Arndam Baneree et al., 2005]. In the hard clusterng, each data pont belongs to exactly one of the parttons n obtanng the dsont parttonng of the data whereas each data pont has a certan probablty of belongng to each of the parttons n soft clusterng. The parametrc clusterng algorthms are very popular due to ts smplcty and scalablty. The hard clusterng algorthms are based on the teratve relocaton schemes. The classcal K-means algorthm s based on Eucldean dstance and the Lnde-Buzo-Gray (LBG) algorthm s based on the Itakura-Sato dstance. The performance of vector quantzaton technques depends on the exstence of a good codebook of representatve vectors. In ths paper, an effcent VQ codebook desgn algorthm s proposed known as Modfed K-meansLBG algorthm. Ths algorthm provdes superor performance as compared to classcal K-means algorthm and the LBG algorthm. Secton-2 descrbes the theoretcal detals of VQ. Secton-3 elaborates LBG algorthm. Secton-4 explans classcal K-means algorthm. Secton -5 emphaszes proposed modfed K-meansLBG algorthm. The expermental work and results are dscussed n Secton-6 and the concludng remarks made at the end of the paper. 2. Vector Quantzaton The man obectve of data compresson s to reduce the bt rate for transmsson or data storage whle mantanng the necessary fdelty of the data. The feature vector may represent a number of dfferent possble speech codng 137

2 Computer Engneerng and Intellgent Systems ISSN (Paper) ISSN (Onlne) parameters ncludng lnear predctve codng (LPC) coeffcents, cepstrum coeffcents. The VQ can be consdered as a generalzaton of scalar quantzaton to the quantzaton of a vector. The VQ encoder encodes a gven set of k-dmensonal data vectors wth a much smaller subset. The subset C s called a codebook and ts elements C are called codewords, codevectors, reproducng vectors, prototypes or desgn samples. Only the ndex s transmtted to the decoder. The decoder has the same codebook as the encoder, and decodng s operated by table look-up procedure. The commonly used vector quantzers are based on nearest neghbor called Vorono or nearest neghbour vector quantzer. Both the classcal K-means algorthm and the LBG algorthm belong to the class of nearest neghbor quantzers. A key component of pattern matchng s the measurement of dssmlarty between two feature vectors. The measurement of dssmlarty satsfes three metrc propertes such as Postve defnteness property, Symmetry property and Trangular nequalty property. Each metrc has three man characterstcs such as computatonal complexty, analytcal tractablty and feature evaluaton relablty. The metrcs used n speech processng are derved from the Mnkowsk metrc [J. S. Pan et al. 1996]. The Mnkowsk metrc can be expressed as D p p (, Y ) = k = 1 x y p, 1 2 k 1 2 k Where = { x, x,..., x } and Y = { y, y,..., y } are vectors and p s the order of the metrc. The Cty block metrc, Eucldean metrc and Manhattan metrc are the specal cases of Mnkowsk metrc. These metrcs are very essental n the dstorton measure computaton functons. The dstorton measure s one whch satsfes only the postve defnteness property of the measurement of dssmlarty. There were many knds of dstorton measures ncludng Eucldean dstance, the Itakura dstorton measure and the lkelhood dstorton measure, and so on. The Eucldean metrc [Tzu-Chuen Lu et al., 2010] s commonly used because t fts the physcal meanng of dstance or dstorton. In some applcatons dvson calculatons are not requred. To avod calculatng the dvsons, the squared Eucldean metrc s employed nstead of the Eucldean metrc n pattern matchng. The quadratc metrc [Marcel R. Ackermann et al., 2010] s an mportant generalzaton of the Eucldean metrc. The weghted cepstral dstorton measure s a knd of quadratec metrc. The weghted cepstral dstorton key feature s that t equalzes the mportance n each dmenson of cepstrum coeffcents. In the speech recognton, the weghted cepstral dstorton can be used to equalze the performance of the recognzer across dfferent talkers. The Itakura-Sato dstorton [Arndam Baneree et al., 2005] measure computes a dstorton between two nput vectors by usng ther spectral denstes. The performance of the vector quantzer can be evaluated by a dstorton measure D whch s a non-negatve cost D (, ˆ ) assocated wth quantzng any nput vector wth a reproducton vectorˆ. Usually, the Eucldean dstorton measure s used. The performance of a quantzer s always qualfed by an average dstorton D [ (, ˆ v = E D )] between the nput vectors and the fnal reproducton vectors, where E represents the expectaton operator. Normally, the performance of the quantzer wll be good f the average dstorton s small. Another mportant factor n VQ s the codeword search problem. As the vector dmenson ncreases accordngly the search complexty ncreases exponentally, ths s a maor lmtaton of VQ codeword search. It lmts the fdelty of codng for real tme transmsson. A full search algorthm s appled n VQ encodng and recognton. It s a tme consumng process when the codebook sze s large. In the codeword search problem, assgnng one codeword to the test vector means the smallest dstorton between the C codeword and the test vector among all codewords. Gven one codeword t and the test vector n the k-dmensonal space, the dstorton of the squared Eucldean metrc can be expressed as follows: 138

3 Computer Engneerng and Intellgent Systems ISSN (Paper) ISSN (Onlne) k 1 2 k 2 D (, C ) = ( x c ), where t { ct, ct,..., ct } t = 1 t 1 2 k C = and = { x, x,..., x }. There are three ways of generatng and desgnng a good codebook namely the random method, the par-wse nearest neghbor clusterng and the splttng method. A wde varety of dstorton functons, such as squared Eucldean dstance, Mahalanobs dstance, Itakura-Sato dstance and relatve entropy have been used for clusterng. There are three maor procedures n VQ, namely codebook generaton, encodng procedure and decodng procedure. The LBG algorthm s an effcent VQ clusterng algorthm. Ths algorthm s based ether on a known probablstc model or on a long tranng sequence of data. 3. Lnde Buzo Gray (LBG) algorthm The LBG algorthm s also known as the Generalsed Lloyd algorthm (GLA). It s an easy and rapd algorthm used as an teratve nonvaratonal technque for desgnng the scalar quantzer. It s a vector quantzaton algorthm to derve a good codebook by fndng the centrods of parttoned sets and the mnmum dstorton parttons. In LBG, the ntal centrods are generated from all of the tranng data by applyng the splttng procedure. All the tranng vectors are ncorporated to the tranng procedure at each teraton. The GLA algorthm s appled to generate the centrods and the centrods cannot change wth tme. The GLA algorthm starts from one cluster and then separates ths cluster to two clusters, four clusters, and so on untl N clusters are generated, where N s the desred number of clusters or codebook sze. Therefore, the GLA algorthm s a dvsve clusterng approach. The classfcaton at each stage uses the full-search algorthm to fnd the nearest centrod to each vector. The LBG s a local optmzaton procedure and solved through varous approaches such as drected search bnary-splttng, mean-dstance-ordered partal codebook search [Lnde et al., 1980, Modha et al., 2003], enhance LBG, GA-based algorthm [Tzu-Chuen Lu et al., 2010, Chn-Chen Chang et al. 2006], evoluton-based tabu search approach [Shh-Mng Pan et al., 2007], and codebook generaton algorthm [Buzo et al., 1980]. In speech processng, vector quantzaton s used for nstance of bt stream reducton n codng or n the tasks based on HMM. Intalzaton s an mportant step n the codebook estmaton. Two approaches used for ntalzaton are Random ntalzaton, where L vectors are randomly chosen from the tranng vector set and Intalzaton from a smaller codng book by splttng the chosen vectors. The detaled LBG algorthm usng unknown dstrbuton s descrbed as gven below: Step 1: Desgn a 1-vector codebook. Setm = 1. Calculate centrod 1 T C1 = = 1 T. Where Ts the total number of data vectors. Step 2: Double the sze of the codebook by splttng. Dvde each centrod C nto two close vectors C = C (1+ ) 2 1 δ andc2 = C (1 δ ), 1 m. Here δ s a small fxed perturbaton scalar. Letm = 2m. Setn = 0, here n s the teratve tme. Step 3: Nearest-Neghbor Search. Fnd the nearest neghbor to each data vector. Put neghbor to. n the parttoned set P f C s the nearest 139

4 Computer Engneerng and Intellgent Systems ISSN (Paper) ISSN (Onlne) Step 4: Fnd Average Dstorton. After obtanng the parttoned sets P = ( P, 1 m ), Setn =n+ 1. Calculate the overall average dstorton 1, C ), m T ( ) Dn = = ( D 1 = 1 T Where ( ) ( ) ( ) P= {,,..., }. Step 5: Centrod Update. 1 2 T Fnd centrods of all dsont parttoned setsp by 1 T = ( ) C =. 1 T Step 6: Iteraton 1. If ( D 1 D ) / D > ε, go to step 3; n n n otherwse go to step 7 and ε s a threshold. Step 7: Iteraton 2. Ifm = N, then take the codebook Here N s the codebook sze. C as the fnal codebook; otherwse, go to step 2. The LBG algorthm has lmtatons lke the quantzed space s not optmzed at each teraton and the algorthm s very senstve to ntal condtons. 4. Classcal K-means Algorthm The K-means algorthm s proposed by MacQueen n It s a well known teratve procedure for solvng the clusterng problems. It s also known as the C-means algorthm or basc ISODATA clusterng algorthm. It s an unsupervsed learnng procedure whch classfes the obects automatcally based on the crtera that mnmum dstance to the centrod. In the K-means algorthm, the ntal centrods are selected randomly from the tranng vectors and the tranng vectors are added to the tranng procedure one at a tme. The tranng procedure termnates when the last vector s ncorporated. The K-means algorthm s used to group data and the groups can change wth tme. The algorthm can be appled to VQ codebook desgn. The K-means algorthm can be descrbed as follows: Step 1: Randomly select N tranng data vectors as the ntal codevectors tranng data vectors. C, = 1,2,..., N from T Step 2: For each tranng data vector, = 1,2,..., T, assgn to the parttoned sets f = arg mn D(, C ). l l Step 3: Compute the centrod of the parttoned set that s codevector usng C = 1 S S Where S denotes the number of tranng data vectors n the parttoned sets. If there s no change n the 140

5 Computer Engneerng and Intellgent Systems ISSN (Paper) ISSN (Onlne) clusterng centrods, then termnate the program; otherwse, go to step 2. There are varous lmtatons of K-means algorthm. Frstly, t requres large data to determne the cluster. Secondly, the number of cluster, K, must be determned beforehand. Thrdly, f the number of data s a small t dffcult to fnd real cluster and lastly, as per assumpton each attrbute has the same weght and t qute dffcult to knows whch attrbute contrbutes more to the groupng process. It s an algorthm to classfy or to group obects based on attrbutes/features nto K number of group. K s postve nteger number. The groupng s done by mnmzng the sum of squares of dstances between data and the correspondng cluster centrod. The man am of K-mean clusterng s to classfy the data. In practce, the number of teratons s generally much less than the number of ponts. 5. Proposed Modfed K-meansLBG Algorthm The proposed algorthms obectve s to overcome the lmtatons of LBG algorthm and K-means algorthm. The proposed modfed KmeansLBG algorthm s the combnaton of advantages of LBG algorthm and K-means algorthms. The KmeansLBG algorthm s descrbed as gven below: Step 1: Randomly select N tranng data vectors as the ntal codevectors. Step 2: Calculate the no. of centrods. Step 3: Double the sze of the codebook by splttng. Step 4: Nearest-Neghbor Search. Step 5: Fnd Average Dstorton. Step 6: Update the centrod tll there s no change n the clusterng centrods, termnate the program otherwse go to step Expermentaton and Results The TI46 database [NIST, 1991] s used for expermentaton. There are 16 speakers from them 8 male speakers and 8 female speakers. The numbers of replcatons are 26 for utterance by each person. The total database sze s 4160 utterances of whch 1600 samples were used for tranng and remanng samples are used for testng of 10 words that are numbers n Englsh 1 to 9 and 0 are sampled at a rate of 8000 Hz. A feature vector of 12-dmensonal Lnear Predctng Codng Cepstrum coeffcents was obtaned and provded as an nput to vector quantzaton to fnd codewords for each class. There are fve fgures shows comparatve graphs of the dstorton measure obtaned usng LBG algorthm and K-means algorthm and proposed K-meansLBG algorthm. The dstorton measure obtaned by the proposed algorthm s smallest as compared to the K-means algorthm and the LBG algorthm. The proposed modfed KmeanLBG algorthm gves mnmum dstorton measure as compared to K-means algorthm and LBG algorthm to ncrease the performance of the system. The smallest measure gves superor performance as compared to both the algorthms as s ncreased by about 1% to 4 % for every dgt. 7. Concluson The Vector Quantzaton technques are effcently appled n the development of speech recognton systems. In ths paper, the proposed a novel vector quantzaton algorthm called K-meansLBG algorthm. It s used effcently to 141

6 Computer Engneerng and Intellgent Systems ISSN (Paper) ISSN (Onlne) ncrease the performance of the speech recognton system. The recognton accuracy obtaned usng K-meansLBG algorthm s better as compared to K-means and LBG algorthm. The average recognton accuracy of K-meansLBG algorthm s more than 2.55% usng K-means algorthm whle the average recognton accuracy of K-meansLBG algorthm s more than 1.41% usng LBG algorthm. References L.R. Rabner, B. H. Juang [1993], Fundamentals of Speech Recognton, Prentce-Hall, Englewood Clffs, N.J. Tzu-Chuen Lu, Chng-Yun Chang [2010], A Survey of VQ Codebook Generaton, Journal of Informaton Hdng and Multmeda Sgnal Processng, Ubqutous Internatonal, Vol. 1, No. 3, pp S. Furu [1986], Speaker-ndependent solated word recognton usng dynamc features of speech spectrum, IEEE Transactons on Acoustc, Speech, Sgnal Processng, Vol. 34, No. 1, pp S. Furu [1994], An overvew of speaker recognton technology, ESCA Workshop on Automatc Speaker Recognton, Identfcaton and Verfcaton, pp F. K. Soong, A. E. Rosenberg, B. H. Juang [1987], A vector quantzaton approach to speaker recognton, AT&T Techncal Journal, Vol. 66, No. 2, pp Wploa J. G., Rabner L. R. [1985], A Modfed K-Means Clusterng Algorthm for Use n Isolated Word Recognton, IEEE Transactons on Acoustcs, Speech and Sgnal Processng, Vol. 33 No.3, pp Arndam Baneree, Sruana Merugu, Indert S. Dhllon, Joydeep Ghosh [2005], Clusterng wth Bregman Dvergences, Journal of Machne Learnng Research, Vol. 6, pp J. S. Pan, F. R. McInnes and M. A. Jack [1996], Bound for Mnkowsk metrc or quadratc metrc appled to codeword search, IEE-Proceedngs on Vson Image and Sgnal Processng, Vol. 143, No. 1, pp Marcel R. Ackermann, Johannes Blomer, Chrstan Sohler [2010], Clusterng for Metrc and Nonmetrc Dstance Measures, ACM Transactons on Algorthms, Vol. 6, No. 4, Artcle 59, pp Shh-Mng Pan, Kuo-Sheng Cheng [2007], "An evoluton-based tabu search approach to codebook desgn", Pattern Recognton, Vol. 40, No. 2, pp Chn-Chen Chang, Yu-Chang L, Jun-Bn Yeh [2006], "Fast codebook search algorthms based on tree-structured vector quantzaton", Pattern Recognton Letters, Vol. 27, No. 10, pp A. Buzo, A. H. Gray, R. M. Gray, J. D. Markel [1980], Speech codng based upon vector quantzaton, IEEE Transactons on Acoustcs, Speech and Sgnal Processng, Vol. 28, No. 5, pp Y. Lnde, A. Buzo, and R. M. Gray [1980], An algorthm for vector quantzer desgn, IEEE Transactons on Communcatons, Vol. 28, No.1, pp D. Modha, S. Spangler [2003], Feature weghtng n k-means clusterng. Machne Learnng, Vol. 52, No.3, pp NIST, [1991], TI46 CD. Balwant A. Sonkamble receved hs BE (Computer scence and Engneerng n 1994 and M. E. (Computer Engneerng) n Currently he s research scholar at SGGS College of Engneerng and Technology, Vshnupur, Nanded (MS) INDIA. He s workng as Assocate Professor n Computer Engneerng at Pune Insttute of Computer Technology, Pune, Inda. Hs research areas are Speech Recognton and Artfcal Intellgence. 142

7 Computer Engneerng and Intellgent Systems ISSN (Paper) ISSN (Onlne) D. D. Doye receved hs BE (Electroncs) degree n 1988, ME (Electroncs) degree n 1993 and Ph. D. n 2003 from SGGS College of Engneerng and Technology, Vshnupur, Nanded (MS) INDIA. Presently, he s workng as Professor n department of Electroncs and Telecommuncaton Engneerng, SGGS Insttute of Engneerng and Technology, Vshnupur, Nanded. Hs research felds are speech processng, fuzzy neural networks and mage processng. Fgure 1. Comparatve graph for centrod K=4 Fgure 2. Comparatve graph for centrod K=8 143

8 Computer Engneerng and Intellgent Systems ISSN (Paper) ISSN (Onlne) Fgure 3. Comparatve graph for centrod K=16 Fgure 4. Comparatve graph for centrod K=32 Fgure 5. Comparatve graph for centrod K=64 144

9 Ths academc artcle was publshed by The Internatonal Insttute for Scence, Technology and Educaton (IISTE). The IISTE s a poneer n the Open Access Publshng servce based n the U.S. and Europe. The am of the nsttute s Acceleratng Global Knowledge Sharng. More nformaton about the publsher can be found n the IISTE s homepage: The IISTE s currently hostng more than 30 peer-revewed academc ournals and collaboratng wth academc nsttutons around the world. Prospectve authors of IISTE ournals can fnd the submsson nstructon on the followng page: The IISTE edtoral team promses to the revew and publsh all the qualfed submssons n a fast manner. All the ournals artcles are avalable onlne to the readers all over the world wthout fnancal, legal, or techncal barrers other than those nseparable from ganng access to the nternet tself. Prnted verson of the ournals s also avalable upon request of readers and authors. IISTE Knowledge Sharng Partners EBSCO, Index Coperncus, Ulrch's Perodcals Drectory, JournalTOCS, PKP Open Archves Harvester, Belefeld Academc Search Engne, Elektronsche Zetschrftenbblothek EZB, Open J-Gate, OCLC WorldCat, Unverse Dgtal Lbrary, NewJour, Google Scholar