Approach Multiclass SVM Utilizing Genetic Algorithms

Transcription

1 Proceedngs of the Internatonal MultConference of Engneers and Coputer Scentsts 03 Vol I, IMECS 03, March 3-5, 03, Hong Kong Approach Multclass SVM Utlzng Genetc Algorths Boutkhl Sdaou, Kaddour Sadoun Abstract- In ths paper, we nvestgate and evaluate the perforance of a sple approach ult class for support vector achne (SVM) ethod. We present a new archtecture, naed GASVM, and based en genetc algorths, n order to acheve hgh classfcaton effcency for ultclass probles. The proposed paradg bulds a bnary tree, for solvng ultclass probles, by genetc algorths wth the a of obtanng a strategy ultclass SVM wth a reasonable and practcal coplexty n the real probles. Our approach s ore accurate n the creaton of the tree. Further, n the test phase our contrbuton, due to ts Log coplexty, t s uch faster than other ethods n probles that have bg class nuber. For the evaluaton two corpuses are used; TIMIT corpus, where we acheved a recognton rate of 57.54% on the 0 vowels and MNIST datasets who s a recognton rate of 97.73% s acheved. These results are coparable wth the state of the arts. In addton, tranng te and nuber of support vectors, whch deterne the duraton of the tests, are also reduced copared to other ethods. However, these results are unacceptably large for the real applcaton tasks. Keywords: Machne Learnng; SVM; Genetc algorths. K I. INTRODUCTION ERNEL Methods and partcularly Support Vector Machnes (SVM) [4,5,6], ntroduced durng the last decade n the context of statstcal learnng [5,7,8], have been successfully used for the soluton of a large class of achne learnng tasks [9,0] such as categorzaton, predcton, novelty detecton, rankng and clusterng. The SVM was orgnally developed for bnary probles, and ts extenson to ult-class probles s not straghtforward. How to effectvely extend t for solvng ultclass classfcaton proble s stll an on-gong research ssue. The popular ethods for applyng SVMs to ultclass classfcaton probles usually decopose the ult-class probles nto several two-class probles that can be addressed drectly usng several SVMs. The paper s organzed as follows. In secton, support vector achne wll be brefly dscussed for probles classfcaton. In the followng secton, we ntroduce our effcent fraework for ultclass SVM usng genetc algorths. We dscuss soe related successful works n sectons 4. In secton 5, we gve results of prelnary experents on the vowels sets of TIMIT data base and dgts sets of MNIST corpus. Fnally, the last secton s Manuscrpt receved October 3, 0; revsed Deceber 30, 0. Boutkhl SIDAOUI s wth Matheatcs and Coputer Scence Departent, Unversty of Tahar Moulay Sada, BP 38 ENNASR Sada 0000, Algera, phone: ; e-al: b.sdaou@gal.co. Kaddour SADOUNI s wth Coputer Scence Departent Unversty of Scences and Technology USTO-MB, BP 505 Elanouar Oran 3000, Algera (kaddour_sadoun@hotal.co). ISBN: ISSN: (Prnt); ISSN: (Onlne) devoted to conclusons and soe rearks pertanng to future work. II. SUPPORT VECTOR MACHINE Bnary SVM, n ther general for, extend an optal lnear hypothess, n ters of an upper bound on the expected rsk that can be nterpreted as the geoetrcal argn, to non lnear ones by akng use of kernels k(.,.). Kernels can be nterpreted as dsslarty easures of pars of objects n the tranng set X. In standard SVM forulatons, the optal hypothess sought s of the for (). Φ ( ξ ) = α k( x, x ) () Where α are the coponents of the unque soluton of a lnearly constraned quadratc prograng proble, whose sze s equal to the nuber of tranng patterns. The soluton vector obtaned s generally sparse and the non zero α s are called support vectors (SV s). Clearly, the nuber of SV s deternes the query te whch s the te t takes to predct novel observatons and subsequently, s crtcal for soe real te applcatons such as speech recognton tasks. It s worth notng that n contrast to connectonst ethods such as neural networks, the exaples need not have a Eucldean or fxed-length representaton when used n kernel ethods. The tranng process s plctly perfored n a Reproducng Kernel Hlbert Space (RKHS) n whch k(x;y) s the nner product of the ages of two exaple x, y. Moreover, the optal hypothess can be expressed n ters of the kernel that can be defned for non Eucldean data such bologcal sequences, speech utterances etc. Popular postve kernels nclude the Lnear, Polynoal and Gaussan kernels: A. SVM Forulaton n Gven tranng vectors x R, =,...,, n two classes, and a vector y R such that y {, }, Support Vector Classfers [5,6,7,8] solve the followng lnearly constraned convex quadratc prograng proble: ( x, x ) axze W(α) = α, j α j y y jk j under theconstrants:, 0 α C α y = 0 The optal hypothess s: = α = () IMECS 03

2 Proceedngs of the Internatonal MultConference of Engneers and Coputer Scentsts 03 Vol I, IMECS 03, March 3-5, 03, Hong Kong f ( x) = α yk( x, x ) + b (3) = Where the bas ter b can be coputed separately [9]. Clearly, the hypothess f depends only on the non null coeffcents α whose correspondng patterns are called Support vectors (SV). The QP objectve functon nvolves the proble Gra atrx K whose entres are the slartes k ( x, x j ) between the patterns x and x j. It s portant to note, on one hand, that the pattern nput denson d, n the above forulaton, s plct and does not affect to soe extent the coplexty of tranng, provded that the Gra atrx K can be effcently coputed for the learnng task at hand. On the other hand, the patterns representaton s not needed and only par wse slartes between objects ust be specfed. Ths feature akes SVM very attractve for hgh nput densonal recognton probles and for the ones where patterns can t be represented as fxed densonal real vectors such as text, strngs, DNA etc. For large scale corpora however, the quadratc prograng proble becoes quckly coputatonally expensve, n ters of storage and CPU te. It s well known that general-purpose QP solvers scale wth the cube of the proble denson whch s, n our case, the nuber of tranng patterns. Specalzed algorths, typcally based on gradent descent ethods, acheve pressve gans n effcency, but stll becoe practcally slow for probles whose sze exceeds 00,000 exaples. Several attepts have been ade to overcoe ths shortcong by usng heurstcally based decoposton technques such as Sequental nal optzaton SMO [9] pleented n LbSVM package [6]. B. Multclass Extensons Support Vector Machnes are nherently bnary classfers and ts effcent extenson to ultclass probles s stll an ongong research ssue [, 3, 4]. Several fraeworks have been ntroduced to extend SVM to ultclass contexts and a detaled account of the lterature s out of the scope of ths paper. Typcally ultclass classfers are bult by cobnng several bnary classfers. The earlest such ethod s the One-Aganst-All (OVA) [5, ] whch constructs K classfers, where K s the nuber of classes. th The k classfer s traned by labelng all the exaples n th the k class as postve and the reander as negatve. The fnal hypothess s gven by the forula: f ( f ( )) ( x) = arg ax,..., k x (4) ova = Another popular paradg, called One-Aganst-One (OVO), proceeds by tranng k(k-)/ bnary classfers correspondng to all the pars of classes. The hypothess conssts of choosng ether the class wth ost votes (votng) or traversng a drected acyclc graph where each node represents a bnary classfer (DAGSVM) [3]. There was ISBN: ISSN: (Prnt); ISSN: (Onlne) debate on the effcency of ultclass ethods fro statstcal pont of vew Clearly, votng and DAGSVM are cheaper to tran n ters of eory and coputer speed than OVASVM.[4] nvestgated the perforance of several SVM ult-class paradgs and found that the one-aganstone acheved slghtly better results on soe sall to edu sze benchark data sets. Other nterestng works wll be dscussed n secton: related work and dscusson. III. BINARY TREE FOR MULTICLASS SVM Ths approach uses ultple SVMs set n a bnary tree structure [40]. In each node of the tree, a bnary SVM s traned usng two classes. All saples n the node are assgned to the two subnodes derved fro the current node. Ths step repeats at every node untl each node contans only saples fro one class. That sad, untl the leaves of the tree. The an proble that should be consdered serously here s how to construct the optal tree? Wth the a of parttonng correctly the tranng saples n two groups, n each node of the tree. In ths paper we propose a genetc algorth for constructng a bnary tree structure for ultclass SVM. A. Bnary Tree Constructon Genetc algorths (GA) can provde good solutons to any optzaton probles. They are based on natural processes of evoluton. The process of genetc algorth s defned as follows: codng, selecton, genetc operators such as utaton and crossover. The GASVM ethod that we propose s based on recursvely parttonng the classes n two dsjont groups n every node of the bnary tree, and tranng a SVM that wll decde n whch of the groups the ncong unknown saple should be assgned. The groups are deterned by genetc algorth. In the general case, the nuber of parttons nto two parts (groups) of a set of k eleents s gven by the followng forula [46]: k ( ) N _ partonsk, = = ( ) (5) 0!( )! Correspondng constructon of the bnary tree, two cases can be expected: the nuber k s sall n ths case; we calculate all possble parttons and then deduce the optal partton. Where the nuber k s greater than 6 (k> 6), we deterne the optal partton by genetc algorths, because t s possble to cover all possble parttons. B. Prelnary Let s take a set of tranng saples labeled by y { c, c,..., c k }, GASVM ethod starts, n the frst step, wth calculatng k gravty centers for the k dfferent classes. In the second step, the goal s to fnd the rght parttonng of k gravty centers nto two dsjont groups. For that, a process of genetc algorth s appled for each node of the bnary tree as follow: we start fro the root of the bnary tree, we partton the k gravty centers (k IMECS 03

3 Proceedngs of the Internatonal MultConference of Engneers and Coputer Scentsts 03 Vol I, IMECS 03, March 3-5, 03, Hong Kong classes) nto two groups s the ost dsjonted, says the chroosoe havng the axu ftness functon wth genetc algorths. Then, we contnue ths operaton for nodes derved untl no parttonng s possble, that s to say nodes are the leaves of the tree. After that, a functon of overall nerta s calculated for each bnary tree (ndvdual). Indeed, ths functon of overall nerta s gven by the followng forula: f overallnerta = = nodes fun _ fteness( ) (6) Where, fun _ fteness( ) s the ftness functon of the node. {,, 6, 7} vs {3, 4, 5} {, 6} vs {, 7} {3, 4} vs {5} {}vs {6} {}vs {7} {3} vs {4} 5 6 Fg.. shows an exaple bnary tree for seven classes C. Genetc algorths for constructng optal partton Encodng: For genetc algorths to buld optal partton, partton ust be encoded so that genetc operators such as utaton and crossover can be appled. Let y { c, c,..., c k } be the label lst of k gravty centers for the k dfferent classes. For encodng each partton, perutaton encodng s used. In perutaton encodng, every chroosoe s a strng of nubers. In ths paper, each nuber represents a vowel; therefore partton s encoded by a strng of nubers. In ths paper we used the set: {aa, ae, ah, ao, aw, ax-h, ax, axr, ay, eh, er, ey, h, x, y, ow, oy, uh, uw, ux} for datasets TIMIT vowels. The vowels are represented by nubers to 0 respectvely and for MNIST datasets each dgt s represented by nuber. TABLE I: EXAMPLE OF CHROMOSOMES (PARTITIONS) Chroosoe Chroosoe Chroosoe, Chroosoe : two exaples of chroosoe representaton (dgt sequence). Ftness Functon: ftness functon of each partton s the Eucldean dstance between the two groups that partton. Genetc Operators: Crossover and utaton are two basc operators of GA. Perforance of GA very depend on the. Type and pleentaton of operators depends on encodng and also on a proble. The Crossover selects genes fro parent chroosoes and creates a new offsprng. Whereas, utaton operator s defned as changng the value of a certan poston n a strng to one of the possble values n the range. There are any ways how to do crossover and utaton. In ths paper, we are only nterested by crossover to a sngle pont, Sngle pont crossover s selected, tll ths pont the frst part s coped fro the frst parent, and then the rest s coped fro the second parent. For utaton, change n rank; two nubers are selected and exchanged. TABLE II: EXAMPLE OF CROSSOVER WITH A SINGLE POINT Partton Partton Offsprng Offsprng Partton, Partton : two fathers, after crossover wth a sngle pont (at 9 pont), we obtan the two representatons: Offsprng and. But the proble s that soe nubers are repeated and others are ssng, whle all nubers are ncluded n each chroosoe. For ths, correcton s appled to the offsprng to add the ssng nuber and reove duplcaton. TABLE III: EXAMPLE OF MUTATION OPERATOR Before utaton After utaton Before utaton: exaple of chroosoe representaton. After utaton: chroosoe after utaton operator of a sngle dgt (5 by 5). Decodng: Decodng s the reverse operator of the encodng process. In ths paper the best ndvdual s the tree wth the axu overall nerta functon. D. Ipleentaton of Tree SVM The optal tree (best ndvdual) created by genetc algorths s pleented to obtan a ultclass approach. Takes advantage of both the effcent coputaton of the tree archtecture and the hgh classfcaton accuracy of SVMs Utlzng ths archtecture, N- SVMs needed to be traned for an N class proble. Ths bnary tree s used to tran a SVM classfer n the root node of the tree, usng the saples of the frst group as postve exaples and the saples of the second group as negatve exaples. The classes fro the frst clusterng group are beng assgned to the frst (left) subtree, whle the classes of the second clusterng group are beng assgned to the (rght) second subtree. The process contnues recursvely (dvdng each of the groups nto two subgroups applyng the procedure explaned above), untl there s only one class per group whch defnes a leaf n the decson tree. The recognton of each test saple starts at the root of the tree. At each node of the bnary tree a decson s beng ade about the assgnent of the nput pattern nto one of the two possble groups represented by transferrng the pattern to the left or to the rght sub-tree. Each of these groups ay contan ultple classes. Ths s repeated recursvely downward the tree untl the saple reaches a leaf node that represents the class t has been assgned to. ISBN: ISSN: (Prnt); ISSN: (Onlne) IMECS 03

4 Proceedngs of the Internatonal MultConference of Engneers and Coputer Scentsts 03 Vol I, IMECS 03, March 3-5, 03, Hong Kong IV. RELATED WORK AND DISCUSSION There are dfferent approaches for solvng ult-class probles whch are not based on SVM. However, the experental results clearly show that ther classfcaton accuracy s sgnfcantly saller than the SVM based ethods. Several ult-class approaches have been proposed to solve the ult-class proble for SVM. SVM usng the standard forulaton cannot deal drectly ult-class probles. Early pleentatons attepted to treat the ultclass proble are the OVA and the OVO approaches already dscussed n Secton. Most of the recent works are based on the cobnaton of classfers bnares. Interestng pleentatons of SVM ult-classes have been developed n recent years. Such as, two archtectures proposed by [4] and [4], n the frst Cha and Tappert bult the decson tree by genetc algorths where they are consdered each tree as chroosoe. Whle n the second archtecture Madzarov and All proposed a clusterng ethod to construct the bnary tree. Another approach n ths drecton has been presented by Le and Venu n [43], they use a recursve ethod n to buld the bnary tree. In ter of coplexty, OVA approach needs to create k bnary classfers; the tranng te s estated eprcally by a power law [44] statng that T αm where M s the nuber of tranng saples and α s proportonalty constant. Accordng to ths law, the estated tranng te for OVA s: Tranng _ TeOVA α k M (7) Wthout loss of generalty, let's assue that each of the k classes has the sae nuber of tranng saples. Thus, each bnary SVM of OVO approach only requres M / k saples. Hence, the tranng te for OVO s: Tranng _ Te ( k ) α M k OVO k( k ) M α k αm The tranng te for DAG s sae as OVO. In the tranng phase, our approach GASVM has k- bnary classfers (k s the nuber of classes). The rando structure of the optal tree coplcates the calculaton of the tranng te. However, an approxaton s defned as: Let s assue that each of the k classes has the sae nuber of tranng saples. The tranng te s sued over all k- nodes n th the dfferent log ( k ) levels of tree. In the level, there are at the ost nodes and each node uses M tranng saples. Hence, the total tranng te: (8) Tranng _ Te log ( k ) = M α SVMAG αm log ( k ) = α M It ust be noted that the tranng te of our approach does not nclude the te to buld the herarchy structure (bnary tree) of the k classes. In the testng phase, OVA requre k bnary SVM evaluatons and OVO necesstate k( k ) (9) bnary SVM evaluatons, whle DAGSVM perfors faster than OVO and OVA, snce t requres only k- bnary SVM evaluatons. The two archtectures proposed by [4] and [4] and GASVM proposed n ths paper are even fasters than DAGSVM because the depth of the bnary tree s log ( k ). In addton, the total nuber of supports (SVs) vectors n all odels wll be saller than the total nuber of SVs n the others (OVA, OVO and DAGSVM). Therefore t allows converge rapdly n the test phase. The advantage of the approaches presented n [4], [4], [43] and the approach shown n ths paper le anly n the test phase, because t uses only the odels necessary for recognton. Whch ake the testng phase faster. However, n [4], [4] and [43] a proble of local na s clearly present. To avod ths proble, an approach proposed n ths works to fnd the bnary tree, usng the process of genetc algorth to each node of the tree to fnd the rght parttonng nto two dsjont groups, the partal optzaton avods fallng nto a local optu, the detals of the algorth s dscussed n secton 5 above. V. IMPLEMENTATION AND RESULTS In ths paper, we perfored our experents on two corpuses: TIMIT corpus [30] and MNIST corpus [47]. For TIMIT datasets, 0 vowels used n [36] are selected to evaluate our approach. The 0 vowels set are: {aa, aw, ae, ah, ao, ax, ay, axr, ax-h, uw, ux, uh, oy, ow, x, h, y, eh, ey, er}. The 0 classes (handwrtten dgts) of MNIST corpus are: {0,,, 3, 4, 5, 6, 7, 8, and 9}. In all the experents reported below, we perfored 5- fold cross valdaton for tunng SVM hyper paraeters g and C. The GNU SVM lght [6] pleentaton s used for our GASVM Machne used n ths paper, and LbSVM [45] for SVC [39] software s to copare our results. All the experents were run on standard Intel (R) core Duo CPU.00 GHZ wth.99 Go eory runnng the Wndows XP operatng syste. The followng tables suarze our prelnary results, for the classfcaton supervsed of 0 vowels lsted above. ISBN: ISSN: (Prnt); ISSN: (Onlne) IMECS 03

5 Proceedngs of the Internatonal MultConference of Engneers and Coputer Scentsts 03 Vol I, IMECS 03, March 3-5, 03, Hong Kong TABLE IV: RESULTS OF SVM (OVO) FOR 0 VOWELS Test CPU te T C g (%) (s) SVM (OVO) T s kernel type: Gaussan (). C s a paraeter of SVM, g s Gaussan kernel paraeter, the two paraeters were calculated by cross valdaton. Test s recognton rate and CPU te s te of test. TABLE V: RESULTS OF GASVM FOR 0 VOWELS T C g Test CPU te (%) (s) GASVM T s kernel type: Gaussan (). C s a paraeter of SVM, g s Gaussan kernel paraeter, the two paraeters were calculated by cross valdaton. Test s recognton rate and CPU te s te of test. TABLE VI: RESULTS OF GASVM FOR 0 DIGITS T C d VI. CONCLUSION Test (%) CPU te (s) GASVM T s kernel type: Polynoal (), lnear (0). C s a paraeter of SVM, d s Polynoal kernel paraeter, the two paraeters were calculated by cross valdaton. Test s recognton rate and CPU te s te of test. We ntroduced and pleented a novel effcent approach for SVM ultclass. The Bnary Tree of support vector achne (SVM) ultclass paradg was shown through extensve experents to acheve state of the art results n ters of accuracy. The prelnary experents of our bnary archtecture ndcate that the results are coparable to those of other ethods. Although the optzaton of learnng paraeters of SVM can prove the results. However, as can be seen fro the results, tranng tes and especally testng tes are stll excessve preventng SVM fro beng used n speech recognton at least for the te beng. We beleve that n order to prove autoatc speech recognton technology, ore research efforts ust take advantage of the sold atheatcal bass and the power of SVM bnary, and should be nvested n developng general purpose effcent achne learnng paradgs capable of handlng large scale ult-class probles. REFERENCES [] K-F. Lee and H-W, Hon. Speaker-ndependent phone recognton usng Hdden Markov Models ; IEEE Trans, Acoust, Speech Sgnal Processng, vol ASSP-37, N, 989. [] Alex Grave and Jürgen Schdhuber, Fraewse Phonee Classfcaton wth Bdrectonal LSTM Networks. IJCNN, 005. ISBN: ISSN: (Prnt); ISSN: (Onlne) [3] Hakan Erdogan, Regularzng Lnear Dscrnant Analyss for Speech Recognton, 005. [4] J. Morrs and E. Fosler-Lusser, Dscrnatve Phonetc Recognton wth Condtonal Rando Felds ; HLTNAACL, 006. [5] DongSuk. Yuk and Jaes Flanagan, Telephone Speech Recognton Usng Neural Networks and Hdden Markov Models, ICASSP, 999. [6] F-Ghnwa Choueter and R-Jaes Glass, A Wavelet and Flter Bank Fraework for Phonetc Classfcaton, ICASSP, 005. [7] K. Joseph, S. Shalev-Shwartz, S. Bengo, Y.Snger and D. Chazan, Dscrnatve Kernel-Based Phonee Sequence Recognton, ICSLP, 006. [8] Son Kng, Todd Stephenson, Stephan Isard, Paul Taylor and Alex Strahan, Speech Recognton va Phonetcally Featured Syllables, ICSLP, 998. [9] Fe Sha and Lawrence K. Saul, Large Margn Hdden Markov Models For Autoatc speech recognton, NIPS, 006. [0] Fe Sha and Lawrence K. Saul, Coparason of Large Margn Tranng To Other Dscrnatve Methods for Phonetc Recognton by Hdden Markov Models, ICASSP, 007. [] M. Kaal Oar and H-J. Mark, Non-Lnear Independent Coponent Analyss for Speech Recognton, Internatonal Conference on Coputer Councaton and Control Technologes, Orlando, 003. [] H. Nao, V. Saeed and MC. Paul, A Novel Model for Phonee Recognton Usng Phonetcally Derved features, Proceedng EUSIPCO, 998. [3] J. Saloon, k. Son and Mles Osborne, Fraewse Phone classfcaton Usng Support Vector Machnes, ICSLP, 00. [4] Boser, B. Guyon, V Vapnk, A tranng algorth for optal argn classfers. Ffth Annual Workshop on Coputatonal Learnng Theory. ACM Press, Pttsburgh, 99. [5] V. Vapnk, Statstcal Learnng Theory, Wley, New York, 998. [6] Guyon, I. Boser and V. Vapnk, Autoatc Capacty Tunng of Very Large VC-Denson Classfers ; Advances n Neural Inforaton Processng Systes Vol.5 Morgan Kaufann, San Mateo, CA, 993. [7] V. Vapnk and Chervonenks, Theory of Pattern Recognton [n Russan]. Nauka, Moscow, 974. (Geran Translaton: Wapnk, Tscherwonenks, Theore der Zechenerkennung, Akadee-Verlag, Berln), 979. [8] V. Vapnk, 98. Estaton of Dependences Based on Eprcal Data [n Russan]. Nauka, Moscow, (Englsh translaton, Sprnger Verlag, New York), 979. [9] E. Osuna, R. Freund, F. Gros, Tranng Support Vector Machnes, an Applcaton to Face Detecton, n Coputer vson and Pattern Recognton, pp.30-36, 997. [0] T. Joachs, Makng large-scale support vector achne learnng practcal, n Advances n Kernel Methods, B. Schölkopf, C. Burges, A. Sola, (eds.), Cabrdge, MIT Press, 998. [] Danel Hong, Speech recognton technology: ovng beyond custoer servce. Coputer Busness Onlne, March st, 007. [] Ryan Rfkn and Aldebaro Klautau, In defense of one-vs-all classfcaton, Journal of Machne Learnng Research 5, 0-4, 004. [3] J.C. Platt, N. Crstann, and J. Shawe-Taylor, Large argn DAGs for ultclass classfcaton ; In Advances n Neural Inforaton Processng Systes, volue, pages MIT Press, 000. [4] Chh-We Hsu and Chh-Jen Ln. A Coparson of Methods for Multclass Support Vector Machnes. [5] S. Furu. Speaker-Independent Isolated Word Recognton Usng Dynac Features of Speech spectru ; IEEE Trans, Acoustc, Speech, and Sgnal Processng 34, [6] T. Joachs, Makng Large-Scale SVM Learnng Practcal, Software avalable at: [7] A. Ganapathraju, Support vector achnes for speech recognton. PhD Thess, Mssssp State Unversty, USA, 998. [8] N. Sth and M. Gales, Speech recognton usng SVM. Advances n Neural Inforaton Processng Systes, 4, MIT Press, 00. [9] J.S. Garofolo, L.F. Lael and al, TIMIT Acoustc-Phontc Contnuous Speech Corpus. Phladelpha, Lngustc Data Consortu, 993. [30] M. Slaney, Audtory Toolbox verson. Tech. Report#00, Internal Research Corporaton, 998. [3] Ryan Rfkn and al, Phonetc Classfcaton Usng Herarchcal, Feed-forward, Spectro-teporal Patch-based Archtectures. Techncal Report, MIT-CSAIL-TR, 007. [3] R. P. Lppann, Speech recognton by achnes and huans, Speech Councaton -5, Elsever, 997. [33] Martens J. P. Depuydt, Broad phonetc classfcaton and segentaton of contnuous speech by eans of neural networks and IMECS 03

6 Proceedngs of the Internatonal MultConference of Engneers and Coputer Scentsts 03 Vol I, IMECS 03, March 3-5, 03, Hong Kong dynac prograng, Speech councaton, vol. 0, no, pp. 8-90, 99. [34] L. R. Rabner and B-H Juang, Fundaentals of speech recognton, Prentce Hall, 993. [35] P. Moreno, On the use of Support Vector Machnes for Phonetc Classfcaton ; In the proceedngs of ICCASP, 999. [36] R. Rfkn and a, Nose Robust Phonetc Classfcaton wth Lnear Regularzed Least Squares and Second Order Featues, ICASSP, 007. [37] J.C. Platt, Fast tranng of support vector achnes usng sequental nal optzaton. In Adv. n Kernel Methods, Schölkopf, C. Burges, A. Sola eds, 998. [38] H. Peter Graf, E. Cosatto, B. Léon, I. Dourdanovc, and V. Vapnk, Parallel Support Vector Machnes: The Cascade SVM n Advances n Neural Inforaton Processng Systes, 005. [39] LbCVM Toolkt of the proved Core Vector Machne (CVM), whch are fast Support Vector Machne (SVM) tranng algorths usng core-set approxaton on very large scale data sets avalable at: [40] B. Fe, J. Lu. Bnary Tree of SVM: A New Fast Multclass Tranng and Classfcaton Algorth IEEE Transacton on neural networks, Vol. 7, No. 3, May 006 [4] Gjorgj Madzarov, Dejan Gjorgjevkj and Ivan Chorbev. A Multclass SVM Classfer Utlzng Bnary Decson Tree. Inforatca 33, 33-4, 009. [4] Sung-Hyuk Cha and Charles Tappert. A Genetc Algorth for Constructng Copact Bnary Decson Trees. Journal of Pattern Recognton Research, -3, 009. [43] Hansheng Le and Venu Govndaraju. Half-Aganst-Half Mult-class Support Vector Machnes. [44] J. Platt. Fast tranng of support vector achnes usng sequental nal optzaton. In Advances n Kernel Methods - Support Vector Learnng. [45] C-C. Chang and C-J. Ln, LIBSVM Toolkt: a lbrary for support vector achnes, Software avalable at: [46] Jan, A. and Dubes, R, Algorths for Clusterng Data. Prentce Hall Advanced, Reference Seres, 988. [47] The MNIST database of handwrtten dgts has a tranng set of 60,000 exaples, and a test set of 0,000 exaples. The dgts have been sze-noralzed and centred n a fxed-sze age avalable at: ISBN: ISSN: (Prnt); ISSN: (Onlne) IMECS 03