A Novel Linear-Polynomial Kernel to Construct Support Vector Machines for Speech Recognition

Transcription

1 Journa of Computer Science 7 (7): , 20 ISSN Science Pubications A Nove Linear-Poynomia Kerne to Construct Support Vector Machines for Speech Recognition Bawant A. Sonkambe and 2 D.D. Doye Pune Institute of Computer Technoogy, Pune, Maharashtra State, India 2 Department of E and TC, SGGSIE and T, Nanded, Maharashtra State, India Abstract: Probem statement: To accept the inputs as spoken word utterances uttered by various speakers, recognize the corresponding spoken words and initiate action pertaining to that word. Approach: A nove Linear-Poynomia (LP) Kerne function was used to construct support vector machines to cassify the spoken word utterances. The support vector machines were constructed using various kerne functions. The use of we known one-versus-one approach considered with voting agorithm. Resuts: The empirica resuts compared by impementing various kerne functions such as inear kerne function, poynomia kerne function and LP kerne functions to construct different SVMs. Concusion: The generaization performances based on the One-versus- One approach for speech recognition were compared with the nove LP kerne function. The SVMs using LP kerne function cassifies the spoken utterances very efficienty as compared to other kerne functions. The performance of the nove LP kerne function was outstanding as compared to other kerne functions. Key words: Hidden Markov Mode (HMM), neura network, Empirica Risk Minimization (ERM), kerne function, voting agorithm, Modified Fuzzy-Hyper sphere Neura Networks (MFHNN), Support Vector Machines (SVM), hyperpane, Vapnik-Chervonenkis (VC), Linear-Poynomia (LP) INTRODUCTION From ast severa years, the speech recognition research paying a eading roe in more number of appications. Many new techniques emerged incuding Modified Fuzzy-Hyper sphere Neura Networks (MFHNN), Neura Networks (Doye et a., 2002; Soaimani, 2009), Hidden Markov Modes (Ping et a., 2009), Bayesian Networks (Mansouri et a., 20) and Dynamic Time Warping decade to decade to increase the performance of the speech recognition systems but Hidden Markov Mode (HMM) (Rabiner and Juang, 993; Doye et a., 2002) is among the most successfu state of art toos widey used but sti speech recognition systems are far away to achieve high-performance as we as accuracy. The HMM are originay a generative modes because the decisions are based on the ikeihood estimation of the currenty evauated pattern. Thus, HMM requires additionay discriminative approaches to discriminate the speech sampes. The imitation of HMM, is the oss of performance due to the mismatch between training and testing conditions. The Support Vector Machine (SVM) (Carkson and Moreno, 999; Schokopf and Smoa, 2002) is emerged as a new machine earning technique for pattern cassification. The SVMs are based on the discriminative approach which discriminates the patterns by finding the goba minima. The SVM uses Structura Risk Minimization (SRM) principe to construct inear and noninear cassifiers with Vapnik- Chervonenkis (VC) dimension (Vapnik, 998; Cristianini and John, 2000). The VC dimension contros the capacity of the earning machine. The inear and noninear approaches are used to construct the SVMs. In inear methods inner products caed dot products are considered for generating the optima separating hyperpane for cassifying the two casses where as in non inear approach dot products are repaced by kerne functions (Burges, 998; Schokopf and Smoa, 2002) to construct the optima separating hyperpane. In this study we propose to use nove kerne function caed the Linear-Poynomia (LP) (Cristianini and John, 2000) kerne incuding the description about the construction of inear support vector machine and the construction of noninear support vector machine Corresponding Author: Bawant A. Sonkambe, Pune Institute of Computer Technoogy, Pune, Maharashtra State, India 99

2 aong with description of the nove LP kerne function. The description of the cassification approach aso described and asty, expained the detaied experimenta resuts obtained by comparison with basic kerne function. MATERIALS AND METHODS The Hidden Markov modes are most successfu techniques for modeing a speech by determining the speech sound representation. The Cassification probems are treated as compex probems. In cassification probem, the main task is to cassify the probem directy by estimating the decision surfaces. Many researchers have proved that, the Support Vector Machines (SVM) are the most efficient and popuar generaized inear cassifiers used for data cassification. The Support Vector Machines are the machine earning techniques deveoped by Vapnik in 960 s can perform static cassification tasks. The SVMs are appied successfuy for soving pattern recognition probems due to its discriminative nature. The SVMs are aso caed hyperpane cassifiers because it constructs optima separating hyperpane to discriminate between two casses. The earning machines are constructed by nonineary mapping from input vector space to a high dimensiona vector space caed feature space. The SVMs are not designed to hande tempora structure of data. The SVMs has very good generaization abiity that improves the system robustness of speech recognition tasks in noisy environment. The SVMs key property is to minimize the empirica cassification error and maximize the geometric margin simutaneousy. Hence it is aso known as maximum margin cassifier. Construction of inear support vector machines: The earning machines can construct optima predictor through a set of functions. Risk minimization means minimizing the functiona from a given training data that is minimizing the optima parameterization. The Empirica Risk Minimization (ERM) (Vapnik, 998; Danies and Ejara, 2009) is a kind of risk minimization commony used as optimization procedure in machine earning. The optimization process depends on the oss functions because prior joint probabiity distribution is not known. The risk can be determined as a mean error computed from the fixed number of training data. Here, the risk is defined as measures of quaity of a chosen function. ERM is computationay simper than attempting to minimize the actua risk but due to non measuring capacity the machine, if the compexity of a machine increases then a machine over fits the data. J. Computer Sci., 7 (7): , Fig. : Optima separating hyperpane for seaparating two casses ineary In the structura risk minimization, the optima function not ony depends on the oss functions for cacuating the expected risk but aso depends on its structure. Here, the risk is determining through VC dimension which measures the capacity of a earning machine by computing the upper bound. The SRM principe is impemented by constructing the SVMs. The inear cassifier in separabe case the two datasets can be perfecty mapped. The separating hyperpane is caed inear hyperpane separates the given datasets by maximizing the margin. The SVMs are constructed by constructing the binary casses. Consider binary cassification probems by assuming the training data, given beow: (x,y ),(x 2, y 2),...,(x, y ) () Here: x = The input patterns y = Outputs abeed by + and The goa is to find the inear decision function f(x) and the separating hyperpane H, where H: x.w + b = 0 and f(x) = sgn(x.w+b). Where bthe distance of the hyperpane from the origin is aso referred as bias and w is the norma to the decision region aso referred as weights. The vaue of H is cacuated using quadratic programming approach. Figure shows the optima separating hyperpane to find the decision boundaries between the two casses. The margin of the SVM is defined as the distance from the separating hyperpane to the cosest two casses. The margin is equa to 2/ w inequaities. Here, distance between the dotted ines is caed margin and the data points appeared on the dotted ines are caed support vectors. The optima hyperpane is obtained by appying scaing on the parameters w and b because scaing avoids variance among the data vaues. The existence of optimaity is guaranteed by the Karush-Kuhn-Tucker (KKT) (Vapnik, 998) theorem. The main feature of the optima hyperpane is to maximize the margin whie minimizing the empirica risk. For separating the two

3 reaistic datasets inear cassifier in non separabe case considers the sack variabe to find the miscassification errors. Construction of Noninear support vector machines (Sonkambe and Doye, 2008): The non-inear cassifiers can hande the decision boundaries in the compex noninear data very efficienty. The use of kerne functions is essentia to construct optima hyperpanes of non-inear cassifiers. The kernes are positive-definite functions to map data into high dimensiona spaces which increases the computationa power of inear machines. The key advantages of the kernes are firsty, it incorporates prior knowedge of the probem by defining a simiarity measure between two data points. Secondy, kerne function finds the kerne matrix which contains a the information about the input space and thirdy, the number of operations required is not necessariy proportiona to the number of features. There are various kinds of kerne functions used commony for speech cassification. The kerne functions shoud satisfy the mercer s condition which shows the symmetry property. The mapping is achieved through a repacement of the inner product: x i.x j Φ(x i). Φ (x j) The functiona form of the mapping Φ(x), does not need to be known since it is impicity defined by the choice of kerne: i.x j) Φ(x i). Φ (x j) Each choice of kerne wi define a different type of feature space and the resuting cassifiers wi perform differenty on test data, though good generaization. For an SVM with RBF kernes the resuting architecture is an RBF network (Cortes and Vapnik, 995; Mahi and Izabatene, 20). However, the method for determining the number of nodes and their centers is quite different from standard RBF networks with the number of nodes equa to the number of support vectors and the centers of the RBF nodes identified with the support vectors themseves. Formation of New kernes: There are different kerne functions commony used for cassification. In this case we are proposing two nove kerne functions by combining the inear kerne function with poynomia kerne function caed Linear-Poynomia (LP) Kerne (Kurtz, 99; Tan and Wang, 2004) function which formaized as foows: i.x j) k (x i.x j) + k 2(x i.x j) J. Computer Sci., 7 (7): , 20 Where: k (x.x ) = (x.x ) = A inear kerne function i j i j k (x.x ) (x.x ) d 2 i j = i j + = A poynomia kerne function The construction of the optima hyperpane is of the form: 0 0 i i i i= f[x, α ] = α y (x.x) + b (2) Here, b-indicates threshod as a constant and (x i,x) indicates inner product of two input vectors as we as -indicates number of data pairs. The maximum-margin separating hyperpane caed optima hyperpane which reduces the generaization errors. The objective function of our optimization probem is the form: D( α ) = α y y α α (x.x ) Such that: (3) i i j i j i j i= 2 i, j= i i j i= α 0 and α α = 0,i = 0,,..., (4) where, α i are the Lagrange mutipiers which define the weights of the mode as w i = α i y i.. The construction of decision functions are depends on generating the inner product in a feature space which are noninear in their input space as given beow: 0 i i i (5) sup portvectors f (x, α ) = sign( y α.x) + b and are equivaent to inear decision functions in the feature space z (x),...,z (x),... γ 0 i i γ i γ (6) sup portvectors f (x, α ) = sign( y α z (x )z (x) + b The kerne function can be represented as k (x i, x j ) which generates the inner product for the feature space. The commony used kerne functions are. The Linear Kerne function is represented by the inner product given by the equation: i.x j) = (x i.x j) (7) The poynomia kerne has more number of hyperparameters than the RBF kerne which infuences the compexity of mode seection. The Poynomia Kerne function is generated for finding the inner product given by the equation: 993

4 .x ) (x.x ) d i j = i j + (8) Here, d is the poynomia degree which is a positive integer. The LP kerne function can be represented as combined kerne functions of inear and poynomia kernes which is formuated as beow:.x ) = k (x.x ) + k (x.x ) i j i j 2 i j d i.x j) = ((x i.x j) + (x i.x j + ) ) J. Computer Sci., 7 (7): , 20 One-versus-One cassification approach: One-versus- One is one of the most commony used successfu to poynomia kerne function. 994 (9) The decision function can be constructed in the form of: i i i (0) sup portvectors f (x, α ) = sign( y α.x) + b The Radia Basis Function (RBF) kerne nonineary maps input sampes into a higher dimensiona space, which can hande the reation between cass abes and attributes is noninear. The RBF kerne is not efficient when the number of features is very arge as compared to other kerne functions. The Radia Basis Function kerne can represent as: xi x j i.x j) = exp( ) 2 2σ 2 () The decision discriminative decision function is determined by the foowing equation: (2) D(x) = y y α α.x) + b = w.x) i j i j i i i i= i= This gives a decision about the casses to discriminate among them. Cassification approach: The speech recognition probem is a muticass cassification probem where as SVMs are efficienty sove binary cassification probem. There are two approaches to sove muticass probem by using SVM. First, One-versus-One (Ganapathiraju et a., 2000) cassification approach aso caed pair wise cassification by simpy constructs for each pair of casses a cassifier which separates those casses and second, One-versus-A cassification approach (Osuna et a., 997; Chin, 999) by constructing for each cass a cassifier which separates that cass from the reminder of data. A data with the exception of one row is used to train the earning agorithm. approaches for discrimination of casses. The cassifiers required according this approach is equa to k(k-)/2 cassifiers. Where k = 0, that is 45 cassifiers are constructed. The One-versus-One cassification approach is aso caed pair-wise cassification approach where ony pair-wise data points can be considered to discriminate between the two casses. The main feature is that, it reduces the generaization error rate by reducing the number of support vectors hence is faster than the One-versus-A approach. A voting scheme agorithm used with fixed weights to cast one vote in favor the cass. This agorithm force to choose among one cass. These votes are distributed uniformy so that we can cassify the correct casses of the speech signas by considering the highest score. The One-versus-One approach requires more memory space as we as requires more time for training. RESULTS AND DISCUSSION The database was coected from 5 Indians. The database was coected for 0 digits uttered by 5 times. The speech features are extracted and obtained LPC Coefficients. The speech signas were samped at 8 KHz divided into a sequence of data bocks, each bock spanning 20ms and separated by 0 ms. The speech features are extracted and obtained LPC Coefficients and these LPCC were used as a data points for training the SVM. The number speech sampes used for training were 50 from each digit and rests of the sampes were used as testing data for speech signa cassification. We have constructed various SVMs using inear kerne function, Poynomia kerne function and the proposed LP kerne function. When LP kerne function used to construct noninear support vector machines, it gives very good performance as compared to inear kerne function as we as poynomia kerne function. The observation is that, it maximizes the margin with sma fraction vaue increased by 0.0 to 0.00 as compared to the margin obtained by poynomia kerne function. Hence, the LP kerne reduces generaization error drasticay so it discriminates the data points very accuratey as compared to poynomia kerne function. Tabe shows the training performance of the poynomia kerne function for cacuating the support vector whie the training performance of the LP kerne function is shown in Tabe 2. The compared training performance graph is aso shown Fig. 2. The LP kerne aso finds better number of support vectors as compared

5 J. Computer Sci., 7 (7): , 20 Tabe : Generaization performance for one-versus-one cassifier using poynomia kerne function for training data cassification experiment conducted shows more accurate resuts as compared to poynomia kerne function by discriminating the decision boundaries between two speech data points. We considered more accurate approach as a One-versus-One to achieve better performance as compared to One-versus-A approach. In future work, the LP kerne functions can be compared with RBF kerne functions. REFERENCES Tabe 2: Generaization performance for one-versus-one cassifier using LP kerne function for training data Fig. 2: The comparative training performance graph (a) (b) The existing kerne functions such as inear and poynomia kerne functions are impemented in addition to the impementation of LP kerne function for the construction of support vector machines. CONCLUSION In this study, the proposed nove LP kerne function outperforms as compared to poynomia kerne function and inear kerne functions. This kerne function can be considered as a more suitabe for cassifying the noninear signas. The speech 995 Burges, C.J.C., 998. A Tutoria on support vector machines for pattern recognition. Know. Discovery Data Min., 2: DOI: 0.023/A: Chin, K.K., 999. Support vector machines appied to speech pattern cassification. Master s Thesis, Engineering Department, Cambridge University Carkson, P. and P.J. Moreno, 999. On the use of support vector machines for phonetic cassification. Proceeding of the IEEE Internationa Conference. on Acoustics, Speech and Signa Processing (ICASSP), Phoenix, Arizona, USA., pp: DOI: 0.09/ICASSP Cortes, C. and V. Vapnik, 995. Support vector networks. Mach. Learn., 20: -25. DOI: 0.234/ Cristianini, N. and S.-T. John, An introduction to support vector machines and other kerne-based earning methods. st Edn., Cambridge University Press, John Shawe-Tayor, Roya Hooway, London, ISBN: , pp: 204. Danies, K. and D.D. Ejara, Impact of information asymmetry on municipa bond yieds: An empirica anaysis. Am. J. Econ. Bus. Admin., : -20. DOI: /ajebasp Doye, D.D., U.V. Kukarni and T.R. Sontakke, Speech recognition using modified fuzzy hypersphere neura network. Proceedings of the Internationa Joint Conference on Neura Networks, May 2-7, Honouu, HI, USA., pp: DOI: 0.09/IJCNN Ganapathiraju, A., J. Hamaker and J. Picone, Hybrid SVM/HMM architectures for speech recognition. Proceedings of the 6th Internationa Conference on Spoken Language Processing, Oct. 6-20, Beijing, China, pp:

6 J. Computer Sci., 7 (7): , 20 Kurtz, M., 99. Handbook of Appied Mathematics for Engineers and Scientists. st Edn., McGraw Hi, New York, 608. DOI: 0.036/ Mahi, H. and H.F. Izabatene, 20. Segmentation of sateite imagery using RBF neura network and genetic agorithm. Asian J. Appied Sci., 4: Mansouri, M., A. Ganguy and A. Mostashari, 20. Evauating agiity in extended enterprise systems: A transportation network case. Am. J. Eng. Appied Sci., 4: DOI: /ajeassp Osuna, E., R. Freund and F. Girosi, 997. An improved training agorithm for support vector machines. Proceeding of the IEEE Workshop on Neura Networks for Signa Processing, Ameia Isand, Forida, USA., pp: DOI: 0.09/NNSP Ping, Z., T. Li-Zhen and X. Dong-Feng, Speech recognition agorithm of parae subband HMM based on waveet anaysis and neura network. Inform. Techno. J., 8: Rabiner, L.R. and B.H. Juang, 993. Fundamentas of Speech Recognition. st Edn., Prentice Ha, Engewood Ciffs, New Jersey, USA., ISBN: 0: , pp: 496. Schokopf, B. and A. Smoa, Learning with Kernes. st Edn., MIT Press, Cambridge, Mass, USA., ISBN-0: , pp: 644. Soaimani, K., A study of rainfa forecasting modes based on artificia neura network. Asian J. Appied Sci., 2: DOI: /ajaps Sonkambe, B.A. and D.D. Doye, An overview of speech recognition system based on the support vector machines. Proceeding of the Internationa Conference on Computer and Communication Engineering, May 3-5, IEEE Xpore, Kuaa Lumpur, pp: DOI: 0.09/ICCCE Tan, Y. and J. Wang, A support vector machine with a hybrid kerne and minima vapnikchervonenkis dimension. IEEE Trans. Know. Data Eng., 6: DOI: 0.09/TKDE Vapnik, V.N., 998. Statistica Learning Theory. st Edn., John Wiey and Sons, New York, USA., ISBN-0: , pp: