Optimization and Application of Support Vector Machine Based on SVM Algorithm Parameters

Transcription

1 Optimization and Appication of Support Vector Machine Based on SVM Agorithm Parameters YAN Hui-feng 1, WANG Wei-feng 1, LIU Jie 2 1 ChongQing University of Posts and Teecom , China 2 Schoo Of Civi Engineering & Architecture Chongqing Jiaotong Univ.,40074,China @qq.com, iu_jie@163.com Journa of Digita Information Management ABSTRACT: The hospita customer cassification is very important for the integration and aocation of hospita resources, can greaty enhance the maret competitiveness of the hospita. Support Vector Machine (SVM) is an approach to sove cassification probem by using optimization method. Seecting different erne parameters can construct different cassifiers, meanwhie parameters decide their earning and generaization abiity. In order to sove the imitation of seecting parameters by experience, so the partice swarm (PSO) pattern search agorithm is proposed to search optima parameters and tae them into the practice of hospita customer cassification; The PSO mode search agorithm is to combine the advantages of PSO agorithm and pattern search agorithm, PSO mode search agorithm has strong goba search capabiity and good advantage of oca convergence. The resut of experiment shows that this method is not ony efficient, but aso to search the optima parameters achieving a high accuracy, which is an effective method of SVM parameter optimization. Categories and Subject Descriptors: I.1.2: [Theory and agorithms]: Support vector machines; E.1: [Database theory] Data structures and agorithms for data management Genera Terms: Data Structures, Partice Swarm Optimization, Support Vector Machines Keywords: Support Vector Machine, Kerne Parameters Seection, Partice Swarm Pattern Search, Customer Data Cassification Received: 1 November 2012, Revised 27 December 2012, Accepted 30 December Introduction Support vector machines (Support Vector Machine, SVM) is Corinna Cortes and Vapni, equa to the 1995 first proposed it in resoving the sma sampes, noninear and high dimensiona pattern recognition exhibit many unique advantages, and be abe to promote the appication to function fitting and other machine earning probems [1]. In addition, support vector machines is a convex quadratic programming, extreme vaue is obtained to ensure the goba optima soution. In recent years, support vector machines in text categorization, remote sensing image anaysis, faut identification and prediction, time series forecasting, and other fieds have a successfu appication [2,3]. Support vector machine (SVM) cassification, the choice of the erne function is very important, erne function parameter seection determines the cassification performance of the support vector machine, so the choice of parameters is essentia. Previous parameter seection of support vector machine by virtue of experience or Mongoian guess, with a ot of imitations. To overcome this drawbac, Chapea parameter seection method based on gradient descent support vector machine [4],Qi Liang proposed parameter seection method of support vector machine based on ant coony agorithm [5] Wang Xinging and cars accounted for Bin-based networ The attice parameter optimization method [6], these methods in the corresponding iterature experiments proved the effectiveness of various methods, but there are sti imitations of these agorithms, easy to fa into oca minimum point, computationay intensive, ow efficiency defects. With the in-depth deveopment of China s maret economy Journa of Digita Information Management Voume 11 Number 2 Apri

2 and heath care system reform, medica industry competition between heath care worers and patients is no onger a simpe treatment with the therapeutic reationship the characteristics of the service industry is graduay emerging hospita service mode from the disease to the patient transfer and hospita customer data cassification, through the networ to estabish and maintain ong-term good reations of the hospita with customers, dig and effective management of customer resources, to maintain and increase the hospita s maret competitiveness has very important significance. Cassification (to determine which project payed a cataytic roe) can pay a very good reference for the future integration of hospita resources and distribution of hospita customers. In view of this, A partice swarm pattern search agorithm is proposed to optimize the parameters, the agorithm combines the advantages of partice swarm optimization agorithm and pattern search, the agorithm in efficiency and performance than partice swarm optimization and pattern search agorithm, there are a ot The improvement for the support vector machine parameter seection provides a new and effective method. Simuation resuts show that the optimized SVM agorithm in the hospita customer data cassification process is not ony efficient, but the search for the optima parameters to achieve a higher accuracy rate. 2. Effect on SVM agorithm and erne function The main idea of SVM can be summarized as two points: (1) it is anayzed for inear separabe case, through the use of non-inear mapping agorithm for inear inseparabe transformed into ow-dimensiona input space inear inseparabe sampe above the high dimensiona feature space it ineary separabe, which maes high-dimensiona feature space the inear agorithm inear anaysis of the non-inear characteristics of the sampe as possibe; (2) it is based on the structura ris minimization theory in the feature space to construct optima partition hyperpane, maes earning to get goba optimization, and satisfy certain upper bound the expected ris of the entire sampe space to a probabiity. SVM method is by a non-inear mapping p, the sampe space is mapped to a high-dimensiona, even infinite dimensiona feature space (Hibert space), so the probem into a noninear separabe in the origina sampe space in the feature space in ineary separabe probems. Simpy put, the raised dimension and inear. Raised dimension, is mapping the sampes to the high-dimensiona space, under norma circumstances, this wi increase the compexity of the cacuation, and may even cause the curse of dimensionaity, and thus rarey cares. As issues such as cassification, regression, however, is iey not inear processing in ow-dimensiona sampe space sampe set in the high dimensiona feature space by a inear hyperpane inear division (or regression). The raised dimension wi bring computing compexity, the SVM method ingenious soution to this probem: the appication erne function expansion theorem, you do not need to now the expicit expression of non-inear mapping; high dimensiona feature space inear earning machine, compared with the inear mode, not ony amost does not increase the compexity of the cacuation, but aso in a way to avoid the curse of dimensionaity. A thans to the expansion of the erne function and computing theory. Sigmoid erne function from the neura networ was shown to have good goba cassification performance when its parameters to obtain appropriate generaization abiity is very simiar to the Gaussian erne cassification resuts are obvious. Sigmoid erne function three parameters and seect a different combination can construct different performance cassifier. c is to contro the right and wrong stars sampe punishment eve, the vaue of smaer right or wrong stars sampes punishment ower degree cassification hyperpane more simpe, when experience arger error; if the c vaue tends to infinity, then a constraints that must be met in this case means that a training sampes must be correcty cassified, this wi resut in separating hyperpane compex, computationay intensive, time-consuming and more. So the choice of the vaue c to be combined with the specific appication, to meet the cassification accuracy to tae a sma vaue as possibe, so that the decision function is simpe. And the γ change actuay is impied to change the mapping function, thereby changing the spatia distribution of the sampe data sub compexity aso determines the inear cassification achievabe minimum error [8]. The γ Vaue is too sma, a the test sampes become support vector to produce Overfitting Phenomenon ; if the γ vaue is too arge, the support vector machine cassification performance wi drop substantiay, the cassification of the test sampes is amost zero, A sampes are judged as a cass. Decreased number of support vectors, and the correct cassification abiity of the cassifier on the test sampes woud be greaty enhanced if the γ vaue is appropriate, usuay use c and γ match. 3. PSO mode optimized SVM parameters 3.1 PSO agorithm PSO agorithm, each member of the groups caed partices, each partice in a muti-dimensiona search space fight and continue according to the partice s own experience, the partice neighbor s experience or the experience of the whoe group and update their own speed and position, in order to find the optima soution. The corresponding iterative formua is: v i = w v i + c 1 rand 1 ( pbest p i ) + (1) c 2 rand 2 ( gbest p i ) p i = p i d + v i (2) Where, w is the inertia weight; reativey arge w have more goba search abiity, reativey sma w wi ead to a rapid 166 Journa of Digita Information Management Voume 11 Number 2 Apri 2013

3 convergence; c 1 and c 2 are earning factors; rand 1 and rand 2 are random numbers on the interva [0,1]; p is expressed support vector machine parameters c and σ is the current vaue of the current position of the partice; v [v max, v min ] is the veocity of the partices, to determine the profie direction and the size of the next generation of c and σ. In order to improve the speed of convergence of the agorithm, the with shrinage factor χ speed evoution equation (equation(3)): v i = χ{w v i + c 1 rd 1 ( pbest p i ) + c 2 rd 2 ( gbest p i )} 3.2 Pattern Search Agorithm Pattern search agorithm aso nown as Hooe-Jeeves method, the agorithm consists of two mobie process: First, the detection of moving another mobie mode. Probing movement is moved aong the direction of the coordinate axis; mode mobie is put mobie connection aong adjacent two probing points. Two processes can aso be understood as foows: First, determine the favorabe direction of the search; acceeration process, speed up the search is in a favorabe direction. PSO agorithm than the pattern search agorithm has a wide range of goba search capabiity; search starting from the groups, with the impicit paraeism; search using the evauation function vaue inspired; convergence is fast, simpe parameter adjustment; scaabiity, and easy to combine with other agorithms, etc. However, in the oca searching abiity of the agorithm ate Poor, feedbac information is underutiized. The advantages of pattern search pattern search method does not require the objective function to guide, is a direct method, and the iteration is reativey simpe, especiay the abiity of oca search. Pattern search method is compared to PSO optimization agorithm has a high earning accuracy, but the number of iterations, computationay intensive, objective function is easy to fa into oca minimum points and defects and poor goba search capabiity. In this paper, the advantages of these two agorithms, an efficient and accurate seect parameters - PSO pattern search agorithm is proposed. 3.3 PSO Pattern Search Agorithm From the goba view, PSO agorithm is a viabe, good robustness optimization agorithm [9]. from the oca view pattern search agorithm has oca convergence;there can introduce pattern search in PSO agorithm PSO to get rid of the defects fa into oca minimum initia vaue sensitive, improve the agorithm convergence speed and accuracy. The core idea of the PSO pattern search agorithm: first with the PSO agorithm to find an optima parameter, reuse pattern search oca search in the vicinity of the (3) optimum parameters, the search resuts did not meet the accuracy requirements, goba search in the goba scope The search cyces unti you find the optima parameters. PSO pattern search agorithm steps are as foows:the first step, the initiaization popuation size, the partice veocity V i and position P i ; The second step, To cacuate the partice fitness function, the cassification error (formua 4) is as the partices of the fitness evauation function. Cassification error formua: 1 1 (4) n Σ n T i + F i i = 1 In which, T i is the correct cassification of sampes; F i is the wrong number of sampes. The third step, recording minimum fitness function vaue according to the formua (1) and the formua (3) update the positions and veocities of a partices; The fourth step, The three step in the decree P g = P g (c, σ) as the initia vaue of pattern search, And given the unit vector e j ( j = 1,..., n). The initia step ength σ = (σ 1, σ 2,..., σ n ) T > 0, acceeration coefficient γ > 0, coefficient of contraction θ (0, 1) And precision ε > 0, Set = 0, y = P, j = 1; The fifth g step, from y set out., in order to parae to the unit vector e j ( j = 1,..., n) axia detection mobie: (1) if f ( y + σ j e j ) < f (y), order y = y + σ j e j, Otherwise turn to (2); (2) if f (y σ j e j ) < f (y), order y = y σ j e j, Otherwise order y = y; The sixth step, order P g +1 = y, if f (P g +1 ) < f (P g ), Dea with P g +1 aong the direction of the acceeration p = P g +1 P g mode of mobie, order y = P g +1 + γ p, σ +1 = σ, = + 1, turn the fifth step, or the seventh step;the seventh step, if σ ε, then stop the iteration, and then output resuts. Otherwise P g +1 P g, order y = P g +1, σ +1 = σ, = + 1, turn to fifth step; when P g +1 = P g, Mae y = P g +1, σ +1 = θσ, = + 1, turn fifth step; If the maximum number of iterations σ > ε, turn to the second step. 4. SVM agorithm and its customers Cassification appication 4.1 hospita customers data processing The hospita customer data with high dimension and noninear characteristics, dimensions, and its incuded noise wi reduce the performance of the cassifier, and the characteristics of the experimenta data sets, the hospita customer cassification mode based on support vector machine and the waveet transform fitering noise reduction processing on the data set. Lead to the curse of dimensionaity, in order to avoid the T i Journa of Digita Information Management Voume 11 Number 2 Apri

4 high-dimensiona data to support vector machines bring arge computing and high data dimensionaity reduction using ocay inear embedding method. 4.2 PSO mode optimized SVM parameters appication Sigmoid erne function K (x, x i ) = tanh(γ (x x i )) + coef ) is seected as the erne function support vector cassifier, erne function σ is width coefficient. Initiaization setting corresponding parameter range is: σ [0, 2], c [1, 1000]. In the experiment, using PSO agorithm, pattern search agorithm and PSO pattern search agorithm for optimizing parameters and estabish cassification mode to cassify detection. Three agorithms of the earning time and accuracy was shown in tabe1, tabe 2. pattern search agorithm has the strong oca search abiity and convergence speed. PSO pattern search agorithm taes into account the groups history best position, enhancing the oca search abiity of the agorithm, in each iteration of the process, use a pattern search agorithm to optimize and upgrade, the better P g Lead the popuation to better search, improve search efficiency, thus to ensure that the PSO agorithm rapid convergence under the premise of not affecting P g strong goba search abiity. The use of PSO pattern search agorithm is to find optima parameters, the optima parameters are taen around in other parameters are accurate, under the combination of different parameters correspond to the accuracy are shown in Tabe 3, in which the coordinates of the vaue. PSO Pattern search agorithm PSO pattern search agorithm Tabe 1. Study time of three agorithms (min) PSO Pattern search agorithm PSO pattern search agorithm Tabe 2.The detection accuracy rate of three inds of agorithm ( % ) σ c Tabe 3. c and σ Under different combinations of accurate rate curve From tabe 1 and tabe 2 shows, the PSO agorithm training time has ess training time than the pattern search agorithm, but the PSO agorithm earning accuracy without pattern search agorithms. Whie PSO pattern search agorithm on the training time and the accuracy is better than PSO agorithm and pattern search agorithm, it reason is in the parameter optimization process, the PSO agorithm at each iteration is a partice position and veocity of the update, the agorithm increases, the search speed rein in, especiay when the partice size is reativey arge, its search speed wi be sower. Whie the As can be seen from the figure, in the combinations of parameters of the vicinity of the optima combination of parameters aso reaches a high cassification accuracy rate, but did not use PSO mode search agorithm to find the optima parameters to achieve high cassification accuracy. The one hand show that the seection of parameters a great infuence on the performance of the cassifier; the other hand indicate that the PSO mode search agorithm to find the optima parameters is an effective method. 168 Journa of Digita Information Management Voume 11 Number 2 Apri 2013

5 5. Concusion Hospita cients cassified appications of PSO agorithm, pattern search agorithm and PSO mode search agorithm, the earning time and the accuracy of the simuation anaysis and comparison. The resuts found that: PSO pattern search agorithm is appied to the parameter optimization of support vector machine, not ony to avoid the imitations of reying on the experience of seected parameters, but aso shorten the training time of support vector machine, but aso improve the accuracy. Instances prove PSO mode search agorithm is an efficient, accurate parameter optimization method. In addition, the seection of the erne function of the standard has not been standardized, further research direction for the seection of different erne function of the PSO mode search agorithm anaysis. References [1]Peng, Bai., Xi bin, Zhang., Bin, Zhanget a. (2008). The theory of support vector machine and its appications. Xi an: Xi an Eectronic and Science University press. [2] Naiyang, Deng., Yingjie,Tian.(2009). Support vector machine theory, agorithm and deveopment. Beijing: Science Press. [3] Gang, Li., Yu, Wang., Sheng, Zhang, J. ( 2006). Support vector machine in the computer signa cassification using. Computer appication, 26, 6. [4] Chapee, O., Vapni, V., J. (2002). Choosing mutipe parameters for support vector machines Machine Learning, 46. [5] Liang, Qi J. (2008). Ant coony agorithm based on support vector machine SVM parameter seection method. 4,1. [6] Xing ing, Wang, J. ( 2005). Car occupies binsupport vector machines based on the erne parameters Journa of Ocean University of China, 5, 35. [7] Xiaoyun, Zhang., Yuncai, Liu., Gauss, J. (2003). Nucear performance anaysis of support vector machines. Computer Engineering, 29, 8. [8] Wu, Kuoping., Wang, Shengde. (2009). Choosing the erne parameters for support vector machines by the intercuster distance in the feature space. Pattern Recognition, 42, 5. [9] Jia, Wang., Xu.Yuhong, J. (2011).Based on momentum partice swarm hybrid nucear SVM parameter optimization method. Computer appication, 2, 31. Author biographies YAN Hui-feng received a Bacheor and Master degree in computer science from the ChongQing University in China. He is a teacher in Coege of Mobie Teecommunications ChongQing University of Posts and Teecom, His main research interests is in the fied of machine earning, data mining and Pattern Recognition. He has more than 10 pubications to his credit in internationa journas and conferences. WANG Wei-feng received the Mechanica and eectronic engineering master s degree from Changchun University of Science and Technoogy, Changchun,Jiin,China,in 2003.His research interestes incude Pattern recognition and artificia inteigence and Routing protoco for Wireess Sensor Networs. LIU jie received a Bacheor and Master degree in Architecture from the ChongQing University in China. She is a teacher in Schoo Of Civi Engineering & Architecture, Chongqing Jiaotong Univ. Her main research interests is in the fied of Digitized inteigent buiding, data mining, inteigent contro. She has more than 8 pubications to his credit in internationa journas and conferences. Journa of Digita Information Management Voume 11 Number 2 Apri