Classification Of Heart Disease Using Svm And ANN

Transcription

1 fcaton Of Heart Dsease Usng Svm And ANN Deept Vadcherla 1, Sheetal Sonawane 2 1 Department of Computer Engneerng, Pune Insttute of Computer and Technology, Unversty of Pune, Pune, Inda deept.vadcherla@gmal.com 2 Department of Computer Engneerng, Pune Insttute of Computer and Technology, Unversty of Pune, Pune, Inda sssonawane@pct.edu Abstract: fcaton of heart dsease can be useful for the physcans f t s computerzed for the purpose of fast dagnoss and accurate result. Predctng the exstence of heart dsease accurately can save patents lfe. The objectve of ths paper s to analyze the applcaton of AI tools for classfcaton and predcton of heart dsease. The work ncludes the classfcaton of heart dsease usng Support Vector Machne and Artfcal Neural Network. Comparson s carred out among two methods on the bass of accuracy and tranng tme. Ths paper presents a Medcal decson support system for heart dsease classfcaton n a ratonal, objectve, accurate and fast way. The dataset used s the Cleveland Heart Database taken from UCI learnng data set repostory. In the proposed model we classfy the data nto two classes usng SMO algorthm n Support Vector machne and Artfcal Neural Network (ANN). Keywords: Support Vector Machne, Sequental Mnmal Optmzaton,Optmzaton problem, Heart dsease, Artfcal Neural Network. I. INTRODUCTION At present, the number of people sufferng from heart dsease s on a rse. Accurate dagnoss at an early stage followed by proper subsequent treatment can result n sgnfcant lfe savng. New data released by the Natonal Heart, Lung, and Blood Insttute (NHLBI) of the show that especally women n older age groups are more at rsk of gettng heart dsease. A recent study felded Heart dsease can be controlled effectvely f t s dagnosed at an early stage [24]. But ts not easy to do accurate dagnoss because of many complcated factors of heart dseases. For example, many clncal symptoms are assocated wth many human organs other than the heart and very often heart dseases may exhbt varous syndromes. Due to ths complexty, there s a need to automate the process of medcal dagnoss whch can help medcal practtoners n the dagnostc process [1], [2]. To reduce the dagnoss tme and mprove the dagnoss accuracy, t has become more of a demandng ssue to develop relable and powerful medcal decson support systems to support the dagnoss decson process. Bascally medcal dagnoss s complcated process, hence the approach for solvng ths ssue, s to develop such an ntellgent system, such as Support Vector Machne and Artfcal Neural Network[4],[5]. It has shown great potental to be appled n the desgn and mplementaton of decson support system of heart dsease. The system uses features extracted from the ECG data of the patents. The experments however, have been performed takng the Cleveland Heart Database taken from UCI learnng data set repostory whch was donated by Detrano[23]. Results obtaned from support vector machne model are satsfactory. Ths paper presents a medcal decson support system for heart dsease classfcaton. In the proposed model we classfy the data nto two classes usng Support Vector machne and Artfcal Neural Network [21], [22]. The rest of the paper organzed as, support vector machne descrbed n secton 2. Secton 3 ncludes artfcal neural network, n whch functonng of ANN s explaned. Proposed model of MDSS and related work s mentoned and explaned n secton 4. Experments and results are shown n secton 5. Secton 6 has concluson followed by the future work n secton 7. Page 694

2 II. SUPPORT VECTOR MACHINE Support Vector Machne, s a promsng method of learnng machne, based on statstcal learnng theory developed by Vladmr Vapnk. Support vector machne (SVM) used for the classfcaton of both lnear and nonlnear data [6], [7]. It performs classfcaton by constructng a lnear optmal separatng hyperplane wthn hgher dmenson, wth the help of support vectors and margns, whch separates the data nto two categores (or classes). Wth an approprate nonlnear mappng the orgnal tranng data s mapped nto a hgher dmenson. Wthn ths the data from two classes can always be separated by a hyperplane[8]. Relevant mathematcs assocated wth the project s as gven below. Let S represents Medcal decson support system. Ths system provdes classfcaton by two methods, one s SVM and another s ANN. It can be shown n followng form, S= { SVM, ANN} Suppose f s a functon for Support vector machne, then, f: IO where, I s doman ( set of nputs) I= {D, E} D= { X, Y} X= {x 1<=<=n} Y= {y 1<=<=n}.e. D= { (x, y ) (X Y) } 1 E (set of constants) = {C, e} O s co doman (set of output) O = { op 1<=<=n} The support vector machne computes a lnear classfer of the form, Where, W s weght vector X s nput vector B s bas f(x) = WX + b The separatng hyperplane s the plane, f(x) = 0. n Therefore we can say that, any pont from one class les above the separatng hyperplane satsfes, f(x) > 0. In the same way any pont from another class les below the separatng hyperplane satsfes, f(x) < 0. Above equatons were processed to make the lnearly separable set D to meet the followng nequalty, y ( f(x) ) 1, Here the margn m s, 1 m w 2 Usng above equaton, maxmzng margn can be wrtten n the form of optmzaton problem as below: 1 2 mn w Subject to y w. x b 1, 2 w, b Ths optmzaton problem can be solved by usng dual Lagrange multpler, N N N 1 mn ( ) mn y y ( x x ), 2 j j j 1 j1 1 The output of a non-lnear SVM s explctly computed from the Lagrange multplers [17, 9], N u y K( x, x) b, j1 j j where K s a kernel functon. We used Radal Bass Kernel Functon (RBF) [10] here, whch s denoted as follow: K(x, x j ) = exp(-γ x x j 2 ), γ>0 The non-lnearty alter the quadratc form, but the dual objectve functon s stll quadratc n α, N N N 1 mn ( ) mn y y K ( x, x ), 0 C,, y 0. j 2 j j j 1 j 1 1 N 1 Sequental mnmal optmzaton algorthm solves above quadratc programmng problem by repeatedly fndng two Lagrange multplers that can be optmzed wth respect to each other. Sequental mnmal optmzaton (SMO) [5] s an algorthm for effcently solvng the optmzaton problem whch arses durng the tranng of support vector machne. At every step, SMO chooses two Lagrange multplers to jontly optmze, fnds the optmal values for these multplers and updates the SVM to reflect the new optmal values [15]. Functonng of SMO as n the below algorthm. Page 695

3 SMO tranng algorthm: Step 1: Input C, kernel, kernel parameters, and epslon. Step 2: Intalze α = 0 and b= 0 Step 3: Let f(x) = b+ and τ the tolerance. Step 4: Fnd Lagrange multpler α, whch volates KKT optmzaton. Step 5: Choose second multpler and optmze par. Repeat steps 4 and 5 tll convergence. Step 6: Update α 1 and α 2 n one step. α 1 can be changed to ncrease f(x1). α 2 can be changed to decrease f(x2). Step 7: Compute new bas weght b. }whle(tranng error > 0.01 && epoch <25000); Step 6: Compute the tranng accuracy. Step 7: Use ths traned ANNnetwork for testng. Suppose g s a functon for Artfcal Neural Network, then, g: MN where, M s doman ( set of nputs) M= {A, B} A= { X, Y} A= { (x, y ) (X Y) } 1 P III. ARTIFICIAL NEURAL NETWORK An artfcal neural network s a computatonal model based on the structure and functons of bologcal neural networks. Informaton that flows through the network affects the structure of the ANN because a neural network changes, n a sense based on that nput and output. ANNs are consdered nonlnear statstcal data modelng tools where the complex relatonshps between nputs and outputs are modeled or patterns are found.tranng a neural network model essentally means selectng one model from the set of allowed models that mnmzes the cost crteron. There are numerous algorthms avalable for tranng neural network models; most of them can be vewed as a straghtforward applcaton of optmzaton theory and statstcal estmaton. In the proposed model Resllent Backpropogaton algorthm s used for tranng ANN. fcaton algorthm of ANN: Step 1: Intalze the tranng class buffers, nput data buffers, Declare BascNetwork ANNnetwork n ENCOG. Step 2: Extract nput data and update as, d= Input Layers, out= Output Read, Hdden layers = 2d -1. Step 3: Intalze trangset wth InputVal[] and OutputVal[]. Step 4: Use Resllent Propagaton for tranng of neural network. Step 5: epoch = 1; do{ tran the ANNnetork. Epoch++; B={ w j, Δ j (t), E, η, epochs} Let w j =weghts Δ j (t) = ndvdual update value Δ j (t) exclusvely determnes the magntude of the weght-update. Ths update value can be expressed mathematcally accordng to the learnng rule for each case based on the observed behavor of the partal dervatve durng two successve weght-steps by the followng formula: where, 0 < η - < 1 < η +. Whenever the partal dervatve of the equvalent weght wj vares ts sgn, t ndcates that the last update was large n magntude and the algorthm has skpped over a local mnma then, Δ j (t) = Δ j (t) - η Otherwse, the update-value wll do some extent ncrease. If the dervatve s postve, the weght s decreased by ts update value, f the dervatve s negatve, the Page 696

4 update-value s added as shown below: w j (t+1) = w j (t) + Δw j (t) However, there s one excepton. If the partal dervatve changes sgn that s the prevous step was too large and the mnmum was mssed, the prevous weght-update s reverted: Δw j (t) = - Δw j (t-1), f To avod a double penalty of the update-value, set the above update rule by puttng below value n Δ j. The partal dervatve of the total error s gven by the followng formula: Ths ndcates that the weghts are updated only after the presentaton of all of the tranng patterns. Reslent back-propagaton (RPROP) tranng algorthm [21] was adopted to tran the proposed ANN model as mentoned prevously. After the selecton of network, the network has been traned usng reslent backpropagaton tranng scheme. The tranng parameters have been modfed several tmes as explaned above untl the optmum performance has been acheved. Maxmum number of teratons has been set to epochs. IV. PROPOSED MODEL OF MEDICAL DECISION SUPPORT SYSTEM result n speedng up the computaton process. They have hgh tolerance of nosy data. The major dsadvantage of neural networks s that, they have poor nterpretablty. Fully connected networks are dffcult to artculate. Whereas varous emprcal studes of Bayesan classfer n comparson wth decson tree and neural network classfers have found out that, n theoretcal way Bayesan classfers have mnmum error rate n comparson to all other classfers. However, n practce ths s not always the case, owng to naccuraces n the assumptons made for ts use, such as class condtonal ndependence and the lack of avalable probablty data. 2. Pre-processed data The experments are carred out on heart dataset usng Sequental Mnmal Optmzaton n Support Vector Machne. The experments are carred out on heart dataset usng Sequental Mnmal Optmzaton n Support Vector Machne. Heart dsease s dagnosed wth the help of some complex pathologcal data. The heart dsease dataset used n ths experment s the Cleveland Heart Dsease database taken from UCI machne learnng dataset repostory [23]. Ths database contans 14 attrbutes as below: 1. Age of patent, 2. Sex of patent, 3. Chest pan type, 4. Restng blood pressure, 5. Serum cholesterol, 6. Fastng blood sugar, 7. Restng ECG results, 8. Maxmum heart rate acheved, 9. Exercse nduced angna, 10. ST depresson nduced by exercse relatve to rest, 11. Slope of the peak exercse ST segment, 12. number of major vessels colored by flourosopy, 13. thal, 14. Dagnoss of heart dsease. 3. Flow dagram of MDSS The purpose of ths proposed model s to dagnose the heart dsease by classfyng the dataset of heart dsease. Ths classfcaton process s shown n Fgure Related work Medcal decson support system work has been carred on the bass of performance of dfferent methods lke SVM[20], Artfcal neural network, Bayesan classfcaton method, etc. [1], [2]. Neural network algorthms are nherently parallel, whch Page 697

5 mnmal optmzaton n Support vector machne s more effectve than Reslent backpropagaton n Artfcal neural network. The proposed system s tested wth many datasets. The expermental results are shown n the followng fgures. Fgure 1. Flow dagram of MDSS for heart dsease V. RESULT AND DISCUSSION In the proposed model, we used dataset havng 297 total number of patent records. Large part of records n the dataset s used for tranng and rest of them are used for testng. The man dfference between the dataset gven as nput to tranng and testng s that, the nput we are gvng to tranng s the data wth correct dagnoss (14 th feld n the dataset) and whereas the nput data of testng doesn t have the correct dagnoss purposely. The Dagnoss (14 th ) feld refers to the presence or absence of heart dsease of that respectve patent. It s nteger valued feld, havng value 1(absence of dsease) or -1(presence of dsease). So that at the end of testng process we can check the result n the output fle created after testng and verfy the effcency of the proposed model n terms of accuracy. In the proposed system two methods of classfcaton are provded, Support vector machne and Artfcal neural network. Performance of both the classfcaton technques s compared n terms of tme needed for classfcaton and accuracy of the system. From the below analyss we can say that, Sequental Fgure 2: Pe chart of multclass SVM classfcaton 1. Multclass classfcaton by usng Support Vector Machne Followng Table1 gves the result of multclass classfcaton of SVM. Fgure 2 of pe chart shows the performance of the system wth ffth sample from table1, whch gves 100% accuracy. 2. Multclass classfcaton by usng Artfcal neural network Followng analyss done wth, Input Layer = 13, Hdden Layer = 25 and Output Layer = 1. In ths case tranng s done tll error becomes less than or epochs are less than Table2 and Fgure 3 gves the result of ANN multclass classfcaton. Followng Fgure 3 of pe chart shows the performance of ffth sample from table 2 whch gves 65% accuracy. Page 698

6 Fgure 3: Pe chart of multclass ANN classfcaton No. of samples I II III IV V Accuracy Tme Testng Accuracy- (10 samples) No. of samples I Table 1: Performance of the multclass SVM decson support system II III IV V Tranng Accuracy Tme Testng Accuracy Table 2: Performance of the multclass ANN decson support system Page 699

7 VI. CONCLUSION The results of the SVM classfcaton algorthm compared to the ANN classfcaton, are very encouragng. The dfference n the accuracy s notceable. Moreover the dfference n the executon tmes s even more noteworthy. The enhanced performance of the SVM classfcaton s due to the fact that they can avod repettve searches n order to fnd the best two ponts to use for each optmzaton step. It s found that SMO performs better wth hgh accuracy when the data s preprocessed and gven as nput. Appled to the task of solvng classfcaton problem of heart dsease and the features extracted based on statstcal propertes, the accuracy s hgher n proposed SVM classfcaton model whch uses SMO. VII. FUTURE WORK SMO s a carefully organzed algorthm whch has excellent computatonal effcency. However, because of ts way of computng and use of a sngle threshold value t can become neffcent. In future multple threshold parameters can be used to mprove the performance n terms of speed. In case of artfcal neural network, wndow momentum s a standard technque that can be used to speed up convergence and mantan generalzaton performance. Wndow momentum can gve sgnfcant speed-up over a set of applcatons wth same or mproved accuracy. REFERENCES [1] Long Wan, Wenxng Bao, Research and Applcaton of Anmal Dsease Intellgent Dagnoss Based on Support Vector Machne IEEE Internatonal Conference on Computatonal Intellgence and Securty, Pages 66-70, [2] S.N. Deepa, B. Aruna Dev, Neural Networks and SMO based fcaton for Bran Tumor, IEEE World Congress on Informaton and Communcaton Technologes, Pages ,2010. [3] C. Cortes and V. Vapnk, Support-vector network, Machne Learnng, vol.20, [4] S. S. Keerth, S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy Improvements to platt s smo algorthm for svm classfer desgn, Neural Computaton, vol.13,pp: , [5] John C. Platt, Sequental Mnmal Optmzaton: A Fast Algorthm for Tranng Support Vector Machnes, Mcrosoft Research, Techncal Report MSR-TR [6] Gong We Wang Shoubn, Support Vector Machne for Assstant Clncal Dagnoss of Cardac Dsease, IEEE Global Congress on Intellgent Systems, pp: , [7] Ya Gao; Shlang Sun, An emprcal evaluaton of lnear and nonlnear kernels for text classfcaton usng Support Vector Machnes, IEEE Seventh Internatonal Conference on Fuzzy Systems and Knowledge Dscovery (FSKD), Pages: , [8] Qnghua Jang, Guohua Wang, Tanjao Zhang, Yadong Wang, Predctng Human mcrorna-dsease Assocatons Based on Support Vector Machne, IEEE Internatonal Conference on Bonformatcs and Bomedcne, pp: , [9] Browne, K.E.; Burkholder, R.J., Nonlnear Optmzaton of Radar Images From a Through-Wall Sensng System va the Lagrange Multpler Method, IEEE Geoscence and Remote Sensng Letters, Pages: , [10] Gao Daq; Zhang Tao, Support vector machne classfers usng RBF kernels wth clusterng-based centers and wdths IEEE Intrnatonal Jont Conference on Neural Networks, 2007, Pages: [11] Olv L.Mangasaran and Mchael E.Thompson, Massve Data fcaton va Unconstrned Support Vector Machnes Journal of Optmzaton Theory and Applcatons, 131, Pages , [12] P. S. Bradley and O. L. Mangasaran. Massve data dscrmnaton va lnear support vector machnes. Optmzaton Methods and Software, 13:1-10, 2000.ftp://ftp.cs.wsc.edu/math-prog/techreports/98-05.ps [13] Osuna, E., Freund, R., Gros, F., Improved Tranng Algorthm for Support Vector Machnes, Proc. IEEE NNSP 97, Page 700

8 [14] Yqang Zhan, Dnggang Shen, Desgn effcent support vector machne for fast classfcaton, The journal of pattern recognton socety 38, Pages ,2004. [15] Peng Peng, Qan-L Ma, Le-Mng Hong, The Research of the Parallel SMO algorthm for solvng SVM, Proceedngs of the Eghth Internatonal Conference on Machne Learnng and Cybernetcs, Baodng, pp: , July [16] Chn-Jen Ln, Asymptotc convergence of an SMO algorthm wthout any assumptons, IEEE Transactons on Neural Networks, vol.13,ssue 1, pp: , [17] Baxter Tyson Smth, Lagrange Multplers Tutoral n the Context of Support Vector Machnes, Memoral Unversty of Newfoundland St. John s, Newfoundland, Canada. [18] Ya-Zhou Lu; Hong-Xun Yao; Wen Gao; De-bn Zhao, Sngle sequental mnmal optmzaton: an mproved SVMs tranng algorthm, Proceedngs of 2005 Internatonal Conference on Machne Learnng and Cybernetcs, Pages [19] Kjeldsen, Tnne Hoff. "A contextualzed hstorcal analyss of the Kuhn-Tucker theorem n nonlnear programmng: the mpact of World War II". Hstora Math. 27 (2000), no. 4, [20] Deept Vadcherla, Sheetal Sonawane, Decson support system for heart dsease based on sequental mnmal optmzaton n support vector machne Internatonal Journal of Engneerng Scences and Emergng Technologes, Volume 4, Issue 2, Pages: 19-26, [21] Ian H. Wtten, Ebe Frank, Data Mnng, Elsever Inc [22] Jawe Han and Mchelne Kamber, Data Mnng: Concepts and Technques, 2/e, Morgan Kaufmann Publshers, Elsever Inc [23] UCI Machne Learnng Repostory: Heart Dsease Data Set. Dsease [24] Medlne Plus: Informaton related to Heart Dseases. es.html Page 701