An Approach for Building Intrusion Detection System by Using Data Mining Techniques

Transcription

1 Internatonal Journal of Emergng Engneerng Research and Technology Volume, Issue, May 04, PP -8 An Approach for Buldng Intruson Detecton System by Usng Data Mnng Technques Praveen P Nak PG Student, Department of CS&E, AIT, Chkmagalur, Karnataka, Inda Prashantha S J Asst Professor, Department of CS&E, AIT, Chkmagalur, Karnataka, Inda Abstract: Informaton securty s one of the cornerstones of Informaton Socety. Integrty and prvacy of fnancal transactons, personal nformaton and crtcal nfrastructure data, all depend on the avalablty of strong and trustworthy securty mechansms. In recent years, many researchers are usng data mnng technques for buldng IDS. Here, we propose a new approach by utlzng data mnng technques such as neuro-fuzzy and radal bass support vector machne (SVM) for helpng IDS to attan hgher detecton rate. The proposed technque has four maor steps: prmarly, k-means clusterng s used to generate dfferent tranng subsets. Then, based on the obtaned tranng subsets, dfferent neuro-fuzzy models are traned. Subsequently, a vector for SVM classfcaton s formed and n the end, classfcaton usng radal SVM s performed to detect ntruson has happened or not. To llustrate the applcablty and capablty of the new approach, the results of eperments on KDD CUP 999 dataset s demonstrated. Epermental results shows that our proposed new approach do better than Condtonal random felds (CRF) wth respect to specfcty and detecton accuracy. Keywords: Intruson Detecton System (IDS), K-Means, Neuro-fuzzy, SVM, CRF, Data Mnng.. INTRODUCTION An Intruson Detecton System (IDS) s a devce (or applcaton) that montors network and/or system actvtes for malcous actvtes or polcy volatons and produces reports to a Management Staton. Intruson detecton s the process of montorng the events occurrng n a computer system or network and analyzng them for sgns of possble ncdents, whch are volatons or mmnent threats of volaton of computer securty polces, acceptable use polces, or standard securty practces. Intruson Detecton Systems have undergone rapd growth n power, scope and complety n ther short hstory. In recent years, Intruson detecton system has been one of the most sought after research topcs n the feld of Informaton Securty havng huge applcatons n the cooperate world where data ntegrty and securty s a comple ssue. When an ntruder attempts to break nto an nformaton system or performs an acton not legally allowed, we refer to ths actvty as an Intruson. Intruders may be eternal or nternal dependng upon the authorzaton level. Intruson technques may nclude eplotng software bugs or system confguratons, password crackng, snffng unsecured traffc, or eplotng the desgn flaw of specfc protocols. An Intruson Detecton System (IDS) s a system for detectng ntrusons and reportng them accurately to the proper authorty. IDSs are usually specfc to the operatng system that they operate n and are an mportant tool n the overall mplementaton of an organzaton s nformaton securty polcy, whch reflects an organzaton's statement by defnng the rules and practces to provde securty, handle ntrusons, and recover from damage caused by securty breaches.. LITERATURE SURVEY In ths secton, related lterature about machne learnng approach and preparaton of datasets for data mnng actvty wll be revewed and dscussed. Anne George [3], Anomaly detecton has emerged as an mportant technque n many applcaton areas manly for network securty. Anomaly detecton based on machne learnng algorthms consdered as the classfcaton problem on the network data has been presented here. Dmensonalty reducton and classfcaton algorthms are eplored and evaluated usng KDD99 dataset for network IDS. Prncpal Component Analyss for dmensonalty reducton and Support Vector Machne for classfcaton have been consdered for the applcaton on network data and the results are analyzed. The result shows the IJEERT

2 Praveen P Nak & Prashantha S J decrease n eecuton tme for the classfcaton as they reduce the dmenson of the nput data and also the precson and recall parameter values of the classfcaton algorthm shows that the SVM wth PCA method s more accurate as the number of msclassfcaton decreases. W.K. Lee, S.J.Stolfo [4], there s often the need to update an nstalled ntruson detecton system (IDS) due to new attack methods or upgraded computng envronments. Snce many current IDSs are constructed by manual encodng of epert knowledge, changes to IDSs are epensve and slow. Ths paper descrbes a data mnng framework for adaptvely buldng Intruson Detecton (ID) models. The central dea s to utlze audtng programs to etract an etensve set of features that descrbe each network connecton or host sesson, and apply data mnng programs to learn rules that accurately capture the behavor of ntrusons and normal actvtes. These rules can then be used for msuse detecton and anomaly detecton. New detecton models are ncorporated nto an estng IDS through a meta-learnng (or cooperatve learnng) process, whch produces a meta detecton model that combnes evdence from multple models. We dscuss the strengths of our data mnng programs, namely, classfcaton, meta-learnng, assocaton rules, and frequent epsodes. We report on the results of applyng these programs to the etensvely gathered network audt data for the 998 DARPA Intruson Detecton Evaluaton Program V. Jyothsna, V. V. Rama Prasad, K. Munvara Prasad [5], Wth the advent of anomaly-based ntruson detecton systems, many approaches and technques have been developed to track novel attacks on the systems. Hgh detecton rate of 98% at a low alarm rate of % can be acheved by usng these technques. Though anomaly-based approaches are effcent, sgnature-based detecton s preferred for manstream mplementaton of ntruson detecton systems. As a varety of anomaly detecton technques were suggested, t s dffcult to compare the strengths, weaknesses of these methods. The reason why ndustres don t favor the anomaly-based ntruson detecton methods can be well understood by valdatng the effcences of the all the methods. To nvestgate ths ssue, the current state of the eperment practce n the feld of anomalybased ntruson detecton s revewed and survey recent studes n ths. Ths paper contans summarzaton study and dentfcaton of the drawbacks of formerly surveyed works. CHEN Bo, Ma Wu [6], the effectve way of mprovng the effcency of ntruson detecton s to reduce the heavy data process workload. In ths paper, the dmensonalty reducton use of technology n the classc dmensonalty reducton algorthm prncpal component to analyss large-scale data source for reducedmade features of the orgnal data be retaned and mproved the effcency of ntruson detecton. And use BP neural network tranng the data after dmensonalty reducton, wll be effectve n normal and abnormal data dstncton, and acheved good results. Paul Dokas, Vpn kumar [7], n whch they gves an overvew of our research n buldng rare class predcton models for dentfyng known ntrusons and ther varatons and anomaly/outler detecton schemes for detectng novel attacks whose nature s unknown. Dsadvantage of ths paper s that due to the fact that the number of nstances of UR and RL attacks n the tranng data set s very low, these numbers are not adequate as a standard performance measure. It could be based f we use these numbers as a measure for performance of the system. 3. METHODOLOGY Fg. Archtecture for Proposed IDS Internatonal Journal of Emergng Engneerng Research and Technology 3

3 An Approach for Buldng Intruson Detecton System by Usng Data Mnng Technques The system archtecture of proposed technque as shown n the fg conssts of 5 step methodology. Ths can be eplaned as follows.. The nput data set DS needed for epermentaton s prepared by conductng relevance analyss on KDD Cup 999 data set n order to reduce the rrelevant attrbutes / features whch wll not contrbute for ntruson detecton.. The nput dataset s dvded nto Tranng Data set and Testng Data set. The Tranng data s clustered usng K- Means Clusterng nto k subsets where, k s the number of clusters desred. 3. Neuro-fuzzy (FNN) tranng s gven to each of the k cluster, where each of the data n a partcular cluster s traned wth the respectve neural network assocated wth each of the cluster. 4. Generaton of vector for SVM classfcaton, S={D, D,.D N } whch conssts of attrbute values obtaned by passng each of the data through all of the traned Neuro-fuzzy classfers, and an addtonal attrbute µ whch has membershp value of each of the data. 5. Classfcaton usng SVM to detect ntruson has happened or not. The detaled descrpton of each of the steps s elaborated n the followng sub-sectons. 3. Data Collecton Ths secton, t gves an overvew of the data set used for ntruson detecton. Ths data set contans seven weeks of tranng data and two weeks of testng data. The raw data was about four ggabytes of compressed bnary TCP dump data from the of network traffc generated. Ths was processed nto about fve mllon connecton records, each of whch s a vector of etracted feature values of that network connecton. As we know, a connecton s a sequence of TCP packets to and from some IP addresses, startng and endng at some well defned tmes. Ths data set of the fve mllon connecton records was used as the data set for the 999 KDD ntruson detecton contest and s called the KDD Cup 99 data. In partcular, MIT Lncoln Lab s DARPA ntruson detecton evaluaton datasets have been employed to desgn and test ntruson detecton systems. In 999, recorded network traffc from the DARPA 98 Lncoln Lab dataset was summarzed nto network connectons wth 4- features per connecton. Ths formed the KDD 99 ntruson detecton benchmark n the Internatonal Knowledge Dscovery and Data Mnng Tools Competton. The KDD 99 ntruson detecton datasets are based on the 998 DARPA ntatve, whch provdes desgners of ntruson detecton systems (IDS) wth a benchmark on whch to evaluate dfferent methodologes [3]. To do so, a smulaton s made of a facttous mltary network consstng of three target machnes runnng varous operatng systems and servces. Addtonal three machnes are then used to spoof dfferent IP addresses to generate traffc. Fnally, there s a snffer that records all network traffc usng the TCP dump format. The total smulated perod s seven weeks. Each connecton was labeled as normal or as eactly one specfc knd of attack. All labels are assumed to be correct. There were a total of 37 attack types n the data set. The smulated attacks fell n eactly one of the four categores : User to Root; Remote to Local; Denal of Servce; and Probe. Denal of Servce (dos): Attacker tres to prevent legtmate users from usng a servce. Remote to Local (rl): Attacker does not have an account on the vctm machne, hence tres to gan access. User to Root (ur): Attacker has local access to the vctm machne and tres to gan super user prvleges. Probe: Attacker tres to gan nformaton about the target host. Data preprocessng comprses followng components ncludng document converson, feature selecton and feature weghtng. The functonalty of each component s descrbed as follows: [] Dataset prepared wth DOS attack whch nclude smurf, Neptune, back, teardrop and POD png of death attacks /anomaly. [] Feature selecton reduces the dmensonalty of the data space by removng rrelevant or less relevant feature selecton crteron. [3] Document converson- converts dfferent types of documents such as gz, tcpdump to csv fle and arff (Attrbute-Relaton Fle Format) data fle format. Internatonal Journal of Emergng Engneerng Research and Technology 4

4 Praveen P Nak & Prashantha S J [4] Totally we consdered 850 data ponts for our epermentaton. 3. K-Means Clusterng K-means s one of the smplest unsupervsed learnng algorthms that solve the clusterng problem. The obectve s to classfy a gven data set nto a certan number of clusters (assume ntal clusters) fed a pror. The pseudo code for the adapted K-Mean algorthm s presented as below. Choose random k data ponts as ntal Clusters Mean (Cluster center). Repeat 3. for each data pont from D 4. Computer the dstance and each cluster mean (centrod) 5. Assgn to the nearest cluster. 6. End for 7. Re-compute the mean for current cluster collectons. 8. Untl reachng stable cluster 9. Use these centrod for normal and anomaly traffc. 0. Calculate dstance of centrod from normal and anomaly centrod ponts.. If dstance(x, D) > = 5. Then anomaly found ; et 3. Else then 4. X s normal; The k-means clusterng algorthm s based on fndng data clusters n a data set by keepng mnmzed cost functon of dssmlarty measure. In most cases ths dssmlarty measure s chosen as the Eucldean dstance. For each data pont to be clustered, the cluster centrod wth the mnmal Eucldean dstance from the data pont wll be the cluster for whch the data pont wll be a member. K n ( ) C 3.3 Artfcal Neural Network ANN s a bologcally nspred form of dstrbuted Computaton. It s composed of smple processng unts, or nodes, and connectons between them. The connecton between any two unts has some weght, whch s used to determne how much one unt wll affect the other. A subset of the unts acts as Input nodes and another subset acts as output nodes, whch perform summaton and threshold. The ANN has successfully been appled n dfferent felds. The feed-forward neural network traned wth the back-propagaton algorthm s a common tool for ntruson detecton. ANN module ams to learn the pattern of every subset. ANN s a bologcally nspred form of dstrbuted computaton. It s composed of smple processng unts, and connectons between them. In ths study, we wll employ classc feed-forward neural networks traned wth the back-propagaton algorthm to predct ntruson. Fg.. Neuro-fuzzy archtecture A feed-forward neural network has an nput layer, an output layer, wth one or more hdden layers n between the nput and output layer. The ANN functons as follows: each node n the nput layer has a sgnal as network s nput, multpled by a weght. Classfcaton of the data pont consderng all ts attrbutes s a very dffcult task and takes much tme for the processng, hence decreasng the number of attrbutes related wth each of the data pont s of paramount mportance. Eecutng the reduced amount of data also results n decrease of error rate and the mproved performance of the classfer system. The man purpose of the proposed technque s to decrease the number of attrbutes assocated Internatonal Journal of Emergng Engneerng Research and Technology 5

5 An Approach for Buldng Intruson Detecton System by Usng Data Mnng Technques wth each data, so that classfcaton can be made n a smpler and easer way. Neuro-fuzzy classfer s employed to effcently decrease the number of attrbutes. 3.4 Radal SVM Classfcaton In our system, we are employng radal SVM for the fnal classfcaton for the ntruson Detecton. SVM s used as t acheves enhanced results when contrasted to other classfcaton Technques especally when t comes to bnary classfcaton. In the fnal classfcaton, the data s bnary classfed to detect ntruson or not. The nput data s traned wth neuro-fuzzy after the ntal clusterng as we have dscussed earler, then the vector necessary for the SVM s generated. Here n the process, each of the data s fed nto each of the neural classfer to get the output value. That s each of the data s fed nto K number of neuro-fuzzy classfers to yeld K output values. So the data values gets dstorted and after passng through the K neuro-fuzzy classfers, attrbute number of the data n consderaton changes and dmnshes to K numbers where each value wll be the output of the data passng through the respectve neurofuzzy. The vector array S= {D, D,...D N } where, D s the th data and N s a total number of nput data. Here, after tranng through the neurofuzzy the attrbute number reduces to k numbers. D = {a,a,..a k }, here the D s the th data governed by attrbute values a, where a wll have the value after passng through the th neuro-fuzzy. Total number of neuro-fuzzy classfers traned wll be K, correspondng to the K clusters formed after clusterng. we comprse a parameter known as membershp value. Incluson of the membershp value nto the attrbute lst results n a better performance of the classfer. Membershp value µ s defned by the equaton below. k c c c k m Hence, the SVM vector s modfed as S*={D*, D*,.D* N } where S* s the modfed SVM vector whch conssts of modfed data D* I, whch conssts of an etra attrbute of membershp value µ. D* = {a,a,..a k, µ }, Hence the attrbute number s reduced to K+ where K s the number of clusters. Ths results n smple processng n the fnal SVM classfcaton. Ths s due to the fact that nput data whch had 34 attrbutes s now constraned to K+.e. to 6 attrbutes. Ths also reduces the system complety and tme ncurred. Use of radal SVM results n obtanng better results from the classfcaton process when compared to normal lnear SVM. In lnear SVM, the classfcaton s made by use of lnear hyperplanes where as n radal SVM, nonlnear kernel functons are used and the resultng mamummargn hyper-plane fts n a transformed feature space. The correspondng feature space s a Hlbert space of nfnte dmensons, when the kernel used s a Gaussan radal bass functon. The Gaussan Radal Bascs functon s gven by the equaton: ep Where =,,.N. The th nput data pont defnes the center of radal bass functon, the vector s the pattern appled to the nput. σ s a measure of wdth of th Guassan functon wth center. 4. RESULTS AND DISCUSSION A. Screen Shots and Output Ths snap shot (fg 3) shows IDS mnng form n whch user has to open KDD 99 Data set. When the user clcked on perform k-means button then clusters wll be formed dsplayng table ncludng ID, attrbutes, and types of tranng data ponts. There wll be totally 5 number of clusters namely PROBE,DOS(denal of servce),rl(remote to local),ur(user to root),normal. After formaton of clusters, tranng and classfcaton of data ponts s easy and less tme consumng and less complety. Net we have to perform neuro-fuzzy operaton to tran our tranng data. fg 4 shows the neuro-fuzzy form. We consder only seven attrbutes namely servce, logn access, no. of tmes, dst_bytes, connecton status, duraton of connecton and src_bytes. Classfcaton of the data pont consderng all ts attrbutes s a very dffcult task and takes much tme for the processng, Internatonal Journal of Emergng Engneerng Research and Technology 6

6 Praveen P Nak & Prashantha S J hence decreasng the number of attrbutes related wth each of the data pont s mportant. Fg 3. Screen shot for IDS Mnng form At the end, radal SVM classfcaton s performed to detect ntruson happened or not. Ths phase ncludes SVM attrbute selecton step. At the end we wll get SVM Result set, whch shows ntrusons f happened wth ther ID, Attack types, Attack group and score. The measurement used for evaluaton of our proposed technques are True postve (TP), False negatve (FN), True negatve (TN), and False postve (FP). True Postve- A legtmate attack whch trggers IDS to produce an alarm. False Postve- An event sgnalng IDS to produce an alarm when no attack has taken place. False Negatve- A falure of IDS to detect an actual attack. True Negatve- When no attack has taken place and no alarm s rased. The graphcal representaton of comparson of Condtonal Random Felds (CRF) wth our proposed Technque s shown below n fg 5. Ths shows our proposed technque for buldng IDS by usng data mnng technques such as k- means clusterng, Neuro-fuzzy tranng and radal support vector machne (SVM) do better than CRF n terms of performance. Fg 4.. Screen shot for neuro-fuzzy form After performng normalzaton operaton, the net step s to perform neuro-fuzzy operaton as shown n the above fg 3. In whch we consder score, count and threshold values obtaned after normalzaton of the data ponts. Fg 6. Performance comparson chart for SVM and CRF Fg 5. Screen shot for SVM Classfcaton form 5. CONCLUSION In recent years, research on neural network methods and machne learnng technques to mprove the network securty by eamnng the behavor of the network as well as that of threats s done n the rapd force. The large volume of database s ncreasng rapdly resultng n gradual rse n the securty attacks. The current IDS s neffectve to update the audt data rapdly t nvolves human nterference thus Internatonal Journal of Emergng Engneerng Research and Technology 7

7 An Approach for Buldng Intruson Detecton System by Usng Data Mnng Technques reduces the performances. The paper elaborates the archtecture of the Intruson Detecton System along wth features of an deal ntruson detecton system. The study also descrbes the categorzaton and challenges f the IDS. In ths paper we analyzed the neural network approach and the machne learnng approach n overcomng the challenges of the IDS. Further there s need to desgn the system whch wll overcome the current challenges of IDS and also the system must provde a hgh performance n detectng the threats and securty attacks. Presently the applcaton support only 0% KDD CUP dataset. Ths applcaton can be etended to manage more number of records. REFERENCES [] A.M.Chandrasekhar and K.Raghuveer, Intruson Detecton Technque by usng K-means, Fuzzy Neural Network and SVM classfers, presented at Internatonal Conference on Computer Communcaton and Informatcs (ICCCI-03), Combatore, INDIA. [] Sandp Ashok Shvarkar, and Mnnath Raosaheb Bendre, Hybrd Approach for Intruson Detecton Usng Condtonal Random Felds, Internatonal Journal of Computer Technology and Electroncs Engneerng (IJCTEE) Volume, Issue 3. [3] Anne George, Anomaly Detecton based on Machne Learnng: Dmensonalty Reducton usng PCA and Classfcaton usng SVM, Internatonal Journal of Computer Applcatons ( ) Volume 47 No., June 0. [4] W.K. Lee, S.J.Stolfo, A data mnng framework for buldng ntruson detecton model, In: Gong L., Reter M.K. (eds.): Proceedngs of the IEEE Symposum on Securty and Prvacy. Oakland, CA: IEEE Computer Socety Press, pp.0~3, 999. [5] V. Jyothsna, V. V. Rama Prasad, K. Munvara Prasad, A Revew of Anomaly based Intruson Detecton Systems, Internatonal Journal of Computer Applcatons ( ) Volume 8 No.7, August 0. [6] CHEN Bo, Ma Wu, Research of Intruson Detecton based on Prncpal Components Analyss, Informaton Engneerng Insttute, Dalan Unversty, Chna, Second Internatonal Conference on Informaton and Computng Scence, 009. [7] Paul Dokas, Vpn kumar, Data Mnng for Network Intruson Detecton, Proceedng of NGDM., pp.-30, 00. [8] A.M.Chandrashekhar and K. Raghuveer. "Performance evaluaton of data clusterng technques usng KDD Cup-99 Intruson detecton data set Internatonal Journal of Informaton & Network Securty (IJINS), Vol., No.4, pp , 0. [9] Jose Vera, Fernando Morgado Das, Aleandre Mota. Neuro-Fuzzy Systems: A Survey. Proceedng Internal Conference on Neural Networks and Applcatons, 004. [0] Mahbod Tavallaee, Ebrahm Bagher, We Lu, and Al A. Ghorban, A Detaled Analyss of the KDD CUP 99 Data Set, Proceedng IEEE nternatonal conference on computatonal ntellgence for securty and defence applcatons, pp , Ottawa, Ontaro, Canada, 009. Internatonal Journal of Emergng Engneerng Research and Technology 8