Communication Networks: Traffic Data, Network Topologies, and Routing Anomalies

Transcription

1 Commuicatio Networks: Traffic Data, Network Topologies, ad Routig Aomalies Ljiljaa Trajković Commuicatio Networks Laboratory School of Egieerig Sciece Simo Fraser Uiversity, Vacouver, British Columbia Caada

2 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios 2

3 lhr: 535,102 odes ad 601,678 liks 3

4 lhr: 535,102 odes ad 601,678 liks 4

5 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios 5

6 Measuremets of etwork traffic Traffic measuremets: help uderstad characteristics of etwork traffic are basis for developig traffic models are used to evaluate performace of protocols ad applicatios Traffic aalysis: provides iformatio about the etwork usage helps uderstad the behavior of etwork users Traffic predictio: importat to assess future etwork capacity requiremets used to pla future etwork developmets 6

7 Traffic modelig: self-similarity Self-similarity implies a fractal-like behavior Data o various time scales have similar patters Implicatios: o atural legth of bursts bursts exist across may time scales traffic does ot become smoother whe aggregated it is ulike Poisso traffic used to model traffic i telephoe etworks as the traffic volume icreases, the traffic becomes more bursty ad more self-similar 7

8 Self-similarity: ifluece of time-scales Geuie MPEG traffic trace E+06 5.E E+05 4.E+06 bits/time uit bits/time uit 6.E+05 4.E+05 bits/time uit 3.E+06 2.E E+05 1.E time uit = 160 ms (4 frames) 0.E time uit = 640 ms (16 frames) 0.E time uit = 2560 ms (64 frames) W. E. Lelad, M. S. Taqqu, W. Williger, ad D. V. Wilso, O the self-similar ature of Etheret traffic (exteded versio), IEEE/ACM Tras. Netw., vol. 2, o 1, pp. 1-15, Feb

9 Self-similarity: ifluece of time-scales Sythetically geerated Poisso model E+06 5.E E+05 4.E+06 bits/time uit bits/time uit 6.E+05 4.E+05 bits/time uit 3.E+06 2.E E+05 1.E time uit = 160 ms (4 frames) 0.E time uit = 640 ms (16 frames) 0.E time uit = 2560 ms (64 frames) W. E. Lelad, M. S. Taqqu, W. Williger, ad D. V. Wilso, O the self-similar ature of Etheret traffic (exteded versio), IEEE/ACM Tras. Netw., vol. 2, o 1, pp. 1-15, Feb

10 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios 10

11 BCNET packet capture: physical overview BCNET is the hub of advaced telecommuicatio etwork i British Columbia, Caada that offers services to research ad higher educatio istitutios 11

12 BCNET packet capture BCNET trasits have two service providers with 10 Gbps etwork liks ad oe service provider with 1 Gbps etwork lik Optical Test Access Poit (TAP) splits the sigal ito two distict paths The sigal splittig ratio from TAP may be modified The Data Capture Device (NijaBox 5000) collects the real-time data (packets) from the traffic filterig device 12

13 Net Optics Director 7400: applicatio diagram Net Optics Director 7400 is used for BCNET traffic filterig It directs traffic to moitorig tools such as NijaBox 5000 ad FlowMo 13

14 Network moitorig ad aalyzig: Edace card Edace Data Acquisitio ad Geeratio (DAG) 5.2X card resides iside the NijaBox 5000 It captures ad trasmits traffic ad has time-stampig capability DAG 5.2X is a sigle port Peripheral Compoet Itercoect Exteded (PCIx) card ad is capable of capturig o average Etheret traffic of 6.9 Gbps 14

15 Real time etwork usage by BCNET members The BCNET etwork is high-speed fiber optic research etwork British Columbia's etwork exteds to 1,400 km ad coects Kamloops, Kelowa, Price George, Vacouver, ad Victoria 15

16 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios 16

17 Iteret topology Iteret is a etwork of Autoomous Systems: groups of etworks sharig the same routig policy idetified with Autoomous System Numbers (ASN) Autoomous System Numbers: assigmets/as-umbers Iteret topology o AS-level: the arragemet of ASes ad their itercoectios Aalyzig the Iteret topology ad fidig properties of associated graphs rely o miig data ad capturig iformatio about Autoomous Systems (ASes) 17

18 Variety of graphs Radom graphs: odes ad edges are geerated by a radom process Erdős ad Réyi model Small world graphs: odes ad edges are geerated so that most of the odes are coected by a small umber of odes i betwee Watts ad Strogatz model (1998) 18

19 Scale-free graphs Scale-free graphs: graphs whose ode degree distributio follow power-law rich get richer Barabási ad Albert model (1999) Aalysis of complex etworks: discovery of spectral properties of graphs costructig matrices describig the etwork coectivity 19

20 Aalyzed datasets Sample datasets: Route Views: TABLE_DUMP B / IGP : : : :3000 NAG RIPE: TABLE_DUMP B / IGP : :3010 NAG 20

21 Iteret topology at AS level Datasets collected from Border Gateway Protocols (BGP) routig tables are used to ifer the Iteret topology at AS-level

22 Iteret topology The Iteret topology is characterized by the presece of various power-laws: ode degree vs. ode rak eigevalues of the matrices describig Iteret graphs (adjacecy matrix ad ormalized Laplacia matrix) Power-laws expoets have ot sigificatly chaged over the years Spectral aalysis reveals ew historical treds ad otable chages i the coectivity ad clusterig of AS odes over the years 22

23 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios 23

24 Traffic aomalies Slammer, Nimda, ad Code Red I aomalies affected performace of the Iteret Border Gateway Protocol (BGP) BGP aomalies also iclude: Iteret Protocol (IP) prefix hijacks, miss-cofiguratios, ad electrical failures Techiques for detectig BGP aomalies have recetly gaied visible attetio ad importace 24

25 Aomaly detectio techiques Classificatio problem: assigig a aomaly or regular label to a data poit Accuracy of a classifier depeds o: extracted features combiatio of selected features uderlyig model Goal: Detect Iteret routig aomalies usig the Border Gateway Protocol (BGP) update messages 25

26 BGP features Approach: Defie a set of 37 features based o BGP update messages Extract the features from available BGP update messages that are collected durig the time period whe the Iteret experieced aomalies: Slammer Nimda Code Red I 26

27 Feature selectio algorithms Select the most relevat features for classificatio usig: Fisher Miimum Redudacy Maximum Relevace (mrmr) Odds Ratio Decisio Tree Fuzzy Rough Sets 27

28 Feature classificatio Trai classifiers for BGP aomaly detectio usig: Support Vector Machies Hidde Markov Models Naive Bayes Decisio Tree Extreme Learig Machie (ELM) 28

29 BGP: update messages Border Gateway Protocol (BGP) eables exchage of routig iformatio betwee gateway routers usig update messages BGP update message collectios: Réseaux IP Europées (RIPE) uder the Routig Iformatio Service (RIS) project Route Views Available i multi-threaded routig toolkit (MRT) biary format 29

30 BGP: aomalies Aomaly Date Duratio (h) Slammer Jauary 25, Nimda September 18, Code Red I July 19, Traiig Data Dataset Slammer + Nimda Dataset 1 Slammer + Code Red I Dataset 2 Code Red I + Nimda Dataset 3 Slammer Dataset 4 Nimda Dataset 5 Code Red I Dataset 6 30

31 BGP: features Defie 37 features Sample every miute durig a five-day period: the peak day of a aomaly two days prior ad two days after the peak day 7,200 samples for each aomalous evet: 5,760 regular samples (o-aomalous) 1,440 aomalous samples Imbalaced dataset 31

32 BGP features Feature Defiitio Category 1 Number of aoucemets Volume 2 Number of withdrawals Volume 3 Number of aouced NLRI prefixes Volume 4 Number of withdraw NLRI prefixes Volume 5 Average AS-PATH legth AS-path 6 Maximum AS-PATH legth AS-path 7 Average uique AS-PATH legth AS-path 8 Number of duplicate aoucemets Volume 9 Number of duplicate withdrawals Volume 10 Number of implicit withdrawals Volume 32

33 BGP features Feature Defiitio Category 11 Average edit distace AS-path 12 Maximum edit distace AS-path 13 Iter-arrival time Maximum edit distace =, where = (7,..., 17) Maximum AS-path legth =, where = (7,..., 15) Volume AS-path AS-path 34 Number of IGP packets Volume 35 Number of EGP packets Volume 36 Number of icomplete packets Volume 37 Packet size (B) Volume 33

34 Feature selectio: decisio tree Commoly used algorithm i data miig Geerates a model that predicts the value of a target variable based o several iput variables A top-dow approach is commoly used for costructig decisio trees: a appropriate variable is chose to best split the set of items based o homogeeity of the target variable withi subsets C5 software tool was used to geerate decisio trees C5 [Olie]. Available: 34

35 Feature selectio: decisio tree Dataset Traiig data Selected Features Dataset 1 Slammer + Nimda 1 21, 23 29, Dataset 2 Slammer + Code Red I 1 22, 24 29, Dataset 3 Code Red I + Nimda 1 29, Either four (30, 31, 32, 33) or five (22, 30, 31, 32, 33) features are removed i the costructed trees maily because: features are umerical ad some are used repeatedly 35

36 Feature selectio: fuzzy rough sets Deal with the approximatio of fuzzy sets i a fuzzy approximatio space defied by a fuzzy similarity relatio or by a fuzzy partitio The fuzzy similarity relatio Sim(C) is: a x matrix that describes similarities betwee ay two samples computed by the mi operator Computatioal complexity: O( 2 m) is the umber of samples m is the umber of features 36

37 Feature selectio: fuzzy rough sets Dataset Traiig data Selected Features Dataset 4 Slammer 1, 3 6, 9, 10, 13 32, 35 Dataset 5 Nimda 1, 3 4, 8 10, 12, 14 32, 35, 36 Dataset 6 Code Red I 3 4, 8 10, 12, 14 32, 35, 36 Usig combiatio of datasets, for example Slammer + Nimda for traiig leads to higher computatioal load Each dataset was used idividually 37

38 Aomaly classifiers: decisio tree Dataset Testig data Acc trai Acc test Traiig time (s) Dataset 1 Code Red I Dataset 2 Nimda Dataset 3 Slammer Each path from the root ode to a leaf ode may be trasformed ito a decisio rule A set of rules that are obtaied from a traied decisio tree may be used for classifyig usee samples 38

39 Aomaly classifier: ELM Used for learig with a sigle hidde layer feed forward eural etwork Weights coectig the iput ad hidde layers with the bias terms are iitialized radomly Weights coectig the hidde ad output layers are aalytically determied Lears faster tha SVMs by a factor of thousads Suitable for olie applicatios We use all features (37), all cotiuous features (17), features selected by fuzzy rough sets (28 or 27), ad cotiuous features selected by fuzzy rough sets (9 or 8) 39

40 Aomaly classifiers: ELM No. of features Dataset Acc trai Acc Traiig time test (s) Dataset ± ± Dataset ± ± Dataset ± ± Dataset ± ± Dataset ± ± Dataset ± ± hidde uits The biary features are removed to form a set of 17 features 40

41 Aomaly classifiers: ELM No. of features Dataset Acc trai Acc test 28 Dataset ± ± (from 37) Dataset ± ± Dataset ± ± Dataset ± ± (from 17) Dataset ± ± Dataset ± ±

42 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios 42

43 Coclusios Data collected from deployed etworks are used to: evaluate etwork performace characterize ad model traffic (iter-arrival ad call holdig times) idetify treds i the evolutio of the Iteret topology classify traffic ad etwork aomalies 43

44 Coclusios Machie learig algorithms (feature selectio ad classificatio algorithms) are used for detectig BGP aomalies Performace of classifiers greatly depeded o the employed datasets Feature selectio algorithms were used to improve the performace of classifiers For smaller datasets, performace of the ELM classifier was improved by usig fuzzy rough sets Both decisio tree ad ELM are relatively fast classifiers with satisfactory accuracy 44

45 Refereces: sources of data RIPE RIS raw data [Olie]. Available: Uiversity of Orego Route Views project [Olie]. Available: CAIDA: Ceter for Applied Iteret Data Aalysis: {Olie}. Available: 45

46 Refereces: Y. Li, H. J. Xig, Q. Hua, X.-Z. Wag, P. Batta, S. Haeri, ad Lj. Trajković, Classificatio of BGP aomalies usig decisio trees ad fuzzy rough sets, i Proc. IEEE Iteratioal Coferece o Systems, Ma, ad Cyberetics, SMC 2014, Sa Diego, CA, October 2014, pp N. Al-Rousa, S. Haeri, ad Lj. Trajković, Feature selectio for classificatio of BGP aomalies usig Bayesia models," i Proc. ICMLC 2012, Xi'a, Chia, July 2012, pp N. Al-Rousa ad Lj. Trajković, Machie learig models for classificatio of BGP aomalies, i Proc. IEEE Cof. High Performace Switchig ad Routig, HPSR 2012, Belgrade, Serbia, Jue 2012, pp T. Farah, S. Lally, R. Gill, N. Al-Rousa, R. Paul, D. Xu, ad Lj. Trajković, Collectio of BCNET BGP traffic," i Proc. 23rd ITC, Sa Fracisco, CA, USA, Sept. 2011, pp S. Lally, T. Farah, R. Gill, R. Paul, N. Al-Rousa, ad Lj. Trajković, Collectio ad characterizatio of BCNET BGP traffic," i Proc IEEE Pacific Rim Cof. Commuicatios, Computers ad Sigal Processig, Victoria, BC, Caada, Aug. 2011, pp