Communication Networks: Traffic Data, Network Topologies, and Routing Anomalies

Transcription

1 Commuicatio Networks: Traffic Data, Network Topologies, ad Routig Aomalies Ljiljaa Trajković Commuicatio Networks Laboratory School of Egieerig Sciece Simo Fraser Uiversity, Vacouver, British Columbia Caada

2 Simo Fraser Uiversity, Buraby Campus December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 2

3 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 3

4 lhr: 535,102 odes ad 601,678 liks December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 4

5 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 5

6 Measuremets of etwork traffic Traffic measuremets: help uderstad characteristics of etwork traffic are basis for developig traffic models are used to evaluate performace of protocols ad applicatios Traffic aalysis: provides iformatio about the etwork usage helps uderstad the behavior of etwork users Traffic predictio: importat to assess future etwork capacity requiremets used to pla future etwork developmets December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 6

7 Traffic modelig: self-similarity Self-similarity implies a fractal-like behavior Data o various time scales have similar patters Implicatios: o atural legth of bursts bursts exist across may time scales traffic does ot become smoother whe aggregated it is ulike Poisso traffic used to model traffic i telephoe etworks as the traffic volume icreases, the traffic becomes more bursty ad more self-similar December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 7

8 Self-similarity: ifluece of time-scales Geuie MPEG traffic trace E+06 5.E E+05 4.E+06 bits/time uit bits/time uit 6.E+05 4.E+05 bits/time uit 3.E+06 2.E E+05 1.E time uit = 160 ms (4 frames) 0.E time uit = 640 ms (16 frames) 0.E time uit = 2560 ms (64 frames) W. E. Lelad, M. S. Taqqu, W. Williger, ad D. V. Wilso, O the self-similar ature of Etheret traffic (exteded versio), IEEE/ACM Tras. Netw., vol. 2, o 1, pp. 1-15, Feb December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 8

9 Self-similarity: ifluece of time-scales Sythetically geerated Poisso model E+06 5.E E+05 4.E+06 bits/time uit bits/time uit 6.E+05 4.E+05 bits/time uit 3.E+06 2.E E+05 1.E time uit = 160 ms (4 frames) 0.E time uit = 640 ms (16 frames) 0.E time uit = 2560 ms (64 frames) W. E. Lelad, M. S. Taqqu, W. Williger, ad D. V. Wilso, O the self-similar ature of Etheret traffic (exteded versio), IEEE/ACM Tras. Netw., vol. 2, o 1, pp. 1-15, Feb December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 9

10 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 10

11 BCNET packet capture: physical overview BCNET is the hub of advaced telecommuicatio etwork i British Columbia, Caada that offers services to research ad higher educatio istitutios December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 11

12 BCNET packet capture BCNET trasits have two service providers with 10 Gbps etwork liks ad oe service provider with 1 Gbps etwork lik Optical Test Access Poit (TAP) splits the sigal ito two distict paths The sigal splittig ratio from TAP may be modified The Data Capture Device (NijaBox 5000) collects the real-time data (packets) from the traffic filterig device December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 12

13 Net Optics Director 7400: applicatio diagram Net Optics Director 7400 is used for BCNET traffic filterig It directs traffic to moitorig tools such as NijaBox 5000 ad FlowMo December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 13

14 Network moitorig ad aalyzig: Edace card Edace Data Acquisitio ad Geeratio (DAG) 5.2X card resides iside the NijaBox 5000 It captures ad trasmits traffic ad has time-stampig capability DAG 5.2X is a sigle port Peripheral Compoet Itercoect Exteded (PCIx) card ad is capable of capturig o average Etheret traffic of 6.9 Gbps December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 14

15 Real time etwork usage by BCNET members The BCNET etwork is high-speed fiber optic research etwork British Columbia's etwork exteds to 1,400 km ad coects Kamloops, Kelowa, Price George, Vacouver, ad Victoria December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 15

16 Aoym tool The Aoym tool provides optios to aoymize time, IPv4 ad IPv6 addresses, MAC addresses, ad packet legth data Provides optios to aalyze the datasets December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 16

17 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 17

18 Iteret topology Iteret is a etwork of Autoomous Systems: groups of etworks sharig the same routig policy idetified with Autoomous System Numbers (ASN) Autoomous System Numbers: assigmets/as-umbers Iteret topology o AS-level: the arragemet of ASes ad their itercoectios Aalyzig the Iteret topology ad fidig properties of associated graphs rely o miig data ad capturig iformatio about Autoomous Systems (ASes) December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 18

19 Variety of graphs Radom graphs: odes ad edges are geerated by a radom process Erdős ad Réyi model Small world graphs: odes ad edges are geerated so that most of the odes are coected by a small umber of odes i betwee Watts ad Strogatz model (1998) December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 19

20 Scale-free graphs Scale-free graphs: graphs whose ode degree distributio follow power-law rich get richer Barabási ad Albert model (1999) Aalysis of complex etworks: discovery of spectral properties of graphs costructig matrices describig the etwork coectivity December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 20

21 Aalyzed datasets Sample datasets: Route Views: TABLE_DUMP B / IGP : : : :3000 NAG RIPE: TABLE_DUMP B / IGP : :3010 NAG December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 21

22 Iteret topology at AS level Datasets collected from Border Gateway Protocols (BGP) routig tables are used to ifer the Iteret topology at AS-level December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 22

23 Iteret topology The Iteret topology is characterized by the presece of various power-laws: ode degree vs. ode rak eigevalues of the matrices describig Iteret graphs (adjacecy matrix ad ormalized Laplacia matrix) Power-laws expoets have ot sigificatly chaged over the years Spectral aalysis reveals ew historical treds ad otable chages i the coectivity ad clusterig of AS odes over the years December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 23

24 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 24

25 Traffic aomalies Slammer, Nimda, ad Code Red I aomalies affected performace of the Iteret Border Gateway Protocol (BGP) BGP aomalies also iclude: Iteret Protocol (IP) prefix hijacks, miss-cofiguratios, ad electrical failures Techiques for detectig BGP aomalies have recetly gaied visible attetio ad importace December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 25

26 Detectio techiques Statistical patter recogitio: mai disadvatage: difficulty i estimatig distributios of high dimesios Rule-based: require a priori kowledge of etwork coditios example: the Iteret Routig Foresics (IRF) they are ot adaptable learig mechaisms, slow, ad have high degree of computatioal complexity December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 26

27 Aomaly detectio techiques Classificatio problem: assigig a aomaly or regular label to a data poit Accuracy of a classifier depeds o: extracted features combiatio of selected features uderlyig model Goal: Detect Iteret routig aomalies usig the Border Gateway Protocol (BGP) update messages December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 27

28 BGP messages BGP protocol geerates four types of messages: ope, update, keepalive, ad otificatio Oly BGP update messages were cosidered BGP update messages: aoucemets or withdrawals December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 28

29 BGP features Approach: Defie a set of 37 features based o BGP update messages Extract the features from available BGP update messages that are collected durig the time period whe the Iteret experieced aomalies: Slammer Nimda Code Red I December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 29

30 Feature selectio algorithms Select the most relevat features for classificatio usig features scorig algorithms: Fisher Miimum Redudacy Maximum Relevace (mrmr) Odds Ratio (OR) ad exteded/weighted/multi-class odds ratio (EOR/WOR/MOR) Class discrimiatig measure (CDM) Decisio Tree Fuzzy Rough Sets These algorithms measure the correlatio ad relevacy amog features December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 30

31 Feature classificatio Trai classifiers for BGP aomaly detectio usig: Support Vector Machies Hidde Markov Models Naive Bayes Decisio Tree Extreme Learig Machie (ELM) December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 31

32 BGP: update messages Border Gateway Protocol (BGP) eables exchage of routig iformatio betwee gateway routers usig update messages BGP update message collectios: Réseaux IP Europées (RIPE) uder the Routig Iformatio Service (RIS) project Route Views Available i multi-threaded routig toolkit (MRT) biary format December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 32

33 BGP: aomalies Aomaly Date Duratio (h) Slammer Jauary 25, Nimda September 18, Code Red I July 19, Traiig Data Dataset Slammer + Nimda Dataset 1 Slammer + Code Red I Dataset 2 Code Red I + Nimda Dataset 3 Slammer Dataset 4 Nimda Dataset 5 Code Red I Dataset 6 December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 33

34 BGP: features Defie 37 features Sample every miute durig a five-day period: the peak day of a aomaly two days prior ad two days after the peak day 7,200 samples for each aomalous evet: 5,760 regular samples (o-aomalous) 1,440 aomalous samples Imbalaced dataset Features are ormalized to have zero mea ad uit variace This ormalizatio reduces the effect of the Iteret growth December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 34

35 BGP features Feature Defiitio Category 1 Number of aoucemets Volume 2 Number of withdrawals Volume 3 Number of aouced NLRI prefixes Volume 4 Number of withdraw NLRI prefixes Volume 5 Average AS-PATH legth AS-path 6 Maximum AS-PATH legth AS-path 7 Average uique AS-PATH legth AS-path 8 Number of duplicate aoucemets Volume 9 Number of duplicate withdrawals Volume 10 Number of implicit withdrawals Volume December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 35

36 BGP features Feature Defiitio Category 11 Average edit distace AS-path 12 Maximum edit distace AS-path 13 Iter-arrival time Maximum edit distace =, where = (7,..., 17) Maximum AS-path legth =, where = (7,..., 15) Volume AS-path AS-path 34 Number of IGP packets Volume 35 Number of EGP packets Volume 36 Number of icomplete packets Volume 37 Packet size (B) Volume December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 36

37 Normalized scatterig graphs 8 Regular Aomaly 6 Feature Feature 1 December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 37

38 Normalized scatterig graphs 8 Regular Aomaly 6 Feature Feature 2 December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 38

39 Top te selected features Fisher mrmr Odds Ratio ad its Variats MID MIQ MIBASE OR EOR WOR MOR CMD December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 39

40 Feature selectio: decisio tree Commoly used algorithm i data miig Geerates a model that predicts the value of a target variable based o several iput variables A top-dow approach is commoly used for costructig decisio trees: a appropriate variable is chose to best split the set of items based o homogeeity of the target variable withi subsets C5 software tool was used to geerate decisio trees C5 [Olie]. Available: December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 40

41 Feature selectio: decisio tree Dataset Traiig data Selected Features Dataset 1 Slammer + Nimda 1 21, 23 29, Dataset 2 Slammer + Code Red I 1 22, 24 29, Dataset 3 Code Red I + Nimda 1 29, Either four (30, 31, 32, 33) or five (22, 30, 31, 32, 33) features are removed i the costructed trees maily because: features are umerical ad some are used repeatedly December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 41

42 Feature selectio: fuzzy rough sets Dataset Traiig data Selected Features Dataset 4 Slammer 1, 3 6, 9, 10, 13 32, 35 Dataset 5 Nimda 1, 3 4, 8 10, 12, 14 32, 35, 36 Dataset 6 Code Red I 3 4, 8 10, 12, 14 32, 35, 36 Usig combiatio of datasets, for example Slammer + Nimda for traiig leads to higher computatioal load Each dataset was used idividually December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 42

43 Aomaly classifiers: decisio tree Dataset Testig data Acc trai Acc test Traiig time (s) Dataset 1 Code Red I Dataset 2 Nimda Dataset 3 Slammer Each path from the root ode to a leaf ode may be trasformed ito a decisio rule A set of rules that are obtaied from a traied decisio tree may be used for classifyig usee samples December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 43

44 Aomaly classifier: ELM Used for learig with a sigle hidde layer feed forward eural etwork Weights coectig the iput ad hidde layers with the bias terms are iitialized radomly Weights coectig the hidde ad output layers are aalytically determied Lears faster tha SVMs by a factor of thousads Suitable for olie applicatios We use all features (37), all cotiuous features (17), features selected by fuzzy rough sets (28 or 27), ad cotiuous features selected by fuzzy rough sets (9 or 8) December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 44

45 Aomaly classifiers: ELM No. of features Dataset Acc trai Acc Traiig time test (s) Dataset ± ± Dataset ± ± Dataset ± ± Dataset ± ± Dataset ± ± Dataset ± ± hidde uits The biary features are removed to form a set of 17 features December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 45

46 Aomaly classifiers: ELM No. of features Dataset Acc trai Acc test 28 Dataset ± ± (from 37) Dataset ± ± Dataset ± ± Dataset ± ± (from 17) Dataset ± ± Dataset ± ± December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 46

47 Roadmap Itroductio Traffic collectio, characterizatio, ad modelig Case study: Collectio of BCNET traffic Iteret topology ad spectral aalysis of Iteret graphs Machie learig models for feature selectio ad classificatio of traffic aomalies Coclusios December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 47

48 Coclusios Data collected from deployed etworks are used to: evaluate etwork performace characterize ad model traffic idetify treds i the evolutio of the Iteret topology classify traffic ad etwork aomalies December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 48

49 Refereces: sources of data RIPE RIS raw data [Olie]. Available: Uiversity of Orego Route Views project [Olie]. Available: CAIDA: Ceter for Applied Iteret Data Aalysis: {Olie}. Available: December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 49

50 Refereces: Y. Li, H. J. Xig, Q. Hua, X.-Z. Wag, P. Batta, S. Haeri, ad Lj. Trajković, Classificatio of BGP aomalies usig decisio trees ad fuzzy rough sets, i Proc. IEEE Iteratioal Coferece o Systems, Ma, ad Cyberetics, SMC 2014, Sa Diego, CA, October 2014, pp T. Farah ad Lj. Trajković, Aoym: a tool for aoymizatio of the Iteret traffic," i Proc IEEE Iteratioal Coferece o Cyberetics (CYBCONF 2013), Lausae, Switzerlad, Jue 2013, pp N. Al-Rousa, S. Haeri, ad Lj. Trajković, Feature selectio for classificatio of BGP aomalies usig Bayesia models," i Proc. ICMLC 2012, Xi'a, Chia, July 2012, pp N. Al-Rousa ad Lj. Trajković, Machie learig models for classificatio of BGP aomalies, i Proc. IEEE Cof. High Performace Switchig ad Routig, HPSR 2012, Belgrade, Serbia, Jue 2012, pp T. Farah, S. Lally, R. Gill, N. Al-Rousa, R. Paul, D. Xu, ad Lj. Trajković, Collectio of BCNET BGP traffic," i Proc. 23rd ITC, Sa Fracisco, CA, USA, Sept. 2011, pp S. Lally, T. Farah, R. Gill, R. Paul, N. Al-Rousa, ad Lj. Trajković, Collectio ad characterizatio of BCNET BGP traffic," i Proc IEEE Pacific Rim Cof. Commuicatios, Computers ad Sigal Processig, Victoria, BC, Caada, Aug. 2011, pp December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 50

51 THE 29TH ANNUAL IEEE CANADIAN CONFERENCE ON ELECTRICAL AND COMPUTER ENGINEERING MAY, 2016 / VANCOUVER, CANADA Advacig Society through Electrical ad Computer Egieerig The 29 th aual IEEE Caadia Coferece o Electrical ad Computer Egieerig (CCECE 2016) will be held i Vacouver, British Columbia, Caada from May 15 to 18, CCECE is the flagship Caadia coferece for researchers, studets, ad professioals i the area of Electrical ad Computer Egieerig. It is held aually i a Caadia city to dissemiate research advacemets ad discoveries, etwork ad exchage ideas, stregthe existig parterships, ad foster ew collaboratios. CCECE 2016 s geeral theme, Advacig Society through Electrical ad Computer Egieerig, reflects the profoud impact of ECE research o our daily lives. CCECE 2016 s topics iclude:! Bioegieerig! Commuicatios ad etworks! Cotrol ad robotics! Computer ad software techiques! Devices, Circuits, ad Systems! Sigal theory ad sigal processig! Power ad eergy circuits ad systems Paper submissio guidelies: Submitted papers must be upublished ad should ot be submitted elsewhere at the same time. Accepted papers should ot exceed 6 pages i two-colum IEEE Trasactios style. Accepted papers loger tha 4 pages will be charged $100 for each extra page. Papers should be submitted as PDF files through the paper submissio system ( All submitted papers will be peer reviewed by at least three idepedet reviewers. Note: To be published i the CCECE 2016 Coferece Proceedigs ad to be eligible for publicatio i IEEE Xplore, a author of a accepted paper is required to register for the coferece at the full (member or o-member) rate ad the paper must be preseted by a author of that paper at the coferece uless the TPC Chair grats permissio for a substitute preseter arraged i advace of the evet ad who is qualified both to preset ad aswer questios. For authors with multiple accepted papers, oe full registratio is valid for oe paper ad the author pays a additioal $200 dollars for each additioal paper. However, oly oe copy of the proceedigs will be provided. CCECE Foudig Chair Vijay Bhargava (Uiversity of British Columbia) Hoorary Co-Chairs Adreas Atoiu (Uiversity of Victoria) James Cavers (Simo Fraser Uiversity) Herma Dommel (Uiversity of British Columbia) Geeral Co-Chairs Rodey Vaugha (Simo Fraser Uiversity) Rabab Ward (Uiversity of British Columbia) Techical Program Co-Chairs Shahriar Mirabbasi (Uiversity of British Columbia) Ljiljaa Trajkovic (Simo Fraser Uiversity) Tutorials ad Paels Co-Chairs Iva Bajic (Simo Fraser Uiversity) Thomas Johso (Uiversity of British Columbia Okaaga Publicatio Co-Chairs Hamed Shah-Masouri (Uiversity of British Columbia) Vicet Wog (Uiversity of British Columbia) Publicity Chair Dave Michelso (Uiversity of British Columbia) Patroage/Exhibitio Chair Bob Gill (British Columbia Istitute of Techology) Local Arragemets Chair Jeff Bloemik (British Columbia Istitute of Techology) Registratio Chair Cathie Lowell (CL Cosultig) Fiace Co-Chairs Steve McClai (IEEE Vacouver Sectio) Lee Vishloff (Tech-Kows Services) Treasurer Ashfaq (Kash) Husai (Dillo Cosultig) Website ad Social Media Chair: Stephe Makoi (Simo Fraser Uiversity) Importat Dates Tutorial proposals: December 13, 2015 Ivited sessios proposals: December 27, 2015 Regular paper submissio: December 27, 2015 Acceptace otificatios: February 7, 2016 Fial paper submissio: March 6, 2016 December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 51

52 CCECE 2016 Caadia Coferece o Electrical ad Computer Egieerig Importat Dates Tutorial proposals: Dec. 13, 2015 Ivited sessios proposals: Dec. 27, 2015 Regular paper submissio: Dec. 27, 2015 Acceptace otificatios: Feb. 7, 2015 Fial paper submissio: March 6, 2015 December 17, 2015 ICAEE 2015, IUB, Dhaka, Bagladesh 52