KNOM Tutorial Internet Traffic Matrix Measurement and Analysis. Sue Bok Moon Dept. of Computer Science

Transcription

1 KNOM Tutorial 2003 Internet Traffic Matrix Measurement and Analysis Sue Bok Moon Dept. of Computer Science

2 Overview Definition of Traffic Matrix 4Traffic demand, delay, loss Applications of Traffic Matrix 4Engineering, research, SLAs Challenges in Obtaining Traffic Matrix 4Limitation of NetFlow and active probes 4Challenges in measurement and modeling Summay & Future Work 2

3 Definition of Traffic Matrix What is a traffic matrix? 4A matrix built on metric of interest 4Traffic demand matrix How much traffic flows from point A to point B Granularity: PoP, router, link, prefix 4Delay matrix How much delay from point A to point B Granularity: PoP, router, link, end hosts 4Loss matrix How many packets are dropped from point A to point B Granularity: PoP, router, end hosts 3

4 Example : AT&T Latency Matrix CAM CHI DAL DEN LA NY ATL CAM CHI Current Average : 35 msec DAL DEN LA 61 NY Latency in milliseconds 4

5 Traffic Demand Matrix Not part of SLAs 4Hard to obtain 4Few available publicly 5

6 Delay Matrix Usually a matrix of average delay of pings between routers of random selection per PoP 4Average of all PoP-to-PoP delays => SLA At end hosts 4Easy to get using pings between hosts of interest 6

7 Loss Matrix Usually a matrix of average loss rate of pings between routers of random selection per PoP 4Average of all PoP-to-PoP loss rates => SLA At end hosts 4Easy to get using pings between hosts of interest 7

8 Applications of Traffic Matrix Marketing/Sales 4How much traffic does customer A send from point #1 to point #2? Where should customer A buy more capacity from us? 4Is most traffic originating in Korea stay within Korea? What is the trend in international traffic growth? 4What is the performance that customer A sees? Do we have an edge over our competitors? 8

9 Applications of Traffic Matrix Network Operators 4Capacity Planning How much traffic do we have from point A to point B? How much capacity should we add? When should we add more capacity? 4Network Engineering Where is the hot spot? From SNMP What if a link fails from point A to point B? What if we move traffic from point A to point B? 9

10 Applications of Traffic Matrix Customers: SLAs 4What quality of service am I getting? How much delay do I get from ISP A? How much loss do I experience from ISP A? Can I get delay under X ms from ISP A? What is the most popular destination of my traffic? 10

11 Applications of Traffic Matrix Researchers 4Traffic modeling How does TM evolve over time? What is the fanout factor of traffic? How much more capacity do we expect between point A and point B? 4Example: IP over WDM Given physical topology of routers and optical nodes, what is the best virtual topology? Based on traffic demand matrix 11

12 Challenges in Obtaining Traffic Matrix Traffic Demand Matrix 4resource requirements in routers # of concurrently active flows 4resource requirements in measurement infrastructure production rate of flow statistics 4traffic characterization packet/byte rate of original traffic rate o f occurrence of original flows average packet/bytes per original flow 12

13 Resource Requirements Router Memory Resource in Measurement Infrastructure Server NetFlow data Analysis Resource in Router Fast link Traffic reports Network Network Operation Center 13

14 Most Popular Tools of Choice? NetFlow for traffic demand matrix ping for delay and loss matrix 14

15 NetFlow Cisco s proprietary tool 4Not an IETF standard Basic idea 4Based on (src ip, src port, dst ip, dst port, proto) 4Records byte/packet/duration per flow 4Cannot keep up with high speed links 4Can sample every N th packet 15

16 NetFlow Sampling Original Packets time flow #1 flow #2 flow #3 Sampled Packets (every 1/N, N=3) time 16

17 Limitation of NetFlow Scalability 4Historically NetFlow had a performance issue 4Never deployed at the core 4Number of flows in case of DDoS attacks beyond capacity Network melt down 17

18 Number of Active Flows on a OC-48 Link 18

19 Limitation of NetFlow Representativeness 4Can we estimate # of total flows from # of sampled flows accurately? 4Can we estimate # of total WWW flows from # of sampled WWW flows accurately? 4Metrics of interest: # of flows, flow rate, 4Packet sampling reduce effective packet rate save cost: slower memory sufficient (DRAM vs SRAM) 19

20 NetFlow Sampling Original Packets time flow #1 flow #2 flow #3 Sampled Packets (every 1/N, N=3) Flow Splitting time 20

21 Comparison of sparse and non-sparse applications Flow definition 45-tuple = (src ip, src port, dst ip, dst port, proto) 4interflow timeout = T Increase timeout T 4potentially less splitting 4fewer measured flows, more active flows Sparse vs. non-sparse flows 4napster vs. www 4# of mean active flows change differently over T 4No simple model of rate and # active flows based on aggregate traffic rates 4Model sparse and non-sparse flows separately [Duffield03] 21

22 Challenges in Delay Monitoring Not much is known about delay within ISP 4People think they know delay, but... 4Cisco SAA implementation on GSR did not consider clock synchronization, and outputs meaningless numbers Too many paths to cover 4hop-by-hop addition not yet possible 22

23 Limitation of Active Probes Representativeness [Choi04] 4Average? Median? 23

24 Suitable Statistic: Percentile! Min Med Avg Mode detection is hard 4 Difficult to distinguish small from big 4 Don t know how many ahead of them High-percentile 4 represents upper bound for most delay 4 requires a very small number of probes to estimate 99% percentile 24

25 Sampling for Demand Matrix Periodic sampling does not answer: 4What are the top 10 flows? 4What is the most dominant application and who is the heaviest user? 4What is the total # of packet for every flow? 25

26 Hash Function Mapping from a very large space to a smaller space 4h: X Y where X >> Y 4IP address to 10-bit hashed key 45-tuple address to 30-bit hashed key Load factor = collision probability 26

27 What are the top 10 flows? 27

28 Sampling for Elephants [Estan02] All packets Every n-th packet Update entry or create a new one Large flow memory Update existing entry Has entry? no Pass with p ~ size create new entry Small flow memory 28

29 Sampling for Elephants [Estan02] h1 h2 h3 All Large? Flow Memory 29

30 What is the most dominant application and who is the heaviest user? 30

31 Who is using my link? [Estan03] 31

32 Looking at the traffic Too much data for a human Do something smarter! 32

33 Looking at traffic aggregates Src. Dest. IP IP Src. Dest. net Dest. Source IPport Dest. Protoc ol Src. port net netran Destination IP Traffic Aggregating on kindividual packet header fields Ran Source Traffi gives useful results but Which Ran jeff.dorm.bigu.e Destination 11.9% Traffic 4 Traffic reports 1k are not Web port Where network du network always at the right 42.1 cdoes the uses granularity (e.g. individual IP address, subnet, traffic etc.) web come 12 Kazaa 4 Cannot show aggregates tracy.dorm.bigu. library.bigu.edu 6.7% % and from? which defined over one multiple kazaa? 3.12% 27.5% 3 Ssh fields (e.g. which network 2 uses 6.3% cs.bigu.edu which application) 18.1% The traffic analysis 3 tool risc.cs.bigu.edu dorm.bigu.edu should automatically 2.83% 17.8% find aggregates over the Most right fields traffic at goes the right granularity to the dorms Dest. port What apps are used? 33

34 Ideal traffic report Traffic aggregate Web traffic Web traffic to library.bigu.edu Web traffic from ICMP traffic from sloppynet.badu.edu to jeff.dorm.bigu.edu Traffic 42.1% 26.7% 13.4% 11.9% Web is the dominant This The paper library application is This about That s a is heavy a giving Denial the of network administrator a big flash user of Service insightful web attack traffic!! reports crowd! 34

35 Traffic Clusters and Reports Traffic clusters are multidimentional aggregates. Traffic reports give volume of chosen clusters Only those over threshold are reported To avoid redundant data, compress inferrable data (up to error H) Highlight non-obvious aggregates with unexpectedness label 35

36 Structure of regular traffic mix Backups from CAIDA to tape server SD-NAP 4Semi-regular time pattern FTP from SLAC Stanford SD-NAP Scripps web traffic Web & Squid servers Large ssh traffic Steady ICMP probing from CAIDA 36

37 What is the total # of packet of every flow? 37

38 Space-Code Bloom Filter Bloom filter answers set-membership. Space-code bloom filter answers multisetmembership Use a number of virtual Bloom-filters, spread multiplicity information over space. Write-only At OC768, it can work at 5ns SRAM What about storage space at the router? 38

39 Future Work One traffic matrix to rule? 4Can we answer all questions with one matrix? Continuous monitoring 4data export in real-time 4query over streaming data Availability/survivability 4Impliations in SLAs? 39

40 Failures are part of everyday operations Weekly Daily Hourly 40

41 Time between Failures (network-wide) 43%: <1 min 81%: <20 min 41

42 Sources of failures Duration can provide hints, e.g., 4long (>1hour): fiber cuts, severe failures 4medium (>10min): router/line card failures 4short (>1min): line card resets 4very short (<1min): software problems, optical equipment glitches Other hints 4shared equipment (routers, optical) 4 router logs (e.g., SONET alarms), etc. 42

43 Network-wide Failure Duration cumulative fraction of failures 40 % in 1-60sec 40 % in 1-15min 10 % in 15-60min 10 % >1h 43

44 References [Duffield03] N. Duffield, C. Lund, M. Thorup, Properties and Prediction of Flow Properties from Sampled Packet Streams, ACM SIGCOMM IMC, Miami, Oct., 2003 [Choi04] B.Y. Choi, S. Moon, Z.L. Zhang, C. Diot, Analysis of Point-to-Point Packet Delay in an Operational Network, IEEE INFOCOM, Hong Kong, Mar., 2004 [Estan03] C. Estan, S. Savage, G. Varghese, Automatically Inferring Patterns of Resource Consumption in Network Traffic, SIGCOMM 2003 [Estan02] C. Estan, G. Varghese, New Directions in Traffic Measurement and Accounting, SIGCOMM

45 Acknowledgements C. Estan s SIGCOMM 2002 talk. S. Bhattacharyya and G. Iannaconne s ICNP 2003 Tutorial. 45