measurement goals why traffic measurement of Internet is so hard? measurement needs combined skills diverse traffic massive volume of traffic

Transcription

1 measurement goals Traffic Measurement and Analysis () SOI ASIA Lecture 22//26 Kenjiro Cho Sony Computer Science Labs, Inc. for operations trouble shooting diagnosis and tuning of performance, reliability usage report long-term planning of capacity, equipment, cost evaluation for protocol/software/hardware engineering trade-off in design (e.g., buffer size vs. cost) to verify things are working as designed to look for unexpected (important in Internet) for scientific interests (new discoveries) characteristics of delay, throughput, loss modeling (e.g., TCP, web traffic) self-similarity/fractal traffic abundant data, simulation tools 2 measurement needs combined skills goals could be operational, engineering, scientific all unseparable, all skills required knowledge of operational environment engineering of measurement tools output can be facts, findings, new ideas new ideas are not always necessary facts, especially long-term measurement, are valuable but you should have clear goals better to start with real problems to solve there are many issues and problems but some are more important than others why traffic measurement of Internet is so hard? massive, diverse and changing traffic mechanisms at different layers in different time scale interact with each other dynamics Internet mechanisms are adaptive and resilient traditional measurement techniques are often not applicable pathological traffic is not unusual by bugs, misconfigurations, errors, mismatches, accidents we still don t have good understanding 3 4 massive volume of traffic unprecedented scale with unprecedented growth e.g., traffic volume: Mbps traffic 2MB/sec 75MB/minute 42GB/hour TB/day far more data than we can analyze techniques needed to reduce data size filtering: e.g., record only TCP SYN packets aggregation: e.g., flow-based accounting sampling: e.g., record in n packets still, details matter a big impact often comes from small fraction from minor differences diverse traffic large variation in traffic mix between sites backbone vs. access links access line types: fiber, ADSL, modem, wireless, satellite differences in bandwidth, delay, loss typical traffic doesn t exist! 5 6

2 constant change of traffic pattern daily, weekly traffic pattern trend changes over time web completely changed traffic pattern hard to predict future! Traffic (Mbps) :: 3:: total.../ /7 6.../ /8 6:: 9:: dst address 2:: Time 35.../ / / / / /7 5:: 8:: / / /7 2:: :: 4/3 time scale of traffic management long-term capacity planning day pricing (off-time rate discount) session service pricing, admission control, routing round-trip time end-to-end flow control, various timeout mechanisms packet time packet scheduling less than packet time link layer dependent 7 8 Internet dynamics dynamics packet switching statistical multiplexing queueing feedback mechanisms at different layers in different time scale e.g., TCP congestion control scaling property Internet traffic is bursty corelation, long-range dependences traditional measurement techniques often not applicable e.g., independent (memoryless) events, random sampling and 9th-percentile more useful than and stddev try log-scale plots to see scaling property other issues problems often occur at boundaries of different networks cooperation needed but not easy operators vs. researchers different interests and culture build good relationship cost: measurement doesn t come free willingness to invest privacy in traffic data companies often do not publish results 9 commonly-used management tools quick overview ping reachability, round-trip time traceroute path detection tcpdump packet capturing SNMP usage monitoring ping a popular and widely-available tool to check connectivity ICMP-echo request/reply ping responses do not network is working correctly ICMP is not representative of host/network performance 2

3 router architecture fast path: hardware assisted processing slow path: software processing processor switch fabric ping sample output % ping -c PING ( ): 56 data bytes 64 bytes from : icmp_seq= ttl=3 time=22.55 ms 64 bytes from : icmp_seq= ttl=3 time= ms 64 bytes from : icmp_seq=2 ttl=3 time= ms 64 bytes from : icmp_seq=3 ttl=3 time= ms 64 bytes from : icmp_seq=4 ttl=3 time=29.32 ms 64 bytes from : icmp_seq=5 ttl=3 time=27.68 ms 64 bytes from : icmp_seq=6 ttl=3 time= ms 64 bytes from : icmp_seq=7 ttl=3 time= ms 64 bytes from : icmp_seq=8 ttl=3 time=28.33 ms 64 bytes from : icmp_seq=9 ttl=3 time=29.85 ms ping statistics --- packets transmitted, packets received, % packet loss round-trip min/avg/max/stddev = 27.68/ /24.832/7.84 ms 3 4 traceroute exploit TTL (time-to-live) of IP router returns ICMP TIME EXCEEDED to the sender when TTL becomes path may change over time path may be asymmetric reports one of the interfaces of router traceroute sample output % traceroute traceroute to ( ), 64 hops max, 44 byte packets entry ( ).35 ms.38 ms.297 ms 2 foundry2.otemachi.wide.ad.jp ( ).96 ms.63 ms.553 ms 3 cisco5.otemachi.wide.ad.jp ( ) ms ms ms ( ).87 ms.953 ms.73 ms 5 gsr-ote.kddnet.ad.jp ( ).872 ms.663 ms 2.35 ms 6 tr-ote9.kddnet.ad.jp ( ) ms 2.98 ms 2.47 ms ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms 5 6 tcpdump packet capture tool capture the first N bytes of packets flexible filtering e.g., capture only TCP SYN from host X enables detailed analysis huge volume difficult to capture high-speed links tcpdump sample output 4:53: linus.csl.sony.co.jp.484 > S :758898() win <mss 44,nop,wscale,nop,nop,timestamp > 4:53: > linus.csl.sony.co.jp.484: S : () ack win 6384 <mss 336,nop,wscale,nop,nop,timestamp > 4:53: linus.csl.sony.co.jp.484 > ack win <nop,nop,timestamp > 4:53: linus.csl.sony.co.jp.484 > P :45(45) ack win <nop,nop,timestamp > 4:53: > linus.csl.sony.co.jp.484: P :324(323) ack 45 win 6384 [flowlabel xec44] 4:53: > linus.csl.sony.co.jp.484:. 324:764(44) ack 45 win 6384 [flowlabel xec44] 4:53: linus.csl.sony.co.jp.484 > ack 764 win <nop,nop,timestamp > 4:53: > linus.csl.sony.co.jp.484:. 764:324(44) ack 45 win 6384 [flowlabel xec44] 4:53: > linus.csl.sony.co.jp.484:. 324:4644(44) ack 45 win 6384 [flowlabel xec44] 4:53: linus.csl.sony.co.jp.484 > ack 4644 win 5395 <nop,nop,timestamp > 4:53: linus.csl.sony.co.jp.484 > ack 4644 win 58 <nop,nop,timestamp > 4:53: > linus.csl.sony.co.jp.484:. 4644:684(44) ack 45 win 6384 [flowlabel xec44] 4:53: > linus.csl.sony.co.jp.484:. 684:7524(44) ack 45 win 6384 [flowlabel xec44] 4:53: linus.csl.sony.co.jp.484 > ack 7524 win <nop,nop,timestamp > 4:53: > linus.csl.sony.co.jp.484:. 7524:8964(44) ack 45 win 6384 [flowlabel xec44] 4:53: > linus.csl.sony.co.jp.484:. 8964:44(44) ack 45 win 6384 [flowlabel xec44] 4:53: linus.csl.sony.co.jp.484 > ack 44 win <nop,nop,timestamp > 4:53: > linus.csl.sony.co.jp.484:. 44:844(44) ack 45 win 6384 [flowlabel xec44] 4:53: > linus.csl.sony.co.jp.484:. 844:3284(44) ack 45 win 6384 [flowlabel xec44] 7 8

4 SNMP (Simple Network Management Protocol) MRTG SNMP allows a remote user to query information, store information, set traps by UDP (unreliable) standardized set of traffic statistics supported by most of routers, switches, host OS many management/monitoring products MIB (Management Information Base) tree structured database of SNMP objects e.g., interfaces.iftable.ifentry.ifoutoctets standard MIBs and private MIBs get, set, get-next to access MIB supported statistics are limited most counter statistics are hard-coded: e.g., interface counters accessing to MIB objects is expensive 9 popular tool to show SNMP data time series data aggregated over time daily, weekly, monthly inbound/outbound traffic can be used for other types of time series data RRDtool: successor of MRTG 2 real-world data ping round-trip time histogram one day ping measurement from home in Tokyo to university in East Coast most replies: 94-96ms facts 864 queries, 866 replies,.4% loss average rtt: 25ms stddev: 39ms min rtt: 94ms th: 95ms 5th: 96ms 9th: 376ms max: 248ms histogram ping rtt requests: 864 replies: 866 average: 25 ms min: 94 ms th: 95 ms 5th: 96 ms 9th: 376 ms max: 248 ms loss rate:.4% ping round-trip time histogram in log scale daily plot ( and ) log-plot to see scaling property no scaling property in data more stable than service degrades during 22: - 26: histogram ping rtt 5-min average average 5 : 3: 6: 9: 2: 5: 8: 2: : time 23 24

5 tails of distribution of histogram daily plot (min, th, 9th, max) minimum, maximum th-percentile, 9th-percentile to remove outliers 5-min average max 9th th min needs appropriate bin size too small: each bin doesn t have enough samples (e.g., empty bins) too large: only few regions available enough samples needed histogram with samples histogram ping rtt 2 : 3: 6: 9: 2: 5: 8: 2: : time cumulative distribution function (CDF) density function: probability of observing x = P[X = x] cumulative distribution function: probability of observing x or less F(x) = P[X <= x] better than histogram when sample count is not enough or outliers are not negligible.9 sample average revisited accuracy how close to true (population) precision variance in data accurate, not precise precise, not accurate CDF samples samples 27 true x 28 and not equal if distribution is asymmetric average () average over n sample values nx variance μx = n nx i= x i s 2 = (x n i μx) 2 i= standard deviation s (in the same unit as ) sample (for large samples) follows normal distribution central limit theorem (see statistics textbook) x 29 3

6 normal distribution 68% within (-stddev, +stddev) 95% within (-2*stddev, +2*stddev) exp(-x**2/2) confidence interval confidence interval for the provides probabilistic bounds tells how much uncertainty in the estimate Probfc» μ» c 2 g = ff.8.6 σ (c, c2): confidence interval ( - ): confidence level.4.2 e.g., with 95% confidence, the population is between c and c2 traditionally, 95 or 99% is used for confidence level x 68% 95% 3 32 confidence interval (cont d) from central limit theorem if observations are independent and samples come from the same population with and standard deviation then, sample for large samples is normal distribution with and standard deviation /sqrt(n) μx ο N (μ; ff= p n) increase sample size to get more accuracy (x- )/(s/sqrt(n)) for samples from normal populations follows t(n-) distribution how to use confidence interval for applications provide confidence interval to show possible range of from sample and stddev, compute how many trials are needed to satisfy a given confidence interval repeat measurement until a given confidence interval is reached summary: be careful when you use average sometimes, average is not so useful in Internet measurement t(n-) density function α/2 α α/2 -t[-α/2;n-] +t[-α/2;n-] (x-u)/s (x-µ)/s measurement techniques management tools are useful but not designed for measurement next lecture: types of measurement throughput, delay, path, routing data reduction techniques filtering, aggregation, sampling clock and timestamp summary overview of measurement issues operational, engineering, scientific skills are needed popular management tools ping, traceroute, tcpdump, SNMP using real ping data histogram, CDF and confidence interval 35 36