Network Measurement. Xu Zhang Vision & CITE Lab, Nanjing University 2018/10/17 1

Transcription

1 Network Measurement Xu Zhang Vision & CITE Lab, Nanjing University 2018/10/17 1

2 Last Time Basics for Network Transmission Network overview What is a network Circuit Switching Packet Switching Network protocol What is a protocol Protocol stacks Protocol for media transport 2018/10/17 2

3 This Time Last time: network overview Today: Network measurement Today s specifics: Delay measurements Bandwidth measurements 2018/10/17 3

4 Why do we measure the Internet? We cannot improve the Internet if we don t understand it We cannot understand it if we don t measure We cannot build effective models or simulators if we don t measure

5 Long term objectives Monitor the Internet at real time Manage the Internet Monitor and react before things go bad

6 What can we measure in the Internet? Structure Topology (router/network) connectivity, link capacities, link loss, available bandwidth, routing Traffic End-to-end performance, packet arrival process (congestion built-up) Users and applications WWW, peer-to-peer, streaming Malicious behavior Attack patterns, port scans

7 Internet Measurement Challenges (1) Network size: 100,000,000s hosts, 1,000,000s routers, ~30,000 ASes Network Complexity Interaction between components, protocols, applications, users All change over time New applications are added New protocol versions (TCP) New router design (AQM)

8 Internet Measurement Challenges (2) Not engineered for measurement: Initial design had no measurement thinking Distributed management Tendency not to share data Blocking measurement attempts ( don t ping my network ) NATs, Firewalls,

9 End-to-end Delay

10 End-to-end Delay The time required to transmit a packet along its entire path Created by an application, handed over to the OS, passed to a network card (NIC), encoded, transmitted over a physical medium (copper, fibre, air), received by an intermediate device (switch, router), analyzed, retransmitted over another medium, etc. The most common measurement uses ping for total roundtriptime (RTT).

11 Types of Delay Causes of end-to-end delay: Processing delay Buffer delays Transmission delays Propagation delays

12 Processing Delay Required time to analyze a packet header and decide where to send the packet (e.g. a routing decision) Inside a router this depends on the number of entries in the routing table, the implementation of data structures, hardware in use, etc. This can include error verification, such as IPv4, IPv6 header checksum calculations.

13 Queuing Delay The time a packet is enqueued until it is transmitted The number of packets waiting in the queue will depend on traffic intensity and of the type of traffic (bursty or sustained) Router queue algorithms try to adapt delays to specific preferences, or impose equal delay on all traffic.

14 Transmission Delay The time required to push all the bits in a packet on the transmission medium in use For N=Number of sent bits per second, S=Size of packet, d=delay d = S/N For example, to transmit 1024 bits using Fast Ethernet (100Mbps): d = 1024/1x10e8 = micro seconds

15 Propagation Delay Once a bit is 'pushed' on to the transmission medium, the time required for the bit to propagate to the end of its physical trajectory In the majority of cases the propagation velocity is close to the speed of light. The delay of propagation in the circuit depends mainly on the actual distance of the physical circuit For d = distance, s = propagation velocity PD = d/s

16 Sending a 100B packet from A to B? A 1Mbps, 1ms B time=0 Time to transmit one bit = 1/10 6 s Time to transmit 800 bits=800x1/10 6 s 100Byte packet Time when that bit reaches B = 1/ /10 3 s The last bit reaches B at Delay Time = (800x1/10 6 )+1/10 3 s Delay = Transmission Delay + Propagation Delay (Packet Size Link Bandwidth) + Link Latency = 1.8ms

17 Sending a 100B packet from A to B? A 1Gbps, 1Mbps, 1ms? B 1GB file in 100B packets 100Byte packet 10 7 x 100B packets The last bit in the file reaches B at (10 7 x800x1/10 9 )+1/10 3 s = 8001ms The last bit Timereaches B at (800x1/10 9 )+1/10 3 s = ms The last bit reaches B at (800x1/10 6 )+1/10 3 s = 1.8ms

18 Active measurement

19 Simple delay/loss probing with ping 2018/10/17 19

20 Traceroute Regular UDP packets successive TTLs time A B C D E ICMP TTL expired message ICMP port unreachable message

21 C:\>tracert Tracing route to [ ] over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms ms 20 ms 19 ms vxr.tau.ac.il [ ] 3 17 ms 22 ms 20 ms c6509.tau.ac.il [ ] 4 21 ms 19 ms 19 ms tel-aviv.tau.ac.il [ ] 5 19 ms 23 ms 18 ms gp1-tau-fe.ilan.net.il [ ] 6 20 ms 20 ms 20 ms iucc.il1.il.geant.net [ ] 7 69 ms 69 ms 69 ms il.it1.it.geant.net [ ] 8 82 ms 82 ms 82 ms it.ch1.ch.geant.net [ ] ms 98 ms 98 ms ch.at1.at.geant.net [ ] ms 105 ms 105 ms at.hu1.hu.geant.net [ ] ms 112 ms 113 ms hu.hr1.hr.geant.net [ ] ms 115 ms 115 ms carnet-gw.hr1.hr.geant.net [ ] ms 122 ms 123 ms ms 112 ms 119 ms ms 119 ms 119 ms ms 114 ms 113 ms duality.cc.fer.hr [ ] Trace complete.

22 C:\Users\xzhang17>tracert 通过最多 30 个跃点跟踪到 google-public-dns-a.google.com [ ] 的路由 : 1 <1 毫秒 <1 毫秒 <1 毫秒 RT-AC68U-E740 [ ] 2 1 ms <1 毫秒 <1 毫秒 ms 4 ms 3 ms ms * 4 ms ms 2 ms 3 ms ms 29 ms 29 ms ms 36 ms 38 ms ms 41 ms 35 ms ms 46 ms 38 ms ms 33 ms * ms 40 ms 37 ms * * 51 ms ms * 156 ms * 150 ms 151 ms * * * 请求超时 16 * * * 请求超时 17 * * * 请求超时 18 * * * 请求超时 19 * * * 请求超时 20 * * * 请求超时 21 * * * 请求超时 22 * * * 请求超时 23 * * * 请求超时 24 * * * 请求超时 25 * 45 ms * google-public-dns-a.google.com [ ] 26 * * 46 ms google-public-dns-a.google.com [ ] 跟踪完成

23 Active measurement Infrastructure-based Path-fitting-based Coordinate-based

24 Infrastructure-based

25 Triangulation AP1 a b Tracer1 c AP2 a-c < b < a+c? Feasible to estimate distance? -- APs -- Tracers

26 King King estimates RTT between any two hosts in the Internet by estimating the RTT between their domain name servers.

27 King King try to trick name server A into directly querying name server B.

28 Path-fitting-based

29 IDMaps Servers maintain simplified topological map

30 iplane The path from S to D is obtained by composing a path from S with a path going into D from a vantage point close to S (V1). BGP1 and BGP2 are destinations in two random prefixes to which S performs traceroutes.

31 Coordinate-based

32 Global Network Positioning (GNP) Model the Internet as a geometric space (e.g. 3-D Euclidean) Characterize the position of any end host with coordinates Use computed distances to predict actual distances (x 1,y 1,z 1 ) y (x 2,y 2,z 2 ) x z (x 3,y 3,z 3 ) (x 4,y 4,z 4 ) 32

33 Landmark Coordinate y (x 2,y 2 ) L 1 (x 1,y 1 ) L 2 L 3 L 2 Internet L 1 x (x 3,y 3 ) Small number of distributed hosts called Landmarks measure inter-landmark distances L 3 Min. where 33

34 Ordinary Host Coordinate Each ordinary host measures its distances to the Landmarks, Landmarks just reflect pings (x 1,y 1 ) y L 2 (x 2,y 2 ) L 1 L 1 L 3 L 2 Internet x L 3 (x 3,y 3 ) (x 4,y 4 ) Min. 34

35 IDES An N N matrix whose rows are not linearly independent has rank strictly less than and can be expressed as the product of two smaller matrices. we can estimate the network distance from H i to H j by where is X i the i th row vector of the matrix X and Y j is the j th row vector of the matrix Y.

36 2018/10/17 36

37 Bandwidth Metrics NMWG divide bandwidth into four sub-metrics: Bandwidth Capacity Achievable Bandwidth Available Bandwidth Bandwidth Utilization

38 Other Metric Terms Throughput Throughput is the same as achievable bandwidth. Bulk Transfer Capacity (BTC) Defined by RFC 3148 BTC = data_sent / elapsed_time The throughput of a persistent TCP transfer. Each of these metrics can be used to describe the entire path (end-to-end) as well as path s link (hop-by-hop)characteristics.

39 Bandwidth Capacity vs. Achievable Bandwidth Capacity is the maximum amount of data per time unit that the link or path has available, when there is no competing traffic. Achievable bandwidth is the maximum amount of data per time unit that a link or path can provide to an application, given the current utilization, the protocol and operating system used, and the end-host performance capability and load. (Throughput)

40 Bandwidth Capacity vs. Achievable Bandwidth Cont. If a path consists of several links, the link with the minimum transmission rate determines the capacity of the path. While the link with the minimum unused capacity limits the achievable bandwidth. i.e. at high-speed networks, hardware configuration or software load on the end hosts actually limit the bandwidth delivered to the application.

41 Available Bandwidth vs. Bandwidth Utilization Available bandwidth is the maximum amount of data per time unit that a link or path can provide, given the current utilization. Utilization is the aggregate capacity currently being consumed on a link or path. Available Bandwidth = Bandwidth Capacity Bandwidth Utilization

42 BTC vs. Available Bandwidth Available Bandwidth is the amount of usable bandwidth without affecting cross-traffic, whereas, the BTC is measured by sending as much packets as possible, limiting other traffic. BTC is simulating steady state persistent flow, taking considerable time and overhead.

43 BTC vs. Available Bandwidth Cont. The BTC definition assumes an ideal TCP implementation, actually, this doesn t exist, and what BTC measured is the variant of achievable bandwidth.

44 Passive vs. Active measurement Active measurement means that the tool actively sends probing packets into the network. Passive measurement tools monitors the passing traffic without interfering. Passive measurement is appreciated, however, less reliable than active, as it can t extract any data pass through it.

45 Receiver-based vs. Sender-based techniques Receiver-based (end-to-end) techniques usually use the one-direction TCP stream to probe the path bandwidth. Sender-based (echo-based) techniques force the receiver to reply the ICMP query, UDP echo or TCP-FIN.

46 Sender-based technique Advantage: Flexible deployment. Clock needn t synchronized at two ends. Disadvantage: ICMP and UDP echo packets usually be rate-limited or filtered out by some routers. Round-trip is much more possibility influenced by cross-traffic than that of one-way delay Response packets may come back through a different path

47 Receiver-based technique Advantage: More accurate than sender-based technique. Disadvantage: Difficult to deployment. The clock have to be synchronized at two ends.

48 Bandwidth Measurement Technology Packet Dispersion technology packet pair and packet train Self-Loading Periodic streams (SLOPS) Variable Packet Size (VPS) technology VPS even/odd Tailgating technique

49 Packet Dispersion Technique Sender sends two same-size packets back-to-back from source to sink. The packets will reach the sink dispersed by the transmission delay of the bottleneck links if there is no cross traffic. Measuring the dispersion can infer the bottleneck link bandwidth capacity. Note: Bottleneck link can refer to the link with smallest transmission rate, it s also can refer to the link with minimum available bandwidth. We refer the bottleneck link to the first case.

50 Packet Dispersion Technique Cont. Bottleneck bandwidth = packet size/ t

51 Packet Dispersion Technique Cont. If sender sends the packets as one observation sample more than two, called packet train. Tools usually apply robust statistical filtering techniques to find valid samples.

52 Packet pair vs. packet train Packet train is more likely to be interfered by cross traffic than packet pair. Packet train can be used to measure the bottleneck link that is multichannel while packet pair can t deal with. Packet train can reduce the limitation of clock resolution. Sophisticated tools apply both methods in their implementation. i.e. Pathrate

53 Packet Dispersion Technique Cont. Tool Name Active/ Passive bprobe Active Packet pair cprobe Active Packet pair Netest Active Packet pair Protocol Metrics ICMP ICMP UDP Bandwidth Capacity Bandwidth utilization Bandwidth capacity Methodology Path/Perlink Path Path Path FOR MORE INFO... Bprobe and cprobe Nettest

54 Packet Dispersion Technique Cont. Pathrate Active Packet pair, packet train UDP Bandwidth capacity Path Pipechar Active Packet train UDP Available bandwidth Per-link Sprobe Active Packet pair TCP Bandwidth capacity Path FOR MORE INFO... Pathrate Pipechar SProbe

55 Self-Loading Periodic Streams(SLOPS) Sender sends series of packets to the sink at the rate of larger than the bottleneck link available bandwidth. Every packets get a timestamp at sender side. Compare the difference of successive packets timestamp and their arrival times to infer the available bandwidth. Rate-adjustment adaptive algorithm to converge to the available bandwidth.

56 Self-Loading Periodic Streams Cont. Tool Name Active/ Passive Methodology Protocol Metrics pathload Active SLOPS UDP Available bandwidth Path/Per -link Path FOR MORE INFO... Pathload

57 VPS technique cont. Assumption: At least one packet, together with the ICMP reply it generates, will not encounter any queuing delays.

58 Variable Packet Size (VPS) Technique Step1. Sender set TTL=1, send out the packet, and wait for the ICMP TTL-exceeded packet back. Step2. Upon receiving ICMP, estimate the RTT. Estimate the RTT multiple times for various size packets. The minimum RTT of various packets are believed to be the valid sample. Step3. The first link capacity is C=1/b, b is slope of RTT graph. Set the TTL=2,3 n, repeat the process of step1 to 3, to Calculate the C=1/ bi bi-1

59 VPS technique cont.

60 Even-odd VPS The VPS probing technique is not altered, Mathematical trick to improve reliability. For each of the probing sizes, divide the set of samples into even and odd numbers. Calculation is based on even-odd samples. i.e. the even sample of link i, the odd sample of link i+1.

61 Tailgating Technique Tailgating technique divides into two phrase: Phase one: (Sigma phase) Like VPS probing, but for entire path instead of per link. Phase two: (tailgating phase) The largest possible nonfragmented packet (the tailgated) followed by the smallest possible packet (i.e 40 bytes). This causes the smaller packet (the tailgater) always queue behind the larger packet. Reference: Kevin Lai, Mary Baker Measuring Link Bandwidths Using a Deterministic Model of Packet Delay ACM SIGCOMM 2000

62 Tailgating Technique cont. The following condition should met: The large packet should not be queued due to cross traffic. The large packet should have a TTL field set to L (1 n). The tailgater packet should be queued directly after the large packet on link L. The tailgater packet should not queued after having passing link L.

63 VPS Technology Tool Name Active/ Passive Methodology Protocol Metrics bing Active VPS ICMP Bandwidth capacity, loss, delay clink Active VPS/ even-odd UDP Pchar Active VPS UDP, ICMP Bandwidth capacity, Loss Bandwidth capacity, Loss, delay Path/Per -link Path Path Per-link Bing Clink Pchar

64 VPS Technology Cont. Tool Name Nettimer Active/ Passive Active, Passive Methodology VPS/tail gating pathchar Active VPS/eve n-odd Protocol Metrics TCP UDP, ICMP Bandwidth capacity Bandwidth capacity, Loss, delay Path/Per -link Per-link Per-link FOR MORE INFO... Nettimer Pathchar ftp://ftp.ee.lbl.gov/pathchar/

65 Thank you! Q & A! 2018/10/17 65