ANALYSIS OF THE CORRELATION BETWEEN PACKET LOSS AND NETWORK DELAY AND THEIR IMPACT IN THE PERFORMANCE OF SURGICAL TRAINING APPLICATIONS

Transcription

1 ANALYSIS OF THE CORRELATION BETWEEN PACKET LOSS AND NETWORK DELAY AND THEIR IMPACT IN THE PERFORMANCE OF SURGICAL TRAINING APPLICATIONS JUAN CARLOS ARAGON SUMMIT STANFORD UNIVERSITY

2 TABLE OF CONTENTS 1. BACKGROUND RESEARCH Page 1.1. INTRODUCTION PACKET LOSS AND NETWORK DELAY INTERACTION ANALYSIS OF THE CORRELATION BETWEEN PACKET LOSS AND DELAY LOSS CONDITIONED IN DELAY DELAY CONDITIONED IN LOSS CHARACTERIZATION OF DELAY AND PACKET LOSS DELAY MODEL LOSS MODEL TEST PLAN 2.1. OBJECTIVES PHASE 1-NETWORK CHARACTERIZATION PHASE 2-NETWORK EMULATION PHASE 3-PERFORMANCE EVALUATION OF SURGICAL TRAINING APPLICATIONS 15 REFERENCES 16 2

3 1 BACKGROUND RESEARCH 1.1 INTRODUCTION Previous research performed in the area of evaluation of surgical training applications considered the impact that network metrics such as packet loss, delay, and jitter would have on the perceived quality of the application [6]. The research considered in this case analyzed the degradation of the quality that software applications interconnected through the network will experience by varying every network metric individually. We have observed that in some scenarios the different network metrics change simultaneously and consequently their combined effect has a specific impact on the perceived quality of the applications. These scenarios are different from those where the network metrics impact the quality of the application when they are modified individually while keeping the other metrics constant (e.g., just having a specific level of end-to-end latency between two networked applications mainly due to propagation delay experienced in the network connection). Some network metrics such as packet loss and network delay change simultaneously when the networking equipments that interconnect the applications start experiencing increased levels of congestion. In consequence, to account for this interaction it is necessary to go beyond the individual variation of each metric and to analyze the impact of the simultaneous variation of packet loss and delay. An ideal scenario for a network connection used to interconnect the software applications is that this connection is always uncongested. It has been experienced in practice that other network traffic (i.e., internet Traffic) might cause congestion along the end-to-end route that is used to transport the traffic of the software applications. Therefore it is important to understand and analyze the level of correlation that exists between packet loss and network delay as a consequence of congestion and to study the impact that this correlation has in the users perceived quality of software applications used for surgical training. 1.2 PACKET LOSS AND NETWORK DELAY INTERACTION Traditional Internet applications such as telnet and ftp use TCP as their transport layer protocol, and depend on TCP congestion control when there is congestion in the network. In TCP, a sender increases its transmission rate additively until it experiences a packet loss, which is taken as an indication of congestion. The sender then decreases its transmission rate multiplicatively, thus reacting quickly to the inferred congestion. As a result of this behavior, the transmission rate of the TCP sender is determined by the level of congestion in the network. Continuous media applications, on the other hand, usually react to network congestion with less flexibility (if at all), due to their more stringent timing constraints. Continuous media applications can be not only loss-adaptive, but delay-adaptive as well [1]. Such adaptive applications keep track of packet delays, and reflect any change in packet delays in their calculation of playout time. Given the above discussion, it is clear that packet loss and delay have tremendous impact on continuous media applications. Both loss and delay result from buffering within the network. As 3

4 packets traverse the network, they are queued in buffers (thus adding to their end-end delay) and from time to time are dropped due to buffer overflow. Consider then a continuous media packet stream at a buffer that is filling up fast with packets from other traffic sources as well. The packets from the continuous media application continue to be queued up in the growing packet queue together with the packets from other sources. Continuous media packets arriving at the receiver experience progressively higher end to end delays than earlier packets. When the buffer reaches its capacity, packet losses begin to occur. The receiver of the continuous media application thus sees increased delay, and eventually losses. Consider next a scenario in which packets from a continuous media application arrive at a buffer that is already full. In this case, they are dropped. As other sources (e.g, TCP connections) detect congestion and decrease their transmission rate, the queue length at the buffer will decrease, and packets from the continuous media application will start to be queued, rather than dropped. In this scenario, the receiver sees losses followed by high, but possibly decreasing, packet delays. The two examples above are plausible scenarios. In the first example, the receiver could have taken the increased delay as an indication of likely future packet loss; in the second scenario, the decreasing delay following loss indicates a future uncongested period of time. In both cases, the application could adapt its behavior accordingly. But do we expect such scenarios to occur (or be detectable) often in practice? Note that there are a number of implicit assumptions in the discussion above, including a single bottleneck link and presumed behavior by other competing applications. If delay is indeed affected by loss, or vice versa, how temporally close are they? What is causing such correlated behavior in the network? How can we exploit such information at the end-system? These are some of the issues that we address in the following section. 1.3 ANALYSIS OF THE CORRELATION BETWEEN PACKET LOSS AND DELAY Moon et al. [1] analyzed the correlation between packet loss and network delay considering that those events occur in sequence (e.g. delay precedes packet loss and packet loss precedes delay as pointed out in the previous section). In this approach, Moon establishes the analysis considering that from an event perspective, when a packet is dropped by the buffer (e.g, the packet is lost) then its delay cannot be determined. Therefore it is necessary to quantify the correlation between packet loss and delay using a metric that conditions network delay to the occurrence of packet loss. To collect end-end delay and loss data for a continuous media source, Moon sends packets with RTP headers at a fixed, periodic rate between workstations located at two remote end points connected through the Internet. The default RTP header has a fixed length of 12 bytes, and includes the version number, sequence number, media-specific timestamp, and source identifier. The sequence number is increased by one per packet, while the increment of a timestamp is dependent on the payload type. Moon maintains a log at both the source and the destination of real timestamps from the system clock along with the RTP sequence number and media-dependent timestamps; these are used to calculate end-end delay. To minimize the load inside the network, each packet had only the 12-byte RTP header, and did not carry any payload data. The metric used for correlation is the sample mean delay conditioned in loss. We first define the 4

5 unconditioned sample mean delay as follows:.(1) In this case: -N is the length of the trace of packets captured during each measurement session - i is the packet sequence number ( 1 < i < N ) - M number of packets that are delivered to the destination. - di is delay of the packet. Moon introduces a lag in calculating the sample mean delay conditioned on loss. Specifically, the sample mean delay, conditioned on a loss occurring at a time lag j packets in the past, is the sample mean of delay of all packets in the trace that have a loss j packets before them in the trace. The following notation allows to clarify the expression of sample mean delay conditioned on loss: li :indicates whether a packet is lost or delivered, specifically: If li = 0 then di = 0. Therefore the delay of a packet that has been lost is taken as zero. The sample mean delay conditioned in loss is defined as follows: The normalized sample mean delay conditioned in loss is defined as follows: (2)..(3) If the sample mean delay conditioned on loss at a positive lag of j is higher than the sample mean delay (i.e., the delay averaged over all received packets), it means that packets that arrive j packets after a loss have a higher average delay than a randomly chosen packet. That is, a loss occurring j packets in the past can be taken as a precursor to a higher delay later. A literal interpretation of a higher conditional average delay at a negative lag is less intuitive. In such a case, a loss can be thought of as an indicator of higher delay in the past. (This is the situation where high delay precedes the occurrence of packet loss). The sample mean delay conditioned on loss at a positive lag allows us to look at delays of future packets following a loss event. 5

6 1.3.1 Loss Conditioned in Delay To look at future loss conditioned on current packet delay, we can count the number of packets lost at a specific lag from a packet experiencing a given delay. This is expressed by the following:..(4) In this case Cj is counting the losses conditioned on a packet at time lag j having a delay above a given threshold, T. By varying within the range of and, we can observe how sensitive the losses are to the delays. Figure 1 plots the loss conditioned on delay as in (4). The threshold values used in calculation are 1, 1.5, and 2 times the unconditional mean delay. Figure 1 indicates that the loss conditioned on delay as a function of lag, computed via (4), is not particularly sensitive to the threshold value. Figure 1 Loss Conditioned in Delay Delay Conditioned in Loss In case of Delay conditioned in loss Figure 2 plots the normalized sample mean delay of six traces by using the equation (2). The range of the lag is between -500 and 500, which corresponds to 10 seconds in both positive and negative directions. All traces show higher loss-conditioned average delay near lag 0. This can be explained by an FCFS buffering process where packets that enter the queue just before an overflow experience a large queuing delay (since the queue is nearly full). Similarly, those packets successfully entering the buffer soon after an overflow event will also see a large queuing delay, since the queue is still nearly full. Another general observation of the graph is that the loss-conditioned average delay shows periodic behavior, dropping below the value of the unconditional mean delay at a given time lag and then rising again above that value for a larger lag. Traces 2.1 and 2.4 have a trace loss probability of less than Their graphs show a well-pronounced rise between lags of 50 and 100, a few more after lag 100 and when lags are negative. As the loss probability increases from those traces to Traces 2.3, 6

7 2.5, and 2.6 in an increasing order, the correlation graph flattens out. Trace 2.1 Trace 2.2 Trace 2.3 Trace 2.4 Trace 2.5 Trace 2.6 7

8 Figure 2 Delay Conditioned in Loss The analysis of the delay and loss measurements using the sample mean of delay conditioned on loss shows that it is likely that the packet delay would increase in a near future of a packet loss is detected. The loss conditioned on delay, on the other hand, is shown to be not sensitive to the threshold values of delay. Also we observe that some traces exhibit an oscillatory behavior when viewed using the sample mean delay conditioned on loss. 1.4 CHARACTERIZATION OF DELAY AND PACKET LOSS In the previous section we provided a characterization of the correlation between packet loss and delay. In this section we describe models used to characterize each parameter individually. Our motivation is to find models that would provide a mathematical description of the probability distributions for packet loss and network delay respectively Delay Model Bolot [2] proposes an interesting model in order to characterize network delay. The analysis of delay is based on the following queuing model : Figure 3 Bolot Queuing Model Where D represents the propagation delay (fixed) experienced by the packets as they travel from source to destination. The analysis is based on two successive applications of Lindley s recurrence equation. Lindley s recurrence equation expresses the relationship between the waiting times of two successive customers in a single channel queue. Specifically, let Wn denote the waiting time of packet n, Yn is the serving time of packet n, and Xn denote the interarrival time between packets n and n+1. Then..(5) A graphical proof of Lindley s recurrence equation is shown in Figure 4. 8

9 Figure 4 Lindley s Recurrence Equation We will use x+ to denote max(x,0). With this notation the above equation can be rewritten as.(6) We now go back to the queuing model. The model can be though of as a slotted-time model in which slot boundaries are defined by probe arrival times. We assume that the first probe packet arrives at the queue at time ( represents the interval between send times for probe packets), and hence that probe packet arrives at time. We refer to the interval As slot n. We assume that the Internet stream contributes Bits during slot n. Becomes a random variable, which characterizes the traffic pattern of the Internet stream. We further assume that all Bits arrive at the queue at the same time. Clearly Let and Denote the waiting time of the Internet packet. Applying Lindley s recurrence equation to we obtain. (7) In the above equation represents the probe packet length and the service rate of the queue expressed in bits/sec. Applying Lindley s recurrence equation to And, we obtain.(8) Substituting equation (7) into equation (8) we obtain.(9) The term Is positive if the buffer does not empty during the interval. Then the above equation becomes 9

10 (10) The term. Then the above equation becomes is positive if the buffer does not empty during the interval..(11) And hence (12) Thus the probability distribution of, I.e. the probability distribution of the Internet workload over an interval of length, can be estimated from the distribution of. Recall, however, that the above equality holds only if the buffer does not empty during this interval, then equation (12) might not hold. Therefore, it is reasonable to estimate Using equation (12) if is sufficiently small, typically if the product is smaller than some average value of. Figure 8 shows the distribution of for From equation (12), we see that..(13) i.e. is the Internet workload, expressed in ms, received by the server during slot n. We note that is also the interarrival time between packets n and n+1 when they arrive to their destination. The distribution approximates a geometric distribution. 10

11 Figure 5 Distribution of For Loss Model Sanneck et al. Yajnik et al and Bolot et al. recommend use of a Markov model to capture temporal loss dependency. All of them analyzed the 2-state Markov model, also known as the Gilbert model (Figure 2). It is simple to understand and to implement in monitoring applications. Figure 6 Gilbert Model In Figure 6 Is the probability that the next packet is loss, provided the previous one has arrived. Is the opposite. Is the conditional loss probability (clp). Normally. If, the Gilbert model reduces to a Bernoulli Model. From the definition, we can compute And, the state probability for state 0 and 1, respectively...(14) In the Gilbert model they also represent the mean arrival and loss probability, respectively., the probability distribution of loss runs with respect to loss length k, has a geometric distribution: 11

12 To calculate and From a packet trace, one can use loss length distribution statistics. Let, Denote the number of loss bursts having length, where Is the length of the longest loss bursts. Let denote the number of delivered packets, then p and q can be calculated as follows...(15) 12

13 2 TEST PLAN 2.1 OBJECTIVES The main objective of the testing is to perform network tests through the transmission of traffic over a set of specific connections (e.g, Stanford - U. Wisconsin) in order to characterize the packet and network delay characteristics with emphasis on the correlation between these network metrics under different congestion scenarios (every congestion scenario requires a simultaneous specification of delay and packet loss probability distributions). This characterization will be used in order to program a network emulator with probability distributions (for delay and packet loss simultaneously) that will reproduce network conditions under the different congestion scenarios. Finally, surgical training applications will be interconnected through the network emulator in order to study the impact on user quality for every congestion scenario that was previously characterized. The testing process is considered to be carried in three phases, which we describe below. The first three steps to be executed as part of the present test plan are motivated by the procedure followed by previous research performed at Summit on performance evaluation [1]. 2.2 PHASE 1-NETWORK CHARACTERIZATION The diagram below shows the design for the testing to be performed in this phase of the project. Figure 7 Test Design Stanford U. Wisconsin XX XX End-Point1a End-Point1b Console Program End-Point2a Switch Router Switch Router End-Point2b Internet2 (Load Generator) Congestion Controlled by Load Generator Probe Traffic Load Traffic Messaging Traffic (Report of Statistics to Console) Control Traffic (Setting-up of Connections and Control of Load Generation) This design is motivated by Lindley s recurrence equation (described in section 3), which is shown below: 13

14 In this equation represents the amount of bits of the Internet traffic stream and represents represents the amount of bits of the probe packet. As we see the waiting time for packet n depends on how big Becomes. In other words the amount of waiting delay in the queue for probe packets depends on the amount of Internet traffic. In this case the higher is, the higher the waiting delay and, in consequence, the higher the level of congestion. As the queue fills up with bigger queue starts discarding packets which increases packet loss. In the diagram shown in figure 7, the workstation labeled end-point1a is intended to generate probe traffic (traffic to be used to perform the characterization). The workstation labeled end-point2a is intended to generate load traffic. In this case our intention is to generate load in the network connection that exists between the two sites (e.g., to make Bigger) so that the probe traffic will face different congestion scenarios. In this way we can control de amount of congestion that we want to introduce in the connection so that our probe traffic will experience different levels of delay and packet loss. We can perform this control by increasing the amount of throughput and/or the packet size of the load traffic (traffic sent by workstation end-point2a). The workstations end-point1b and end-point2b work as reception points. We intend to use a performance tool program (Chariot or Agilent Application Analyzer) to generate probe traffic as well as load traffic through the end-point agents. Workstation end-point1a will also run the Console program, which is the one that will collect the statistics generated by the network testing between end-point1a and end-point1b. This Console will also control amount of load generated by endpoint2a. The objective is to generate a set of probability distributions for delay and packet loss at different congestion levels. Based on the analysis performed by Moon with regards to the correlation between delay and loss, we will first characterize probability distributions for network delay conditioned to different levels on average packet loss. Therefore each level of average packet loss will define a congestion level. Each congestion level will provide a probability distribution for delay and another for packet loss. 2.3 PHASE 2-NETWORK EMULATION Once we capture of probability distributions is completed in phase-1, we will develop a program (based on C++) that will emulate a network that will reproduce the different congestion levels previously characterized. The first approach for this phase will be to modify the source code of NistNET in order to introduce the capability of programming probability distributions for delay and loss. the 14

15 2.4 PHASE 3-PERFORMANCE EVALUATION OF SURGICAL TRAINING APPLICATIONS Once the emulator program is finished, we will study the impact on user perceived quality for the applications interconnected through the emulator program. The impact on quality will be studied under the different congestion scenarios that will involve simultaneous variation of packet loss and network delay metrics according to the characterization performed in phase 1. 15

16 REFERENCES [1] Correlation of Packet Delay and Loss in the Internet. Sue B. Moon, Jim Kurose, Don Towsley Technical Report 98-11, Department of Computer Science, University of Massachusetts, Amherst, MA [2] Characterizing End-to-end Packet Delay and Loss in the Internet. Jean-Chrysotome Bolot. Journal of High Speed Networks, vol2, no3,pp December [3] Modeling of packet Loss and Delay and Their effect on Real Time Multimedia Service Quality. Wenyu Jiang, Henning Schulzrinne. Department of Computer Science Columbia University. [4] Impact of Packet Spacing Time on Packet Loss under Loss Window Size for FEC Based Applications. Teruko Miyata, Harumoto Fukuda, Satoshi Ono. IEICE TRANS.INF. & SYS., VOL.E82-D, No4 APRIL [5] Measurement and Modelling of the Temporal Dependence in Packet Loss, Maya Yajnik, Sue Moon, Jim Kurose and Don Towsley. Department of Computer Science University of Massachusetts. [6] Performance of Remote Anatomy and Surgical Training Applications Under Varied Network Conditions. David Gutierrez, Amol Shah, and Dale A. Harris. SUMMIT / Department of Electrical Engineering Stanford University. 16