Evaluation of BART: a real-time available bandwidth measurement method

Transcription

1 Master Thesis (D Level) Evaluation of BART: a real-time available bandwidth measurement method By: Daniel Rodriguez Retamosa M.Sc. Computer Science with Software Engineering Profile Department of Computer Science and Electronics (IDE) Mälardalen University, Sweden dra05002@student.mdh.se daniel_rodriguezretamosa@yahoo.es Supervised by: Andreas Johnsson Department of Computer Science and Electronics (IDE) Mälardalen University, Sweden andreas.johnsson@mdh.se

2 Evaluation of BART Page 2 of 55 Abstract Proliferation of advanced network applications and services requires a more efficient use of the available bandwidth. The increasing access to the Internet and the greater network bandwidth give cause for the raising of streaming media applications, which need means of knowing the bandwidth and link capacity estimates in order to adapt the send stream to the current network status. End-to-end available bandwidth estimates over a network path can be obtained by active measurement methods. Probe packets are repeatedly injected into the network path with some probe traffic intensity and time-stamped on the source as well as on the receiving-end. Further, statistical analysis of the inter-packet separation on arrival will produce the available bandwidth and link capacity estimates. The major aim of this thesis work is to present and further evaluate an implementation of the BART (Bandwidth Available in Real-Time) method on a wired network laboratory placed at the Department of Computer Sciences and Electronics (IDE). BART is capable of keeping track of the available bandwidth over the network by using a Kalman filter technique along with an active probing scheme for maintaining an updated estimate over time. Furthermore, a survey of integrating available bandwidth measurement methods with live streaming media applications (e.g. Web cast: Internet radio and TV, video stream ) is made within this work.

3 Evaluation of BART Page 3 of 55 Acknowledgements First of all, I would like to thank my supervisor at the Department of Computer Science and Electronics (IDE), Andreas Johnsson for his guidance, trouble-shooting sessions, on-line meetings, good advice and availability. I really hope you liked the experience of working with a guy from down there. I really enjoyed it, learnt and improved in my work at MDH during the period of almost two years. I would also like to take this opportunity to thank my parents Hilda y José Luis for throwing me out of the house and encouraging me to come to Sweden. It is almost two years since that happened. I cannot thank you enough dad for giving the hard-working spirit in me and showing me that alarm clocks are not only a part of the bedside table decoration. I cannot thank you enough mum for the gift of life and for teaching me the pursuit for perfection on all fronts. I thank you both for being my source of inspiration and for your never-ending love and support. I also want to apologize to my dearly beloved Sylvia for the endless wait. Side by side in the distance, always putting coal in the fire. I thank you for your unconditional love and infinite patience. Je t aime. Last but not least, I want to say a few words to the rest of my family and all those friends back in Spain I did not see for such a long time. You made me feel you as though I was always with you. Thank you. Daniel Rodriguez Retamosa MDH, Västerås, April 2007

4 Evaluation of BART Page 4 of 55 TABLE OF CONTENTS List of figures... 6 List of tables Introduction Motivation Thesis overview Theoretical Background IP-based networks Basic network measurement theory Why network measurements? Available bandwidth and link capacity Cross-traffic interaction Measurement methods Passive measurements Active measurements The active measurement process Active probing schemes Areas of use of end-to-end bandwidth measurement applications Addressed issues Related work State-of-the-art measurement tools Bandwidth estimation tools Link capacity estimation tools BART Descriptive overview The BART method Basic features How to use BART? The network model and BART estimation algorithm The network model The BART algorithm Kalman filtering Tuning of the BART characteristics...34

5 Evaluation of BART Page 5 of Evaluation of BART Experimental configuration The testbed Measurement goals The measurement model Experimental results and analysis Problems Final considerations Bandwidth estimation in streaming media applications Future work Summary and Conclusions References... 53

6 Evaluation of BART Page 6 of 55 List of figures Figure 2.1 Example network overview Figure 2.1.a. Encapsulation process. Figure 2.1.b. A generic router architecture Figure 2.3. Classification of bandwidth measurement techniques Figure The two-step bandwidth estimation process. Figure The relation between the offered probe-traffic intensity and the inter-packet strain. Figure 4.2. Cyclic operation of the Kalman filter. Figure Illustration of the measurement testbed. Figure. 5.2.a bytes probe packet size, 17 probe packets/train, 1 train per second. 3 Mbps cross-traffic exponential distribution. Figure. 5.2.b bytes probe packet size, 17 probe packets/train, 1 train per second. 3 Mbps cross-traffic Pareto distribution. Figure. 5.2.c bytes probe packet size, 17 probe packets/train, 1 train per second. 5 Mbps cross-traffic exponential distribution. Figure. 5.2.d bytes probe packet size, 17 probe packets/train 1 train per second. 5 Mbps cross-traffic Pareto distribution. Figure 5.2.e bytes probe packet size, 17 probe packets/train, 1 train per second. 7 Mbps cross-traffic exponential distribution. Figure 5.2.f bytes probe packet size, 17 probe packets/train, 1 train per second. 7 Mbps cross-traffic Pareto distribution. Figure 5.2.g bytes probe packet size, 9 probe packets/train, 2 trains per second. 3 Mbps cross-traffic exponential distribution. Figure 5.2.h bytes probe packet, 9 probe packets/train, 2 trains per second. 3 Mbps crosstraffic Pareto distribution. Figure 5.2.i bytes probe packet size, 9 probe packets/train, 2 trains per second. 7 Mbps crosstraffic Pareto distribution Figure 5.2.j bytes probe packet size, 3 probe packets/train, 8 trains per second. 3 Mbps crosstraffic exponential distribution. Figure 5.2.k bytes probe packet size, 3 probe packets/train, 8 trains per second. 5 Mbps cross-traffic Pareto distribution. Figure l bytes probe packet size, 17 probe packets/train, 1 trains per second. 3 Mbps cross-traffic exponential distribution in both bottleneck links. Figure m bytes probe packet size, 17 probe packets/train, 1 trains per second. 3 Mbps cross-traffic Pareto distribution in both bottleneck links. Figure n bytes probe packet size, 9 probe packets/train, 2 trains per second. 3 Mbps cross-traffic exponential distribution in both bottleneck links.

7 Evaluation of BART Page 7 of 55 Figure o bytes probe packet size, 3 probe packets/train, 8 trains per second. 3 Mbps cross-traffic exponential distribution in both bottleneck links. Figure p bytes probe packet size, 3 probe packets/train, 8 trains per second. 3 Mbps cross-traffic Pareto distribution in both bottleneck links. Figure 6. Layering of real-time protocols. List of tables Table 3.1. Available bandwidth estimation tools Table 4.2. Specific a priori and a posteriori equations of the Kalman filter Table a. Degree of intrusiveness of probing schemes Table b. Cross-traffic size distribution Table a. Measured MSE (Mean Square Error) for several probing schemes and cross-traffic configurations in a simple bottleneck scenario Table b. Measured MSE (Mean Square Error) for several probing schemes and cross-traffic configurations in a multiple bottleneck scenario. Table 6. Common audio and video formats.

8 Evaluation of BART Page 8 of Introduction The importance of the Internet in today s society is huge. It is widely used around the world, the number of network applications is huge and it is seen by many as an indispensable research tool. As the Internet technologies spread and develop, user s machines performance increases and the cost of broadband connections decreases, providers are able to offer more extensive services and solutions to both final users and corporations. End-users demand quality and speed of transmission when using the Internet, especially nowadays that file sharing and real-time audio and video applications are leading a big change in the way we traditionally used networks. Network topologies are not uniformly distributed, i.e., different users might have access to the Internet at different speeds. Moreover, as the number of users grows over a specific network they are forced to share a certain broadband capacity. Therefore, it is evident that network congestion inflicts a negative effect on the transmission rates and quality at independent destinations. Every single end-user has their own bandwidth needs and those may be met independently. Many multimedia applications are bandwidth-sensitive, i.e. they require a given available bandwidth to work properly. There are many others that make a more efficient use of the available bandwidth such as electronic mail, file transfer and Web transfers applications. The latter group may use adaptive techniques to encode data at a rate that matches the current available bandwidth. For example, let us consider a trendy IP telephony application encoding at the maximum rate for human voice (33 kbps). The data must be delivered to the receiving end at the same rate. If such an amount of available bandwidth is not available the application can make use of as much or as little bandwidth as is available at that moment by encoding voice at a different rate. Active measuring of network properties as the available bandwidth is important, depending on the circumstances, for network diagnosis, route selection, measuring the present transmission rate (i.e. the so called available bandwidth), network performance tuning or future performance predictions, so the provider has means of adjusting the amount of data and transmission speed rates accurately to fit both the network status and users demands. Advanced end-to-end available bandwidth measurement methods can be used for such purposes as well as for network applications such as live streaming audio and video Motivation This work focuses on the evaluation of BART, a state-of-the-art end-to-end measurement method over a test wired network. End-to-end measurement sessions have been run in order to prove the accuracy of BART in single and multi-hop scenarios and reveal possible problems and constraints of the measurement scheme. BART relies on self-induced congestion (see section 2.2.2) and it is sensitive to cross-traffic flowing across the network path inspected along with probe packets. The use of both probe packet pairs or larger packet trains and the way BART deals with bursty crosstraffic traversing the end-to-end path to produce more precise measurements in real-time are major issues that have been considered in order to evaluate BART in depth over wired paths. The motivation behind this thesis work, though, is to investigate the aforementioned issues from bandwidth measurements. This report details the progress and outcomes that this project has made towards this goal.

9 Evaluation of BART Page 9 of Thesis overview This thesis is organized in such a way that an insight into the basics of the bandwidth measurements discipline is given in Section 2, together with the different existing measurement methods within this field of study and their most common real-world applications and Section 3 describes some related work. In Section 4, the BART method is presented and described in depth. The author s contribution can be found in Section 5. A measurement process has been carried out in order to evaluate BART and reveal important aspects of its performance by analysing the results obtained. Section 6 discusses the suitability and benefits of integrating end-to-end available bandwidth measurement methods with live stream media applications. Uncovered issues within this work are stated in Section 7 and the thesis work concludes with a final summary and conclusion in Section 8.

10 Evaluation of BART Page 10 of Theoretical Background This section gives a brief introduction to the foundations of IP networks and network measurements field of study. Essential concepts and useful definitions are given throughout the text for a better insight and understanding of assumptions and analysis at further sections of this work IP-based networks IP-based networks are of great importance in today s society of information and IP-based solutions are both flexible and economical. Therefore, the most sensible thing would be to go through the basics of IP networks first and foremost, since networks are the physical environments where measurements are to take place. The following information has been extracted from [1], [2] and [3], simplified and summarized for clarity and overall understanding. Figure 2.1. Example network overview. I. Basics in network communication As a starting point, we identify the underlying network components upon which this technology is built: Nodes also called hosts or end systems. A node is a computing device such as desktop PC s, laptops, workstations and so-called servers that store and transmit information, e.g. web pages or messages. Links connect hosts. There are many types of communication links, which are made up of different types of physical media, e.g. twisted pairs, coaxial cables, fibre optics or radio waves (via satellite links).

11 Evaluation of BART Page 11 of 55 Switches network devices for establishing communication between nodes on a network. The two most used switches in today s Internet are routers and link-layer switches. In concrete a router is a computer networking device that buffers and forwards data packets across a network towards their destinations. Such a process is known as routing. Usually, hosts are not directly connected to each other via a single communication link. From the sending host to the receiving end system, the data may traverse a sequence of communication links and switches. Such a sequence is known as a route or a path through the network. In an IP network, an end-to-end path is a sequence of hops that connects a source host to a destination host. An example end-to-end path is the route HostA-Router1-Router3-HostB in Figure 2.1. A hop is called the way data packets take from one router or any other intermediate network device to another in the network. That is to say, the communication link in between two network nodes. We must not forget that all the information traversing the path is encapsulated in the shape of data packets or datagrams. II. Network communication Hosts, packets switches and other pieces of the Internet run protocols that control the sending and receiving of information within the Internet. There are several different protocols that can be utilized together with each network topology for communication purposes. Aside from identifying the standards of communications between the network devices, a protocol sets the technical specifications needed to transmit data within a network. To transmit a message to another device in a network, the message is split into data packets. These data packets are then transmitted via the communication media and reassembled at the receiving end. End-to-end data transmission comprises two essential processes: packet encapsulation and network switching. They are both explained below. II.a. Protocols stack. The Internet s principal protocols are collectively named after two of the most important protocols used by it: the Transmission Control Protocol (TCP) and the Internet Protocol (IP). Basically, the TCP/IP reference model defines a hierarchical structure consisting of a set of five layers: physical, link, network, transport and application (see Figure 2.1.a). When taken together, the protocols of the various layers are called the protocol stack and any application that supports TCP/IP will also be able to communicate over any IP-based network. IP is a connectionless protocol where each packet is treated as a separate entity, just like in the postal service. Each network device has at least one IP address that uniquely identifies it from all other devices on the network. In this manner, intermediate nodes can correctly address packets sent from the source to the destination. Hence, IP enables data packets to be transmitted across and between local area networks. Each IP packet includes both a header and the message data itself, where the header specifies the source, the destination, and other information about the data. When data coming from the top layer is to be transmitted it is progressively added to additional headers comprising information about different aspects of the end-to-end transmission process. It is said that the data packet is encapsulated in order to translate the data coming from upper layers into a suitable form to be physically transmitted at a later time.

12 Evaluation of BART Page 12 of 55 Figure 2.1.a. Encapsulation process. Every layer adds its own data headers to the message. At the sender s side, the process is carried out from top to down bottom layers below, whereas at the receiver s side the message is unwrapped in the reverse order. The transport layer carries application-layer messages between the client and server sides of an application. There are two major transport-layer protocols, TCP and the User Datagram Protocol (UDP). We will come back to them later on. The network layer is responsible for adding an IP header to transport-layer segments comprising source and destination-end addresses as well as routing information so as the network-layer datagrams can successfully be guided through the network path. The IP network layer implements the IPv4 protocol and its natural successor IPv6, among others such as the Address Resolution Protocol (ARP) and the Internet Control Message Protocol (ICMP). The link layer will add its own header information and create a link-layer frame that will be passed down to the physical layer for transmission. The link-layer header contains information about the characteristics of the link the data frames are to be transmitted through. For instance, link layers include Ethernet and Point-to-Point (PPP) protocols. II.b. Packet switching and queueing IP-based networks utilize a packet-switched network technology, which uses capacity much more efficiently and the IP network service transmits datagrams between intermediate nodes and also between different IP networks (disregarding their physical media or protocols) using IP routers 1. In the latter case, they are specifically known as gateways. But, what is inside a router? Figure 2.1.b shows a generic router architecture consisting of four different components. Intuitively, routers receive data packets on one of its incoming links (input interface) and forward those data on one of its outgoing links (output interface). 1 The testbed used for the measurements consisted of three Linux routers, which are simple computers running Linux OS set to forward probe traffic and cross-traffic (UDP datagrams) to their respective destination hosts within the test network s IP address domain.

13 Evaluation of BART Page 13 of 55 Figure 2.b. A generic router architecture. In practice two or more incoming/outgoing links gather and store packets in input/output queues 2. Incoming packets are forwarded to the switching fabric from the input queues. The switching fabric is the combination of hardware and software that connects the router s ingoing links to its appropriate outgoing links. A routing processor supports the switching by executing routing protocols and managing the routing tables where information to compute the next hop for a packets is stored. Routing tables can be static or dynamic depending on the routing protocols used. Small and invariable networks use static routing tables and protocols like the Routing Information Protocol (RIP). Dynamic routing protocols are applied to larger networks where the topology varies over time and hence simple hardcoded routing tables are inefficient. An example of dynamic routing protocol used in Internet is the Open Shortest Path First (OSPF). Notice that the routing process occurs at the network layer (layer 3 in Figure 2.1.a) Most routers store the entire incoming packets before they can begin to transmit the packet via any outgoing link. This technique is called store-and-forward transmission and induces a per-packet delay onto the routing process. Moreover, given that packets from different links can reach the routers at the same time, output queues may be busy and therefore packets may suffer from output queuing delays. These delays are variable and depend on the level of congestion in the network. Since storage capacity at network routers is limited, packet loss will occur under higher levels of congestion when one or more router queues are full and the IP router is forced to discard either the incoming packet or any of the buffered packets. The Internet provides a best-effort approach to deliver packets in a timely manner, but it does not make any guarantees at all. A best-effort network tends to preserve both timing and ordering in which packets were sent. 2 A first-in-first-out (FIFO) queuing policy has been assumed for the input and output queues (limited buffers). A FIFO discipline implies that incoming data packets are forwarded to output queues in order of arrival. That is, if a packet P 1 reached a router before certain packets P 2 and P 3, P 1 will be forwarded first.

14 Evaluation of BART Page 14 of 55 III. UDP vs. TCP To compensate for the packet loss occurrence, hosts rely on transport-layer protocols since neither link layer nor the network layers provide any means for retransmitting lost packets. By using TCP, end hosts can create reliable connections to one another that ensure full in-order data transfers, meaning the delivery of data without error and in the proper order. TCP also includes congestion-control services to provide a fair use of network bandwidth. On the other hand, UDP is a packet-based, connectionless, best-effort service. It provides no reliable data transfer so that a source never knows which packets will ever reach the destination. Unlike TCP, it makes neither flow control nor congestion control. The aforementioned TCP features have some implications for real-time applications, as a measurement method or a VoIP application. Congestion-control mechanisms regulate the transmission rates not to overload TCP connections over the network. Nevertheless, that can have a important effect on loss-tolerant applications that have a minimum required bandwidth constraint (real-time audio and video). The use of a fully reliable protocol to take over the transmission, as TCP is, may slow down the streaming several 100s of milliseconds so compromising the quality of service (QoS) demanded by end-users. Let us remember the fact that for real-time properties to be guaranteed, a network with QoS must be used to provide fixed delay and bandwidth. For these reasons, the chosen transport protocol for real-time applications is usually the UDP. Moreover, UDP is used for example by applications like the Simple Network Management Protocol (SNMP), Domain Name System (DNS) and the Network Time Protocol (NTP), while TCP is used by the File Transfer Protocol (FTP) and Simple Mail Transfer Protocol (SMTP). Within the following subsections, the reader is provided with an overview of the network measurements research area, with focus on active bandwidth estimation category in IP-based networks Basic network measurement theory Bandwidth estimation is a non-trivial network performance measurement technique where accuracy is difficult to obtain, particularly in large and high-speed networks. That is partly due to the number of existing bandwidth-related metrics: Capacity, Available bandwidth, Bulk Transfer-Capacity (BTC) and achievable TCP Throughput, among others [4]. State-of-the-art bandwidth estimation tools such us Pathload [5], Pathchirp [6], TOPP [14][15] or BART [7] measure and analyse some or all of these metrics to make accurate bandwidth measurements by only applying traditional besteffort services. On the other hand, the most common metrics typically measured by Internet Service Providers (ISP) are latency (propagation delay, one way delay, round-trip time (RTT), queuing delay), packet loss, TCP throughput, link utilization and availability [10][24] Why network measurements? Network measurements attempt to reveal the heterogeneous nature of the Internet. Different network topologies make bandwidth conditional on distribution and inflict variability on other network performances, e.g. end-to-end delay, packet loss, etc. Therefore, some of the challenges of network measurements are:

15 Evaluation of BART Page 15 of 55 Improvement of end-to-end performance, enabling advanced network capabilities (e.g. dynamic tuning of transport protocols) Network performance prediction, aiming a more efficient utilization of networks. In this work we focus on active bandwidth measurements (see Figure 2.3). As a first approach, active network measurement systems are based on the injection of probe traffic to be transmitted between, at least, two end-points in the network, providing end-to-end performance information and helping us to improve our understanding of Internet traffic dynamics Available bandwidth and link capacity At this point, we cannot allow ourselves any delay in giving definitions of essential concepts mentioned previously in the text: link capacity, available bandwidth and cross-traffic. The definitions of the metrics are taken from [4] and will be quoted constantly in further sections, so there is a need to formally set what we exactly refer to. A. Link capacity The capacity C is the maximum transmission bit rate at which data packets can be transmitted by a single communication link. Both the physical transmission media and the hardware used in transmission are factors that restrict the capacity of a link. An end-to-end path may be composed of more than one hop. An intermediate hop is incapable of delivering full link capacity to a switch s network layer (layer 3 in Figure 2.1.a). Instead, a lower transfer rate is achievable because of link-layer packet encapsulation, which in fact induces an overhead on transmitted data. This overhead varies for different link-layer protocols such as PPP (8 bytes) or Ethernet (38 bytes). Therefore, for a given end-to-end path of H hops, the hop that delivers the minimum link-layer throughput limits the capacity of the whole end-to-end path. To say: C = min i 1,..., H = C i (1) where C i denotes the capacity of link i. The hop having the minimum capacity is called the narrow link. In what follows, we will refer to the narrow link as the bottleneck link of the end-to-end-path [8]. B. Available bandwidth A key concept that we will now turn our attention to is the available bandwidth of a link and end-toend path. Intuitively, the available bandwidth is the unused portion of the link capacity. However, this metric needs a meaningful definition to be fully understood. The available bandwidth is a dynamic metric that changes over time and strongly depends on the instantaneous utilization of a link at a time t (a link is either idle or transmitting packets at its maximum speed). Hence, if we intend to measure the available bandwidth we must take into account the average utilization of the link as a function of time.

16 Evaluation of BART Page 16 of 55 u i (t 1,t 2 ) = t 1 1 t 2 t t 2 1 u(x)dx (2) where t 2 = t 1 +τ and u(x) determines the link is idle (0) or transmitting (1) at time x. Given an end-to-end path of H links, the average available bandwidth A i of a hop i in the interval (t 1, t 2 ) is: A i = (1 - u i ) C i (3) Thereafter, the available bandwidth of the whole end-to-end path is the minimum A i. A = min i 1,..., H = A i (4) The hop having the minimum available bandwidth is called the tight link [8]. Among the factors that impact the measurement of the available bandwidth we find the routing changes (capacity changes), sensitivity to transient link errors or effects of the overall traffic load on the probe traffic [4]. C. Cross-traffic In a simplistic way, cross-traffic is defined as all the representative non-probe traffic flows traversing a network, e.g. TCP flows in the Internet Cross-traffic interaction The regular traffic flowing through a network is the most determining factor for bandwidth estimation techniques. Cross-traffic burstiness (variability) induces congestion on network switches and thus the accuracy of the available bandwidth estimates may be affected due to increased oneway delays of probe packets or probe packets loss. In order to describe cross-traffic behaviour, a number of variables must be taken into account. Far from being a constant process, cross-traffic interaction is quite variable. In experimental testbeds, cross-traffic features can be controlled but unfortunately that is not the case in larger and more heterogeneous networks like the Internet, where in turn the cross-traffic nature and load varies along with different applications and users accessing a shared portion of the network. The following basic factors determine the behaviour of the crosstraffic [11]: Intensity and packet size distributions. Depending on the cross-traffic generator, the crosstraffic flows are injected into the network at different rates (dissimilar separation between consecutive packets) and are composed of packets of diverse lengths. Intensity distributions normally follows statistical functions from uniform (light) to exponential or Pareto (bursty). Flow length. A flow is a set of packets flowing through the network path. It is in relation to the time a certain cross-traffic flow remains active. Aggregation. The degree of aggregation grows along with the number of cross-traffic flows

17 traversing the network concurrently. Evaluation of BART Page 17 of 55 It has been mentioned that cross-traffic packets interacts with probe packets at network switches. The work in [16] describes the interaction patterns observed from measurements and study how they affect the analysis phase performed by available bandwidth measurement methods. Many other research papers also inspect the implications of cross-traffic in probing techniques from both perspectives the cross-traffic and the probing methodology [29,31] Measurement methods The network measurement field of study embraces two main groups of measurement techniques: passive and active measurement methods. Figure 2.3 shows a complete classification of the existing branches in this area and the category being tackled within this work (highlighted in aquamarine). Figure 2.3. Classification of bandwidth measurement techniques The first attempt at using bandwidth estimates can be tracked back to 1996, when Bprobe/Cprobe [12] were introduced as tools for server selection based on capacity estimation. A year later, in 1997, Pathchar [13] appeared and it was introduced as a per-hop network capacity estimation tool. From that time up to the present day, the network measurement area has matured and a number of passive and active methods and tools have been developed and implemented to give shape to this research area. We now present the two most important ways of measuring network characteristics and performance.

18 Evaluation of BART Page 18 of Passive measurements Passive measurement systems attempt to measure the behaviour and performance of packet streams by monitoring the traffic without modifying it [25]. In comparison to the active measurements, the passive measurements do not inject probe packets into the network. Instead, they rely on observations of user-generated traffic, performed at network devices (e.g. servers, switches and routers). Passive measurements thus require access to those devices to retrieve information corresponding to the most typical metrics collected using some of the popular passive measurement techniques. Collecting Simple Network Management Protocol (SNMP) data is one of these techniques. SNMP data provides switch-level and router-level information such as availability, utilization, packet errors and discards: Throughput refers to the amount of data carried on the link commonly measured in raw bits per second (bps) or refined for statistical analysis. As an example of the latter is the measurement of traffic type distribution by application-layer protocol (HTTP, SMTP, FTP ). Utilization the proportion of the link being used during a time interval relative to the link capacity Others packet size distribution, packet timing, packet loss rate, etc. gives a more detailed insight into the network performance The main drawback of passive measurement methods is that their measurements of the network metric of interest only correspond to a concrete point in time over a certain link inside the network path that has recently registered user traffic. Moreover, the use of these passive tools is restricted to those people having full privileged access to the core network and network devices. An example of well-known passive measurement tool is MRTG [26], a tool for overseeing the traffic load over network links. The available bandwidth can then be easily calculated using (3) if the link capacity is known Active measurements Active probing has become one of the main ways in which the performance of IP networks such as the Internet is measured. Active measurements and analysis of networks provide a valuable insight into network end-to-end performance that could not be obtained through passive measurement based systems. All active measurement methods rely on the concept of self-induced congestion, which is the injection of so-called probe traffic (UDP-based probe packets) with a predefined dispersion (interpacket separation) into the network. Probe traffic is intrusive to some degree, i.e. it causes little congestion for a short time period of time, so it is important to tune active measurement methods to achieve a good accuracy relative to the overhead they induce on the network path. At the sender, every injected probe packet is time-stamped and their final dispersions (samples 3 ) measured at the cooperating receiving end. From the known packet sizes, and the measured timestamps of emission and reception, an estimation algorithm is able to infer the end-to-end available bandwidth. Probe traffic is affected by the portion of the network it traverses and therefore the initial dispersion may change due to cross-traffic [16] and/or the bottleneck spacing effect [27][28]. The 3 According to [41], a sample is the separation in time between probes packets n and n-1 in a pair/train.

19 Evaluation of BART Page 19 of 55 bottleneck link slows down the transmission (higher service time) so the packet dispersion will increase (the dispersion is inversely proportional to the probe rate). That is, the packet inter-arrival time measured at the receiver is determined by their spacing on the bottleneck link. Typical end-to-end metrics measured by active methods are: Latency - It is about the time needed for a packet to reach the destination. It involves components such us propagation delay, queueing delay or service delay. A very traditional application used for measuring network latency is the ping application. One-way packet delay - It is the time from the source sending a packet across an IP network to the destination. One-way packet loss. It is the portion of packet lost along an end-to-end connection. Bandwidth-related metrics. Link capacity, available bandwidth or TCP throughput. The latter measures the amount of data carried by a TCP connection. A metric, the Bulk Transfer Capacity (BTC) is defined for performance issues and represents the maximum achievable throughput by a single TCP connection [4]. As in most cases, models are powerful to some extent but also fall victim to shortcomings. Active measuring has the advantage of being very flexible, given their ability to adapt the probing scheme to focus on different measurement metrics or bandwidth estimates. The analysis of those metrics is also faster and easier since the data generated by the process is largely reduced compared to passive monitoring. In turn, the main drawback of active measurement methods is that probing induces an overhead that perturbs the traffic in some way that is intended to be measured. It is important though to reduce as much as possible the degree of intrusiveness. Among well-known active end-to-end-available bandwidth estimation tools we find Cprobe [12], Pathload [5], Spruce [30][8], TOPP [20][33] and the most recent Pathchirp [6], IGI [29] and BART [7] methods. The basic essentials of some of them will be briefly explained in Section 3. Nowadays, there exists a new trend based on coordinating active and passive measurement studies. For instance, work in [22] presents the Change-of-Measure Based Passive/Active Monitoring (CoMPACT Monitor), which combines estimates of the actual performance seen by users are obtained based on both active and passive measurement data The active measurement process In this section we present the basics of the active end-to-end bandwidth estimation process. Active measurement tools such as TOPP, Pathchirp or BART usually have two phases of operation. The first one corresponds to the probing session, in which probe traffic is sent across the network and captured at a receiving end. In the second phase, called the analysis phase, end-to-end bandwidth estimates are calculated based on the packet dispersions previously measured at the receiver s end during the probing phase. The analysis phase typically performs statistical calculations on the inter-packets separation values and in some cases filtering of useless data or system noise.

20 Evaluation of BART Page 20 of 55 I. Probing phase. Following the notation in [31], a probe-traffic sender injects probe packets into the network path at an input rate R i and packet size L. The initial inter-packet separation is then given by δ i = L/ R i. As explained in section 2.3.2, the predefined packet dispersion may change after the packets have traversed the network, i.e. the dispersion δ o at the receiver, seen as the inter-arrival time between two consecutive probe packets, is δ o = δ i + δ i. Then the output rate at the receiver can be easily computed as R o = L/ δ o. II. Analysis phase. In this phase the relation between the initial and final packet dispersions are inspected in the following way: a) If δ o > δ i, the probe traffic caused congestion in the network and therefore the input probe rate R i is greater than the available bandwidth A ( R i > A ). b) If δ o = δ i, no congestion is induced and therefore the input probe rate R i is less or equal to the available bandwidth A ( R i A ). Figure The two-step bandwidth estimation process The relation δ o / δ i describes the rate response curves in the case of congested/notcongested paths. Consider the specific case of b) where the ratio δ o / δ i is equal to 1. In case a) the value of the relation will deviate from 1. Let us concentrate now on the sloping segment in Figure To derive the value of the available bandwidth, active applications point out to the intersection of the two segments. The point beyond which the sloping segment starts to deviate from unity, will be considered the actual value of the available bandwidth in the path.

21 Evaluation of BART Page 21 of 55 Obviously, applications use their own approaches to this process so the way they perform both phases may slightly differ from each other. However the underlying idea applies to each of them. A detailed statement still has to be made about active measurements. All existing estimation tools and techniques can be classified into one of the following approaches under the aforementioned probing framework: direct probing and iterative probing 4. Direct probing In this approach, each probing stream is injected at a fixed input rate and produces a unique sample of the available bandwidth given by the formula: A = C t - R i Ct Ro 1 (5) where C t is the known tight link capacity. Spruce and Delphi [32] are methods that perform direct probing. Iterative probing In this approach, no samples of the available bandwidth are generated. Instead, probe streams are injected with varying input rates to infer the available bandwidth from the observation of the value the different probe streams converge to. A simple binary search may be used to deduce that value from the range of stream rates. Methods that apply such a technique are TOPP, Pathload, Pathchirp and BART Active probing schemes A number of schemes for estimating capacity and available bandwidth have been developed. According to the work in [4], the most widespread and representative techniques are variable packet size probing (VPS), packet pairs/train dispersion (PPTD) and self-loading periodic streams (SLoPS). Implementations of these techniques can be found in the tools listed in Table 1 (section 3.1). The performance of each technique usually provides insights into how the network reacts to a certain traffic pattern. For a good understanding of the interaction patterns between probe and crosstraffic packets, the author recommends the reading of [16]. Variable packet size probing (VPS) The methodology proposed by VPS consists of applying probing to measure the capacity of every link (hop) in a path. This task is carried out by measuring the variation of packets Round- Trip Times (RTT) from the source to each link as the probe-packet size increases. This technique has no means of dealing with the problem of hidden bottlenecks (invisible hops at the IP layer) along the path and therefore it may underestimate the capacity. 4 Direct and iterative probing approaches are equivalent to the Probe Gap model (PGM) and the Probe Rate model (PRM) respectively [8].

22 Packet pair/train dispersion (PPTD) Evaluation of BART Page 22 of 55 The packet-pair probing scheme is a good mechanism to measure the capacity of and end-toend path. The mechanism consists of sending two consecutive equal-sized packets through the network to the receiver. As the packets flow across the network path, significant competing cross-traffic or bottleneck spacing effect may increase the initial gap between packets in a pair. The dispersion values are used at the receiver to compute the bottleneck bandwidth estimates. The packet-train probing scheme is based on the same principle as the injection of packet pairs. The inter-arrival times of packets in a train are measured at the receiver s end from which the available bandwidth estimates can be easily derived. The difference with the packet-pair approach is that trains consist of multiple packet pairs sent back-to-back and the dispersion between packets in a train can be equal or proportionately decreasing. There are interesting studies of the effects of cross-traffic on PPTD in [24] and [31]. Self-loading periodic streams (SLoPS) It is a modern technique for measuring end-to-end available bandwidth. The probing session consists of sending periodic streams of equal-sized packets at different rates and measuring variations in one-way delays of such streams Areas of use of end-to-end bandwidth measurement applications The end-to-end nature of active systems makes them suitable to be applied in many ways. Active bandwidth estimation tools provide network operators with useful information on network status and performance. Other areas that can also benefit from end-to-end available bandwidth measurement applications are: Service-Level-Agreements (SLA s) check and verification of the service between ISP s and customers. The capacity offered by ISP s must be upgraded based on the bandwidth demands of users. On the other hand, SLA s enable users to verify the speed of the Internet connection provided by ISP s, e.g. by using TPTest [21]. Generic server selection active available bandwidth measuring is applied in order to ascertain the server that offers the shortest download time. Real-time transport and streaming see Section 6. Peer-to-peer networks aims the construction of suitable application-layer topologies based on the bandwidth available between peers. End-to-end admission control checks for sufficient bandwidth prior to the sending of data between end hosts. A session is accepted if the probes are received with no or moderate loss. TCP improvement the work in [23] introduces active available bandwidth measurement techniques to accurately and quickly adapt the TCP start-up send rate to the available bandwidth. Another example of improving TCP performance is described in [24]. TCP packets can be used for in-line bandwidth measurements, decreasing the amount of probe

23 Evaluation of BART Page 23 of 55 traffic and showing negligible effects on other network traffic. Active methods can also be applied to enhance congestion control mechanisms. In the case under discussion, given the rise in popularity of wireless technologies, researchers are starting to turn their attention towards wireless measurements [17]-[20], [35]. In this way, active available measurement methods could be applied to mobile applications Addressed issues The following sections 5 and 6 contain the author s contribution to this work. As mentioned before, in Section 5 an implementation of BART [7], a state-of-the-art method for active available bandwidth estimation has been evaluated. In Section 6, a theoretical insight into the modern streaming media technology is given together with a survey of some specific streaming media applications that can make use of BART.

24 Evaluation of BART Page 24 of Related work The author has made the effort to compile the necessary and basic theory and information about the bandwidth measurements field of study throughout the entirety of Section 2 of this work. The most important and widely used measurement techniques, schemes and tools have been presented and properly explained to give the readers an insight into the foundations and mechanisms of bandwidth-oriented measurements. More information about theoretical issues, different techniques and measurement tools can be easily found in research papers and links used as references for this work. The bandwidth estimation area has grown and matured in recent years. As a consequence, the field has expanded and many new tools and research projects have emerged. Although we mostly focus on the evaluation of BART, many other state-of-the-art end-to-end available bandwidth measurement tools are based on the same principles and aim to serve the same purposes. Comparative evaluations between implementations of different active measurement tools have been carried out. [5] and [6] present and test the performance of Pathload and Pathchirp respectively with simulations and Internet experiments. The TOPP method is presented and tested in [33] to estimate bottlenecks and the available bandwidth of congested links. Pathchar is addressed in [13] as a tool to infer important characteristics of Internet paths. Jacob Strauss [30] also has a paper comparing Spruce, Pathload, and IGI. Finally, a previous evaluation of BART in [7] shows a good comparative evaluation with Pathchirp in testbed and Internet scenarios. In the next section, 3.1, we attempt to show which active measurements tools are more frequently used for bandwidth and link capacity estimation, as well as some of their most important characteristics State-of-the-art measurement tools The taxonomy of active available bandwidth estimation tools together with the metrics and probing methodologies used for the measurements is shown in the following table. Tool name Measurement metric Methodology Pathchar Clink Pchar Per-hop capacity Per-hop capacity Per-hop capacity Variable Packet Size Variable Packet Size Variable Packet Size Bprobe Nettimer Pathrate Sprobe Cprobe Pathload TOPP IGI Pathchirp BART TReno Cap Ttcp Iperf Netperf End-to-end capacity End-to-end capacity End-to-end capacity End-to-end capacity End-to-end Available Bandwidth End-to-end Available Bandwidth End-to-end Available Bandwidth End-to-end Available Bandwidth End-to-end Available Bandwidth End-to-end Available Bandwidth Bulk Transfer Capacity Bulk Transfer Capacity Achievable TCP Throughput Achievable TCP Throughput Achievable TCP Throughput Packet Pairs Packet Pairs Packet Pairs & Trains Packet Pairs Packet Trains Self-Loading Periodic Streams Self-loading Packet streams (Pairs) Self-Loading Periodic Streams Self-Loading Packet Chirps Self-loading Packet streams (Pairs or Trains) Emulated TCP Throughput Standardized TCP Throughput TCP connection Parallel TCP connections Parallel TCP connections Table 3.1. Available bandwidth estimation tools [4]

25 Bandwidth estimation tools Evaluation of BART Page 25 of 55 The tools described below are all classified under the category of iterative probing and rely on the concept of self-induced congestion to calculate the available bandwidth on end-to-end paths. TOPP The basics of the TOPP operation are the injection of same-sized well-separated probe packet pairs into the network and the measurement of the resulted dispersion in a pair at the receiver s end. TOPP runs one-way measurements and performs a regression-based analysis on the measured dispersions to calculate the equation of the sloping segment (see section ), thus obtaining the available bandwidth estimates. Dietopp [20] is an implementation of the Trains of Packet Pairs (TOPP) model [14][33]. DietTopp does not use probe-packet pairs but probe-packet trains instead. Pathchirp Pathchirp injects so-called chirps into the network path. A chirp is a probe train with exponentially decreasing separation between probe packets. A statistical analysis based on packet queueing delays (FIFO policy assumed in this model) performed at the receiver s end gives a per-chirp estimate of the available bandwidth over an end-to-end path. The advantage of chirps over probe trains using fixed dispersions between probe packets is that a chirp explores a whole range of input rates within one chirp [6]. BART (Bandwidth Available in Real-Time) BART uses a real-time approach to obtain estimates of the available bandwidth.. The underlying probing model is based on the TOPP model with one difference: BART injects probe trains into the network and uses an analysis mechanism based on Kalman filtering, which enables us to describe the sloping segment by means of a constantly updated state vector. More detailed explanations of the BART mechanism and Kalman filtering are given in Section Link capacity estimation tools The most representative of the direct probing methods is probably Pathchar but there are other tools such us Pathrate [34] that are capable of estimating the tight link capacity on and end-to-end path. Furthermore, tools like TOPP and BART are able to generate estimates of the link capacity since for this model, the bottleneck link capacity is considered to be the inverse of the sloping segment in Figure Pathchar It is a per-hop capacity estimation tool that attempts to discover link characteristics (latency, bandwidth, propagation delay, queue delay). It sends variable packet size probes over defined periods of time to each hop in a path. The measuring performed by Pathchar focuses on the round-trip delay between the release of UDP packets with a certain life expectancy (TTL) and the receipt of their corresponding ICMP error messages from the network switches and routers [25]. More about Pathchar in [13].

26 Evaluation of BART Page 26 of 55 Pathrate Pathrate is a tool that can estimate the capacity of network paths. An important feature of Pathrate is that it is able to estimate the path capacity even when the path is heavily loaded with cross-traffic. The methodology used by Pathrate has two phases. Firstly a packet-pair measurement session using packets of different sizes is launched in order to create a set of possible capacity values. Secondly, a packet-train measurement session is performed in order to obtain the average dispersion rate (lower bound) of the path. Finally, Pathrate estimates the capacity as the strongest capacity value relative to the dispersion.

27 Evaluation of BART Page 27 of BART The main objective of this section is to provide a good explanation of the network model used by the BART method along with the presentation of the main features that make this state-of the-art measurement method suitable for real-time tracking of available bandwidth. The Kalman filter technique supports and contributes largely to this purpose wherefore it is a key element that demands our attention Descriptive overview BART was developed within the Evalunet project, sponsored by Vinnova and the KK foundation, as a result of the collaboration between Ericsson, SICS and Mälardalens University and its development is related to the Ericsson participation in the research project EVERGROW (supported by the European Comission) within the period BART code is written in C++ and runs under some GNU/Linux distributions such as Fedora Core or Ubuntu. Nevertheless, it has been run under some versions of BSD (FreeBSD), MacOS and Solaris. More information about BART is accessible at The BART method BART (Bandwidth Available in Real-Time) is a method for estimating the dynamic available bandwidth between a sender and a receiver (end-to-end) over a packet-switched network [7]. TOPP is considered to be the precursor of BART and it likewise relies on self-induced congestion. BART is regarded as an active measurement method. Consequently, it does not require access to any intermediate network device or link within the end-to-end path. It simply requires pieces of software running at both the sender and the receiver s ends. BART works by maintaining estimates of the available bandwidth and link capacity. Those estimates are continuously updated for each new probe packet sequence monitored at the receiver s end. The measuring of the time dispersions in a probing pair or train upon arrival is known as sampling. For every new sample, the Kalman filter is applied in order to keep the estimates updated in real-time Basic features The following set of bullet points state the most important features of the estimation mechanism performed by BART: Implemented in C++ under the principles of socket programming. Nevertheless, no communication is needed from the receiving end once the injection of probe packets has started. Lightweight application. The BART algorithm is computationally simple, requiring a few floating-point operations at each iteration. It also minimizes the memory requirements for calculating every new estimate.

28 Evaluation of BART Page 28 of 55 Network-friendly. The amount of probe traffic injected is small enough to avoid overloading the network excessively. Producing fast new estimates. Light computations allow a faster performance. Easily tuneable. Some parameters can be adjusted to meet a variety of needs. The probe packet size, number of probe packet pairs per measurement, organization of probes in pairs/trains, covariance matrix of the process noise or temporal characteristics such as the distribution of the probing intensity can be tuned for either agility or stability of the estimates at will How to use BART? BART is easy to configure. The configuration of the probing session is specified to the receiver software in the shape of adjustable input parameters, which allow to perform a number of different measurement sessions adequate to the purposes of the application at hand. Basically, BART enables the control of: The input probing intensity distribution. It is possible to send probe packet streams with equal inter-packet dispersion within configurable probe rate ranges [max, min]. The organization of each measurement. It allows launching either packet-pair or packettrain executions by defining the number of packets in each stream. The duration (in seconds) of the probing session. The size (in bytes) of each probe packet in a pair/train. The number of packet-pair or packet-train streams generated per second (intermeasurement separation). The covariance matrix of the process noise. The log files, which the measured data will be stored in. BART generates text-based information for every single probing stream processed at the receiver. Here we have a real example output produced by BART: psxt unknown Navb= AB= / / / / / / / / / The first number is the normalized probe rate. 0 corresponds to the minimum probe rate and 1 to the maximum probe rate. - The third is the variance of the strain estimate. - The acronyms mean: p = packet size, s = send rate, x = cross traffic type, t9 = there should be 9 pairs of time-stamps (one per train belonging the stream). The unknown statement means just that BART does not know what kind of cross traffic is actually flowing on the network (not used for calculations).

29 Evaluation of BART Page 29 of 55 - Navb represents the normalized bandwidth estimate produced by BART and AB is the absolute available bandwidth estimate. - The number before the / is the send timestamp on the sending computer and the number after the / corresponds to the receive timestamp on the receiving computer The information stored in the log files is thereafter used as input data to Matlab scripts that contains the coded Kalman filter equations. The scripts extract such information from the files in order to generate Matlab graphs depicting the final BART available bandwidth and link capacity estimates in a clear and intuitive format (right-most diagram in Figure ). Even though, the BART code also calculates the available bandwidth in real-time, previous to the use of Matlabs scripts The network model and BART estimation algorithm The network model The design of BART has been done according to the network model described at [7][39]. The model considers and/or assumes the following: The end-to-end network path is a sequence of first-come-first-served (FCFS) hops. Each hop consists of a transmission link and an intermediate network device (e.g. routers) implementing a first-in-first-out (FIFO) queuing policy. Each hop along the network path is allocated a constant capacity C i. Each link carries a load of cross-traffic X i and a load of probe-traffic, which are assigned a fair portion of the link capacity according to their intensity. Adding the existing dynamic cross-traffic load on each link to the calculation of the available bandwidth, we redefine equations (3) and (4) as: B = B i = (C i - X i ) min (C i = 1,..., H i - X i ) (5) Network traffic is transmitted in the form of discrete packets. Cross-traffic interacts with probe traffic at the FIFO queues of the bottleneck link. Congestion is thus understood as the situation in which the incoming traffic is greater than the capacity of the outgoing link. The offered probe-traffic intensity and the received probe-traffic intensity are denoted by u and r respectively. According to the process that was explained at section , the ratio u/r characterizes the threshold of congestion of the link inspected. Furthermore, by repeatedly applying it to successive links provides a way of estimating the available bandwidth of the entire path: if it is possible to determine the minimum offered rate u that makes the bottleneck link be congested, i.e. the point where the ratio starts to deviate from unity. That value of u corresponds to the available bandwidth estimate.

30 Evaluation of BART Page 30 of 55 u r 1 = 1 u + C X C ( u B) ( u > B) (6) Nonetheless, BART estimates are based on the inter-packet strain ε of received probe packets. The strain ε is the measure of the inter-arrival times of packets in probe pairs/trains and it is related to the ratio u/r by: u/r = ε + 1 (7) which means that the degree of congestion in the bottleneck is equivalent to the deviation of the strain from zero. Any variation of ε responds to interacting cross-traffic flowing across the end-to-end path. Figure The relation between the offered probe-traffic intensity and the inter-packet strain allows the estimation of the end-to-end available bandwidth. Kalman filtering is to compute the available bandwidth from information provided by the state vector of the system, which is updated in real-time. The state vector α = β x (8) contains all the information needed to describe the sloping straight line in Figure The state of the system will be estimated from successive measurements of the strain and its variance, given that we can also express the strain as ε = α u + β. It can be easily understood that our system and the measurements taken evolve along with time. This dynamic behaviour is described by the following system equations: x k = f(x k 1 ) + w k 1 (9) z k = h(x k )+ v k (10)

31 Evaluation of BART Page 31 of 55 where x k is the state of the system at instant k, z k is the k-th measurement (the measured inter-packet strain), w is the process noise and v is the measurement noise that affects the network model. Functions f and h represent the system evolution model and the measurement model respectively, and they will be computed using matrices in the Kalman filtering procedure. As it is suggested by formulas (9) and (10), the Kalman filter takes a previous estimate x k 1 and a new generated measurement z k in order to calculate a new estimate of the system state x k. See section 4.2 for an introduction to the Kalman filter basics, an insight into the filter procedure and its detailed equations. Regarding the strain measurements, we normally expect to measure a zero value of ε when the probe traffic rate u is less than the actual available bandwidth and a larger value that grows in proportion to the degree of congestion induced by a value of u also larger than the available bandwidth. Such a formalism can be described by the following piecewise linear model: 0 = αu + β ( u B) ε (11) ( u > B) When the Kalman filter estimates the variables α and β (the state), the new available bandwidth estimate B is automatically obtained since those system variables are related to B by the equation 0 = α B + β The BART algorithm The BART algorithm [7] performs the following five steps: 1. The receiver initializes the state vector x, the available bandwidth B and the state error covariance matrix P for x. 2. According to the setup, the sender generates a sequence of probe packets at an intensity rate u. Every sequence of probe packets, which are timestamped on sending and on arrival, is called a measurement. The value of u is enclosed in the probe packets. 3. For each received probe sequence, the receiver retrieves the value of u from the packets and compares it with B. If u < B no updating is performed and the execution returns to step 2. Otherwise, if u > B, the receiver computes the average strain ε of the packets in the sequence and its variance R. An estimate of the process noise covariance matrix Q is also computed at this step. 4. The strain ε and its variance R, are forwarded as input values to the Kalman filter together with Q. The filter then updates the estimates of the state vector x and the error covariance matrix P (Table 4.2). 5. The receiver produces a new available bandwidth estimate B using the updated value of x to find out the point where the sloping straight line crosses the u-axis (Figure 4.1.3).

32 Evaluation of BART Page 32 of 55 The relation between the state variables and the available bandwidth estimate in (11) is not fully linear. In general, iterative probing converges to a single available bandwidth estimate and BART solves this non-linearity by convergence (asymptotic approximation to the point where u = B). The principle is the continuous application of Kalman filtering to measurements for which u > B Kalman filtering The following information about the Kalman filter methodology and its uses has been taken from [36]. For a more concrete mathematical background, related work and implementations of the Kalman filter, visit the web site at [37]. The Kalman filter is a widely used mathematical method in computer graphics and radar detection systems. Nevertheless, it can also be applied to tracking and estimating the state of a system based on discrete measurements, according to the design of the BART model. Since the available bandwidth changes over time on a network path, the BART measurement model suffers from a strong non-linearity in its outcomes. Furthermore, BART is a network-oriented application, which does not take the inherent noisy nature of network measurements into account. Such nonlinearity can be handled by tuning certain parameters to filter and reduce the effects produced by system noise. In this way, the Kalman filter applies a predictor-corrector model that aims to minimize the estimated error (as a function of process and measurement noises) between measurements and optimize every new estimate of the system state. The process carried out and the equations applied are the following: Figure 4.2. Cyclic operation of the Kalman filter and equations [36]. The general problem is the estimation of the system state vector x dictated by a linear stochastic equation: n R, whose evolution is

33 Evaluation of BART Page 33 of 55 x k = Ax k 1 + Bu k 1 + w k 1 The functions representing the evolution of the systems are considered to be linear at the boundaries of the region congestion, so the Kalman filter can be optimally applied. m The set of measurements z, depends linearly on the state vector, that is: R z k = Hx k + v k The variables w k and v k are independent of each other and represent the process and measurement noise respectively. Well, as explained above, the state of the system is tracked and estimated along discrete points in time k. Kalman filtering is carried out in a cyclic process combining time updates (prediction) and measurement updates (correction) as an evolving mechanism to achieve estimates in real-time. The prediction operations project the latest estimate (state and error covariance) ahead of time. Thereafter, the latest measurement is used to adjust the projected estimate (predicted state). In order to get the best result at the latter (correction step), the first task to undertake is to calculate the Kalman gain K k. The Kalman gain represents the weight (importance) given to the new measurement as opposed to the predicted evolution of the previous measurement. To fully understand those operations, we need to distinguish between the a priori and a posteriori variables at each step k of the process: State estimate A priori A posteriori xˆk- Xˆk 5 State estimate error covariance matrix P - k = E[e - k e -T k ] P k = E[e k e T k ] State estimate error e - k = x k xˆke k = x k xˆk Table 4.2. Specific a priori and a posteriori equations of the Kalman filter Equations in Figure 4.2.a. are coded in the BART receiver s source code as well as in a Matlab script called kalman.m. The point with running two instances of the filter at different steps of the analysis is that the Matlab script allows the tuning of the process covariance matrix of the process noise Q for performance without the need for repeating the same measurement process over and over. We remind the reader that there exists the possibility of setting the values to Q as input for every new measurement. The way Q can be tuned is described in section below. 5 Being E[x k ] = xˆk. The same holds for xˆk- with E[xˆk- ] = xˆk-

34 Evaluation of BART Page 34 of Tuning of the BART characteristics The operation of the Kalman filter is governed by two important parameters: The process covariance matrix of the process noise w (Q) The process noise is defined as the change of the system state between two consecutive measurements [36]. The system may fluctuate due to variations of the cross-traffic or the link capacity in the network, which cannot be prevented. Nevertheless, Q can be adjusted to adapt the performance of BART and handle this variability. Q is a symmetric matrix, Q = Q Q Q 12 Q where Q 21 = Q 12 and Q 11, Q 22 > 0 22 There is plenty of freedom to set the value of Q. For the evaluation carried out in this work and according to the observations in [38] we selected Q 21 = Q 12 = 0 since the choice produces no worsening of BART estimation. Likewise, no gain in performance is observed when Q 12, Q 21 > 0. The choice of a suitable value of Q 22 is essential when tuning Q for improving the performance of the BART tool whereas Q 11 is preferably to be set a constant and small value (e.g. Q 11 < 10-4 ). Agility (fast adaptation) in estimation is achieved, in cases of high variability, by weighting new measurements heavier, thus having a larger value of 22. On the other hand, good stability is achieved having a smaller value of 22. In this case, with low variability in the network path, new measurements are likely to be less accurate and predictions are weighted heavier. The variance of the measurement noise v (R). The measurement noise is defined as the deviation from the average inter-packet strain measurements [36]. This parameter cannot be tuned for agility or stability of the BART behaviour in the way Q is set. R is, though, a necessary input parameter to the Kalman filter, which acts as an indicator of the variations in the measurement noise that determines the setting of Q for stability purposes. Q Q

35 Evaluation of BART Page 35 of Evaluation of BART The evaluation has been carried out over the performance of the BART implementation version 0.3 in a laboratory testbed. The forthcoming sections will describe the working environment in which the measurement sessions took place, the approaches and tools used and the focus given to the evaluation. The results section will present the most important findings from experiments and tackle the analysis of observations, giving a reasoned perspective of the most outstanding problems encountered throughout the entire process Experimental configuration The testbed The testbed used in this work consist of five computers running Linux, three routers and fullduplex Ethernet links to make up a multi-hop scenario as shown in the figure below, where one can also check the capacity of each wired link and the role performed by each host in the measurements. The primary goal in building this testbed is to study end-to-end network performance data reported by the BART tool to empirically correlate them with network events in a controlled monitoring network infrastructure. Figure Illustration of the measurement testbed. The measurements have been carried out in two contexts within a unique multi-hop scenario: experiments with one and two bottlenecks along the end-to-end path. The premises for both sets of measurements are described next: