Policy-based Injection of Private Traffic into a Public SDN Testbed

Transcription

1 Intitut für Techniche Informatik und Kommunikationnetze Adrian Friedli Policy-baed Injection of Private Traffic into a Public SDN Tetbed Mater Thei MA Advior: Dr. Bernhard Ager, Vaileio Kotroni Supervior: Prof. Dr. Bernhard Plattner

2 2

3 Acknowledgement I would like to thank my advior Dr. Bernhard Ager and Vaileio Kotroni for their help and great upport during carrying out my thei and Prof. Dr. Bernhard Plattner for giving me the opportunity to write thi thei in the Communication Sytem Group. Alo I would like to thank Hildur Ólafdóttir for helping me with ome debugging. Special thank go to my parent, my roommate and her boyfriend for alway giving me good advice and upport. 3

4 4

5 Content 1 Introduction 9 2 Environment OpenFlow OFELIA PAL PAL proxy FlowVior Mininet Goal and Challenge Goal of the PAL Deign and Build of the Arbitrator Performance Meaurement Correctne Evaluation Software Deign Deign of the Arbitrator Integration with PAL Flow Policie and Guarantee Forwarding flow to experiment Handling of new flow Flow table Arbitrating between experiment Change made in the PAL proxy Reerve flow pace in LUT and requet original header Interecting flow pace workaround Handle error meage and rejected rule Input port i part of flow pace Timeout cheduler Interface between PAL proxy and Arbitrator Requet flow injection Requet original header Inform PAL proxy about deletion of a flow Requet Arbitrator to hortcut flow Implementation choice Inight Inight in OpenFlow Inight in FlowVior Evaluation Environment Arbiterbench Connection to the PAL proxy Performance Evaluation Reult Time delay

6 6 CONTENTS Ditribution of the time delay Maximum handleable flow rate Comparion with flow arrival at the TIK network Correctne Evaluation Future Work Poible improvement to the Arbitrator Poible improvement to the PAL proxy Deployment in ETHZ OFELIA iland Summary 41

7 Lit of Figure 2.1 Privacy and Availability Layer before introduction of the Arbitrator. A preented by Kotroni et al [3] Privacy and Availability Layer after introduction of the Arbitrator Flow chart for handling new flow entering the tetbed Flow chart for handling new flow leaving the tetbed Schematic depiction of the flow travering the tetbed Time delay between packet-in and packet-out for the firt event of each bi-flow travering the tetbed. The blue curve how an overload of the Arbitrator. The peak reult from time needed reorganizing the internal data tructure Time delay between packet-in and packet-out ummed up on all event of a biflow travering the tetbed. Again, the blue curve how the overload of the Arbitrator and the peak from reorganizing the internal data tructure are viible Time delay between packet-in and packet-out for the firt event of each bi-flow travering the tetbed. We look at fewer flow and the overload-creating flow rate ha been omitted in thi plot. We ee higher delay with lower flow rate, we ruled out the mot obviou caue for them and didn t further evaluate it in order to not wate more time Time delay between packet-in and packet-out ummed up on all event of a biflow travering the tetbed. Alo here we look at fewer flow and we omitted the overload-creating flow rate. The higher delay with the lower flow rate are alo viible here ECDF for the time delay between packet-in and packet-out for the firt event of each bi-flow travering the tetbed. For the high flow rate 90% of the delay are below 10 m. Some mode are viible for the lower flow rate ECDF for the ummed up time delay between packet-in and packet-out for all event of each bi-flow travering the tetbed. For the high flow rate 90% of the delay are below 25 m Time delay between packet-in and packet-out for the firt event of each bi-flow for everal high flow rate. A clear eparation between 460 flow and 470 flow i viible. We aume thi i about the maximum flow rate the Arbitrator i able to handle Time delay between packet-in and packet-out for the firt event of each bi-flow for high rate. The yellow curve how, at that flow rate the backlog created by reorganizing the internal data tructure cannot be completely worked off Ditribution of delay between packet-in and packet-out for the firt event of each bi-flow for high rate. A expected, higher flow rate typically how a higher delay Flow rate at the TIK network for different time cale. (Plot provided by Bernhard Ager) 38 7

8 8 LIST OF FIGURES

9 Chapter 1 Introduction Software Defined Networking (SDN) i an emerging technology. New exciting networking protocol can eaily be be experimented with uing thi technology. Alo new routing algorithm may be deployed uing SDN enabled witche without having to replace the witch hard- or oftware. SDN enabled witche alo find application in large virtualization environment. Some large corporation e.g. Google already tarted deploying SDN enabled witche in their data center. For thi upcoming technology, we ee the need for teting environment. In general, many teting environment exit for SDN application. But none of thee tetbed allow teting with real uer traffic. Teting with real uer traffic i needed, becaue modelling real uer traffic mean jut making a implification of reality. Feedback i only available, when real traffic i involved and the uer experience additional delay and outage of hi Internet connection due to the inertion of experimental SDN application. But why i it difficult to ue real uer traffic? Uer traffic i enitive, uing it in a tetbed may caue privacy iue, furthermore the uer may have ome requirement on availability of hi Internet connection. Thu we need to guarantee privacy and availability. Even when privacy and availability are guaranteed, the uer till ha no incentive to donate traffic to the tetbed. Therefore a marketplace i needed. A introduced by Kotroni et al [3] a PAL proxy i available with an important part miing, a traffic Arbitrator, thi Arbitrator we wanted to build. The tak of thi thei i to deign, implement and evaluate the Arbitrator. Tak of the Arbitrator are to forward uer traffic to the tetbed, arbitrate between experiment and hortcut traffic on policy violation. Uer hould be able to tate guarantee for their traffic and experimenter hould be able to make requirement to the traffic. Evaluation conit of a performance analyi, including a comparion with the nature of traffic of a real network and a correctne evaluation. We have een that our implemented Arbitrator would be able to handle traffic coming from a typical /23 network uch a the TIK network. Moreover we learned ome inight about the OpenFlow protocol and that it i a ub-optimal choice to be ued in a privacy layer. We had to reimplement ome of FlowVior behaviour inide the PAL proxy uch a it i feaible now to build the PAL proxy inide or around FlowVior. Chapter 2 decribe technologie ued in the thei, Chapter 3 tate the goal and challenge. The deign of the implemented Arbitrator i explained in Chapter 4, performance and correctne of the Arbitrator get evaluated in Chapter 5. Chapter 6 preent poible future work and Chapter 7 conclude the thei with a ummary. 9

10 10 CHAPTER 1. INTRODUCTION

11 Chapter 2 Environment In thi chapter we decribe technologie ued in thi thei. The OpenFlow protocol, which controller ue to communicate with the witche, i decribed in Section 2.1. A teting facility for OpenFlow i OFELIA which we decribe in Section 2.2. In Section 2.3 we preent a extenion to OFELIA, the PAL. A different tet uite for SDN application i decribed in Section OpenFlow Openflow i a protocol to control hardware witche. The OpenFlow tandard i managed by the Open Networking Foundation and it i called being an enabler for Software Defined Network. We ued OpenFlow verion 1.0 [2], but our technique hould be adaptable to newer verion of the protocol. OpenFlow witche are imple and fat packet forwarding device, the logic ha been moved to an external controller. The witche have a flow table with rule matching flow and each rule may apply a lit of action to the packet of the flow. A rule can match eleven header field with either an explicit value or a wildcard. A ubnet mak can alo be ued to match the two header field ource and detination IP addre. The action can rewrite header field and output packet on a pecified port. The witch i connected with the OpenFlow protocol to an OpenFlow controller. If no rule matche a packet the witch forward the packet to the OpenFlow controller. The controller may then decide what to do with the packet, apply action to the packet and output it on a pecified port, it alo may intall rule in the witch for matching further packet of the flow. 2.2 OFELIA OFELIA 1 i an experimental facility baed on OpenFlow. There exit everal interconnected OFELIA iland at European univeritie. Thee iland typically conit of everal OpenFlow enabled witche and hot providing virtual machine. The phyical network provided by the witche i plit into lice by FlowVior. FlowVior i a tool peaking OpenFlow and can be put between OpenFlow enabled witche and controller. Thu allowing different controller to control the witche, each having it own lice. OFELIA ue VLAN-ID for licing, each experiment ha it own VLAN-ID. 2.3 PAL The goal of the Privacy and Availability Layer (PAL) i to enable experimenter to tet SDN application with real uer traffic. And guaranteeing the uer privacy and availability for the traffic. The PAL ha been introduced by Kotroni et al [3] and conit of everal part. We have FlowVior a part of the OFELIA iland. Then there i the PAL proxy which we will explain in 1 OFELIA webite: 11

12 12 CHAPTER 2. ENVIRONMENT Experimenter' controller Experimenter' controller Privacy and availability layer PAL proxy Gatekeeper witch Uer FlowVior Internet Figure 2.1: Privacy and Availability Layer before introduction of the Arbitrator. A preented by Kotroni et al [3]. Section And there i a gatekeeper witch 2. Figure 2.1 how an example tetbed with the PAL, the PAL proxy and a gatekeeper witch PAL proxy The PAL proxy i the entity between FlowVior and the experimenter. All OpenFlow command ent by the experimenter to hardware witche get checked by the PAL proxy. The PAL proxy maintain internal flow table, replicating the flow table of the witche. The PAL proxy reerve uer related flow travering the tetbed. After a ucceful reervation, the PAL proxy will diallow command from experimenter, which violate a flow privacy guarantee. The PAL proxy ue Lookup Table (LUT) for the reervation of uer flow. We will further decribe the LUT in Section FlowVior In order to eparate each experiment from each other and to have multiple controller connection to each witch, we intalled FlowVior between the PAL proxy and the tetbed witche. Each experiment ha it own controller connection per witch, it own lice and thu it own VLAN-ID aigned. 2.4 Mininet Mininet 4 i a framework for teting SDN application. It emulate a network and let you define a cutom network topology coniting of virtual hot, witche and link. Mininet iolate hot through network namepace provided by the Linux kernel. Wherea all procee of the different hot run on the ame operating ytem, only with jut different virtual network interface attached to them and with them having aigned different network addree. Thi lightweight virtualization allow Mininet to cale to run thouand of hot on a modet computer. 2 Depending on the etup there may be a ingle gatekeeper witch or two eparate witche, one for the uer ide and one for the Internet ide. Our implemented olution upport both etup. 3 A more detailed decription of the PAL proxy i available in [4] 4 Mininet webite:

13 2.4 Mininet 13 Switche are et up uing Open vswitch. Open vswitch i a production ready virtual witch for Linux and upport among other the OpenFlow protocol. Thi kernel module baed witch allow fat witching with a rate of more than 2 GB/. Although witching performance doe not matter for our correctne evaluation tet. Mininet allow the uer to launch procee on it hot, which allow eay teting of the topology and the attached controller. Mininet provide an interactive conole which allow a uer to control the Mininet environment and launch program on it virtual hot. With thi mechanim a uer can eaily run ping, netcat or iperf to end traffic from a virtual hot through the virtual network to another virtual hot and ue tcpdump at any virtual hot or witch to inpect the communication path.

14 14 CHAPTER 2. ENVIRONMENT

15 Chapter 3 Goal and Challenge In thi chapter we will decribe the different goal of the thei. The main goal i to deign and implement a traffic Arbitrator (Section 3.2). In order to do that we firt had a look at the goal of the whole PAL (Section 3.1) and at lat decided how to integrate it in the PAL. Furthermore the Arbitrator needed ome teting. We teted two different apect, which are a performance analyi with it goal decribed in Section 3.3 and a correctne evaluation with it goal decribed in Section Goal of the PAL The goal of PAL a a whole i to enable experimenter to experiment with real uer traffic without being able to violate policie which each uer tate for one traffic. The PAL conit of everal part, one part of it i the PAL proxy. The PAL proxy wa already available before the tart of thi thei, but it needed ome improvement. The PAL proxy job i to check the OpenFlow command ent by experimenter to the real hardware witche in the tetbed for policy violation. The PAL proxy i jut one part of the PAL and we needed an entity forwarding uer traffic to the OpenFlow enabled witche in the tetbed. For thi job we have a gatekeeper witch, which ha acce to real uer traffic. Thi gatekeeper witch i alo OpenFlow enabled and need a controller. Thi controller we wanted to build. 3.2 Deign and Build of the Arbitrator The Arbitrator i the controller for the gatekeeper witche. Figure 3.1 how how the PAL hould be extended with the Arbitrator. It ha the job to intruct the gatekeeper to forward uer flow to the tetbed. Each uer may define ome required guarantee for one flow. And each experiment may define ome requirement on the traffic. The Arbitrator mut then choe an experiment to forward the flow to regarding the guarantee and the requirement. It may alo choe to hortcut the flow if no other option i available. The Arbitrator need to be integrated into PAL. It need to control the Gatekeeper witch uing the OpenFlow protocol and it need to communicate with the PAL proxy about flow, their policie and poible violation. 3.3 Performance Meaurement High delay make webite load luggihly and uer notice quickly, epecially at univerity network where uer are ued to an Internet connection with round-trip time to a lot of erver with only a few milliecond. Google howed, that increaing the delay of loading a webite by 500 m caue them 20% traffic lo and amazon howed that increaing the delay of loading their webite caued them 15

16 16 CHAPTER 3. GOALS AND CHALLENGES Experimenter' controller Experimenter' controller Privacy and availability layer PAL proxy Arbitrator Gatekeeper witch Uer FlowVior Internet Figure 3.1: Privacy and Availability Layer after introduction of the Arbitrator. a lo in 1% of ale 1,2. With today ubiquitou preence of the world wide web, a lot of effort i made by many Internet companie to improve the experience for uer urfing the web. With large content ditribution network thee companie try to bring the content a near to the uer a poible, all in order to make the web fater. By reducing the geographical ditance and the number of hop between the erver and the uer PC one i able to lower the latency. Not only many content ditribution network get deployed to reduce latency, but alo new protocol are reearched on with that goal. For example SPDY [1] by Google. Delay i important, for a regular web browing eion delay will have a large impact on the uer experience. Therefore it i an important goal of thi thei to evaluate how the inertion of an OpenFlow tetbed into the uer Internet connection would impact the uer experience, due to the additional delay created by the Arbitrator. Delay i important for each ingle uer, independent of how many uer are participating and thu donating traffic to the tetbed. However it doe not tell how many uer can be handled imultaneouly. To better undertand how many uer can be handled we further tudy the flow arrival rate a well a the total amount of flow the Arbitrator can handle. 3.4 Correctne Evaluation There are many threat in the Internet. Computer crime i long not anymore jut committed by individual for peronal interet. Organized group perform criminal action in the Internet due to commercial interet. Be it to teal confidential information from unencrypted communication for indutrial epionage or peronal data for committing identity theft or be it to intall maliciou oftware on a uer computer to gather confidential data from the computer torage, gather acce to one online banking account or miue one computer to act a a ource for further attack or jut for ending unolicited commercial . If an attacker i able to oberve or even control a uer Internet connection, he would be able to perform everal attack. For intance, if an attacker i able to change content of a uer Internet connection, i.e. if one can inject traffic, an attacker can redirect uer to maliciou webite and trick them into downloading maliciou oftware, which eventually get executed by the uer computer to intall a back-door. If an attacker i able to jut oberve the uer Internet traffic, one may be able to teal confidential information from unencrypted communication. Alo ome form of traffic injection i till poible from other place, even when the attacker can t manipulate

17 3.4 Correctne Evaluation 17 traffic at the oberving location. Even when an attacker only ha acce to connection data but not to content, one may till be able to miue gathering from thi data to perform criminal action. For example, with uch data an attacker i able to learn which webite a uer viit, thi knowledge may be ued for blackmailing, if a uer conume inappropriate content. In general, reearcher are jut noy and we would not expect them doing any harm with uer traffic they experiment with. But ince we only have limited control over who might run experiment in our tetbed, we have to aume anyone in the Internet can regiter and run potentially maliciou experiment. Becaue of the preence of uch threat in the Internet, it i important for the Arbitrator to work a pecified and not to compromie a uer privacy and ecurity, when untruted third-party experimenter hould be allowed to perform their experiment in a public tetbed. Therefore it i a goal of thi thei to evaluate correctne of the Arbitrator.

18 18 CHAPTER 3. GOALS AND CHALLENGES

19 Chapter 4 Software Deign Becaue the gatekeeper witch need an entity controlling them, we needed to build the Arbitrator. We will dicu the deign of the Arbitrator in Section 4.1. The Arbitrator could either be built into the PAL proxy, or a a eparate application. We choe to build it a a eparate application (we will motivate that in Section 4.1.1). Therefore the PAL proxy needed to be aware of an external Arbitrator and needed to be extended in that way (we will dicu that in Section 4.2). Furthermore we needed to deign an interface between the Arbitrator and the PAL proxy (thi will be explained in Section 4.3). Furthermore we dicu ome inight epecially about the OpenFlow protocol learned during deign in Section Deign of the Arbitrator In thi ection we explain the deign of the Arbitrator and dicu the deciion taken Integration with PAL There wa a deign deciion we had do to at firt. We had two option, hould the Arbitrator be integrated into the PAL proxy or hould we build it a a eparate application? Integrating the Arbitrator into the PAL proxy would certainly be eaier, becaue then there wouldn t be the need for a communication interface. Building it a a eparate application on the other hand would implify teting. Alo future refactoring of the PAL proxy would be made eaier with a eparate application. Therefore we decided to build the Arbitrator a a eparate application Flow We define a flow a all packet having the 5-tuple (ource IP addre, detination IP addre, tranport protocol, ource port, detination port) in common. Thi definition correpond to the packet in one direction of a TCP connection. Alo UDP tream are caught by thi requirement. We alo tated a requirement for the time between two equent packet, which we ll dicu later. Flow are unidirectional, but mot communication pattern are bi-directional. When there i a flow in the backward direction of another flow and it belong in ome mean of their protocol to that flow, we call it the revere-flow of that flow. In cae of TCP or UDP the flow and it correponding revere-flow have the ame value in the tranport protocol header field and the two value in the IP addre header field and the two value in the port header field are wapped, repectively. The ICMP protocol may be ued by router to inform the ender of a flow about error, care hould be taken to handle thee error meage under the ame policy a the flow they belong to. Both flow, a flow and it correponding revere-flow, together contitute a o-called bi-flow, in the cae of TCP thi i equivalent to a connection. Becaue we mut keep information about a bi-flow in the Arbitrator a long a the lat rule of a flow belonging to the bi-flow ha expired and removed from the gatekeeper witch, we tate a an additional requirement for a flow, that in 19

20 20 CHAPTER 4. SOFTWARE DESIGN a bi-flow the maximum time interval between the arrival of two equent packet mut be lower than a pecified value Policie and Guarantee A uer donating traffic to the tetbed may require guarantee for hi traffic or part thereof. Currently we upport the following guarantee: direct, no-niff and allow. The direct guarantee mean not to deliver thi part of the traffic to the tetbed at all, a hortcut rule will be intalled a flow appear in the tetbed. The allow guarantee doe not warrant for anything and the no-niff guarantee protect the content of the traffic. Another requirement of the uer may be, not to allow the tetbed to do header pace rewriting. A policy include a tatement for matching a flow, a guarantee the PAL proxy mut warrant for and whether the tetbed i allowed to rewrite the header pace Forwarding flow to experiment When the Arbitrator intall a flow mod rule to the gatekeeper witch for forwarding a flow to the tetbed, it add two action to that rule, the firt action i to add a VLAN-ID to the packet, the econd action i to output the packet on the port facing the tetbed. The Arbitrator intall an oppoite rule for flow leaving the tetbed, alo with two action, the firt i to remove the VLAN-ID and the econd i to output the packet to the outide port Handling of new flow The Arbitrator work purely event baed. All it action are triggered from event from the outide. Event it will react upon may either come from the gatekeeper witch or from the PAL proxy. The interface to the PAL proxy with the event that the Arbitrator will handle will be decribed in Section 4.3. There are two poible event from the gatekeeper witch, the Arbitrator ha to react to. Firt, there i a packet-in event, whenever a new flow travere the gatekeeper witch, and econd there i a flow removed event when an intalled flow rule wa removed due to it idle timeout. A new flow ha to be handled, whenever the Arbitrator receive an OpenFlow packet-in event packet from the gatekeeper witch. When the flow i leaving the tetbed, a requet (requet will be explained in Section 4.3) i ent to the PAL proxy to get to know the original header pace of the flow before header pace rewriting might have changed the flow header pace in the tetbed. Then the Arbitrator ue the header pace of the flow to look it up in it internal flow table. If no entry wa found a new one i created and for flow entering the tetbed, an experiment ha to be choen to end the traffic to (arbitrating between experiment i decribed in Section 4.1.7). For flow leaving the tetbed, the experiment i already known from the VLAN-ID in the packet header. For flow entering the tetbed the Arbitrator look up the policy of that flow and ak the PAL proxy if forwarding the flow to the tetbed would violate the given policy. If it conforming to policy, the action for that flow will be forwarding to the tetbed, ele the action would be to hortcut the given flow. Flow leaving the tetbed are alway forwarded to the outide. The Arbitrator then output the packet received by the packet-in event according to the choen action and intall a flow mod rule in the witch, for forwarding future packet of that flow. In every cae, the Arbitrator update it internal flow table, with header pace information, choen experiment, action taken, and which rule i already intalled in the witch. Figure 4.1 and Figure 4.2 how the equence of operation in handling new flow with more detail Flow table When a flow i forwarded to an experiment in the tetbed, the correponding revere flow hould alo be forwarded to the ame experiment. The Arbitrator need to be able to find an exiting flow, when it revere flow travere the gatekeeper witch. In order to do that, we introduced

21 4.1 Deign of the Arbitrator 21 new flow at gatekeeper witch from uer ide or internet ide get packet (or buffer id) get lock on flow table create new flow table entry with VLAN ID and ref counter = 1 decide which VLAN/experiment thi flow hould go to no i a matching flow or revere flow in the flow table? (get entry) tore header pace in flow table no i ref counter >1? increment ref counter in flow table no ye ye releae lock on flow table tore packet i rule already intalled on witch? inform PAL proxy about new flow with VLAN ID and ak if it i policy conformant to forward thi flow to the tetbed releae lock on flow table ye output packet according to rule get lock on flow table i it policy conformant? no releae lock on flow table prepare hortcut rule ye output packet according to rule prepare forwarding rule with adding VLAN ID and forwarding to tetbed i ref counter >1? ye output all tored packet according to rule and decrement ref counter accordingly no update action/rule in flow table intall rule on witch decrement ref counter in flow table releae lock on flow table Figure 4.1: Flow chart for handling new flow entering the tetbed.

22 22 CHAPTER 4. SOFTWARE DESIGN new flow at gatekeeper witch from tetbed get packet (or buffer id) ak PAL proxy about original header get lock on flow table create new flow table entry with VLAN ID and ref counter = 1 no i a matching flow or revere flow in the flow table? (get entry) tore header pace in flow table no i ref counter >1? ye increment ref counter in flow table ye prepare forwarding rule with removing VLAN ID and forwarding to the outide on witch releae lock on flow table tore packet no i rule already intalled on witch? ye ye doe VLAN ID match? no drop packet and log a warning releae lock on flow table output packet according to rule output packet according to rule i ref counter >1? ye output all tored packet according to rule and decrement ref counter accordingly no intall rule on witch decrement ref counter in flow table releae lock on flow table Figure 4.2: Flow chart for handling new flow leaving the tetbed.

23 4.2 Change made in the PAL proxy 23 experiment guarantee h rewriting information for flow information for revere flow (a) Information tored for each bi-flow. action for flow information for part entering tetbed information for part leaving tetbed (b) Information tored for each flow, twice for each bi-flow. h of flow part rule i intalled refcount packet tore (c) Information tored for each part of a flow, four time for each bi-flow. Table 4.1: Information tored in a flow table entry. an internal flow table. The flow table conit of flow table entrie, for each bi-flow travering the gatekeeper witch, a flow table entry i created. Table 4.1 how the information tored in a flow table entry. It conit of the experiment (VLAN- ID), the guarantee and if header pace rewriting i allowed, action taken for the flow in each direction, header pace when entering the tetbed and leaving the tetbed for each flow. For each part of a flow, it i tored in the flow-table if a correponding rule i already intalled in the gatekeeper witch. More information tored for each part of a flow are a reference count for handling concurrency iue, and a reference to a lit of tored packet alo for handling concurrency iue Arbitrating between experiment Each experiment ha requirement to the policie of flow ent to it. One may require a maximum guarantee, be it to be able to acce all the payload of each packet of the flow or the header may be enough. Or one may need to rewrite header pace. Whenever an experiment ha to be choen to end the flow to, we create a lit of experiment the guarantee and the ability to rewrite header pace i uable to and take a random experiment from that lit. 4.2 Change made in the PAL proxy Thi ection explain the change needed in the PAL proxy to integrate the Arbitrator into the PAL and give an overview over important bug found and fixed or worked around in the PAL proxy Reerve flow pace in LUT and requet original header For implementing the requet to reerve flow pace in the PAL proxy and to get the original header pace of a flow, not much had to be done in the PAL proxy, mot of it wa already there. The PAL proxy ha a topology of node. Every node in the topology ha a correponding witch and contain a flow table and a LUT. The flow table in the PAL proxy node reemble the flow table in the correponding witche. The LUT entrie track uer related flow a they travere the tetbed and reerve their flow pace. They contain reference to previou and next LUT entrie, current header pace of the flow when it enter the node and the original header pace of the flow. In order to reerve a flow, we create a LUT entry in the firt node and perform a path validation. The flow pace i then propagated through the virtual topology and linked LUT entrie are created accordingly. Becaue the lat witch header pace rewriting wa not recorded in any LUT, we had to extend the topology and include the gatekeeper witch. The gatekeeper node doen t need a flow table. Looking up a flow original header pace now involved jut looking up the flow in the gatekeeper node LUT.

24 24 CHAPTER 4. SOFTWARE DESIGN Interecting flow pace workaround We intalled FlowVior between the PAL proxy and the tetbed witche, thu rule ent by an experimenter firt pa the PAL proxy and then pa FlowVior until they reach the witch. FlowVior enforce each experimenter to ue it own header pace, which i called a lice. OpenFlow rule ent to the witch contain a match and a lit of action. FlowVior interect the header pace of the match and the lice, if the reult i the empty header pace, the rule i rejected, otherwie the reult i ued a the new match. If an action would rewrite header pace to be not part of the lice, the rule i rejected 1. There wa a deign error in the PAL proxy, FlowVior change matche before it pae them to the witche, thi wa not conidered in the PAL proxy. We ued the VLAN-ID field for licing rule from the experimenter. When an experimenter end a rule with a wildcard in the VLAN-ID field in the match, the PAL proxy would accept it, update it internal flow table accordingly and forward it to FlowVior. FlowVior will then accept it and change the VLAN-ID field in the match to the experimenter VLAN-ID. If another experimenter end a rule with the ame match with the wildcard in the VLAN-ID field, the PAL proxy will update it internal flow table and modify the firt experimenter entry and pae it to FlowVior. FlowVior will then accept thi rule too and change the VLAN-ID field in the match to the econd experimenter VLAN-ID. Thi give u two problem. Firt the internal flow table of the PAL proxy i not in ync with the flow table in the witch, becaue FlowVior change rule and thee change are not conidered in the PAL proxy. Second, entrie in the internal flow table of the PAL don t hold aociation to the experimenter who created them, thu allowing an experimenter to change another experimenter flow table entrie. To work around thee problem we enforced VLAN-ID in rule from the experimenter in the PAL proxy. Matche in rule containing a wildcarded VLAN-ID were changed to contain the experimenter VLAN-ID, rule with matche already containing the experimenter VLAN-ID were accepted a uch and rule with matche containing another VLAN-ID were rejected. Thi workaround olve both problem, rule wont get changed anymore in the FlowVior and each flow table entry i aigned to an experiment due to the value in it VLAN-ID field. Thi workaround limit the poibilitie of experiment to run. We are only able to aign one VLAN-ID per experiment and experiment can t hare VLAN-ID. Both are poible in OFELIA but rarely ued Handle error meage and rejected rule The OpenFlow protocol pecifie, that a witch hould end no repone, when a flow mod meage ucceeded, only in the cae of an error it hould end an error meage. Flow mod meage from experimenter caue the PAL proxy to immediately update it internal flow table. Later error meage generated by unucceful flow mod meage get ignored by the PAL proxy itelf, and jut forwarded to the experimenter. Such that the flow table in the witch and the one in the PAL proxy get out of ync. An OpenFlow barrier requet ent to a witch, make the witch to proce all o far unproceed meage and then repond with a barrier reply. Thu making all error meage caued by unucceful flow mod meage ent before the barrier requet being ent before the barrier reply. We ued OpenFlow barrier requet to fix thi problem. After each flow mod meage, we end a barrier requet and wait for the barrier reply. In between, the change to the flow table i in a tranaction tate, it get rolled back, when an error meage i received, and it get committed, when the barrier reply i received. Becaue other meage hould not operate on inconitent flow table entrie, all other requet get queued Input port i part of flow pace In order to do policy violation checking, the PAL proxy emulate an OpenFlow enabled witch and maintain a copy of the witch flow table. OpenFlow matche, a part of a witch flow table, are deigned to match twelve field 2. Eleven of thoe twelve field match field in the 1 At leat with FlowVior verion 1.0 and newer. 2 OpenFlow Switch Specification 1.0 [2] in Table 2 on page 3.

25 4.3 Interface between PAL proxy and Arbitrator 25 Ethernet frame, a ent through the network. The other field i to match the phyical input port a packet enter the witch. Flow pace in the PAL proxy i alo repreented by thee twelve field. The PAL proxy maintain LUT to reerve uer flow pace, it doe thi by inerting a LUT entry for a witch, then check flow table entrie of that witch to find the next witch in the path of that flow. Thi i done recurively until a gatekeeper witch i reached. The PAL proxy didn t conider to update the input port in the LUT flow pace a the flow travel through the tetbed. Thi caued matche in the flow table without wildcard in the input port field to behave incorrectly. We fixed thi by updating the input port field in the LUT entrie on each hop Timeout cheduler Becaue rule in the PAL proxy may expire due to an idle timeout and may caue a policy violation when they get removed, the PAL proxy need to watch the timeout of the rule. OpenFlow witche collect tatitic about flow, thee flow tatitic contain a time value to idle before expiration. The PAL proxy chedule an action to get tatitic of a rule ome econd before it idle timeout, and rechedule if needed. If the rule would expire oon, it doe a path validation, if that fail, it tell the Arbitrator to intall a hortcut rule matching the flow 3. In the original verion of the PAL proxy thi wa planned, but the implementation wa not finihed. Scheduling wa implemented, but retrieving tatitic from the witch wa miing. One did not conider the licing of FlowVior, each experimenter can only get tatitic about rule in the lice he ha acce to. In order to not have to find out the correct experiment for each rule and omit the need to filter out tatitic repone to the experimenter we introduced an admin lice in FlowVior. The admin lice conit of the all wildcarded flow pace, thu it ha acce to all rule in the witche. The PAL proxy then ue the admin lice connection to retrieve tatitic about the rule. 4.3 Interface between PAL proxy and Arbitrator Becaue we choe the Arbitrator to be a eparate application which i not included in the PAL proxy (we did dicu that in Section 4.1.1), but they till depend on each other and need to exchange information at ome point, we needed to deign a communication interface between the two. We want to be able to handle new event from the gatekeeper witch while having requet from the Arbitrator to the PAL proxy from earlier event till pending. In the cae when ome requet are pending, new event not needing communication with the PAL proxy hould get handled immediately by the Arbitrator, i.e. the Arbitrator hould not block while waiting for requet to complete. Therefore we choe an aynchronou requet and repone communication model for the communication between the Arbitrator and the PAL proxy. In order not to let the PAL proxy idle wait for a new requet after a repone ha been ent and therefore enable it to quickly work off requet, we choe to ue pipelining. Even when there are pending requet from the Arbitrator to the PAL proxy, new event needing communication with the PAL proxy will caue the Arbitrator to end requet, which then need to get queued at the PAL proxy. We wanted a future PAL proxy with a multi-threaded deign to be able to handle requet out of order. Thu we had to make ure, the Arbitrator will handle repone correctly, when they arrive at a different order a their correponding requet. On the requeting part, each requet i aigned an incrementing requet id. Thi requet id i then included in the requet data which i ent to the reponding part. On the requeting part the requet id i then ued a a key for toring the requet data in a pending requet lookup table. Together with the requet data itelf a handler function i tored in thi table. When the requeted action ha been performed and there i a repone, it i ent to the requeting part and alway include the requet id. When a repone i received by the requeting part it take the requet id and look up the 3 More on that can be found in [3].

26 26 CHAPTER 4. SOFTWARE DESIGN entry in the pending requet lookup table and call the handler function included in the entry with the repone payload a an argument. There are three different type of requet performed by the Arbitrator to the PAL proxy. Requet to inject a flow into the tetbed, requeting the original header of a flow leaving the tetbed, and informing the PAL proxy about removal of a flow. In addition one type of requet i performed by the PAL proxy to the Arbitrator, namely requeting the Arbitrator to hortcut a flow. The ret of thi ection decribe the detail of the four type of requet and the choice we made for the implementation Requet flow injection Whenever there i a new flow to be injected to the tetbed, the Arbitrator need to ak the PAL proxy if injecting the flow into the choen experiment caue a policy violation. Flow injection we named thi requet type. The repone to uch a requet tate if it either policy violating or not. The payload of thi requet type contain the VLAN-ID of the experiment we want thi flow to be injected, the 5-tuple header, the ource where thi flow i coming from, either uer or Internet, and the guarantee, we want thi flow to be warranted for. The repone to uch a requet i either the forward tatement or the hortcut tatement. The forward tatement tell the Arbitrator that the PAL proxy uccefully reerved thi flow pace in it lookup table and it i now afe to forward thi flow to the tetbed, the PAL proxy now alo expect to be informed about removal of thi flow to be able to releae ytem reource. The hortcut tatement tell the Arbitrator that the PAL proxy wa not able to reerve thi flow pace in it lookup table, meaning that forwarding thi flow would caue a policy violation and that the Arbitrator hould hortcut thi flow intead of forwarding it to the tetbed Requet original header Whenever a new flow i leaving the tetbed, the Arbitrator need to look it up in it flow table. If the experiment handling thi flow i allowed to do header pace rewriting, the Arbitrator need to know the header pace of the flow before it entered the tetbed. The PAL proxy reerve each uer flow and keep track of header pace rewriting action within it LUT. All the information about original header pace i tored in the PAL proxy. The Arbitrator can ue the requet type named original header to retrieve thi information from the PAL proxy. The payload of thi requet type contain the VLAN-ID, the 5-tuple header and the detination, where the flow i headed to, either uer or Internet. The repone to uch a requet contain the VLAN-ID and the 5-tuple header of the flow when it entered the tetbed Inform PAL proxy about deletion of a flow The PAL proxy ue ytem reource for reerving flow pace in it LUT. Each LUT entry ue ome memory, additionally policy check demand computation time depending on the number of LUT entrie intalled. In order to not exhaut ytem memory we want to releae thee ytem reource a oon a poible. The gatekeeper witch inform the Arbitrator a oon a a flow rule expired due to the idle timeout and wa removed from the witch table. The Arbitrator then update it internal flow table. If a rule jut removed belong to a flow travering the tetbed the Arbitrator end a requet of the type flow remove to the PAL proxy to inform it to give up the flow pace reervation of that flow and free ued ytem reource. Becaue no repone i needed by the Arbitrator for thi type of requet, the PAL proxy doe not end repone to uch requet. Therefore thee requet are not put in the pending requet lookup table. The payload of the requet contain the VLAN-ID, the 5-tuple header and the ource where the flow originally wa coming from, either uer or Internet.

27 4.4 Inight Requet Arbitrator to hortcut flow When the PAL proxy encounter a policy violation for an exiting flow pace reervation, it ha to be able to inform the Arbitrator that it i not afe anymore to forward thi flow to the tetbed. Thi can happen when a rule time out on the witch and get caught by the timeout cheduler. The PAL proxy then end a requet type named hortcut to the Arbitrator to inform it about uch a policy violation. Becaue no repone i needed by the PAL proxy for thi type of requet, the Arbitrator doe not end repone to uch requet. Therefore thee requet are not put in the pending requet lookup table. And becaue thi i the ingle requet type performed by the PAL proxy, we could omit the implementation of a pending requet table on the PAL proxy ide. The payload of the requet contain the VLAN-ID, the 5-tuple header and the ource, where the flow i originating from, either uer or Internet Implementation choice Becaue we were already uing the POX library at both end, we choe to ue POX meenger component. The meenger component provide bi-directional channel. Becaue of our requet and repone communication model, we choe to ue two channel, a requet and a repone channel. Requet are ent to the requet channel and if there i a repone to a requet, the other part end the repone to the repone channel. The POX meenger component i built uing a TCP connection and end JSON encoded data. For the time being only the erver part wa implemented. Therefore on the PAL proxy ide we had to extend the meenger component to act a a client and connect to a erver. 4.4 Inight A we have een in the previou Section, the OpenFlow protocol i a ub-optimal choice for the purpoe of a privacy layer, it deign make it difficult to handle every cae correctly. Furthermore we had to rebuild part of FlowVior in the PAL proxy. In thi Section we dicu more inight learned while deigning the Arbitrator and debugging the PAL proxy Inight in OpenFlow A we have een in Section 4.2.3, the OpenFlow protocol ha no upport to acknowledgement flow mod meage a controller end to the witch. In order to be able to keep the flow table in the witche and in the PAL proxy in ync, a mechanim for acknowledging thee flow mod meage wa needed. We had to ue barrier requet for that purpoe, which certainly ha an impact on performance. A dicued in Section 4.2.5, handling of idle timeout a defined in the OpenFlow protocol i very limiting for our purpoe. There i ome kind of aymmetry in handling of idle timeout of rule in the OpenFlow protocol. Rule are intalled by the controller, but get removed by the witch a oon a timeout occur and the controller i notified after the removal of the rule. Thi iue wa already known from the previou verion of the PAL proxy, but the propoed olution did not conider licing of the FlowVior Inight in FlowVior A een in Section FlowVior doe make change to rule ent from the experimenter controller to the witche. We had to reemble thee change in the PAL proxy and therefore duplicating behaviour of FlowVior inide the PAL proxy. The main ue of FlowVior how it i ued currently in the PAL, i to multiply a controller connection to different experimenter.

28 28 CHAPTER 4. SOFTWARE DESIGN

29 Chapter 5 Evaluation Latency i an important factor in the uer experience of one Internet connection, a we have dicued in Section 3.3. One goal of our performance analyi i to meaure the delay added by inerting the Arbitrator to one Internet connection, o we will be able to etimate the impact on the uer experience. In order to minimize meaurement artifact, we needed a controlled environment for meauring performance. It need to be adjutable and it need to be able to create a pecific load. Deploying the PAL etup in ETHZ OFELIA tetbed would have been poible but would have created too much effort to generate load and meaure performance. Additionally there i no tandard tool readily available for our purpoe to generate a pecific load on an OpenFlow controller and to meaure it performance. Therefore we decided to build uch a tool by our own. The tool generate OpenFlow packet-in event and handle repone accordingly. Thi Arbitrator benchmarking tool i fat enough not to interfere with the meauring reult. Thu teting the tool itelf for it performance wa required. Additionally we evaluated the correctne of the implemented Arbitrator, which i dicued in Section 5.5. All the Performance tet were done on a Lenovo Thinkpad T410 with an Intel Core i7 CPU running at 2.67 GHz and 4 GB of Memory. Furthermore to get a table and predictable environment we diabled hyper-threading in the ytem BIOS to make ure not to encounter cheduling interference. We alo did configure the CPU not to enter lower C-State and ued the performance CPU governor in the Linux kernel. Both technique would reduce power uage of the ytem when idling, but could fudge our tet reult. The Operating Sytem ued wa Ubuntu Linux LTS. The Linux kernel wa verion and the intalled Python verion wa Environment We wanted to meaure the performance of the Arbitrator. Therefore we had to figure out, how we could take the Arbitrator out of it intended environment and put it into a benchmark environment. Since the Arbitrator i a tandalone application, which wa deigned a a modular part of the PAL from the beginning, with only a few connection to other part of the PAL. Thee connection are the bet point to meaure performance at becaue meauring at thee point include meauring communication overhead and it i not required to intrument the Arbitrator. We had a cloer look at the outide connection of the Arbitrator and dicovered, the only relevant in- and output of the Arbitrator i the connection to the gatekeeper witch (Figure 5.1 how a implified verion of the Arbitrator and the gatekeeper witch in their intended environment). There i alo a connection from the Arbitrator to the PAL proxy (not depicted in the Figure) and we will dicu it impact in Section 5.3. Becaue of thi ingle relevant connection, where OpenFlow i ued, it i ufficient to replace the gatekeeper witch by a benchmarking tool which we call Arbiterbench. 29

30 30 CHAPTER 5. EVALUATION Arbitrator Uer Tetbed Gatekeeper Internet Figure 5.1: Schematic depiction of the flow travering the tetbed. 5.2 Arbiterbench We had to figure out, which OpenFlow packet the gatekeeper witch end to the Arbitrator, therefore we looked at the flow travering the tetbed. When a bi-flow i ent through the tetbed it travere the gatekeeper witch four time in total. Firt a flow initiated by the uer travere the gatekeeper witch when it enter the tetbed and travere the gatekeeper witch a econd time a it leave the tetbed. When the detination hot in the Internet repond, it will generate a econd flow, a o called revere-flow with IP ource and detination addree wapped and, if applicable, ource and detination port wapped. Thi revere-flow travere the gatekeeper witch a it enter the tetbed and again a it leave the tetbed. The red arrow in Figure 5.1) repreent thee flow. In the Arbitrator intended environment the arrival of the firt packet of each flow at the gatekeeper witch caue the witch to end an OpenFlow packet-in event packet, which make it two for each flow and four for each bi-flow. All future packet of a flow will not caue new packetin event, a flow-mod rule will then be intalled in the gatekeeper witche and all packet of that flow will be forwarded accordingly. The four packet-in event are a wort-cae cenario. Thi can actually be reduced by half, if we not only intall the flow-mod rule matching the flow jut encountered but alo we would preintall a flow-mod rule for the correponding revere-flow on the gatekeeper witch, becaue the header for matching the revere-flow are already known at thi point. It can even be reduced to a quarter, in the cae where no header pace rewriting i done inide the tetbed 1 where we will be able to intall all four required flow-mod rule at once, but one may not forget to requet inertion of the revere-flow to the tetbed at the PAL proxy if applicable. Our Arbiterbench imulate thi behaviour and generate an OpenFlow packet-in event packet, a a new flow initiated by the uer entering the tetbed at the gatekeeper witch would do, and end it to the Arbitrator. The Arbitrator then will handle thi imulated new flow event and will repond to the benchmarking tool accordingly with a OpenFlow packet-out and a flow-mod command packet. The benchmarking tool will then meaure the time delay between the packetin event and the packet-out command. For each packet-out command received, the Arbiterbench will generate a new OpenFlow packet-in event packet repreenting the bi-flow next traveral of the gatekeeper. Thi i done for every packet-out except for the one repreenting the fourth traveral of a bi-flow. The Arbiterbench generate the firt OpenFlow packet-in event packet with a contant rate. Additionally it will generate packet-in event after receiving a packet-out command from the Arbitrator, if required to. So for each packet it generate with a contant rate, it will generate three other in addition, equentially. The Arbiterbench end an OpenFlow packet-in event packet, calculate the time interval between ending time of thi and the next uch packet and leep thi amount of time. In the meantime another thread handle incoming packet-out and flow-mod command, meaure time delay and generate new packet-in event accordingly. It i deigned a a tandalone application. Becaue we had experience with the POX library and etimated it i fat enough for that purpoe we ued the ame patched verion of the POX library a the PAL proxy wa uing to build the Arbiterbench. In a later point we howed it i indeed 1 The experiment traffic requirement tell if header pace rewriting i performed in the tetbed.

31 5.3 Connection to the PAL proxy 31 fat enough for our purpoe. In addition we re-ued ome code of the PAL proxy, epecially for doing the northbound communication, the part which i reponible for the communication with an OpenFlow controller. In the hope to reduce CPU uage and make it poible to tet higher flow rate, we alo teted ending in batche of ten packet, meaning ending ten packet and then wait an equivalent longer time interval until the next ten packet-in event hould be ent. Teting with batche howed artifact, the delay howed a mode at about 65 m either produced by the benchmarking tool itelf or by the Arbitrator. Thu, it wa for our meaurement purpoe not uable and we deactivated it. We have verified that the performance of the Arbiterbench i ufficient and it meaurement reult are not influenced by ome co-proceing iue on the computer. We have een that our benchmarking tool generate packet fat enough, becaue it alway caued coniderably le CPU uage than the Arbitrator. The Arbiterbench and Arbitrator are only able to ue one CPU at a time and the etup wa running on a dual-core ytem. 5.3 Connection to the PAL proxy We wanted to tet both the delay inide the Arbitrator and it communication delay, the one between the gatekeeper witch and the Arbitrator a well a the one between the Arbitrator and the PAL proxy. We didn t require the PAL proxy to be fully functional for the tet. Having a running PAL proxy would have required u to create a model for teting the PAL proxy internal tructure and tet reult would trongly depend on that. Therefore we built a dummy PAL proxy for the performance analyi tak. Thi dummy PAL proxy repond to every flow inertion requet with the forward tatement, which mean every flow hould be forwarded to the tetbed. And it repond to every original header pace requet with the header pace the requet wa aociated with, uch a no flow i ubject to header pace rewriting in the tetbed. 5.4 Performance Evaluation Reult Time delay Figure 5.2 how the time between the generation of the firt packet-in event and the reception of the correponding packet-out event for different flow rate. The peak in time delay which are viible in all the curve tem from the internal data tructure of the Arbitrator, becaue we ued a Python dictionary, which get reized every now and then. Thi could be avoided with warming up the data tructure before ue. The curve at the top how that 500 flow create an overload on the Arbitrator. The Arbitrator i not fat enough to handle thi rate and the backlog i increaing and with it the total delay. Even in the time interval between higher delay due to dictionary reizing the delay i riing, from thi we conclude even a warmed up data tructure would create an overload with thi flow rate. The plot of the other three packet-in/packet-out delay of each bi-flow look imilar, the firt packet-in/packet-out delay i only a bit higher than the other, thu we omitted them for brevity. Becaue we wanted to know the total delay added by the Arbitrator to a bi-flow, we ummed up the 4 meaured delay and plotted it in Figure 5.3. Becaue the peak are hifted in time for the different packet-in event of a bi-flow, the top curve howing the overload cae look much flatter here. Figure 5.4 how a zoomed verion of Figure 5.2 which how the time delay of handling the firt packet-in in a bi-flow, where we only include the lower rate and look only at the firt 5000 flow. The cyan line with a rate of 50 flow how the mot delay, and the green one with a higher rate of 200 flow how the lowet delay. We expected thee to be about equal. We turned off power aving feature of the CPU a decribed in Section 5.2, therefore we aume it ha to do with either cheduling of the Linux kernel or other power aving feature, which can t be turned off. Figure 5.5 how a zoomed verion of Figure 5.3 which how the total time delay of handling a bi-flow, alo here we omit the high flow rate and only look at the firt 5000 flow.

32 32 CHAPTER 5. EVALUATION 9 50 flow flow flow time delay () flow bi-flow Figure 5.2: Time delay between packet-in and packet-out for the firt event of each bi-flow travering the tetbed. The blue curve how an overload of the Arbitrator. The peak reult from time needed reorganizing the internal data tructure flow flow 200 flow time delay () flow bi-flow Figure 5.3: Time delay between packet-in and packet-out ummed up on all event of a bi-flow travering the tetbed. Again, the blue curve how the overload of the Arbitrator and the peak from reorganizing the internal data tructure are viible.

33 5.4 Performance Evaluation Reult 33 time delay () flow 100 flow 200 flow bi-flow Figure 5.4: Time delay between packet-in and packet-out for the firt event of each bi-flow travering the tetbed. We look at fewer flow and the overload-creating flow rate ha been omitted in thi plot. We ee higher delay with lower flow rate, we ruled out the mot obviou caue for them and didn t further evaluate it in order to not wate more time flow 100 flow 200 flow time delay () bi-flow Figure 5.5: Time delay between packet-in and packet-out ummed up on all event of a bi-flow travering the tetbed. Alo here we look at fewer flow and we omitted the overload-creating flow rate. The higher delay with the lower flow rate are alo viible here.

34 34 CHAPTER 5. EVALUATION probability flow 100 flow 50 flow time delay () Figure 5.6: ECDF for the time delay between packet-in and packet-out for the firt event of each bi-flow travering the tetbed. For the high flow rate 90% of the delay are below 10 m. Some mode are viible for the lower flow rate Ditribution of the time delay Figure 5.6 how ECDF for the three lower rate of the firt 5000 flow each for the delay of the firt packet-in event. The highet rate with 200 flow how about 90% of the delay are below 10 m. The rate with 100 flow how mode at 10 m, 20 m and 35 m. The rate with 50 flow how mode at 20 m, 35 m, 65 m and 85 m. Figure 5.7 how the ECDF for the three lower rate of the firt 5000 flow each for the ummed up delay of all the packet-in event in a bi-flow. At a flow rate of 200 flow about 90% of the delay are below 25 m. There are no clear mode viible for all the rate Maximum handleable flow rate In order to find the maximum flow rate, the Arbitrator i able to handle, we meaured time delay with different flow rate around the expected maximum. Figure 5.8 how again the time delay of handling the firt packet-in event in a bi-flow for different rate between 400 flow up to 500 flow. One can clearly ee a plit between the rate 460 flow and 470 flow. At a rate of 460 flow the Arbitrator can not work off the whole backlog created by the peak time created by reordering the internal data tructure, but in the interval between the peak, one can ee, the delay i decreaing, thu we aume, with a warmed up data tructure the Arbitrator would be jut able to handle 460 flow. In contrat to that at a rate of 470 flow the delay i even increaing in the time interval between the peak, thu the Arbitrator i not able to handle that flow rate. Figure 5.9 how only the firt 5000 flow and only the rate between 400 flow and 450 flow. With the highet of that rate the Arbitrator i building a mall backlog over time, a can be een in the plot the curve i not getting near zero after a while in comparion to the other curve. But the Arbitrator would, a een before, alo be able to handle thi rate with a warmed up dictionary. 440 flow i alway getting near zero, thu it i about the highet rate the Arbitrator can handle without a warmed up data tructure. Figure 5.10 how the ECDF of the tet with high rate. A expected the higher rate typically how a higher delay.

35 5.4 Performance Evaluation Reult probability flow flow 50 flow time delay () Figure 5.7: ECDF for the ummed up time delay between packet-in and packet-out for all event of each bi-flow travering the tetbed. For the high flow rate 90% of the delay are below 25 m time delay () flow 460 flow 410 flow 470 flow 420 flow 480 flow 430 flow 490 flow 440 flow 500 flow 450 flow bi-flow Figure 5.8: Time delay between packet-in and packet-out for the firt event of each bi-flow for everal high flow rate. A clear eparation between 460 flow and 470 flow i viible. We aume thi i about the maximum flow rate the Arbitrator i able to handle.

36 36 CHAPTER 5. EVALUATION time delay () flow 410 flow 420 flow 430 flow 440 flow 450 flow bi-flow Figure 5.9: Time delay between packet-in and packet-out for the firt event of each bi-flow for high rate. The yellow curve how, at that flow rate the backlog created by reorganizing the internal data tructure cannot be completely worked off probability flow 440 flow 430 flow 420 flow 410 flow 400 flow time delay () Figure 5.10: Ditribution of delay between packet-in and packet-out for the firt event of each bi-flow for high rate. A expected, higher flow rate typically how a higher delay.

37 5.5 Correctne Evaluation Comparion with flow arrival at the TIK network We compared the flow arrival rate at the TIK network with the capability of our etup. A can be een on Figure 5.11 the TIK network 2 how on a regular Thurday during the emeter a baeline of about 10 flow and about twice that much during working hour. When looking at a bin ize of 1 econd we have peak up to 210 flow. With a bin ize of 0.1 econd we ee peak up to 1200 flow. The 1200 flow during 0.1 econd correpond to 120 flow. Auming our ytem can handle 460 flow, the backlog of 120 flow will be worked off in about 260 m. Thu we aume our ytem can handle the traffic of the nature of the TIK network with introducing mall delay. 5.5 Correctne Evaluation In order to tet the Arbitrator and the PAL proxy a tetbed wa et up in the very beginning of the work. The goal of the tetbed i to evaluate the correctne of the Arbitrator in combination with the PAL proxy. It wa alo an important aid during development and debugging to find programming error. The tetbed wa built uing the Mininet framework. In the beginning teting wa done manually, by tarting each component by itelf and uing Mininet interactive conole to end traffic through the virtual tetbed, but thi i a tediou tak. To overcome thi, a imple tet cript ha been written. The tet cript take care of etting up the Mininet tetbed and launche all the needed component which are the Arbitrator, the PAL proxy and ome experimenter controller. It then ue tcpdump on ome link to capture traffic for later analyi. For teting, we configured ome tatic policie in the Arbitrator. We configured policie to allow ome traffic being forwarded to the tetbed and other to hortcut the traffic and not let it be forwarded to the tetbed. The tet cript then end packet through the virtual network and ue captured traffic by tcpdump to evaluate the path the traffic took. If it matche the tatically configured policy then the tet pae, otherwie it fail. The manually performed a well a the automatically performed tet were ucceful and the implemented program behave a pecified. 2 Two high traffic hot have been omitted from the meaurement.

38 38 CHAPTER 5. EVALUATION Figure 5.11: Flow rate at the TIK network for different time cale. (Plot provided by Bernhard Ager)