Implementing an OpenFlow Switch With QoS Feature on the NetFPGA Platform

Transcription

1 Implementing an OpenFlow Switch With QoS Feature on the NetFPGA Platform Yichen Wang Yichong Qin Long Gao Purdue University Calumet Purdue University Calumet Purdue University Calumet Hammond, IN Hammond, IN Hammond, IN ABSTRACT The goal of the project is to implement a switch by modifying the arbiter component in the NetFPGA board to improve the performance of whole network. This new switch provides the Quality of Service (QoS) feature for live streams so that video and voice information will be transferred with a higher priority, and users can get a more satisfying network service. Rate Dynamic RAM.[11] The NetFPGA enables researchers and students to build working prototypes of high-speed, hardware-accelerated networking systems. Gigabit Ethernet switches can be implemented on this platform. Figure 1 shows what the card looks like. 1. INTRODUCTION As networks play an increasingly important role in our daily life, people are paying more and more attention to their networking speed, flexibility, and stability. Although lots of IM (Instant Messaging) applications support functions like VoIP (Voice over IP) and Online Video Conferences, traditional switches consider all those real-time data as the same as others. Therefore the users may dissatisfy with the communication experiences, especially when the networking devices reach high loads. To solve this problem, a new module of switch will be implemented which will classify data first, put them into different priority queues, and finial forward them according to different queues. By doing so, users can enjoy the online communication services at a higher quality. 2. BACKGROUND The hardware parameters and some existing standards which are utilized in the project are introduced in this part Hardware The NetFPGA card is the hardware platform used in the project. The NetFPGA is a PCI card that contains a large Xilinx Virtex-2 Pro FPGA, 4-Gigabit Ethernet ports, Static RAM, and Double-Date Figure 1. Photo of NetFPGA. The abstracted infrastructure of NetFPGA card is shown in Figure 2. Figure 2. Abstract Description of NetFPGA. NetFPGA only provide the driver for Linux officially, so all the works are done on CentOS Existing Standard To achieve the goals, some existing standards are reviewed and highly utilized in the project

2 Figure 3. IPv4 Packet. OpenFlow OpenFlow is an open standard that allows researchers to run experimental protocols in production networks. [12] It is in the process of being implemented by major switch vendors and used today by universities to deploy innovative networking protocols in their campus networks. It is added as a feature to commercial Ethernet switches and provides a standardized interface which allows researchers to run experiments. Internet Protocol (IPv4) The Internet Protocol (IP) is a protocol used for communicating data across a packet-switched internetwork using the Internet Protocol Suite, also referred to as TCP/IP.[6] IP is the primary protocol in the Internet Layer of the Internet Protocol Suite. It is responsible to deliver different datagrams (packets) from one source host to the destination host solely based on their IP addresses. For this purpose, the Internet Protocol defines addressing methods and the contents for datagram encapsulation. are real-time data, so they should be forwarded immediately to avoid discontinuousness. While other packets forwarding uses accuracy as the primary goal rather than latency, which means they can be forwarded later. The IPv4 supports packet classifications by using the ToS (Type of Service) bit in Differentiated Services Field, which provides different priorities to different kinds of packets (Referred to Figure 3). The new switch will take advantage of this field to distinguish the packet type. The Internet Protocol runs at the Network Layer, so the switch will de-capsulate the packets and do the classification according to its ToS value in the IP header. Figure 4 is a diagram that briefly shows the data path of each phase when a packet arrives. The packet composition under IPv4 protocol is shown in Figure DESIGNATION The designing details of all components involved in the project are illustrated in this part Strategy of Processing Packets The main idea of QoS (Quality of Service) is to provide different priorities to different types of packets to lower the delay of high-priority packets. Ethernet packets may have different types, such as video packets, audio packets, and normal packets. Since video and audio packets Figure 4. Strategy of Dealing with Each Packet

3 3.2. Components The development of the OpenFlow switch can be clearly divided into three major parts: Hardware, Controller, and Monitor Software. For testing and contrasting purposes, some other related components are also included in the project. Hardware The hardware (NetFPGA card) will provide the support for QoS at a very fundamental level. It must have the ability to handle all three operations mentioned above in Figure 4. In addition, it should also be able to respond the instructions sent from the controller and do corresponding operations. The original user-data-path of OpenFlow switch without the QoS feature can be abstracted as shown in Figure 5. Ethernet MAC Ports Input Arbiter Output Port Lookup Output Queues Figure 5. Abstracted Structure of User-data-path used in OpenFlow Switch. In order to provide different treatments to different packets, packet classification must be done as soon as it is completely received (buffered). A new component will be inserted between the Ethernet MAC Port and Input Arbiter to do the packet classification, and the Input Arbiter will be redesigned to have the ability to forward different types of packets based on their priorities. Controller The controller will interact with the hardware and be involved in the entire processing procedure. It will send instructions to the hardware, such as START, STOP, ENABLE QOS, DISABLE QOS, and so on. What s more, it can gather the first 128 bytes of each packet from the PCI Bus for monitor software to distinguish different types of packets if needed. In order to improve the flexibility of the switch and lower the pressure of the host, which acts as the OpenFlow Switch, out-of-band control module is used. In this module, the controller and switch are running on two hosts, so that it is easy to control the behavior of the switch. Meanwhile, it will greatly increase the stability because it becomes much easier to diagnose the problems when the switch malfunctions. The relationship between the switch and the controller is shown in Figure 6. Figure 6. Structure Between Controller and OpenFlow Switch. Monitor Software A GUI-based monitor software is implemented in order to get the performance data for contrast and further optimization. Java is used to implement the monitor in this project. The GUI monitor software will collect and display the performance data shown in Table 1. The performance data can be collected from several different sources. When the packets are de-capsulated by the operating system, it is very easy to capture and distinguish the type of each packet; the packet cap function can be implemented by using pcap library. Since the sender-side terminal knows how many packets it sent and the receiver-side terminal knows how many packets it received, it is able to get the percentage of Packet Loss. By synchronizing the time of terminals first, the accurate switching time of every packet can be obtained. Finally, the average delays of two packet types can be easily calculated. Other Related Components (1) Test Applications Suite After implementing the hardware and the controller, the switch should work properly. In order to test the performance of the new switch and make comparison to the old one, several applications used for testing proposed will also be developed

4 Normal Packets Stream Packets Stream Packets Sent Received Sent Received (%) Packet Loss (%) Normal Packets Delay(ms) Stream Packets Delay(ms) Table 1. The Main Frame of GUI Monitor Software. The suite contains a sender application used to generate traffic, a receiver application used to receive the traffic and calculate the performance data, and two time synchronization applications used to sync the time among multiple hosts. (2) Shared Library Java cannot access the registers in the NetFPGA card directly, so a shared library will be implemented as an interface to connect the monitor software and the card. Here C programming will be used to implement the bridge and acquire the data from the card. The program will be compiled into shared object in Linux, which is similar to the dynamic link library in Windows. By doing so, the monitor software in Java is able to access the NetFPGA via the shared library Project Structure In conclusion, the project has six components. The relationship among the components is briefly shown in Figure IMPLEMENTATION The implementation details of each component mentioned in design part are discussed and introduced in this section Hardware In order to support the priority control feature, a new component used to pre-analyze each packet is implemented as mentioned in the design part. The pre-analyzer component will classify and buffer two different types of packets. At the same time, the algorithm of Input Arbiter is redesigned to meet the requirements of the feature. Processing Procedure The user-data-path components receive packets from eight ports [four physical Ethernet MAC ports and four logical CPU DMA (Direct Memory Access) pipelines]. For each pipeline, the new components will conduct two operations: remove the VLAN (Virtual LAN) tagging field and test the ToS bit. After this kind of classification, all the data and QoS indicators will be passed to input arbiter through eight different pipelines. Figure 7. Project Structure

5 Figure 8. New Components Between Ethernet MAC Port and Input Arbiter. The Input Arbiter will choose which pipeline to forward its packet to the next processing component by using a well-designed algorithm. The algorithm will schedule the sequence of the packets in queues, so that the low-priority packets arrived earlier may be forward later than the high-priority packets arrived later. Therefore, QoS is implemented to some extent. The new structure with the added components is shown in Figure 8. VLAN Remover Component The switch will identify the type of packets by Differentiated Services field in the IP header. But the size of data frame can be various if VLAN technology is used, which increase the difficulty for next component to analyze. To simplify the logic of next component, this component is added to remove the VLAN tagging field in the data frame. Pre-analyzer Component After the VLAN tagging field has been removed, the offset of Differentiated Services field is absolute. If any bit of it is non-zero, the QoS indicator value of the corresponding pipeline will passes 1 to the Input Arbiter, otherwise passes 0. Arbiter Algorithm The original arbiter algorithm is round-robin algorithm, which means the arbiter checks eight pipelines one by one. If current checking pipeline has packets, the arbiter will choose it and forward one of them at a time. Then check the next pipeline. The purpose of this project is to treat packets differently, so the fair scheduling algorithm will not be used. A new algorithm based on (1) the types of packets and (2) the number of cycles that the packets have been waited is designed. The new algorithm maintains two kinds of priority scales, one is 4 for the QoS packets, and the other is 2 for the normal packets. Each pipeline has its own priority value. When a new packet arrives at the front end of one pipeline, the priority will be set to the scale according to the type of packet. If the pipeline has packet ready but is not chosen by the Input Arbiter, the corresponding scale will be added to the priority value of the pipeline. If the pipeline is chosen, the priority will be reset to 0. The Input Arbiter always chooses the pipeline which has the largest priority value. If more than one pipeline has the same largest number at the same time, the pipeline which has the lowest port number (from 0-7) will go first. This algorithm considers QoS and normal packets at the same time, and it well balances the entire performance of the switch Controller NOX 0.6.0, an OpenFlow switch controller, which is based on the latest stable version of - 5 -

6 OpenFlow protocol, is chosen as the controller in the project. Also, a Debian-based operating system will be deployed to run the controller software. Ubuntu is chosen in the project and the switch will contact with the controller remotely via another Ethernet. The official support platform of NetFPGA is CentOS, while the official support platform of OpenFlow is Debian-based Linux, such as Ubuntu. So it really takes time to solve the capability problem. Another important reason why use the out-of-band control module is that NOX cannot be successfully complied under CentOS. It will be much easier to configure the controller if the right operating system is chosen. it When the switch can reach the controller, the switch should be working properly. Next, several instructions which used to control the behavior of the switch, such as enable or disable the QoS features will be implemented. The API provided by NOX is used to implement this. int send_openflow_command(const datapathid&, const fp_header*, bool block) const; All the required programming references can be found in OpenFlow specification document. When the instruction is sent to the switch, the hardware will recognize it and respond to it Test Application Suite The test application suite consists of three components: packet sender, packet receiver and time synchronization programs. Packet Sender Packet sender can generate both QoS packets and normal packets. All the sending parameters can be easily changed by configuring send.ini file. These parameters include the destination host address (the receiver s address), the number of packets needs to be sent, the size of each packet, the sending rate, the ratio of normal packets to all packets, and the monitor address (the host running the monitor software). While the packet sender sending packets by specified parameters, it reports the statistic data to the monitor software every one second (by default) simultaneously. Packet Receiver The packet receiver receives packets from the Ethernet network and analyzes them. The packet type and the packet delay can be calculated when the packets arrived. The receiver will also report its status to the monitor software every one second (by default) while receiving packets. Time Synchronization Programs The packet delay can be measured by knowing when it is sent and when it is received. To solve this problem, the packet sender encapsulates the send-time in the packet and sends it to the receiver. When the packet is delivered, the receiver will know both the sending and receiving time. In order to measure the delay as accurate as possible, it is very essential to synchronize the time of two hosts. One of the computers in the network is chosen to work as a time server, which means the time on that machine is assumed as the standard time. Before any experiment conducts, all the other computers should synchronize time with the server first. By doing so, the delay measurement will become much more accurate Shared Library Since the monitor software in JAVA cannot access the registers of NetFPGA directly which is mentioned before, several native functions which are written in C language are compiled into the shared object file. The functions in shared object file are able to read the registers, so that the performance data of each port stored on the board can be acquired. The GUI program will invoke those functions via the shared object, get the data needed, and show them on the GUI Monitor Software The GUI monitor shows performance data from two sources: one is the NetFPGA registers; the other is the sending or receiving terminals

7 Figure 9. GUI Monitor Software. To get the performance data from the NetFPGA registers, a mechanism of JAVA called JNI (Java Native Interface) is used to invoke the native functions defined in the library file. JNI allows native methods to be written to handle situations when an application cannot be written entirely in the Java programming language, e.g., when the standard Java class library does not support the platform-specific features or program library.[19] By invoking functions in the shared library, the performance data stored in the NetFPGA registers can be obtained and shown out in the upper panel including bytes sent, bytes received, sending rate, and receiving rate of each port. To measure the packet loss and the packet delay, the monitor must be able to get data from the sending and receiving terminals. UDP protocol is used to communicate each other within the test applications. The monitor program starts four threads and each of them listens to a specified port. The program will check each port every one second (by default) to gather the performance data including packets sent, packet received, bytes sent, bytes received, sending rate, receiving rate, and delays of both normal packets and QoS packets via sockets. Those data are shown in the lower panel after processing. A screenshot of the monitor software is shown in Figure CONTRASTING EXPERIMENTS The switch is implemented successfully, next is to implement a comprehensive experiment to see the actual performance of the switch with QoS feature. A real Ethernet networking environment is set up to make contrast experiments to test the effect of the QoS feature and the impact of stream packets ratios on switch performance Performance Test A networking environment with only one sender and only one receiver is set to test the maximum throughput of the OpenFlow switch. The result is disappointing. The maximum transmission rate can only reach 24 MB/s, even if using the official binary file. Theoretically, the Xilinx FPGA chip has the ability to handle 8 Gbps throughput. We tried different NIC cards, different cables, and different operating systems, but the result is the same. So it may be the card designation problem, probably in the Input Arbiter component

8 Loss (%) 20% 40% 60% 80% 100% 120% 140% 160% 200% Maximum Delay (ms) By changing the number of input/output ports, a formula used to indicate the transmission rate is derived: Rate = 24 MAX(# of in ports, # of out ports ) MB/s Packet Delay Under Different Loads QoS Packets Normal Packets 5.2. Contrasting Experiments QoS for live streams is the most innovative and critical feature in the new switch. In order to test whether it works, and how much improvement it can exactly achieve, the performance data mentioned in Table 1 will be gathered and analyzed. Meanwhile, an OpenFlow switch without the QoS feature is configured as a contrast. To measure the performance of the new switch, two performance data are considered in this test: Packet Delay and Packet Loss. Since the performance may be various under different switch loads, or different packet ratios, those factors are also considered in the experiments to make it more comprehensive. Mark the OpenFlow switch with QoS feature as Switch 1, and the normal OpenFlow switch without QoS feature as Switch 2. The performance data of QoS packets in Switch 1 is indicated by blue lines, the performance data of normal packets in Switch 1 is indicated by red lines. Since all the packets in Switch 2 are treated equally, so the performance data of both packet types are indicated by green lines in the figure. The experiments are conducted under the same networking environment (same pc terminals with same cables). Use the data shown on monitor software to compare and analyze. Figure 10 and Figure 11 show the differences of three parameters. Figure 10. Packet Delay Contrast under Different Loads (Lower Is Better) Low Load Switch Load (%) Packet Loss Comparsion Medium Load High Load Switch Load Over Load QoS Pkts, Switch 1 Normal Pkts, Switch 1 Pkts, Switch 2 Figure 11. Packet Loss Contrast under Different Loads (Lower Is Better). From the figures, the actual transmission quality of QoS-Required packets is much better on Switch 1 than Switch 2. This reflects in the performance data; both the packet delay and the packet loss of QoS packets are lower than those of the normal packets, especially when the switches reach the high load. It is very understandable that the performance improvement for stream packets is based on the performance decline for normal packets as a result. To indicate the effect of the QoS feature more clearly, the packet delay parameter is chosen to make comparison between the two kinds of packet under the overload condition

9 Percentage (%) Advantage of QoS Packet Delay Advantage Ratio (Percentage of QoS Packets) Figure 12. Advantage of QoS Packet Delay. When the QoS packet is only a little portion (11%) of the entire traffic, the delay of QoS packets is 28% lower than normal packets. As the ratio of QoS packets becomes higher, the effect becomes less and less clearer. Another important point for this project is to find a performance balance between QoS packets and normal packets. In order to solve this problem, performance data under different packet ratios should be collected to optimize the switching algorithm. So that users can get a comfortable experience under more different networking conditions. 6. CONCLUSIONS The new switch can improve the performance of a network by strengthening the effect for applications using video and voice data. For users who use these applications, a more understandable video and voice communication experiences with less delay and noise can be achieved. The monitor software on the PC will show users the performance statistics and specific details about how the switch works. This switch can be deployed in the networks which people have high demands on video and voice communication, or which people use those applications frequently. For some networks, this kind of switches may not improve the performance at all. So in order to deploy the switches into more generic networks, more studies on the algorithms should be researched in the future. 7. REFERENCES [1] Wikipedia, Networking Switch. (September 2009). [2] LinuxHPC.org and Cluster Resources, Network Switch. edia/alphabetized/n/network-switch.php (September 2009). [3] V. O. K. Li, J. Fu. Chang, K. C. Lee and T. S. Yang, A survey of research and standards in high-speed networks /abstract (September 2009). [4] Wikipedia, OSI_Model. (September 2009). [5] Cisco, Internetworking Basics. king/technology/handbook/intro-to-internet. html (February, 2008). [6] Wikipedia, Internet Protocol Suite. l_suite (September 2009). [7] Protocols.com, TCP / IP Reference Page. Data (September 2009). [8] Wikipedia, IEEE (September 2009). [9] N. L. Binkert, A. G. Saidi, S. K. Reinhardt, Integrated Network Interfaces for High-Bandwidth TCP/IP &coll=ACM&dl=ACM&CFID= &CFTOKEN= (2006). [10] J. Paris, V. Gulias, A. Valderruten, A High Performance Erlang TCP/IP Stack &coll=ACM&dl=ACM&CFID= &CFTOKEN= (2005) [11] netfpga.org, NetFPGA Learn More. /OneGig/LearnMore (January 2010). [12] openflowswitch.org, What is OpenFlow? e/ (September 2009). [13] J. Naous, D. Erickson, G. A. Covington, G. Appenzeller, N. McKeown, Implementing an OpenFlow Switch on the NetFPGA Platform. ANCS '08: Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems. 2008, pp [14] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker and J. Turner. OpenFlow: Enabling Innovation in Campus Networks. ACM - 9 -

10 SIGCOMM Computer Communication Review, Vol. 38, Issue 2, Apr. 2008, pp [15] OpenFlowSwitch.org, OpenFlow Switch Specification. penflow-spec-v0.9.0.pdf (July 20, 2009). [16] Wikipedia, Computer network programming. work_programming (September 2009). [17] Girish Venkatachalam, High-Performance Networking Programming in C. Linux Journal, Vol. 2007, Issue 163, Article No.6. November [18] Gokhale, M. Stone, J. Arnold, J. Kalinowski, M. Stream-Oriented FPGA Computing in the Streams-C High Level Language. Field-Programmable Custom Computing Machines, 2000 IEEE Symposium. 2000, pp [19] Wikipedia, JNI. (April 2010)