The impact of transport protocol, packet size, and connection type on the round trip time

Bachelor of Science in Computer Science with specialization in Game Programming June 2017 The impact of transport protocol, packet size, and connection type on the round trip time Tobias Kling Faculty of Computing Blekinge Institute of Technology SE 371 79 Karlskrona, Sweden i

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Degree of Bachelor of Science in Computer Science with specialization in Game Programming. The thesis is equivalent to 10 weeks of full-time studies. Contact Information: Author(s): Tobias Kling E-mail: tobbe.kling@hotmail.com University Advisor: Francisco Lopez Luro Department of Creative Technologies Faculty of Computing Internet : www.bth.se Blekinge Institute of Technology Phone : +46 455 38 50 00 SE 371 79 Karlskrona, Sweden Fax : +46 455 38 50 57

Abstract While developing networking for games and applications, developers have a list of network specific requirements to be met as well as decide how to meet them. It is not always easy to decide what protocol is best suited for a given network configuration, or what is the best size of a data packet. By performing a comparative analysis, it becomes possible to identify how protocols, packet size, and network configuration impact the one-way travel time and throughput of a given implementation. The result shows how the different implementations compared against each other and the analysis tries to determine why they perform as they do. This gives a good overview of the pros and cons of how TCP, TCP(N), UDP, and RakNet, behave and perform over LANs and WLANs with varying packet size. Keywords: Transport protocol, Network Configuration, Packet Size, Comparison

ii

List of Figures 5.1 Ethenet TCP Delay Results............................. 12 5.2 Ethenet TCP(N) Delay Results........................... 13 5.3 Ethenet RakNet Delay Results............................ 13 5.4 Ethenet UDP Delay Results............................. 14 5.5 Switch TCP Delay Results.............................. 15 5.6 Switch TCP(N) Delay Results............................ 16 5.7 Switch RakNet Delay Results............................ 16 5.8 Switch UDP Delay Results.............................. 17 5.9 Private wifi TCP Delay Results........................... 18 5.10 Private wifi TCP(N) Delay Results......................... 19 5.11 Private wifi RakNet Delay Results.......................... 19 5.12 Private wifi UDP Delay Results........................... 20 5.13 Public wifi TCP Delay Results............................ 21 5.14 Public wifi TCP(N) Delay Results.......................... 22 5.15 Public wifi RakNet Delay Results.......................... 22 5.16 Public wifi UDP Delay Results........................... 23 5.17 Ethernet TCP Throughput Results......................... 24 5.18 Ethernet TCP(N) Throughput Results....................... 25 5.19 Ethernet RakNet Throughput Results........................ 25 5.20 Ethernet UDP Throughput Results......................... 26 5.21 Switch TCP Throughput Results.......................... 27 5.22 Switch TCP(N) Throughput Results........................ 28 5.23 Switch RakNet Throughput Results......................... 28 5.24 Switch UDP Throughput Results.......................... 29 5.25 Private wifi TCP Throughput Results........................ 30 5.26 Private wifi TCP(N) Throughput Results...................... 31 5.27 Private wifi RakNet Throughput Results...................... 31 5.28 Private wifi UDP Throughput Results....................... 32 5.29 Public wifi TCP Throughput Results........................ 33 5.30 Public wifi TCP(N) Throughput Results...................... 34 5.31 Public wifi RakNet Throughput Results...................... 34 iii

Contents List of Figures iii 1 Introduction 1 1.1 Background...................................... 1 1.2 Problem Description................................. 1 1.3 The Scope of the Thesis............................... 1 1.4 Research Question.................................. 2 2 Previous Work 3 3 Theory 4 3.1 What impacts the problem.............................. 4 3.2 Network Configuration................................ 4 3.3 Network Transport Protocols............................ 5 3.4 Packet Size...................................... 6 4 Method 7 4.1 Method Description.................................. 7 4.2 Execution....................................... 8 4.3 Expected Results................................... 10 5 Result 11 5.1 Delay Results..................................... 12 5.1.1 Ethernet.................................... 12 5.1.2 Switch..................................... 15 5.1.3 Private Wifi.................................. 18 5.1.4 Public Wifi.................................. 21 5.2 Throughput Results.................................. 24 5.2.1 Ethernet.................................... 24 5.2.2 Switch..................................... 27 5.2.3 Private wifi.................................. 30 5.2.4 Public Wifi.................................. 33 6 Analysis 35 6.1 Packet Travel Time Test............................... 35 6.2 Throughput Test................................... 37 iv

7 Conclusion 39 7.1 Summary....................................... 39 7.2 Future Work...................................... 40 A Bachelor Networking 43 A.1 main.cpp........................................ 43 A.2 NetworkService.h................................... 55 A.3 NetworkService.cpp.................................. 56 A.4 NetworkData.h.................................... 57 A.5 WinsocModule.h................................... 59 A.6 WinsocModule.cpp.................................. 62 A.7 RakNetModule.h................................... 88 A.8 RakNetModule.cpp.................................. 90 B Modified RakNet Big Packet Test 97 B.1 BigPacketTest.cpp.................................. 97 C Hardware List and Network settings 106 D Literature study 108 v

vi

Chapter 1 Introduction 1.1 Background A common and popular feature for today s video games is the ability to play together with players from across the world. Being able to play the same game, at the same time, with friends or strangers is the main selling point of many games. The games that support this feature stretch from turned based strategy games[1][2] to a fast paced shooter[3][4]. To make the experience enjoyable for the players, the game has to always feel responsive even though the data might have to travel from one side of the planet and back again. Depending on the game, the requirements for transport time and reliability of the data will vary. This is why it is important for the developer of a games networking system to know what can be done to optimize the game s networking to best suit its requirements. 1.2 Problem Description There are many variables that determine the total time it takes for one packet of data to be sent from one system to another. Some of the variables can not be interacted with and are therefore constants, such as the time it takes the data to physically travel from one device to another. There are also other variables that can be changed and optimized to make the total travel time faster. Determining the best combination of these variables is the key to making an effective and fast communication system for video games. 1.3 The Scope of the Thesis For this paper, a literature study will be conducted using the Snowball method described by Claes Wohlin in his paper Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering [5] to identify earlier works in the field that are relevant to this paper and the experiment. The results of the literature study can be found in Appendix D. The selected papers from the literature study will be further discussed in Chapter 2. After studying the results of the literature study, three independent variables have been identified to impact the Packet Delay and will, therefore, be the main focus for the experiment. The three variables are Transport Protocol(TP), Packet Size, and Network Configuration. These 1

variables will be discussed in more detail in Chapter 3. This paper will use a comparative analysis to compare different real combinations of these variables. Each combination will vary its Transport Protocol, Packet size used and Network Configuration to identify any changes to its Packet Delay and Throughput. The research goal of this analysis is to identify which combination would be best suited for given network requirements based on Packet Delay and Throughput. In this paper, there will be two test scenarios, one that will measure the Packet Delay, and one that will test the Throughput of the implementation. The first scenario will have the combinations to only to send a single packet at a time and measure the one-way travel time of that packet. This is to determine the sending speed and potential differences in packet delay between Network Configurations. The second scenario will have the combinations send one gigabyte of data to measure the throughput of the combination. The experiment will be executed using a test bed where different combinations of the variables can be tested and data will be collected on Packet Delay, Bandwidth, Packet Loss, and CPU Work Load. More information will be provided in Chapter 4. Finally, the data will be presented in graphs in Chapter 5. The results will be analyzed in Chapter 6 and finally, a summarized answer to the research questions will be presented in Chapter 7. 1.4 Research Question To summarize; there will be two research questions that this paper will try to answer. 1. What is the impact of different combinations of transport protocols and packet sizes on the packet delay and bandwidth? 2. How does the network configuration impact on the packet delay? 2

Chapter 2 Previous Work In their book, Performance Analysis of Computer Networks [6], Matthew Sadiku and Sarhan Musa talk thoroughly about the techniques and performance measurements normally used to analyze network performance. They discuss the three performance evaluation techniques of a network, measurement, analytic modelling, and simulation and the pros and cons in different situations. They also describe the inner workings of different topologies for both LAN and WLAN which are the network configurations this paper will mainly focus on. The methods used in this paper will be based on the guidelines discussed in this book. In the paper TCP and UDP Performance over Wireless LAN [7], by George Xylomenos and George Polyzos, a similar experiment to the one in this paper is performed. By doing a real implementation of both TCP and UDP while varying the packet size and processing power, the authors try to identify how the performance of the system changes depending on the variables. The goal of the journal is to identify performance problems in realistic conditions for UDP and TCP but does not try to compare the protocols against each other to determine the optimal choice in the given scenario. The experiment that will be performed in this paper will be similar to the experiment executed in the journal, but with some additional variables such as network configuration, and with the goal to compare the results against each other to identify the best combination of variables for each scenario. There are also multiple papers that perform performance measurements with TCP and/or UDP, but not with real implementations [8, 9, 10, 11]. Many researchers seem to prefer to simulate their experiments using software such as The Network Simulator (ns-2) [12]. Using an established Simulation program such as NS2 for an experiment is in most cases both faster and more cost effective than to making a real implementation. It also avoids outside factors that might skew the results. This is also why a simulation is not always the best way to go. A simulation in most cases does not reflect reality since there is a lack of unexpected outer factors that would impact the final results. This is why in this paper the experiment will be based on a real implementation instead of running a simulation. Based on the literature study conducted, there are no more papers related to this thesis. There are more papers that do conduct network measurements and comparisons like the papers mentioned earlier[8, 9, 10, 11], but the author decided there were not much credit to include more papers that were not directly related to this thesis. 3

Chapter 3 Theory 3.1 What impacts the problem To determine what design decisions a developer has to take, the author has identified three independent variables that will have an effect on the Packet Delay. By doing a comparative analysis of different combinations of the variables, it will be possible to find the individual impact on the Packet Delay for each one of them. The variables that will be tested in this thesis are; Network Transport Protocol, Packet Size, and Network Configuration. 3.2 Network Configuration Network Configuration is the network s physical structure. This means how two computers are connected to each other, more often than not, by Ethernet cables, switches, and routers. With bigger networks such as the Internet, it is between these devices where most of the Packet Delay time is created. The time for a packet of data to traverse a network grows bigger the more connections and network devices it has to travel through. Each device adds further latency to the Packet Delay. This is because each device that the packet has to travel through, will have to read and redirect each packet that it handles. When the input is higher than the reading capabilities of the router, a buffer is created to store incoming packets while it reads the oldest packet in the buffer. In public networks where there may be thousands of users at the same time, the device buffer may get close to full and will, therefore, create a noticeable delay for incoming packet since it has to wait in the buffer to be processed. If a buffer is full when a packet arrives, the arriving packet will be discarded and lost forever[13]. This can create big problems for a system if it is not programmed to handle packet loss, and even if it handles the loss, it will create further delays. The redirection of the packet takes time as well. The device has to read the packet and extract the destination from it, checking its internal network map, and then redirecting the packet to the next device. The time it takes for a device to do this is not big[14], but it is close to a constant time that will be multiplied by the number of devices that the packet will traverse through. In this thesis, there will be four types of network configurations: 1-1 with Ethernet cable without any intermediate device which will serve as the baseline for the experiment, 1-1 with Ethernet cable with switch, 1-1 on a dedicated WIFI (only the two computers on it), and 1-1 on public WIFI (other computers on the WIFI). 4

3.3 Network Transport Protocols Network Transport Protocol, or just simply Transport Protocol(TP)[15], is the description of how the given data should be packeted, sent, received, and handled if lost. Following will be a detailed description of the protocols that will be used for this thesis and how they behave compared to each other. In this thesis, we will focus on the two most common TPs and two variations on them and how they impact the Packet Delay and the bandwidth of the system. The protocols used in this thesis are TCP, TCP(NoDelay), UDP, and UDP implemented with RakNet. All of them are well documented, thoroughly tested, and they are considered a standard for each of their fields. Therefore it is considered safe to use these protocols for the experiment. TCP (Transmission control protocol)[16] is a reliable protocol that guarantees that the packets always arrives at their destination. This is done by adding an acknowledging system which requires the receiver of the packet to send an ack(acknowledge) packet to the sender to tell it that the packet has arrived. If the original sender does not receive an ack packet, it will assume that the sent packet was lost and will resend it. This method is reliable but slow since TCP also confirms in what order packets should arrive. So if a series of packet 1,2,3,4, and 5 is sent in that order and packet 1 is lost, the other packets will have to wait until packet 1 is resent and arrives before being processed. This slows down the communication a lot. TCP also have another way to prevent overflowing buffers. The congestion window and receive window of TCP creates congestion control which tries to avoid packet loss by throttling the number of packets sent each frame by different means. The sender uses the congestion windows to define how many packets that can be sent each frame, and when a successful send is completed it slightly increases its size. But when the sender detects packet loss, it halves the congestion window, assuming that it is because of a buffer got full. By halving the throughput, TCP hopes to let the buffer process the stacked packets in its buffer before trying to pressure it again with a higher number of packets. The receive window is used by the receiving system and announces for the sender how many bytes are still left in the receiver buffer. This helps the sender to know how many packets it can send before it would fill up the receiver buffer. TCP(NO). The second protocol is the same TCP as in the first, but with one flag changed, NO DELAY. When this flag is used on initialization, Nagle s algorithm[17] will be turned off. Nagle s algorithm is used to take small packets and put them into a buffer until there was enough of data to fill a larger TCP packet. This way, the overhead was minimized for the cost of speed. By disabling Nagle s algorithm in TCP, it sends small packet a lot faster which can be useful for games that require small and fast packets to function. UDP. The third protocol is UDP (User Data Protocol)[18]. UDP is a protocol that focuses on sending the data packets as fast as possible without regards if they arrive at the destination or not. In contrast to TCP, UDP does not handle lost packages, but processes packages as they arrive and is therefore often used for games that do not care for the order that the data arrives in, but about what data is the most up-to-date. RakNet. The fourth and final protocol is UDP implemented with the RakNet[19] library. RakNet is a high performance, cross platform, game network engine which is designed to reliably handle packet loss and congestion control. This will increase the reliability of the UDP protocol which should prove interesting for the experiment results. 5

3.4 Packet Size Packet size is the size of the data packet. Each packet contains a header with a fixed size besides the data that will be transported. In some cases, it can be good to put different data in the same package to save the overhead of having multiple packets and headers and only send one larger packet instead of multiple small ones. The negative side of this is that one big packet will take up much space in the buffers which may create bigger packet loss if the buffer is not large enough to handle the incoming packets. The theoretical max size for TCP and UDP are around 64 kb (TCP 65535 bytes[16] and UDP 65507 bytes[18]), but packets of this size are not commonly used. That is because of the MTU (Maximum Transmission Unit), which is the maximum size of data that can be sent at a time. By effect, this means that if data with a size larger than the MTU is passed from the transport layer to the network layer, the data will be fragmented into smaller packages that are provided with the IP header with the final destination address. These packages will be the ones to be sent over the physical link. In this paper, the MTU size used for the experiment was 1500 bytes for all scenarios. 6

Chapter 4 Method 4.1 Method Description There are three independent variables in that will be tested in this experiment: transport protocol, packet size, and the network configuration between the processes. The transport protocol variable will be tested with four values: UDP, TCP, TCP with the NO DELAY flag active which disables Nagle s algorithm, and UDP implemented with RakNet. The packet size will range between set values that will be the same for every combination of Transport Protocol and connection type. The final packet size is determined by the network interface MTU value as mentioned in Chapter 3 Theory. The MTU value will be set through windows command console (cmd) and will be covered in detail later in this chapter under the Execution Section. There will be four types of network configurations: 1-1 with Ethernet cable without switch which will serve as the baseline for the experiment, 1-1 with Ethernet cable with a switch, 1-1 on a dedicated WIFI (only the two computers on it), and 1-1 on public WIFI (other computers on the WIFI). There will be two different scenarios that will test the different combinations of variables. The first scenario will measure the travel time of a single packet. Here, the packet sizes will be identical for all the protocols and network configurations. This will determine what combination creates the fastest single packet transport time. In the second scenario, the system will have to send a large amount of data (1GB) over the network. This is to measure the bandwidth used and the reliability of the different transport protocols. To transport 1 GB of data, the packet size for TCP, TCP(NO), and UDP will use one set of packet sizes while RakNet will use another set. This is because RakNet handles large packet transfers differently and sends larger segments at a time instead of individual packages. The RakNet part of the experiment will be based on their own example project, Large Packets and the modified file can be found in appendix B. Even though the packet sizes will differ, the purpose of the second scenario is to determine the effective bandwidth used, but it is worth noting the difference in the handling of the data. In none of the scenarios will data validation be a factor. For the experiment, the payload of the packet will be filled with throwaway values and will not be processed by the receiving system. This is to save processing power and keep the throughput as fast and consistent as possible. A program will be implemented to give full control over all the variables in the experiment. The program will give the user the options to choose the transport protocol, the amount of data to send, packet size, the IP address to connect to, and if the program will send or receive data. The source code of the program can be found in Appendix A. The collection of data will 7

be handled by windows own Performance Monitor tool which allows the user to create a user defined data collector set where the system s performance data can be collected. The environment that the experiment will be executed in will be two identical computers with Windows 10 operating systems. Both computers will run an instance of the program, one set to send data and the other to receive data. This way, full control can be had over both the sending and receiving of packets and can, therefore, be measured reliably. This means that the sending and receiving sides will not perform any other computation other than sending and receiving data. This is to isolate the impact of each variable on the packet delay (PD) as possible. To study the basic latencies that are introduced by additional network devices such as switches and routers, the experiment will first start with a simple one to one Ethernet cable connection. Then by adding more devices that the data has to go through, each device s latency can be determined. By testing all different combinations of these variables and measuring the Packet Delay and the bandwidth, and then plotting the results to graphs will grant the answer to the first research question. The connection type of 1-1 with Ethernet cable without switch will serve as the baseline for the PD since the effect of external disturbances will be minimal. By comparing the baseline with the other connection types, the impact of the connection can be deducted and answer research question two. The experiment will only study one peer to one peer communication. This is to try to isolate the impact of the variables as much as possible in each scenario. Therefore, the result of this experiment will suit one to one connections more, but the experiment could still be done with other network configurations such as a server-client setup. Since there will be a risk for packet loss for UDP in combination with some of the network configuration options, the loss of data statistics will also be collected and presented together with the bandwidth in the analysis. 4.2 Execution Hardware Setup: This part of the paper is a step-by-step tutorial how the experiment was performed. The hardware used in the experiment can be found in Appendix C. Set up two computers with Windows 10 operating system and install the network measuring program (In this paper the computers had the same specifications).chose which computer will be the sender and which will be the receiver and then start the network measuring program with the following parameters on the corresponding computer: Sender: -s -*protocol* -ip *ip-address of receiver* -p *number of iterations* Receiver: -r -*protocol* -p *number of iterations* The Sender has three parameters. The -s is to tell the program that it will be the sender and will therefore also require the -ip parameter to set the ip-address it will connect to. The Receiver uses -r instead of -s to set itself up as the Receiver. Both programs will have the same protocol parameter which will define what transport protocol 8

will be used by the program. There are four options: -u (UDP), -t (TCP), -tn (TCP(NO)), and -rn (RakNet). The last parameter is in what mode the program will run in. Both the programs will have to have the same transport protocols or the programs will fail. The -p parameter is to set the program in ping mode which means that it will measure the travel time of a single packet. Entering -ps instead of -p will put the program into throughput mode which is used for the second scenario. The -ps parameter will be followed by the size of the packets for the throughput test. The MTU of both computers were checked before the experiment started so that it was set to the standard size 1500 bytes for Ethernet communication. This means that if a packet larger than 1500 bytes is trying to be sent, it will be fragmented into smaller packages as discussed in Chapter 3: Theory. When all combinations packet size and protocol has been tested on one network configuration, it is time to switch over to the next one and redo the test. In this experiment four network configurations were used, 1-1 with Ethernet cable, 1-1 with Ethernet cable with a switch, 1-1 on a dedicated WIFI (only the two computers on it), and 1-1 on public WIFI (other computers on the WIFI). Packet Travel Test: To perform the packet travel test, all that is needed is to enter the correct parameters for both the Sender and Receiver. The Receiver has to be started before the Sender for it to connect correctly. After that, the program will measure the speed of four different sized packets. The sizes are 4, 512, 1024, 1500, and 2048 bytes. This is to identify if there is any difference in speed when the packet size is smaller or larger than the MTU of 1500 bytes. The one-way travel time is calculated by using a variation of Cristian s algorithm described in Flaviu Cristian s paper (Probabilistic clock synchronization)[20] which is to take the total travel time and divide it by two, assuming that the travel time is the same both ways. In this paper, the experiment also used four different amounts of iterations to see if there would be any notable differences in speed and in UDP case, packet loss. Data was collected for 100, 500, 1000, and 100.000 iterations. The time will be measured in nanoseconds as the overhead this thesis is trying to examine is very fast. By using nanoseconds it will be possible to identify even smaller changes to the total travel time. When one test is completed, switch the topology and redo the test until all protocols, packet sizes, and topologies have been measured. The results will be printed into text files next to the executable file. Throughput Test for TCP, TCP(NO), and UDP This section is only for the protocols TCP, TCP(NO), and UDP. This is because RakNet uses another program to send large packages and will therefore have its own section below this one. In the throughput test, it is needed to collect some more data than in the packet Travel test. To do this, open the Performance Monitor program provided by Windows 10 and create a new User defined Data Collector Set and add the following Performance Counters to the set: NetworkAdapter/Bytes sent per second NetworkAdapter/Packets sent per Second NetworkAdapter/Current Bandwidth Processor/User Time in percent 9

The coloured part of the Counter names may vary depending on the computer and what parts it uses. This allows one to collect information on how the network adapter and CPU work while the program is transferring the 1GB of data between the two computers. A user would want to set a directory where the resulting data file would be saved. The parameters are similar to those in the first test. The only real difference is the -p parameter and should be changed to -ps followed by the size of the packets that should be sent. In the experiment, there were three sizes used, 10240, 30720, and 61440 bytes. The difference in size is trying to create overhead by forcing the transport protocol to fragment the packages into smaller packages to fit the MTU of 1500 bytes. Unlike the Packet Travel Test, there will only be one iteration of each combination of variables since the transfer times will allow the Data Collector Set to collect more than enough data to draw conclusions from. Before starting the program start the Data Collector Set and then the program. When the program has successfully sent 1GB of data, the user will have to stop the Data Collector Set manually. The result files from the program will be next to the executable like in the first test, while the Data Collector Set result file will be at the predefined location. Throughput Test for RakNet As mentioned above, RakNet sends large data packages a bit differently than the author s program. Instead of letting the user send individual packets of a certain size, it defines large segments that is then split up in smaller packages of that the API determines suitable. In this experiment, the RakNet example project was used and modified to send 1GB of data. The modified RakNet project can be found in Appendix B. The author chose the segment sizes of 40 MB (41943040 bytes), 80 MB (83886080 bytes), and 160 MB(167772160 bytes). The sizes were chosen to see if the RakNet API would handle the sizes differently. The example project also starts it connection differently from the author s program. The Sender only initialize the communication but it is the Receiver that send the segments back to the Sender. this means that to collect the throughput data with the Data Collector Set, it will have to be run on the Receiver computer instead of the Sender as in the other tests. Even though the way that the data is sent and handled, the collected data will still be relevant to compare the differences in throughput against the other transport protocols. As the experiment will take much longer time than the one-way travel time experiment, the time unit will be changed from nanoseconds to milliseconds. 4.3 Expected Results The result will be presented in a series of graphs. There will be one graph for each type of implementation and the result will be based on the data collected in the two test scenarios the implementations will be put through. This will give a good and easy overview of the speed of each combination of transport protocol and packet size for every connection type. An informed answer will be obtained how the different implementation behaves individual and compared to each other. Other outcomes for the experiment could be that the results are very similar to each other and the author can not deduct any noteworthy impact on the Packet Delay or throughput in any combination of independent variables. In this case, the experiment will have to be revised to find another method of measuring and/or and identify new independent variables. The paper TCP and UDP Performance over Wireless LAN [7], that was discussed earlier in Chapter 2, shows that changing the variables should lead to different results for both in single packet speed and throughput which is in line with the author s expectations. 10

Chapter 5 Result In this chapter, all the data collected will be displayed in graphs. The results will be divided into two categories, Delay Results which is the data collected on the one-way travel time of a single packet, and Throughput Results which will show the throughput of the connection while transferring one gigabyte of data. 11

5.1 Delay Results As mentioned above, this section will show the average one-way travel time for a single packet between the two systems. The packets were sent one by one while the travel time is calculated for each one. The average time will be calculated for 100, 500, 1000, and 100.000 iterations. In this all observations and analysis will be based on the 100.000 iteration measurement since it is based on the largest amount of data-points. Data was collected for smaller number of iterations as a compliment and to identify if there was an increase or decrease in packet sending speed when more packets where sent. The black vertical line at the top of each bar is a visualization of the standard deviation of time for all the packets sent. This is to show the how the different packet times differ throughout the experiment. 5.1.1 Ethernet The result for the Delay test for the Ethernet network configuration. Here there are no intermediate devices between the computers since they are directly connected by a 1Gbit Ethernet cable. Figure 5.1: Ethenet TCP Delay Results 12

Figure 5.2: Ethenet TCP(N) Delay Results Figure 5.3: Ethenet RakNet Delay Results 13

Figure 5.4: Ethenet UDP Delay Results 14

5.1.2 Switch The result for the Delay test with a switch between the two computers. This adds an additional device that has to handle the data packets before they arrive at their destination. With added overhead, there should be an increase in travel time for the packets which hopefully can be seen in the graphs. Figure 5.5: Switch TCP Delay Results 15

Figure 5.6: Switch TCP(N) Delay Results Figure 5.7: Switch RakNet Delay Results 16

Figure 5.8: Switch UDP Delay Results 17

5.1.3 Private Wifi The result for the Delay test on a private wifi connection. By introducing a not as fast of a connection as a physical one, the packets one-way travel speed should be a lot slower. There were only the two computers that executed the experiment on the network to avoid network traffic that could interrupt the test. Figure 5.9: Private wifi TCP Delay Results 18

Figure 5.10: Private wifi TCP(N) Delay Results Figure 5.11: Private wifi RakNet Delay Results 19

Figure 5.12: Private wifi UDP Delay Results 20

5.1.4 Public Wifi The result for the Delay test on a public wifi connection. With he introduction of public network traffic, it should be expected that this network configuration should prove to be the slowest. The increase in traffic should fill up buffers and increase the time the packets have to wait before being processed. Figure 5.13: Public wifi TCP Delay Results 21

Figure 5.14: Public wifi TCP(N) Delay Results Figure 5.15: Public wifi RakNet Delay Results 22

Figure 5.16: Public wifi UDP Delay Results 23

5.2 Throughput Results The throughput test will focus on the throughput of the system while transferring one gigabyte of data. The size of the data packets that were sent varied between 10240, 30720, and 61440 bytes with the exception for RakNet as discussed in Chapter 4. This way it should be possible to identify how the throughput varies depending on the size and network configuration and which combination is superior in what scenario. 5.2.1 Ethernet The result for the Throughput test on an Ethernet connection. The physical connection should be the fastest and most stable connection out of all the other network configurations. The speed is only limited by the cables transfer speed and the receivers reading speed. Figure 5.17: Ethernet TCP Throughput Results 24

Figure 5.18: Ethernet TCP(N) Throughput Results Figure 5.19: Ethernet RakNet Throughput Results 25

Figure 5.20: Ethernet UDP Throughput Results 26

5.2.2 Switch The result for the Throughput test with a switch between the systems. Now with more data traffic on the network, there is a higher risk for buffer overflow which would result in packet loss and longer transfer times. The size of the packets might have a big impact in how the buffers will handle them selfs and the resulting throughput. Figure 5.21: Switch TCP Throughput Results 27

Figure 5.22: Switch TCP(N) Throughput Results Figure 5.23: Switch RakNet Throughput Results 28

Figure 5.24: Switch UDP Throughput Results 29

5.2.3 Private wifi The result for the Throughput test where the systems are connected over a private wifi with no other traffic. With the higher output of the sender, the router buffers will have a harder time to process the incoming data packets. Figure 5.25: Private wifi TCP Throughput Results 30

Figure 5.26: Private wifi TCP(N) Throughput Results Figure 5.27: Private wifi RakNet Throughput Results 31

Figure 5.28: Private wifi UDP Throughput Results 32

5.2.4 Public Wifi The result for the Throughput test where the systems are connected to a public wifi where other network traffic will be present. As mentioned before, the increase of traffic on the network will result in more packet loss and that, in turn, will lead to lower throughput. Figure 5.29: Public wifi TCP Throughput Results 33

Figure 5.30: Public wifi TCP(N) Throughput Results Figure 5.31: Public wifi RakNet Throughput Results 34

Chapter 6 Analysis 6.1 Packet Travel Time Test Ethernet: With a Direct Ethernet cable connection between the two systems, the fastest protocols were TCP (Fig. 5.1) and TCP(N)(Fig. 5.2) as they had close to identical one-way travel times for all packet sizes used. No clear difference in speed when packets larger than the MTU was used so direct impact of packet fragmenting. The second fastest was UDP (Fig. 5.4) with only a couple of thousands of nanoseconds slower than TCP. And the slowest was the RakNet (Fig. 5.3) implementation with units of tens of thousands nano-seconds slower than UDP for all packet sizes. All of the protocols had very low delay variation which is most likely because the internal buffers of the systems never got filled up since only a single packet was handled at a time. Switch: When a Switch is introduced between the two systems and connected by Ethernet cables, there is no clear fastest protocol. TCP (Fig. 5.5) and TCP(N) (Fig. 5.6) are faster than UDP (Fig. 5.8) and RakNet (Fig. 5.7) for 4 and 512 bytes packets but TCP takes a great drop in speed for 1024 and 1500. UDP and Raknet, on the other hand, seems to have much more stable speed throughout all packet sizes even if they are slower in most cases compared to TCP. The general decrease in speed for all protocols is because of the introduction of the switch. Since the packet now has to queue in another buffer and be read again before being sent on to the intended system. The Switch seems to add 20.000-30.000 nanoseconds (0.02-0.03 milliseconds) delay for the one-way trip. The reason for TCP s drop in speed for 1024 and 1500 bytes packets might be caused by fragmenting overhead or Nagle s algorithm waiting for more data before sending the packet. Nagle s algorithm seems unlikely since it would already be active in all the other packet sizes. Fragmenting also seems unrealistic since the other protocols did not see a similar decrease in speed. Also, the speed is increased back to normal for TCP and TCP(N) for the 2048 byte packets. One should expect the fragmenting to happen to the 1500 and 2048 byte packets since the MTU was set to 1500 bytes. The only logical thing left that could impact the travel time of the packet is other programs interfered with the experiment. Since the experiment was executed on computers with a Windows operating systems, there are background processes that might have interfered. 35

Wifi Private: The two systems are connected with a private wireless network with no other devices connected to it. Relatively even between all protocols even though there are some areas for all protocols that are irregular. RakNet (Fig. 5.11 ) have a really fast time for 4 bytes packet with 750.000 ns (0.75 ms) but then goes back up to 1.100.000-1.200.000 ns (1.1-1.2 ms) for all other packet sizes. TCP (Fig. 5.9) has a similar oddity with the 512-byte packet at 1.000.000 ns (1.1 ms) compared to the other packets of around 1.200.000 ns (1.2 ms). TCP(N) (Fig. 5.10) became slower for 1500 and 2048 byte packets with around 100.000 ns compared to the earlier packets and finally UDP (Fig. 5.12) becomes faster at the 512 and 1024 bytes packets with around 200.000-300.000 ns (0.02-0.03 ms). But the consistent fastest protocol is UDP followed by RakNet with TCP and TCP(N) in the last place. The packet loss for UDP was lower than 0.01% of 100.000 packets which is only ten packets lost which are a minuscule loss overall. With the data not travelling through a cable anymore, a further delay has been added across the board and brings the travel time consistently into the millisecond time range. Compared to the earlier Network Configurations, now with wireless networks, several outside variables have been introduced into the calculation. A major factor is that the connection between computer and router is not a constant. The connection will be weaker and stronger over the course of the experiment and will impact the one-way delay of a packet. The oddities noticed above are probably the result of varying connectivity of the wireless network. The variables impacting earlier measurements still apply but are harder to identify when the connectivity plays a much bigger factor into the final travel time. Wifi Public: For the final Network Configuration, the systems were connected using BTH s public student wireless network Eduroam. On this network, there are a lot of nodes to go through and different users competing for access at the same time. RakNet (Fig. 5.15) was clearly the fastest of all the protocols in both speed and consistency. TCP (Fig. 5.13) and TCP(N) (Fig. 5.14) had similar speeds for packets of the same size but the speed between different packet sizes varied with around 200.000 ns (0.02 ms). UDP (Fig. 5.16 ) which is the absolute slowest of the four protocols are similar to RakNet in that they both are very consistent in their one-way travel time. For UDP, the loss rate was only 0.000014% which is only seven packets lost for all packet sizes combined. Except for UDP, the overall travel times seem to have gone down which would mean a couple of things. Eduroam s connection is much more stable than the router used for private wifi, and despite having more traffic on the network, it still delivers the packets faster. This can be because of the processing power of the Eduroam system and be further helped if there were not a lot of traffic at the time of the experiment. This could also be the reason why UDP is so much slower than the other Protocols. The UDP part of the experiment might have been executed during a time of high traffic on the network. Authors note; The experiments were done on different days but mostly on the same time of the day to have as consistent traffic as possible. The experiments were performed between 10 am to 3 pm, which is the time where most students are active at BTH campus. This is to have as much network traffic as possible to differ the public wifi connection from the private connection. 36

6.2 Throughput Test Ethernet: RakNet (Fig. 5.19) as by far the highest throughput of all the implemented protocols even if the throughput varies from time to time. UDP (Fig. 5.20) has the second largest throughput and the size of the individual packet does not seem to impact the throughput in any meaningful way. For TCP (Fig. 5.17) and TCP(N) (Fig. 5.18) on the other hand, their throughput speed is greatly impacted by the size of the packets sent. It also seems like Nagle s algorithm can clearly be seen as having an impact on the consistency of the throughput. While TCP(N) has a relatively even throughput, TCP has a very uneven throughput. This is because the protocol waits for more data before sending a larger packet and that can be seen as the larger steps between the higher tops on the graph. Overall, for TCP and TCP(N) it seems to greatly gain throughput by increasing the packet size while UDP only gets a marginal increase. This is probably because of the way the packets are packeted and sent in UDP compared to TCP. The reason why RakNet is so much faster than the other protocols is probably because it is an actual commercial product that has been developed, optimized, and thoroughly tested by professionals while the implementations for TCP, TCP(N), and UDP are implemented by the author of this paper and is therefore not as thoroughly tested and optimized. Switch: No real change can been seen for the RakNet (Fig. 5.23) and UDP (Fig. 5.24) implementation from the earlier Ethernet test. It seems like TCP(N) s (Fig. 5.22) throughput varies much more than before. This can be because of the switch buffers getting filled up by all the data and communication packets TCP uses, and since more packets are sent since Nagle s algorithm is turned off, more traffic is created and create longer queues and possible packet loss. There is also an increase in throughput for regular TCP with the packet size of 10240 bytes(fig. 5.21) but no explanation was found to why this happens. Wifi Private: UDP (Fig. 5.28) has overtaken RakNet (Fig. 5.27) in throughput for certain packet sizes. When UDP send packets of 10240 and 30720 bytes, it is faster or as fast as RakNet, but when sending 61440 bytes packets, the packet loss rate increased to 240%. This means that for every packet that arrived at its destination 2,4 packets never arrived and had to be re-sent. It is unexpected that this first occurs when the packets get larger than 30720 bytes, but this might be caused by the router buffers getting filled up and then dropping incoming packets and since there is no back-off method in place for the UDP implementation in keeps sending packets that continue to get dropped. For TCP (Fig. 5.25) and TCP(N) (Fig. 5.26) there seems to be a similar problem. The throughput is increased from 10240 bytes packets to 30720 bytes, but when larger packets are sent, throughput drops. The reason that TCP(N) gets even lower throughput can already be explained by the extra packets created by not packeting smaller packets together and therefore creating more traffic to overflow the router s buffers. The spikes in the TCP and TCP(N) throughput look like the effect of the congestion window halving the throughput when packet loss is detected, and packet size seems to have a large impact on the packet loss. The 10240-byte packets have relatively small spikes compare to the 61440-byte packets. Smaller packets seem to have lower drop rate than the larger packets. 37

Wifi Public: For this part, data could not be collected at all for UDP since the drop rate was so high that the program never succeeds in transferring a whole Gigabyte of data to the destination. For the remaining protocols, the throughput dropped drastically but RakNet (Fig. 5.27) still has the highest. But for some reason, the sum of RakNets total throughput in this experiment have exceeded the 1GB of data that should have been sent. The total amount of data sent is close to 2 GB. The reason for this is unknown but it is most probable that there is another process running in the background that disturbed the measurement of data sent since there is no packet loss for 41943040 and 83886080 bytes segments and only a single packet for the 167772160 bytes segment. Packet size seems to have much lesser or no impact at all on the throughput for TCP (Fig. 5.25) and TCP(N) (Fig. 5.26). It is also hard to identify the impact of Nagle s algorithm and the congestion window seems to behave similarly for both TCP and TCP(N) which would imply similar packet loss. It is unknown why the drop rate for UDP got so high but it could probably be explained by two factors. It could be high network traffic at the time of the experiments. The author also suspects that BTH might have an anti-ddos system in place, and by sending a lot of UDP packets through the system might have triggered it and stopped further packets. 38

Chapter 7 Conclusion 7.1 Summary During this experiment, the protocols have been compared against each other in two categories in an effort to determine which one of them is best suited to what kind of a scenario. The categories are Single Packet Travel Time and Throughput. Ethernet A simple 1-1 connection with an Ethernet cable is a high bandwidth and low latency and can, therefore, be suitable for any of the tested protocols. That said, TCP and TCP(N) have the fastest one way trip of a packet while UDP has the highest Throughput. There are also some additional concerns to consider before choosing which protocol to use. Even though the TCP protocols have the fastest one-way travel time compared to UDP, TCP creates additional traffic in the form of ACK packet that is not really needed since there is no packet loss over LAN. In that way UDP is superior since it only sends the data given to it and nothing else, keeping the amount of data sent to a minimum. TCP(N) should also increase the effectiveness of sending small packages at a high frequency since it will not wait for other packages or a timeout before sending the packet. The downside with this is that it will create more overhead and network traffic because of the additional ACK packets used. Switch As for the Ethernet, TCP and TCP(N) has the fastest one-way travel time and UDP the highest throughput. The travel time of packets is still very fast and all the protocols are usable. With the introduction of the switch buffer, there is a greater emphasis on keeping the amount of packet on the network to a minimum to avoid filling up the switch buffer. In this experiment, there were no instances of packet loss for UDP, but there is the possibility that there could occur packet loss. For the same reason, TCP(N) is not as optimal as regular TCP, since the extra ACK packets created increases the risks for packet loss. Wifi Private With the introduction of wifi network connection, UDP has the fastest one-way travel time for a packet. The main reason for this is that there is no extra synchronization as for TCP, TCP(N), and RakNEt, but this may be a big problem if the user absolutely requires the specific data sent or the order of the packets matter. UDP is good for applications that update at a high frequency, such as First Person Shooters. If reliability is a concern, TCP, TCP(N), and RakNet are the way to go since they implement synchronized communication between the systems. This can be seen when moving on to the throughput. UDP has the highest throughput but still is 39