Evaluating TCP with Corruption Notification in an IEEE Wireless LAN

Transcription

1 Evaluating TCP with Corruption Notification in an IEEE Wireless LAN A diploma thesis submitted to the Institute of Computer Science, University of Innsbruck for the degree of Engineer in Computer Science presented by Mattia Rossi, BTech supervised by Dr. Michael Welzl November 2006

2

3 To my parents i

4 ii

5 In general, an implementation must be conservative in its sending behavior, and liberal in its receiving behavior. (Jon Postel) iii

6 iv

7 Abstract Reliable transport protocols like TCP perform very well on wired computer networks; packet loss is primarily caused by congestion and much less frequently due to physical or link layer problems. On the other hand, in wireless links TCP performance is degraded due to excessive packet loss on the physical layer, and congestion control mechanisms react even where no congestion exists. It has been suggested to enhance the performance of TCP over wireless links by adding a checksum that covers the packet header, enabling the TCPreceiver to detect errors in the payload and inform the sender of such occurrences. This way, excessive TCP window retransmissions could be avoided and congestion control mechanisms could adapt in a more appropriate way. Such mechanisms have been implemented and tested on wired or simulated wireless networks, which do never reflect real wireless environments, as errors are usually generated randomly and do not follow some real measured behavior. This thesis shows measurements of TCP with an additional header checksum on sample TCP connections over real IEEE wireless links and presents results which are quite different from the ones reported in simulation studies. v

8 vi

9 Contents Abstract Contents v vi 1 General introduction 1 2 Background Introduction The draft Related work Link layer approaches TCP improvements ELN in other protocols than TCP Concluding discussions Summary and conclusions Implementation Introduction The hostap driver The hostap driver source and frame reception process Changes to the hostap driver The implementation of the draft The implementation of the corruption detection option The implementation of the corruption notification option The changes to congestion control Summary and conclusions vii

10 viii CONTENTS 4 Measurements Introduction Equipment Software Hardware Preliminary measurements Indoor measurements Outdoor measurements Final measurements Indoor measurements Outdoor measurements Summary and conclusions Conclusions and outlook 81 A Commands to setup the prism card 85 Bibliography 86 Acknowledgments 93 Curriculum Vitae 95

11 Chapter 1 General introduction While on wired computer networks the probability of packets loss due to interferences and other physical problems from one node to another is very low, in wireless networks the physical transport reliability of such a link between two nodes is not given. Atmospheric and other radio interferences as well as reflections and attenuation of the radio waves are possible causes which can result in lost or corrupted packets. These problems, which will always exist with radio waves, lead to a high percentage of packet loss in wireless LAN links. The packet loss varies due to the quality of the radio link, which again varies with the distance to the transmitting station. Moreover the architecture of a building and the material used for construction massively affect the quality indoors due to the already mentioned reflection and attenuation problem. While on wired networks the physical reliability, which reflects the link quality, is static unless somebody is tampering with the cable, in wireless links the signal strength may vary all the time. Moving around with a wireless station like a notebook changes the link behavior considerably, and the mentioned atmospheric influences like rain or humidity as well as trees and other vegetation affect outdoor links too. Lastly mobile devices like cellphones and devices like microwaves produce interferences, as well as other WLAN stations which accidentally work on the same frequency. All these influences are not static and continuous, but of random time and duration. This constitutes a big problem for the TCP congestion control algorithms, which interpret packet loss as a congestion on the link and slow their transfer rates down. Algorithms like Reno and NewReno [1][2][3][4] significantly reduce the rate if there is a signal loss. Then the connection gets into slow start and may recover only slowly even if the signal strength 1

12 2 General introduction is high again. But in the meantime the signal could be lost again before the mechanisms reach congestion avoidance mode, where packets are sent at maximum rate. Observing this worst case scenario, it can be seen that the TCP connection may never reach its maximum troughput. The additional selective acknowledgments [5] should improve the performance of these congestion control problems allowing quick recover from multiple packet losses, but the described congestion control problems cannot be solved with it. There are many approaches to improve TCP over wireless links, as the original TCP protocol [6] (developed in the early 80 s) was designed mainly for low bandwith wired connections. The ideas range from changing TCP congestion control mechanisms as in the case of TCP Westwood [7], splitting the TCP connections between wired and wireless links as in indirect TCP [8] or caching the connection as in the Berkeley SNOOP protocol [9], to link layer enhancements which use forward error correction (FEC) and automatic repeat requests (ARQ), concluding with explicit loss notification based approaches (ELN). Explicit loss notification is the idea to let the link layer inform upper layers about packet loss or corrupted data, such that upper layer protocols may be able to take measures to better adapt to the situations. The herein discussed internet draft [10] is such an ELN based approach, which adds a special pseudo header checksum to the TCP packet. In this way corrupted packets passed from the lower layers can be detected and the sender can distinguish between packet loss due to congestion and packet loss due to wireless link errors. The idea of adding such an extra checksum to the TCP header is not new, a similar approach was already proposed by the name of TCPHACK [11] - but while the measurements of TCPHACK were only made in a wired network where packet loss was simulated, this thesis aims to conduct measurements in real IEEE b/g wireless LANs to eventually show benefits of the draft. Usually one problem of simulated wireless environments is the use of too few variables, because using all possible variables which refelect real wireless environments and affect a wireless link, would render a simulation exceedingly complex. So the outcomes of simulations are still theoretical and never practical, while this thesis tries to show a practical application of an improvement to the TCP protocol. The point that the many explicit loss notification based ideas have not been tried in real wireless networks made it absolutely necessary for us to take this next step. It made it also necessary to use the most advanced technologies possible, thus not using reduced settings in wireless media which do not reflect a typical industrial wireless setup just to provide a possible better outcome.

13 3 The ELN approach states that lower layers have to pass information of packet loss to upper layers. As in the approach of this thesis corrupted packets are needed to be informed about that loss, also special hardware is needed which doesn t discard such packets at link layer level. There are some papers and technical reports which discuss the appropriate hardware for this type of measurements within an IEEE link layer technology and the appropriate driver used, which is a prism2/2.5/3 chipset based wireless LAN card, and its hostap 1 [12] or the wlan-ng driver 2 [13] for linux. As with linux kernel the hostap driver was included in the kernel source tree, it seemed logical to use the hostap driver instead of the older and no longer maintained wlan-ng driver. On his webpage Jean Tourillhes suggested to use the more current hostap driver too, which uses some code from the wlan-ng driver and is able to interact with the linux wireless tools 3 which is the standard toolkit in linux to control based WLAN cards. In IEEE link layer technology, the improvements for data transfer on lossy links which rely on corrupted packets conflict with security mechanisms of the link layer. In the IEEE wireless LAN standard security mechanisms have been taken into consideration. A user has the possibility to encrypt every single packet which is sent over the air. The encryption is optional, but if it is used, the packet will be encrypted at the link layer before being checksummed, which results in a completely unusable packet if the packet has been corrupted on its way to the receiver. Decrypting a corrupted packet would result in a random bit distribution within the packet, thus into complete nonsense. So it is obvious that the idea of the draft will only work on link layers which do not use any encryption. IEEE is not the only wireless link layer technology available. The outcome of the tests in this thesis only consider WLANs; for other technologies the results may be completely different. Still, the results of the thesis may give general indications about the reality on wireless links and corruption notification. They will not only be useable for proving the idea of the draft, but also for deployers of other protocols which tend to use corrupted packets for further processing. 1 The hostap page refers no longer to the hostap driver itself, but is dedicated mainly to the WPA supplicant and hostapd tools 2 The linux wireless tools homepage formerly hosted the linux-wlan-ng driver too, which has now been removed due to the incorporation of the hostap driver into the kernel source tree. 3 This information has also been removed from the site.

14 4 General introduction This thesis is structured in three main parts: (1) The draft and related work are explained as well as the discussions about pros and cons, to give a detailed overview on what the thesis is about. (2) The implementation of the draft is described with all the changes to the linux TCP stack and the hostap driver. (3) The preliminary measurements and the final measurements are given, which show the results of the implementations in a real wireless environment. The next chapter (chapter 2) gives a detailed description of the basis of this thesis, describing the Internet draft, and all the related work and discussions that came up with it.

15 Chapter 2 Background 2.1 Introduction This chapter gives a detailed explanation of the corruption notification draft [10] (further simply called the draft), and its purposes. As similar ideas were already proposed in the past huge part of this chapter is covered by the related work (section 2.3,) in which an overview and some explanations why the argument is so prevailing will be given. So much work leads obviously to many questions about the possible advantages and disadvantages, which are shown and explained in section The draft As already described in chapter 1 the draft is an improvement of TCP which is based on the explicit loss notification (ELN) idea. The explicit loss notification approach is to use corrupted packets in wireless environments as indication of non-congestion-related packet loss within a link, as a truly lost packet itself is not detectable. There are two players in the game, the link layer and the transport layer with the TCP protocol. The draft itself does not specify the technology of the link layer, except that it has to be a wireless technology. For the measurements in this thesis the IEEE WLAN technology was chosen, but any other wireless link layer technology could theoretically be used. The important claim to link layer technologies is the possibility to accept corrupted packets and to be able to pass them to upper layers, as well as the possibility to control link layer automatic repeat requests (ARQ). The link layer ARQ is the retransmission of packets if no link layer ACK for a 5

16 6 Background successful reception of a packet is sent to the source, and times out after a certain number of retries. It is important to be able to switch off link layer ARQ because, if a corrupted packet arrives, no link layer ACK would be sent and thus the packet would be retransmitted, until the packet arrives without corruption or the retransmission limit is achieved, which then could result in a corrupted, but also in a lost packet. In the last case it would be possible to lose the possibility of an ELN. In section 2.4 the link layer ARQ problem discussion is explained further. The additions chosen for TCP are two options which are used to notify the sender of packet loss at the receivers side. The first option, the corruption detection option (CDO) is a rather big option, as it carries a 4 byte CRC32c checksum and needs the obligatory 2 bytes for the option type and the option length (see fig. 2.1). The type number of the CDO is 27 and the total length is 6 bytes. Figure 2.1: The corruption detection option as specified by the draft The CRC32c checksum by Castagnoli et al. [14], which is the same as used in the SCTP protocol [15][16], covers a subset of the TCP header and the pseudo header used by the regular TCP checksum: the source and destination address of the IP header, the source and destination port, the sequence and acknowledgement number, the CWR, ECE, ACK, RST, SYN and FIN control bits and the ECN field [17]. This additional checksum gives the receiver the possibility to accept corrupted packets. Figure 2.2 shows the pseudo header. If only one TCP connection was available, this header checksum would not be necessary, as it would only be important to know that the protocol used is TCP to send a corruption notification. But simultaneous connections need additional information like the port number to be able to send the corruption notification to the correct TCP connection, which is stored in the fields covered by the mentioned header checksum. The CRC32c checksum was chosen instead of shorter checksums as used for regular TCP checksumming or Adler-32, because discussions in the IETF caused some motion in that direction, as shown by decisions made in [16] and [18]. In the draft the idea to use a checksum algorithm able to correct bit errors is mentioned, as the amount of data covered is only small, and thus such an algorithm could be computed fast enough. This idea is not further investigated and might be future work, but it is questionable, because in [19] it is

17 2.2 The draft 7 Figure 2.2: The pseudo header covered by the corruption detection option checksum stated that errors do not occur bitwise but in bursts of byte length, which such an algorithm might also be unable to recover (explained in section 2.3). The special pseudo header checksum has to be computed before the normal TCP checksum is calculated, which covers the whole TCP packet with header and payload including the corruption detection option. The corruption detection option could be included in every TCP frame which is sent over the network, although it is useful only when used on frames within an established connection, as this is the part where congestion control comes into operation. Within an established TCP connection there are only two types of packets: ACKs and data packets. In the draft it is also stated, that it is probably not that useful to add the corruption detection option to plain acknowledgement frames, as those frames do not carry any data. If such a packet is marked as corrupted, the error may affect with a high probability the TCP, the IP and the link layer header if any. It is quite improbable that the error affects only the IP and/or link layer header. If this happens, then the ACK part is not corrupted and can be used like a normal ACK, but the main importance of the mechanism lies in adding the CDO to data packets. The implementation in chapter 3 also shows problems on using the CDO on ACKs due to the use of SACK and the timestamp option [20]. Obviously the CDO leads to processing and header size overhead, therefore the option should not be used in wired networks, but on lossy links only. As the option works only in the presence of corrupted packets which are delivered to the TCP protocol, it is necessary that the link shows this capabilities. This could be achieved by letting the sender know that the receiver is behind such a lossy link and therefore has this capability. So the

18 8 Background Figure 2.3: The small corruption notification option specified in the draft sender may enable the options explicitly for that connection. At the receiver side the corruption detection option is ignored unless the regular TCP checksum of a packet carrying this option fails. In that case the header checksum is verified, and if that fails too, the packet is discarded. Otherwise the packet is a corrupted packet which indicates packet loss on the link used by the TCP connection. The information about the connection can be extracted from the intact parts of the TCP header, such that a correct corruption notification can be sent. The fields which are not covered by the checksum have to be ignored, as they could contain corrupted information. This would mainly affect other TCP options such as the timestamp option [20]. The corruption notification option (CNO) is a 2 byte option, which is the minimal length for a TCP option. It contains just the option type (type 28) and the length as seen in figure 2.3. This option is sent by the receiver of a corrupted segment to inform the sender about packet loss due to a noisy link. The corruption notification option is designed to be as small as possible, and as an alternative it could also be put into the reserved field of the TCP header. It is enough for the receiver of the ACK to know that the packet belonging to that uniquely identifiable ACK was lost, such that a single bit could suffice. Additionally as explained above, the use of the corruption detection option is possible also on ACKs, and if an ACK has to carry both, the CDO and the CNO, the header size overhead would increase unnecessarily. Similar considerations include the idea of using a option type like SACK [5] and carrying additional information about the sequence numbers of corrupted packets. The advantage is that lost ACKs which were carrying a CNO could be compensated through this, but the packets would be bigger. The benefits of such an extended option may not be worth the overhead. Thus,this variant is not specified in the draft, which simply states that an ACK may carry both options at the same time. The receiver of an ACK segment carrying a corruption notification option has to carry out two operations, as stated in the draft: to recalculate the round trip time (RTT) and the retransmission timeout (RTO), and to retransmit the corrupted data segment. The

19 2.2 The draft 9 standards for the RTO and RTT calculations are given in [21]. These two operations should already be an improvement for the connection over a lossy link. If the retransmitted segment arrives at the receiver side in time, congestion control reaction might be circumvented, as the timers were updated, and the ACK lets the sending process continue its normal operation. Problems arise if the retransmitted packet gets completely lost without generating a new ACK with a CNO, this means the retransmitted packet could not be recovered by the header checksum or did not arrive at the receiver side. Then the simple retransmission is not enough, and the congestion control mechanism should be able to use the information obtained before by the corruption notification, e.g. by not reducing the congestion window as long as the retransmitted packet is not acknowledged. This congestion control response is not specified in the draft; there are ongoing discussions on how congestion control should react on corruption and congestion on wireless links, e.g on the DCCP mailing list. The main idea is still to react less conservatively upon packet loss in the presence of corruption notifications than in the presence of congestion; this means that upon reception of a corruption notification, halving the window size is an unnecessary measure. On the other hand the idea to keep on sending without restrictions on lossy links when packet losses arise, does not seem to be a good choice as stated in a note by Sally Floyd on the DCCP mailing list. The update of the RTT and hence the RTO is already considered as a step in direction of better reaction to a corruption notification for congestion control mechanisms like NewReno. The biggest issue with the corruption notifications in TCP is the interaction between this mechanism and the security mechanisms of the lower layers. As already described in chapter 1 the link layer should not use any encryption which encrypts the complete segment or at least the sensitive part of the packet, which are the fields covered by the special pseudo header specified in the draft, may not be encrypted. Decryption on corrupted packets would destroy any useful information contained in this sensitive part. The same restrictions apply to encryption at the network layer. As IPSec encrypts at least the whole packets IP payload (which contains the TCP header) if the Encapsulation Security Protocol (ESP) with encryption is used, any decryption will result into a corrupted TCP header if any part of the encrypted IP payload has changed. If the IP payload is not encrypted, but only authentication with the authentication header (AH) is used, the authentication of a corrupted packet will always fail, as the data has changed. These restrictions limit the application of the TCP corruption notification to unencrypted and unsecured lower layers. If data security is still desired, it may be switched on at the application layer with TLS/SSL [22][23].

20 10 Background 2.3 Related work A huge amount of research has been done in the field of improving data transfer over wireless links. The approaches are different, and not all are TCP related. There are ideas to improve the behavior of the link layer, of the network layer, and of the transport layer. These ideas vary from changes to common protocols to the development of new protocols. Some of them also use corrupted packets for improvements of the behavior over wireless links Link layer approaches In their technical report [19] Winstein and Riemann have shown that it is possible to recover broken wireless frames with forward error correction. The idea is based on link layer ARQ. Usually link layers try to retransmit packets until they aren t received correctly. The IEEE WLAN technology for example uses a checksum over the entire segment to verify its integrity. This is necessary, as the standard actually gives the possibility to encrypt the packets for improved security (see section 2.2). So if an encrypted packet is received with errors, the packet has to be discarded, as the encryption is used on the whole segment, and a decrypted corrupted segment would be unusable. In this technical report, corrupted wireless LAN packets were received by using the same prism based WLAN cards as for this thesis and also the same driver. The aim of the work was to extend the range of an outdoor wireless link, by using forward error correction at the link layer to recover corrupted packets which appear when the signal strength is weak. The protocol used in the tests was the ICMP (ping) protocol, so no huge data transfer and higher protocols were tested, as it was not needed for the purpose. The tests were made by using Reed-Solomon error correction which also uses some extra bytes of overhead to be able to correct byte errors. The chosen code was RS(255,191) which resulted in 191 message bytes and 64 redundancy bytes for packets with 255 bytes. The authors made some preliminary tests with one way measurements outdoors on the top of two buildings, where they were already able to improve the distance of 70% and to decrease the packet loss ratio by 75%. This means that the loss rate of packets fell from 68% to 17%, which seems to be a good value, but following the statements in the work, this loss ratio is still too high for TCP to be able to work properly. Therefore the authors strongly encourage to use TCP improvements in addition to the use of link layer FEC. The work of Winstein and Riemann shows that FEC is a good approach to improve the link layer quality, but also that it is not enough. This work additionally shows the restric-

21 2.3 Related work 11 tions which come along with measurements in real wireless test setups. For instance prism cards, the only WLAN cards able to pass corrupted frames to the host, can do that only if connected in ad-hoc mode. If the cards are in managed mode, they will disconnect automatically after the reception of the first corrupted frames. The other restriction is the operation of the cards at 1 Mbit/s. This is not the typical setup of an industrial environment, where the cards (type b) are usually operated at full speed with 11 Mbit/s. According to the authors, the operation at 11 Mbit/s causes long bursts of lost packets instead of bit errors in packets (measurements shown in [24]). As this would slow down a data transfer more than the corruption notification would ever improve, this operation mode is not applicable for the measurements in this thesis. The combination of the link layer FEC and the TCP corruption notification is questionable, as FEC would already correct most of the corrupted packets, and therefore few could be used for the notification at transport layer. In this way it can be said, that FEC does not reduce real packet loss, but it reduces the amount of corrupted packets TCP improvements The most similar approaches to the one specified in the draft are based on explicit loss notification (ELN) like the TCP header checksum (TCPHACK) approach. There are also ideas to split the TCP connection when it encounters a lossy link and additions to or complete changes of the congestion control algorithms. Changes to congestion control TCPWestwood [7] (short TCPW) is a congestion control algorithm that is thought to perform very well on wireless links. The algorithm exploits two basic concepts: the end-to-end estimation of the available bandwidth, and the use of this estimation to set the slow start threshold and the congestion window, which comes into operation after three duplicate ACKs or an RTO timeout. The duplicate ACKS (short dupack) are the acks which are sent after a new received segment which is out of order. An out of order segment can not be acknowledged correctly; the sent ACK still acknowledges only the last in-order segment. This results into a duplicate ACK at the senders side and is considerated as a strong indicator of congestion. While the Reno based algorithms simply halve the congestion window after three dupacks, TCPWestwood tries to select a slow start threshold and a congestion window that is proportional to the connection rate at the time of congestion. This means

22 12 Background that the response to congestion is adapted at the variation of the transfer rate. This is possibly a good approach even for lossy links, as it could be able to recover faster and not reduce the rate so drastically upon sudden packet loss; a typical behavior of wireless links. The problem of TCPW is that it needs an initial learning time, and does still reduce the rate upon longer packet losses which do not belong to congestion. This means that it is not able to distinguish between packet loss due to link errors and packet loss due to congestion, which limits the performance improvement. The selective acknowledgement options [5] are another try to improve the TCP performance using the standard congestion response of Reno or NewReno. These TCP options carry the information about multiple lost TCP packets from a single window to the sender, such that on a packet retransmission, only the missing packets may be sent instead of a whole window with multiple packets. On the reception of a duplicate ACK, if this ACK carries the SACK option, the ACK also acknowledges the packet to which it belongs. So multiple packet retransmission is avoided, and TCP may recover faster from loss. This approach improves the transfer rate by reducing retransmissions and avoiding excessive reduction of the sending rate. Although this standard already found its way into the TCP stack of the main operating systems because it is considered a major improvement for every kind of network, it does not perform much better on lossy links. Like TCPW it is not able to tell the difference between a lossy and a congested link. As SACKs are also used in the linux kernel s TCP stack, the implementation of the draft has to deal with them, which led to some restrictions regarding the use of the corruption detection option in ACKs (see chapter 3). Split connections Two well known protocols implement the idea of splitting the TCP connection if it passes over a lossy link, such that the part of the connection over the wireless link may react differently upon packet loss than the part on a wired link. The indirect TCP [8] splits the connection completely, in a manner that the connection between the server and the access point (taken as more current example) has its own TCP connection and its own congestion control, which hides the real connection between the AP (access point) and the mobile host. This means that whatever connection the mobile host uses to connect to the AP, the AP will use TCP to connect to the server. So from the server side it looks like a TCP connection to the mobile host, which is not true. This approach is not strictly an improvement to TCP as it cuts off the connection, and the idea permits to use a completely different protocol

23 2.3 Related work 13 than TCP over the lossy link, and at least intends to use a different congestion control mechanism over that link. This protocol was tested by using TCP and a different protocol on the wireless link, which resulted in no significant difference [25]. Therefore the use of a split connection is questionable, and it seems that it is better to use a complete TCP connection and apply special adaptions when it encounters a lossy link. Additionally with indirect TCP it is possible that there are a large number of cached packets at the AP, as the wired connection could be much faster than the wireless, if that link has excessive packet loss. The packets on the wired network may be acknowledged faster than the same packets on the lossy link, and therefore the AP risks to run out of memory. This shows that the application of such protocols is quite limited and not scalable. A similar approach as I-TCP is the so-called snoop protocol [9]. The snoop protocol introduces a module, the snoop agent, which is installed on a lossy link. This agent monitors the packets passing trough a TCP connection and caches all packets that have not yet been acknowledged. If dupacks arrive at the snoop agent (signaling packet loss), these dupacks are suppressed and the packet is retransmitted. This hides the non-congestion-related packet loss from the sender, which may focus completely on congestion and is not confused by noise related losses. This approach does not actually split the connection, but only hides a certain part. This protocol is situated on the link layer, and is therefore a link layer protocol which is aware of higher transport layers, namely the TCP protocol. This is exactly the opposite of the idea of the draft, which tries to let the transport layer become aware of link layer problems. The snoop protocol behaves well on connections which pass over lossy links, but the implementation and the installation effort are high, as it is actually a link layer technology. Improvements on the transport layer may be preferable. ELN related improvements Ideas for wireless TCP improvements on the transport layer are often based on the explicit loss notification (ELN). ELN is a mechanism by which the reason for the loss of a packet can be communicated to the TCP sender. In particular, it provides a way by which senders can be informed that a loss happened because of reasons unrelated to network congestion, so that sender retransmissions can be decoupled from congestion control. There are also proposals for improvements of TCP with notifications from the link layer. The loss notification in these cases is always intended as the reception of corrupted packets. Such packets are lost for a TCP connection, as the TCP checksum would fail and thus the packet be discarded.

24 14 Background Ziegler et al. [26] tried to improve the TCP connection by counting the corrupted packets at the receiver side and by sending this number to the sender with the ACKS, which then knows whether the loss was due to congestion or due to link problems. The mechanism proposed in this paper only reacts on duplicate ACKs, because a retransmission timeout is considerated a strong indicator for network congestion and therefore the procedure invoked by it is not changed. The standard procedure for TCP with NewReno congestion control is to send out segments depending on the size of the congestion window, and to await the acknowledgments. The ACKs that arrive acknowledge either a new segment, leading to an increment of the window size depending on the state of the sender (Slow Start or Congestion Avoidance), resetting the dupack counter and transmitting new segments, or a previously acknowledged segment, and thus are counted as a dupack. If a normal ACK is received, obviously the procedure is not changed by the authors. The changes are applied upon reception of a dupack: the sender then has to check whether there is any corruption notification sent with this dupack. The corruption notification, as mentioned before, is the counter of corrupted packets at the receiver side. The sender can then compare the received counter with the counter memorized at the last corruption notification retransmit. The difference between the counters tells the sender about the amount of lost packets on the radio link. If there is no difference, the packet loss happened due to congestion instead. If that happens, the sender proceeds as usual and triggers fast retransmit/fast recovery on the third dupack. Obviously all three dupacks are monitored, as it could happen that not every lost packet is a corrupted one, and therefore the correct loss notification could arrive only with the third dupack. If loss was experienced on the wireless link, the procedure is another: first the lost packets are retransmitted (it might be that the corruption counter signals more than one lost packet), then the number of the packets in flight (the packets already sent but not yet acknowledged) and the size of the congestion window are stored. As the TCP source will receive several dupacks before the acknowledgments for the retransmitted packets arrive, the source has no information about network congestion. The stored number of packets in flight is used to allow the same number of dupacks at the sender side, before it triggers the normal congestion control mechanisms. If more dupacks than the stored number of packets in flight are received, the sender may not be able to distinguish between congestion and packet loss due to link problems anymore, thus for safety reasons the normal congestion control procedure is initiated. Every received dupack increases the window size by one

25 2.3 Related work 15 segment in order to allow the sender to send new packets without having to wait only for the recovery ACK. This does not increase the transfer rate, as dupacks do not acknowledge new segments, but keeps the transfer rate constant. When the recovery ACK arrives at the sender, the sender then has to check whether congestion occurred in the meantime for the sent packets or not. This has to be done with a conservative assumption, as the dupacks received while waiting for the recovery ACK can not inform about that. The assumption is that if a partial recovery ACK (an ACK which does not acknowledge all of the lost segments but only a part) arrives, then there could be congestion and the fast recovery/fast retransmit procedure has to be invoked. If instead the recovery ACK is complete and acknowledges all the lost segments, the congestion window is reset to the stored state, and the connection operates as usual. In this way the halving of the window size is bypassed, as the packet loss could appropriately assigned to link problems. The authors of this TCP-ELN have tested the implementation on a simulated network with good results. The idea differs from the one specified in the draft in that it includes detailed information on the reaction of congestion control, which is not specified in the draft. The idea to stop the congestion window reduction while there is a corruption notification was also taken into consideration for the implementation in this thesis. It was also tried to avoid the congestion window reduction as long as dupacks arrive due to radio link problems (see chapter 3). In a second paper [27] Ziegler et al. focus on performance and on protocol analysis of TCP-ELN. As already mentioned, in simulated environments the idea performs very well; as stated in the analysis it is up to 400% better than normal TCP. In the analysis the security of the protocol was also considered, with the outcome that it is easily possible to send false corruption notifications to the sender in order to keep its transfer rate up despite congestion, which would result in unfairness to other participants. It also takes the possibility of fragmented packets into consideration, both at the MAC and the network layer. The general outcome of this analysis is that in every case, all IP packets have to be collected and reassembled for the possibility to extract the TCP information to generate a corruption notification event. The security and fragmentation aspects both have to be taken in consideration also for the draft, as these are possible issues in typical networks. However both considerations have no weight for the outcome of this thesis, as there packet fragmentation did not occur in the reported measurement studies and the security aspects were not part of the tests. The TCP-ELN approach does not specify the use of simultaneous TCP connections which may be the result of using different applications at the client or of using different

26 16 Background Figure 2.4: The header checksum option of TCPHACK services at the same time. It also does not specify how the information could safely be extracted from a corrupted TCP packet in order to generate a reliable corruption notification. This safety is provided by an idea which was published approximately a year before the draft: a TCP header checksum called TCPHACK [11]. The main idea of TCPHACK is the same as in the draft: to use a checksum over the TCP header only, which may be checked on corrupted packets to be able to safely extract TCP sensitive information and to generate a correct corruption notification for the sender. Like the draft, TCPHACK uses two TCP options to provide a checksum and corruption notification. The first option (option type 14) is used as an enabling option and carries a checksum. Sent with a SYN packet, the option signals to the other party that there exists the possibility of using that option (the value of the option is ignored), whereas when sent with data packets within an established connection, the option includes the header checksum for this packet. Other than in the draft, TCPHACK uses the same checksum as TCP uses to checksum the whole packet, namely a 16 bit checksum, whose 1 s complement is stored as the option value. This option is needed as in the draft to determine a corrupted packet, but other than in the draft the whole option is shorter by 2 bytes, thus has a length of 4 bytes (depicted in fig. 2.4). This little detail makes a huge difference in the area of usage of the options. Unlike in the implementation of the draft, with the shorter checksum it is possible to use the option also on ACKs together with the timestamp option and SACK (see chapter 3). If the regular TCP checksum fails, the header checksum is tested, and if it is okay, the packet may be classified as corrupted. If both checksums fail, the only thing that can be done is to discard packets, as it could be possible that it is a misrouted packet, which must also not be part of the TCP connection. Without the confirmation of the correctness of the sensitive parts of the TCP packet, it would also not be possible to send a corruption notification to the correct TCP source. The corruption notification is sent to the source via a special ACK containing the corruption notification option (option type 15) which, other than in the draft, is not only a flag which notifies of the corruption, but a 6 byte option, carrying the 32 bit sequence number of the corrupted segment in its 4 byte value field as

27 2.3 Related work 17 Figure 2.5: The corruption notification option of TCPHACK shown in figure 2.5. This sequence number permits the selective retransmission of exactly the corrupted segment. The retransmission of the segment is done within the present congestion window, which is not changed upon reception of a corruption notification. Following the statement of the authors the congestion window would not be reduced even if many of the special ACKs arrive, as the ACKs would not be counted as dupacks. Like the TCP-ELN idea TCPHACK was implemented and tested in a simulated environment with random and burst errors, whose generation however is not based on real measurements. The burst errors are described as burst corruptions, which lets guess that every packet could be at least received, but in a corrupted state. The random error distribution in the packet then might result in a broken TCP header or not, which means that there are no real bursts of packet loss, but only grouped corrupted packets. This is a behavior which absolutely does not reflect a real wireless link behavior, as will be shown in chapter 4. In these simulated environments TCP-ELN and TCPHACK perform well and therefore also the idea of the draft should work well. As we will see, the real environment shows other results ELN in other protocols than TCP The ELN approach, or more precisely the corruption notification has already been taken in consideration by two other protocols: DCCP [18] and SCTP [15]. The datagram congestion control protocol (DCCP) is an unreliable datagram protocol like UDP which includes congestion control. The problem which caused this protocol to emerge is related to the huge amount of UDP traffic in the Internet due to online gaming, streaming media, and voice over IP (VoIP). Such traffic is sometimes absolutely unfair compared to TCP traffic, because it lacks congestion control. A UDP sender usually just sends its packets without any restrictions, with a high probability to congest a small link, but without reacting to that problem. The following datagrams just get lost and congestion is never dissolved. A TCP connection which traverses the same link is in trouble, because its congestion control mechanisms come into operation and reduce the sending rate, and it isn t able to recover fast enough when

28 18 Background the congestion is gone, as the UDP datagrams may congest the link again. This may let a TCP congestion succumb completely. The behavior of TCP in that case is similar to its behavior on a very lossy link. The problem should be avoided by adding congestion control to UDP flows, enabling them to avoid congesting a network, and therefore to behave in a more fair way. There are two congestion control mechanisms defined for DCCP for now, TCP-like congestion control [28] and TCP-friendly rate control (TFRC) [29]. The design of the DCCP protocol is rather similar to TCP with its three way handshake for setting up a connection and its congestion control mechanisms, although it is a datagram protocol which does not realize reliable transport. The use of congestion control however confronts the DCCP protocol with the same problems as TCP: how to adapt the sending rate in the presence of packet loss which is not congestion related, or rather how to distinguish the causes of that loss. Even if DCCP resembles the TCP protocol, there are certain aspects where the two protocols differ relevantly. This includes the possibility to add more options, the use of different acknowledgement types within the two congestion control mechanisms, and denial of service (DoS) protection in DCCP. The half-closed states and simultaneous open as in TCP are eliminated. As DCCP is a datagram protocol it has no concept of a window and already acknowledges data when the header of the packet has been checked, while in TCP the complete data already has to be queued for the application. But the most interesting difference for this thesis is DCCP s built in ability to distinguish between different kinds of packet loss. The data dropped option lets an endpoint declare that a packet was dropped because of corruption, because of receive buffer overflow, or due to some other reasons. This means that the DCCP protocol already opted for the use of ELN. However DCCP also does not yet specify how to react on all drop codes which are sent to the source. The defined drop codes are code 0, code 1 and code 2. The 0 drop code is intended to signal no drop at all. The congestion control has to react as if the ACK would be a regular one. Drop code 1 signals that the packet is dropped because the connection was cut off by the application. This means that the application which was listening for that packet stream has received enough packets and is not listening any further. The reaction of congestion control should be to stop sending data. Drop code 2 signals a full buffer at the receiver side. It is a sign of congestion (albeit not inside the network), but the sender may slow the sending rate not as severely as it would in other cases. The exact behavior depends on the congestion control algorithm. The drop codes where the congestion control response is not specified

29 2.3 Related work 19 are the codes from 3 7. Drop codes 4 6 are reserved, drop code 3 and 7 are specified as corrupt and delivered corrupt respectively. For now the guideline to react on that drop codes is to implement the same reaction as for an ECN notification. The congestion control response to a corruption notification is also left open in the draft. The specifications may be written after ample experiments like in TCP-ELN or this thesis. The detection of corruption itself is done as specified in he draft and in TCPHACK by checking the header checksum. The header checksum is weaker than the one described in the draft, it is the same 16 bit checksum as in TCPHACK and TCP. A packet with a bad complete checksum but with a good header checksum is classified as corrupted, while a bad header checksum lets the packet be discarded. The second protocol which opted for the use of corruption notifications is the SCTP protocol with the enhancement of packet drop reporting [30]. The SCTP protocol is a transport protocol which aims to replace TCP, as it is a reliable transport protocol with improved security aspects and other enhancements. The main focus of SCTP is similar to TCP, to assure that all packets sent are also received, and to avoid congestion, but it also supports features like multiple streams within the same connection, message bundling and multihoming. It is also secured against flooding and masquerading which are techniques used by denial of service and distributed denial of service attacks. The interesting part of the SCTP protocol in this case however, is as in the DCCP protocol, the congestion control mechanism. The congestion control to be used by SCTP is defined on the basis of the Reno congestion control, but it is left free to any implementation to use also other algorithms. SCTP additionally uses so called gap ack blocks, which may be compared to the TCP SACKs. As SCTP is able to have multiple data streams within a connection (a so called association ) of two endpoints, the congestion control in SCTP is applied to the whole association, and not to single streams. This makes sense, as all the streams traverse the same links and would experience the same congestion. To be able to understand SCTP, it has to be explained that, other than TCP which uses one fixed 20 byte header plus possible options, the SCTP packet has a common header followed by multiple chunks which may contain control information or data, with every chunk having its own special header, which resembles the TCP option format. The chunks have a chunk type, chunk flags, and a chunk length, followed by the chunk value. This mode of creating a packet already shows a much higher flexibility than TCP, also from the number of chunks to be used, which is only limited by the connections maximum transfer unit, while

30 20 Background the number of options in TCP is limited by the size of the offset field, which could easily be exceeded as shown in chapter 3. The chunks in SCTP are handled like the options in TCP: if one of the participants does not understand the chunk type, it must ignore it. So are the aforementioned gap ack blocks part of the SCTP SACK chunks, but they have the same meaning as the whole TCP SACKs. SCTP SACKs acknowledge single data chunks even if they are delivered out of order, but data is only considered as fully acknowledged if an SCTP SACK with a cumulative transmission sequence number (cumulative TSN) set acknowledges a data chunk. This resembles the TCP congestion window, but other than in TCP (without SACKs), where the cwnd value controls the upper bound between the highest acknowledged sequence number and the latest DATA chunk that can be sent within the congestion window, in SCTP this value controls the amount of outstanding data. This leads to different implementations of fast-retransmit/fast-recovery. The biggest difference between TCP and SCTP is probably multi-homing, which is the possibility to reach the same receiver at more than one transport address. It may be possible that different addresses also lead to different paths to the same endpoint, and therefore a data stream may encounter different congestion or link problems, thus may need different congestion control parameters for each path. In SCTP, a new treatment of congestion control for multihomed receivers is therefore specified. This multihoming problem does not affect the calculation of the congestion window, but it may affect the receiver window. Even if SCTP needs to apply some changes to the congestion control mechanisms due to the different protocol design and the additional possibilities like multihoming, the main algorithm is still the same as in TCP. Therefore the problems with SCTP are the same as with TCP: how to make a difference between real congestion and packet loss due to a lossy link? The answer is given in an enhancement to SCTP which uses the ELN idea [30]. The idea follows the DCCP drop code technique by introducing a new chunk type to SCTP which is able to inform the sender of the specific type of packet loss. The introduction of new chunks in SCTP is as easy as introducing new options in TCP, as the chunk my be ignored if not understood. If instead both participants know about the 0x81 packet drop chunk, the information within that chunk may be used to react properly to that certain type of packet loss. The packet drop chunk contains more than just the information about the dropped packet type it also contains the corrupted or dropped packet itself, which may be truncated to fit into the new SCTP packet. The important fields of the drop packet chunk are the flag field, which triggers various properties

31 2.3 Related work 21 of the chunk information, the link bandwidth or maximum rwnd field, which gives information about the bandwith or the receive window at the other end, the send data on queue field, which informs the sender about the data queue length at the receivers side, that is all the delivered but not yet TSN-acknowledged data, and finally the data field which contains either the whole or the truncated corrupted SCTP packet. It can be seen that the information sent back by such an SCTP chunk is much more than what is sent back by a small TCP ACK with options. Also, the information needed to get a working SCTP association is nothing more than the source and the destination port, as well as the verification tag, which is a sort of checksum that identifies the sender and the receiver for security reasons (e.g. SYN flood protection), contained in the common header. Therefore the identification of a corrupted packet is different from the draft, TCPHACK or DCCP. In SCTP the CRC32c checksum covers the whole SCTP packet with common header and following chunks. If this checksum fails, the common header is examined. The information of the source and destination port and the verification tag allow SCTP to find out the correct association. If this test fails, the packet is dropped, because there would be no possibility to send back a drop packet chunk to the correct source. If the correct association could instead be determined, every information necessary for the drop packet chunk could be extracted by the associations control block (TCB) [31] at the receiver side, without any more information needed by the corrupted packet itself. It could be said that the verification tag of the common header resembles the header checksum. The high configurability of SCTP packets using chunks makes the use of such corruption notification messages more interesting, as the information which could be passed to the source is much higher. This also leads to higher flexibility on adapting congestion control mechanisms. The rwnd which may be sent with the correct flags set can inform the sender to adapt it immediately, the bandwith value may lead to more correct adaptations of the cwnd, and the original packet sent back may let the sender selectively retransmit the dropped one. Obviously there are lots of security aspects taken in consideration for SCTP, which however is beyond the scope of this thesis. There is no detailed specification for congestion control behavior in the drop packet chunk draft, except that the congestion window must not be changed upon the reception of such a drop packet chunk, if the RTT is or has ever been measured to be less than or equal to the RTO value. The SCTP protocol specifies many new things which result in a very complex specification also of the drop packet chunks.

32 22 Background The DCCP and SCTP protocols show a high interest on the use of corruption notification for the possibility to detect different packet loss causes, a need which raised due to the always more important wireless link layer technologies. Yet the benefit of such ELN approaches has still not been shown in real wireless networks, and therefore it is not even sure if the corruption notifications are that useful in the end. This thesis helps to clarify this situation. 2.4 Concluding discussions There have been lots of discussions about the usefulness of corruption notifications on the transport layer. The only clear problem is that there exist different types of packet loss in networks, and they can not all be treated the same in congestion control mechanisms. But anyhow it is still unclear which approach for improving connections over lossy links is the best. Should it be the link layer approach, by hiding lost packets from the transport layer, trying to deliver every packet reliably, or should the network layer try the reliable transport, such that the transport layer may just focus on plain congestion related packet loss or is the ELN approach at transport layer still the best idea? In any case, the hardware has to be able to receive corrupted packets in order to be able to apply improvements. Otherwise neither a link layer nor any other improvement can be done - but the possibility to accept such corrupted packets at the link layer is limited by the hardware, and until there is not enough evidence that corrupted packets may be useful, the hardware developers will not implement this possibility. On the other hand, it is not possible to show the benefits of any improvements without having hardware which accepts corrupted packets for testing purposes. The problem is similar to the chicken-egg problem. Up to now the only known hardware for IEEE WLANs which is able to accept corrupted packets are the prism2/2.5/3 chip based b cards, also used for the measurements in this thesis. In UDPlite [32] the requirements for link layer technologies are specified more precisely. UDPLite is a lightweight UDP protocol designed for improvements of streaming media. The use of such streaming media and the use of better and better codecs, makes it reasonable for some applications to receive corrupted packets instead of having them dropped. A good audio or video codec is able to correct missing information in the stream, so if media streams are sent with UDP, where some packets may be dropped, the codec is still able to produce a more or less useful output anyways. Sometimes however it is easier for the

33 2.4 Concluding discussions 23 codec to reassemble a part of the stream from a group of datagrams, where the datagrams do not contain gaps because of a dropped packet, but contain partially corrupted data due to corrupted packets. Using the partially corrupted data from corrupted packets could therefore result in better streaming quality. To be able to receive the corrupted data on upper layers, the link layer must also be capable of accepting corrupted data. UDPlite specifies the link layer requirements, which are used as basis for the definition of the lower layer requirements in the draft and also other protocols which can use corrupted packets. First of all, it is specified that a lower layer protocol may not discard packets with errors in the insensitive part. Insensitive part refers to the packet s payload. UDPlite specifies the sensitive part as the IP header and the UDP header, in the draft it is specified as the IP and parts of the TCP header. As stated in the UDPlite specification, on links which support partial error detection, the lower layers have to use a strong checksum mechanism to detect errors in the sensitive part, to be able to drop such packets which are useless and dangerous. On links without such partial detection capabilities, errored frames always have to be dropped. It is also specified that implementing partial error detection only makes sense on error-prone links, i.e. radio links. The UDPlite specification also considers the security view of the use of corrupted packets. The problems are the authentication and encryption mechanisms which need an entirely intact packet in order to be able to check the authenticity as already described in section 2.2. The second point are the link layer encryptions, which are mainly used in wireless links to avoid packet sniffing, and therewith information theft. If even only few bits of an encrypted packet are damaged, usual encryption/decryption mechanisms are not able to avoid a spread of errors. A decrypted and partially corrupted packet would therefore result in a completely corrupted packet. There exist some encryption mechanisms which are able not to spread errors called stream ciphers. If such stream ciphers are not used, the encryption has to be turned off for the purpose of receiving corrupted frames. It has to be observed that omitting an integrity check would lead to a loss of confidentiality. If encryption is needed, it should be thought of using an application or transport layer encryption like TLS, for example. These specifications and the idea of the use of corrupted packets also for something else than for corruption notification at least should be a start in solving the chicken-egg dilemma explained above. If there are that many ideas which can profit from corrupted frames, hardware manufacturers and designers should begin to think about building in the

34 24 Background possibility of corrupted frame delivery from the link layer to upper layers, and the possibility to switch off ARQ. This problem and further discussions of the different kinds and needs of the reception of corrupted packets for higher layer purposes are discussed in [33]. The link layer dilemma however is not the only discussion about the advantages or disadvantages that the corruption notification might bring. In the discussions about IPv6, arguments against the abolishment of link layer checksums and ARQs, which are the basis for corruption notifications, were already raised. The main argument in this discussion is that the inner network of the Internet, which is the network clamped by the routers, has little computing power, and since the IP header changes on every hop by decreasing the time to live (TTL), the IP checksum has to be recomputed on every hop. This is an excessive requirement for the routers, if we believe the developers of IPv6, as the link layers usually use checksums and ARQ anyway, and therefore the probability of a corrupted IP header is not given. Thus a checksum in IPv6 has been omitted. The problem of IP packets with wrong or changed addresses should be solved by the upper layer protocols, which have to calculate the checksum as e.g. TCP already does over a pseudo header which includes the IP address fields and additionally the upper layer packet length and next header field, which identifies the type of the upper layer protocol. This may be enough for the packet to be delivered correctly at the endpoint, but fields which are important at every hop like the flow label field, which is used for quality of service, the version field which is used to identify the IP protocol version used, and the hop-by-hop options which can define nodes where the packet has to pass along its way to the destination, remain unchecked by the protocol itself. The assumption of IPv6 that underlying protocols have to check for packet integrity, conflicts with the IP over everything everything over IP notion. It also aggravates the link layer chicken-egg problem, as it encourages the use of checksums and ARQ. Note that IPv6 was designed before UDPLite. Now we have two protocols which encourage link layer designers to do two different things. As the typical behavior of link layers is to use checksumming and ARQ, it will be hard work to convince the hardware manufacturers to omit these mechanisms as needed in UDPLite or also for the DCCP and SCTP drop codes. The above cited paper also considers the problem, that no protocol which uses corruption notification actually describes how to react to this notifications. This is due to the fact that not every corruption notification is to be understood as strictly not congestion-related loss. Some congestion types are known which may actually cause corrupted packets, for

35 2.4 Concluding discussions 25 example congestion on shared wireless links. This shows that more investigations about how to react to corruption notifications still need to be done. In test setups like the ones for TCP-ELN, TCPHACK and the ones in this thesis, the corruption notification clearly shows non-congestion-related packet loss on the wireless link, and therefore a correct adaptation of congestion control could be implemented. Yet for specifications the ideas and test results are not enough. The biggest problem for resolving the link layer questions, is that link layer checksums and ARQs can sometimes perform poorly and sometimes work well. This means that, if we look at a simple direct radio link, we have retransmissions at the link layer level which assure data integrity, and then additional checks at the transport layer with TCP. This is a control loop on top of a control loop, which causes additional overhead, and may also cause unnecessary latency. The TCP congestion control mechanisms depend on the principle that a packet should only be sent into the network if a packet has left the network. This is done by updating and calculating the round trip time (RTT) of TCP packets, which therefore has a direct impact on the sending rate. Link layer ARQs would artificially prolong the RTT, by spending time on packet retransmissions, which would negatively affect the TCP throughput performance. On the other hand, if the TCP connection has more than one hop, where a partial link might be a wireless connection like an IEEE WLAN building to building bridge, and a second partial link might be a continental satellite connection, the use of link layer ARQs in the WLAN would be of great help. The retransmissions in this link would improve the performance of TCP, as the packet would not have to be resent completely if lost on that part of the connection resulting into a huge RTT, but may arrive with some proportionally small and hence hardly noticeable delay to the receiver, with a resulting RTT that is lower than in the other case. It can be seen that the use of link layer ARQ may be dependent on the path of the packet and thus the involved nodes must be aware of the path length and its composition to be able to react properly a condition which is impossible to fulfill with strict layering. It is possible to send information along with the network layer flows, but with the protocols actually in use, the link layer is not able to distinguish between different flows traversing the same link. With the introduction of the SCTP protocol, this could be changed, as SCTP is designed to be able to distinguish between flows due to its multi-homing capabilities.

36 26 Background Ideally link layers should trigger their ARQ capabilities for each flow based on the signal-to-noise ratio and the send rate, which would result in a link layer that is aware of packet size, RTT, steady-state loss event rate and the TCP retransmit timeout value (RTO value). The advantage of using corruption notifications, this means letting the link layer pass corrupted data to upper layers, has to be compared to the overhead of the link layer having to consider all the mentioned variables. If a benefit from corrupted packets could be expected, the link layer should switch off ARQ, otherwise it should use it a challenge which the link layer can hardly manage on its own. Recalling the end-to-end argument, there are two complementary goals in a network: higher level layers may organize lower level layers to their needs to achieve their specific goals efficiently and lower level layers, which should support a broad range of independent applications situated on higher level layers, should provide only resources of broad utility across applications and provide information of ressource sharing and resolution of ressource conflicts. These two goals could be translated into high level protocols communicating their requirements to the lower layers, and lower layers should use mechanisms of broad utility while reacting to commands of higher layers. This means, since only higher layers are aware of their own needs and requirements, link layers should use ARQ with different degrees of persistence, which may be tuned and/or disabled based on a message from the higher layers. There exist some possibilities of such inter-layer communication to control link layer ARQ. The effort started in the IETF under the name of Triggers for Transport (TrigTran) and further continued as Access Link Intermediaries Assisting Services (ALIAS) is about messages between transport end points and the network in between as performance enhancement. In this context the notifications for available or unavailable links were discussed, as well as a message for non congestion related packet loss. There is no precise outcome of this discussion, since as discussed above, there is no clear decision regards the interpretation of such non-congestion-related packet loss messages. It is not specified when such a message should be sent and who should receive the notification, and the discussion also raised issues about timescale and scalability. In [34] it was identified as a bad idea to let the nodes in the inner network send such information. In the UDPLite specification, as explained above, it is stated that the link layer should be aware of the UDPLite protocol in the packets, and switch ARQ on and off by packet inspection. With this method there is still the problem of the necessary per flow state, which is known not to be scalable. The common idea to improve that is to let only the routers on the outer edges keep per flow states the specified inter layer communication and let the core routers work as normal.

37 2.5 Summary and conclusions 27 There are enough other possibilities to let the link layer know that corrupted data is useful to the upper layer, but every method seems to have its distinct disadvantages. One possibility to avoid the link layer s need of transport protocol detection would be e.g. to set a bit in the IP header which signals the possibility of accepting corrupted data. The problem is that IP has no bits in the header to spare, but alternatively an IP option would be another possibility. This again would result in a higher RTT [35][36] which as explained above is not desired. On the other hand, an IP option included in every packet could avoid the per-flow state and this could possibly outweigh that problem, but as some routers seem to accept only a certain number of options per second, it is a problem to add the option to every packet. If the option is not sent with every packet, per-flow state is still necessary, and the advantage over detecting the transport protocol is lost. It seems that the best methods for inter-layer communication and link layer behavior have still to be found. Many tests in different scenarios have to be carried out to find out which is the best idea. The measurements presented in this thesis are a part of these tests, as they show the performance of the corruption notification in TCP, and use a link layer hardware which is able to switch off ARQ and accept corrupted data. In these tests ARQ is not switched off dynamically by transport layer requests or by the detection of the presence of a wireless link; it is switched off manually from within user space. Also the possibility to pass corrupted data to the upper layers is switched on manually. 2.5 Summary and conclusions This chapter gave a good overview of how prevailing the topic of corruption notifications is, and that much work has to be done to bring it to effectiveness. There were lots of discussions about the chicken-egg problem at link layer. The outcome is that more tests have to be performed in order to show the applicability of the link layer changes. The tests presented in this thesis use an IEEE WLAN card which has the possibility to use the described link layer changes. This makes it possible to implement the draft and perform measurements in real environments. The outcome of these measurements are a first big indicator if the changes to the link layer and the use of corruption notifications are profitable. In the next chapter (chapter 3) the changes to the WLAN card driver for enabling the reception of corrupted data are explained as well as the implementation of the draft.

38 28

39 Chapter 3 Implementation 3.1 Introduction In this chapter the implementation of the draft as well as the hostap driver is described. As explained in the previous chapters, the hostap driver and the prism2/2.5/3 chipset based WLAN cards give the possibility to accept corrupt frames. There exists also the possibility of switching off link layer ARQ, but this feature may also be present on other IEEE WLAN cards. The first section (section 3.2) provides the information about the hostap driver, and the changes applied to it to be able to correctly pass corrupt packets to the upper layers. A procedure was added to be able to record some statistical data for the measurements. The second section (section 3.3) shows the implementation of the draft and the flow diagrams which give a better overview of the draft s workflow. The implementation was done using the linux kernel sources [37] with the gentoo patchset [38], which however does not affect the hostap driver, or the linux kernel TCP stack. The applications made in this thesis are applicable to any kernel of version or above. 3.2 The hostap driver As already explained in chapter 1, the hostap driver is shipped along with the linux kernel sources, as of version The hostap driver is designed to be able to use all the possibilities that the prism chipset offers, the driver itself is generic for that kind of chipset and does not depend on a certain manufacturer of the complete WLAN card. The driver source files may be found in the kernels drivers/net/wireless/hostap/ directory. 29

40 30 Implementation The hostap driver source and frame reception process Since the hostap driver is poorly documented except for the source code comments, the best help is still a cross reference produced by the lxr engine [39]. It is possible to generate a browseable source code directly by applying the engine on the own kernel sources, or to use an already generated cross reference web page. For linux kernels to the browseable cross reference can be found at the lxr project s home page; for kernel and which are well suited for this project, they can be found at [40]. The hostap driver files It is not that easy to understand how the hostap driver works if one is not familiar with kernel coding standards and coding mechanisms. The driver itself is well written, but is poorly commented, even though the comments do give some general important information about certain functions. As the prism2/2.5/3 chipset is a special one, and may be used to act as an access point (AP) if set to master mode, the hostap driver includes more features than a standard driver for cards which can act only as stations (STA). So the driver is split up into 20 files, of which 14 are sources and 6 header files. After compiling it generates two kernel modules. There are different groups of files in the hostap directory. Some files interact directly with the hardware, which are the ones with cs, plx and pci suffixes. These three different files are necessary, because the prism chipset is shipped with PCMCIA cards for notebooks (like the one used here), or with PCI cards for desktop PCs. The PLX is a PCMCIA to PCI adapter for desktop PCs, for which some special functions are required, so it has its own code. Other files allow the use of the card as an AP, which are the ones with ap suffix. The hw, ioctl and info files are the ones that interact directly with the hardware files. The first is required to process the raw packets, to register the functions to the kernel subsystem (for example functions to tasklet lists or interrupts) and so on. The second one is for interaction with the control registers of the card. The exact function of the registers in the prism chipset can be found in the prism driver programmer s manual [41]. The third one processes the information which the chipset firmware extracts from management frames and passes to the host. The main, proc and download are the main functions for initialization of the driver and the card, the functions that manage the output to the /proc filesystem, and the file

41 3.2 The hostap driver 31 that manages the possibility of firmware updates, for which a tutorial can be found at [42]. The files containing the tag are the ones that process packets at the MAC layer. For a better understanding, they are split into a tx file for transmitting packets, and a rx file for receiving packets. The mod-tagged files are used to generate correct module aliases and hardware specific information. The main frame reception functions The functions of main interest in the whole hostap driver construct are the hostap rx skb function in the hostap hw.c file and the hostap rx function in the homonymous file hostap rx.c (furthermore the struct hostap rx status defined in hostap h is needed to transport information between the functions). These two functions are the functions which handle received packets. The hostap rx skb function is not bound to an interrupt; it processes the packets from a queue where they have been put by a previous function prism2 rx. This function already drops some frames that are of invalid length or partial frames which occur by a failed read from the card s buffer allocation path (BAP) which contains the packets in the hardware and creates a socket buffer (skb). The hostap rx skb instead already deals with the skb, and performs tests related to the packets header. It drops packets that are shorter than the header, unless the card is set to monitor mode, and with the same restriction it drops corrupt frames which are masked by HFA384X RX STATUS FCSERR. After that the hostap rx status struct is filled with the information that came with the header, like reception time, signal and noise information as well as the transfer rate 1. Then the hostap rx function is called, with the skb (the packet) and the status information as argument. As more prism chipset based cards may be used with the hostap driver at the same time, the device name matters a lot, and so it is always passed with the information. The second function hostap rx is a huge 400 liner. This function extracts the MAC layer information from the packet and makes it ready to be processed by upper layers. But before it reaches the stage of information extraction, the packet passes a variety of tests, which could cause it to be dropped. As this chapter is intended to give a overview of the hostap driver and the project related functions, it would be superfluous to explain 1 Transfer rate in this case means the fixed maximum rate at which participants can communicate. In the case of this thesis the rate is fixed at 11Mbit/s.

42 32 Implementation every single case distinction. The main work of this function is to identify wrongly passed packets; this means, packets which are passed from the card to the host, but, for example, not belonging to the host, or which are not data packets. After the MAC addresses are correctly assigned to the skb, the ethernet header is stripped from the packet with the eth type trans function, and the protocol information is set to the skb->protocol field. After that the packet (which usually contains IP and TCP, UDP or ICMP) is passed to the upper layer via netif rx Changes to the hostap driver The first changes were made to both the functions described in section 2.3. The most important thing was to enable the reception of corrupt frames and the second was to test if the corrupt packet then contained a working IP Header, and in a second step if certain fields of the TCP Header were not corrupt too. The activation of the reception of corrupt frames was done in the hostap rx skb function while the rest was done in the function hostap rx. Enabling reception of corrupt frames The prism2/2.5/3 chipset passes corrupt frames to the host when in monitor or promiscuous mode. As monitor mode does not allow to associate to an AP in managed mode or to another station in ad-hoc mode, promiscuous mode had to be used to be able to set up a TCP connection. The simplest way to be able to further process corrupt frames in the driver was to replace the code in listing 3.1 with the code in listing 3.2 in the hostap rx skb function, in file hostap hw.c. In order to pass the information to the other function (hostap rx in the file hostap rx.c) that a packet is corrupt, the hostap rx status struct gained one more variable which holds the mentioned information. For comparison the original struct (listing 3.3) and the expanded struct (listing 3.4) are shown. In the hostap rx skb function the struct is named stats. if ( le16_to_cpu ( rxdesc -> status ) & HFA384X_RX_STATUS_FCSERR && (! local -> monitor_allow_fcserr local -> iw_mode!= IW_MODE_MONITOR )) goto drop ; Listing 3.1: The original if statement in function hostap rx skb

43 3.2 The hostap driver 33 if ( le16_to_cpu ( rxdesc -> status ) & HFA384X_RX_STATUS_FCSERR ) stats. fcserr = 1; else stats. fcserr = 0; Listing 3.2: The new if statement in function hostap rx skb struct hostap_80211_rx_status { u32 mac_time ; u8 signal ; u8 noise ; u16 rate ; /* in 100 kbps */ }; Listing 3.3: The original hostap rx status struct struct hostap_80211_rx_status { u32 mac_time ; u8 signal ; u8 noise ; u16 rate ; /* in 100 kbps */ u16 fcserr ; }; Listing 3.4: The new hostap rx status struct with the fcserr field The new if-clause tests if the currently processed packet is corrupt by masking the packet s status information with HFA384X RX STATUS FCSERR. This mask shows the least significant bit of the status information, which is the one that indicates whether the frame is corrupt or not. The information then is set in the new stats field by setting it to 1. If the currently processed frame is okay, the stats field is set to 0. After that the frame goes on to be further processed by the hostap rx function. Verifying IP and TCP checksums As the reception of corrupt frames is enabled, the packets have to be checked for their consistency by verifying the IP checksum. If the checksum doesn t fail, the TCP checksum is controlled and if this checksum fails the new TCP pseudo header checksum of the draft is verified. These steps are all done in the hostap rx function in file hostap rx.c. The frames passed to this function have to pass several checks, where they could be discarded. Most of these checks are necessary for a correct handling even of the broken packets, but there exist some tests which could discard a broken packet due to wrong information in the wireless header. As this could easily happen, but it is not desirable to discard corrupt frames due to such tests, these tests are only done for correctly received frames.

44 34 Implementation if (! rx_stats -> fcserr ){ if ( type!= IEEE80211_FTYPE_DATA ) { } if ( type == IEEE80211_FTYPE_MGMT && stype == IEEE80211_STYPE_AUTH && fc & IEEE80211_FCTL_PROTECTED local -> host_decrypt && && ( keyidx = hostap_rx_frame_decrypt ( local, skb, crypt )) < 0) { printk ( KERN_DEBUG "%s: failed to decrypt mgmt :: auth " " from " MACSTR "\n", dev ->name, MAC2STR (hdr ->addr2 )); goto rx_dropped ; } if ( hostap_rx_frame_mgmt ( local, skb, rx_stats, type, stype )) goto rx_dropped ; else goto rx_exit ; } Listing 3.5: The test if frames other than data frames are passed to the host. The further processing and decryption of management frames is only for AP mode The part shown in 3.5 tests if a packet is a data packet or not, and if it is not performs tests related to the AP functionality of the card. If a packet is marked as corrupt (flag rx stats->fcserr is set) it should not be processed in the function, because the information in the type and stype fields of the connection information struct might be unreliable, and the packet could be dropped, or worse passed to the management frame process function with all its wrong data. In this cases the choice is to keep every corrupt packet until the IP and TCP checksum tests were made. An other similar construct in the function is listed in 3.6, where again a test is made to check whether the frame is a data frame or not. If it is not a data frame, we will find it out with the IP checksum, so we keep every broken frame again. For the standard good packets these two tests are necessary for the hostap driver to work correctly. if (! rx_stats -> fcserr ){ if ( stype!= IEEE80211_STYPE_DATA && stype!= IEEE80211_STYPE_DATA_CFACK && stype!= IEEE80211_STYPE_DATA_CFPOLL && stype!= IEEE80211_STYPE_DATA_CFACKPOLL ) { if ( stype!= IEEE80211_STYPE_NULLFUNC ) printk ( KERN_DEBUG "%s: RX : dropped data frame " " with no data ( type =0x%02x, subtype =0x%02x)\ n", dev ->name, type >> 2, stype >> 4); goto rx_dropped ; } } Listing 3.6: An other test for checking if the frame is a data frame or not After some more packet testing which does not affect corrupt packets (like tests for encryption), we come to the last part of the function, which is the most important one. The

45 3.2 The hostap driver 35 original function does a simple thing: it strips the ethernet header from the skb->data part (with eth type trans) and sets the protocol (ETH P IP) into the appropriate socket buffer field. Then the device is set in the socket buffer, and the whole buffer is sent to the IP layer. The original construct is made of four lines of code (listing 3.7). if ( skb ) { skb -> protocol = eth_type_trans (skb, dev ); memset (skb ->cb, 0, sizeof (skb ->cb )); skb ->dev = dev ; netif_rx ( skb ); } Listing 3.7: The original part of the hostap rx function, which passes the skb to the IP layer In this part of the function the tests for the checksums were inserted. At first, preliminary test were made by checking the skb->protocol field for ETH P IP. This gave a negative result, as the field always returned ETH P which is in fact the LLC protocol. Even the packets captured with wireshark (a tool for packet capture and analysis) [43] showed the same results. After some investigation on the byte fields of the corrupt packets, the problem was solved: corrupt packets have an ethernet I header which automatically sets the length of the packet and the LLC protocol value in the header, followed by the correct protocol number and the IP header, while the good packets use the ethernet II header, which lacks these 4 bytes. Figure 3.1 shows these bytes. Figure 3.1: Screenshot of wireshark with a corrupt packet, showing the ethernet I header. The bytes marked by the two frames are the ones that usually are in the ethernet II header and the value 8 tells that the following data belongs to the IP header. It is interesting that even if the ethernet destination address is wrong, the packet contains a correct IP header. The simple solution is to strip 4 more bytes from the packet, and to extract the IP header directly. A test for the protocol is obsolete, due to the checksum test that is performed anyway. The protocol is set in the socket buffer after the frame was identified as a good

46 36 Implementation IP packet. The IP header is checksummed and the result verified, as shown in listing 3.8. The result of the action is logged to the kernel message buffer for statistical analysis. If the IP checksum is correct, the TCP checksum is verified, otherwise the packet is dropped, to avoid unnecessary further processing. The test for the IP checksum is only done for packets of size bigger than 1494 bytes, which were indicated by some tests to be TCP data packets. This avoids unnecessary checksum calculations, but limits the use of the driver to a direct connection between the two participants, where the packet size reaches always the same maximum transfer unit. if ( skb ) { skb -> protocol = eth_type_trans (skb, dev ); if ( rx_stats -> fcserr ) { if (skb ->len >= 1494) { skb_pull (skb,4); skb ->nh. raw=skb ->data ; iph=skb ->nh. iph ; if ( unlikely ( ip_fast_csum (( u8 *) iph, iph ->ihl ))){ printk ( KERN_DEBUG " IP checksum FAILED \n" ); goto rx_dropped ; } else { printk ( KERN_DEBUG " IP checksum OK!\0 " ); skb -> protocol =htons ( ETH_P_IP ); Listing 3.8: Stripping 4 more bytes from the data part, extracting IP header and verifying the checksum The verification of the TCP checksum is similar to the IP checksum. The TCP header is extracted from the socket buffer, and the checksum is calculated and tested. The functions used for both procedures are the standard checksumming functions of the protocol implementations in the kernel. If the TCP checksum is valid (which is very improbable, in consideration of the fact that the packet is classified as corrupt), the packet is passed to the IP layer for further processing. Otherwise the newly implemented TCP pseudo header checksum is verified. The new one is a 32 bit checksum which only covers certain parts of the TCP and IP header, which are: ECN field, source address and destination address from the IP header and source and destination port, sequence and acknowledgement number and the CWR, ECE, ACK, RST, SYN and FIN control bits from the TCP header. This checksum implementation is part of the full implementation of the draft, more precisely the first part of the corruption detection algorithm, and is therefore described in section 3.3. The behavior of this test is the same as for the TCP test. The results are always written to the kernel message buffer and the packet is sent to the IP layer for further processing. The packet will then be discarded at the TCP layer. Listing 3.9 shows the last part of the driver related changes.

47 3.3 The implementation of the draft 37 th=( void *) iph + iph ->ihl * 4; if( unlikely ( tcp_v4_check ( th,skb ->len -iph ->ihl *4, iph ->saddr,iph ->daddr, csum_partial (( char *) th, skb ->len -iph ->ihl *4, 0)))){ struct tcp_cdo_phdr * phdr ; u32 crc32c_calculated, crc32c_from_opt ; u32 * ptr ; u8 ecn ; if (( phdr = ( struct tcp_cdo_phdr *) kmalloc ( sizeof ( struct tcp_cdo_phdr ), GFP_ATOMIC )) == NULL ) { printk ( KERN_DEBUG " Not enough memory to allocate pseudo header \n" ); } < fill pseudo header - simple assignments of th and iph fields to phdr fields > /* calculate crc32c checksum */ crc32c_calculated = cdo_crc32c_check (( u8 *) phdr, sizeof ( struct tcp_cdo_phdr )); ptr =( u32 *)( th + 1); if (!th -> syn ){ ++ ptr ; crc32c_from_opt =ntohl (* ptr ); if ( crc32c_calculated == crc32c_from_opt ) printk ( KERN_DEBUG " TCP PHDR csum OK \n" ); else printk ( KERN_DEBUG " TCP PHDR csum FAILED \n" ); } else printk ( KERN_DEBUG " SYN or SYN / ACK \n" ); } else printk ( KERN_DEBUG " TCP csum OK \n" ); Listing 3.9: The TCP and TCP pseudo header check. Continuation of the else part of listing The implementation of the draft In this section the implementation of the draft in the linux kernel is described. The implementation is split into 3 parts, the implementation of the corruption detection option (further called CDO), the implementation of the corruption notification option (further called CNO), and the implementation of the congestion control changes, which are not defined in the draft, but were nevertheless included as a proof-of-concept. The congestion control algorithm used is the linux kernels standard NewReno algorithm. No tests have been made with other congestion control algorithms which are available for this kernel The implementation of the corruption detection option The corruption detection option is a 32 bit CRC32c checksum [14] covering the already mentioned fields of the pseudo header (see section 3.2.2), which is sent within a TCP option. As the draft says that the option MAY be included in any packet, some choices were made which are related to the kernel s TCP implementation and when to add this option. As outgoing TCP SYN packets use different functions in the kernel than packets which belong to an already established connection, it was decided not to add the option to SYN or SYN/ACK packets for the sake of simplicity. Another issue is the fact that the option itself

48 38 Implementation Figure 3.2: Sending the corruption detection option uses 8 bytes in the implementation, because the checksum uses 4 bytes, and the option information as type and length use 2 bytes and have to precede the checksum. As far as there isn t any option (and there is none in the standard linux kernel) which needs only 2 bytes, those remaining 2 bytes in before the option information bytes must be filled with no operation (NOP) options. The same alignment solution can be found in the timestamp option [20] implementation. As packets in a TCP connection of two linux systems always carry the timestamp option, and it is possible to use selective ACKs (SACKs) [5], the header size of an ACK packet with all these options and additionally the corruption detection option could arrive at 64 bytes, which exceeds the limit of the TCP header size. Because of this it was decided not to add the option to the ACKs and thus only to data packets, which means to packets which are bigger than the header size only. The flow diagram for the corruption detection option is shown in figure 3.2. The CRC32c checksumming function was borrowed from the SCTP [15] [16] implementation, which again borrowed much of the code from the CRC32c library of the kernel. This function splits the data into chunks of 1 byte size and computes the checksum. As the pseudo header with the ECN [17] part of the IP header would not be a multiple of 8 bits,

49 3.3 The implementation of the draft 39 it was decided to cover the whole type of service field [44] to bring the header to a multiple of 8 bits. With the correct alignment of the fields in the pseudo header struct the header reaches an exact length of 24 bytes which is a multiple of 4 bytes or 32 bits as needed. As listing 3.10 shows, the pseudo header struct is included in the include/linux/tcp.h file, which already contains the struct for the standard TCP header and the TCP socket. ct tcp_cdo_phdr { be32 saddr ; be32 daddr ; u16 source ; u16 dest ; u32 seq ; u32 ack_seq ; u16 ecn ; # if defined ( LITTLE_ENDIAN_BITFIELD ) u16 res1 :4, doff :4, fin :1, syn :1, rst :1, psh :1, ack :1, urg :1, ece :1, cwr :1; # elif defined ( BIG_ENDIAN_BITFIELD ) u16 doff :4, res1 :4, cwr :1, ece :1, urg :1, ack :1, psh :1, rst :1, syn :1, fin :1; # else # error " Adjust your <asm / byteorder.h> defines " # endif }; Listing 3.10: The ecn field was put after the sequence numbers for correct alignment # define CRC32C_POLY 0 x1edc6f41 # define CRC32C (c,d) (c=(c > >8)^ crc_c [(c^( d ))&0 xff ]) static inline u32 cdo_crc32c_check ( u8 * buffer, u16 length ) { u32 u32 crc32 i; = ~( u32 ) 0; } for (i = 0; i < length ; i++){ CRC32C ( crc32, buffer [i ]); } return crc32 ; Listing 3.11: The simple CRC32c shifting function with the preprocessor macro The checksumming function was included in the header file which contains the tcp function prototypes, namely include/net/tcp.h. As this header file is included in every file that belongs to the IPv4 TCP stack, this function can be called from every other function within this context. The checksum function consists of the CRC32c table and the simple CRC calculation function as shown in listing As the checksum itself has to be calculated just before the standard TCP checksum which needs parts of the IP layer, but possible TCP options are set in functions before the packet is sent to the IP layer or to a function able to extract information from the socket belonging to the IP layer, the operation is split in two: in a first part the place is reserved by setting the option with an empty checksum, while in a second one the pseudo header checksum is calculated and the correct option is set instead of the placeholder.

50 40 Implementation This operation affects some functions in the net/ipv4/tcp output.c file and one function in the net/ipv4/tcp ipv4.c file. The first file contains the functions to calculate the maximum segment size of a tcp packet and uses the information of the IP s MTU (maximum transfer unit) for that. The first thing to do was to decrease the MSS by 8 byte, so the whole packet size with the corruption detection option would not exceed the MTU. Otherwise the packet would be dropped at the IP layer, and never find its way to the net. The safest way was to subtract the 8 bytes directly from the MTU before the MSS is calculated as shown in listing int tcp_mtu_to_mss ( struct sock *sk, int pmtu ) { struct tcp_sock *tp = tcp_sk (sk ); struct inet_connection_sock int mss_now ; * icsk = inet_csk ( sk ); } /* Calculate base mss without TCP options : It is MMS_S - sizeof ( tcphdr ) of rfc1122 */ mss_now = pmtu - icsk ->icsk_af_ops -> net_header_len - sizeof ( struct tcphdr ); /* Clamp it ( mss_clamp does not include tcp options ) */ if ( mss_now > tp -> rx_opt. mss_clamp ) mss_now = tp -> rx_opt. mss_clamp ; /* Now subtract optional transport overhead */ mss_now -= icsk -> icsk_ext_hdr_len ; /* Then reserve room for full set of TCP options and 8 bytes of data */ if ( mss_now < 56) mss_now = 56; /* Now subtract TCP options size, not including SACKs */ mss_now -= tp -> tcp_header_len - sizeof ( struct tcphdr ); mss_now -= TCPOLEN_CDO_ALIGNED ; return mss_now ; Listing 3.12: Reserving 8 bytes for the CDO option in the mtu to mss function in tcp output.c It was decided to set the corruption detection option the first option in the options chain, such that the correct place to write the real checksum could be found easily and fast in the corresponding function. The options for packets in the established state are set in the function tcp build and update functions which is called from tcp transmit skb. The placeholder for the corruption detection option is set just before the call, so it can be detected by the length of the packet whether the packet is a data packet or a simple ACK. Listing 3.13 shows the relevant part. Once the work in the net/ipv4/tcp output.c file is done, the checksum has to be calculated and written before the standard TCP checksum is calculated. This is done in the tcp v4 send check function in file net/ipv4/tcp ipv4.c as shown in listing After this function the packet makes its way down to the IP layer. Like in the function before the

51 3.3 The implementation of the draft 41 packet is tested for not containing a SYN and for its length, to be sure that it is a correct packet which already holds the CDO placeholder. static int tcp_transmit_skb (...){... /* corruption detection option part - increase header * only in not SYN packets and packets with a payload */ if ((! unlikely (tcb ->flags & TCPCB_FLAG_SYN ))&&( skb ->len )){ tcp_header_size += TCPOLEN_CDO_ALIGNED ; }... if ( unlikely (tcb ->flags & TCPCB_FLAG_SYN )) { tcp_syn_build_options (( u32 *)( th + 1), tcp_advertise_mss ( sk), ( sysctl_flags & SYSCTL_FLAG_TSTAMPS ), ( sysctl_flags & SYSCTL_FLAG_SACK ), ( sysctl_flags & SYSCTL_FLAG_WSCALE ), tp -> rx_opt. rcv_wscale, tcb ->when, tp -> rx_opt. ts_recent ); } else { /* build CDO with placeholder for checksum */ placeholder =0; ptr =( u32 *)( th + 1); if (skb ->len > tcp_header_size ){ * ptr ++ = htonl (( TCPOPT_NOP << 24) ( TCPOPT_NOP << 16) ( TCPOPT_CDO << 8) TCPOLEN_CDO ); * ptr ++= placeholder ; } tcp_build_and_update_options (ptr, tp, tcb -> when ); TCP_ECN_send (sk, tp, skb, tcp_header_size ); } Listing 3.13: Setting the CDO in front of the remaining options on data packets void tcp_v4_send_check ( struct sock *sk, int len, struct sk_buff * skb ) { struct struct inet_sock * inet = inet_sk ( sk ); tcphdr * th = skb ->h. th; struct tcp_cdo_phdr * phdr ; u32 u32 crc32c =0; * ptr ; } if ((! th -> syn )&&( skb -> len > th -> doff * 4)){ /* fill pseudo header for crc32c checksum */... crc32c = cdo_crc32c_check (( u8 *) phdr, sizeof ( struct tcp_cdo_phdr )); ptr =( u32 *)( th + 1); * ptr ++ = constant_htonl (( TCPOPT_NOP << 24) ( TCPOPT_NOP << 16) ( TCPOPT_CDO << 8) TCPOLEN_CDO ); * ptr ++ = htonl ( crc32c ); kfree ( phdr ); } if (skb -> ip_summed == CHECKSUM_HW ) { th ->check = ~ tcp_v4_check (th, len, inet ->saddr, inet ->daddr, 0); skb ->csum = offsetof ( struct tcphdr, check ); } else { th ->check = tcp_v4_check (th, len, inet ->saddr, inet ->daddr, csum_partial (( char *) th, th ->doff << 2, skb -> csum )); } Listing 3.14: The pseudo header is filled with information and the CRC32c checksum is calculated, followed by the standard TCP checksumming procedure

52 42 Implementation Figure 3.3: The reception of a data packet and the eventual sending of the special ACK Obviously this additional option causes the data segment to pass through the slow path implementation of the TCP stack at the receivers side, but this should not affect the speed that much The implementation of the corruption notification option The corruption notification option is always sent when a packet is received, which contains a corrupt TCP header, but a correct pseudo header checksum. So the relevant information can be extracted: source and destination port, sequence and acknowledgement numbers, as well as the header length. With all this information a simple ACK packet can be built, carrying the corruption notification option. Originally it was envisioned to carry the timestamp options too, but as the options are not covered by the pseudo header checksum, this information would not be reliable. After sending the ACK packet, the received data packet is discarded, as the data contained are corrupt. Figure 3.3 shows the flow diagram for the reception of a corrupt data frame and the sending of the ACK containing the CNO. This simple sequence is done in the function tcp v4 rcv in the file net/ipv4/tcp ipv4.c as shown in listings 3.15 and It is not defined in the

53 3.3 The implementation of the draft 43 draft how the acknowledgement number should be generated. As this is usually the sequence number of the received data plus the length of the data, precisely this calculation is done. The data length is reliable even if the packet is corrupt and the header checksum does not cover it because it is covered by the IP checksum. This choice was made as sending back the old sequence number would result in a duplicate ACK at the source, which is not the behavior described in the draft. if (( skb -> ip_summed!= CHECKSUM_UNNECESSARY && tcp_v4_checksum_init ( skb ))){ if (skb ->len > th ->doff * 4 ){ /* fill... pseudo header */ crc32c_calculated = cdo_crc32c_check (( u8 *) phdr, sizeof ( struct tcp_cdo_phdr )); ptr =( u32 *)( th + 1); if (!th -> syn ){ ++ ptr ; crc32c_from_opt =ntohl (* ptr ); if (( crc32c_calculated == crc32c_from_opt ) && (!th -> ack )){ kfree ( phdr ); goto phdr_ok ; } } kfree ( phdr ); } goto bad_packet ; } Listing 3.15: If the TCP checksum fails for an incoming packet the pseudo header checksum is calculated phdr_ok : /* from tcp_v4_send_ack */ } memset (& rep.th, 0, sizeof ( struct tcphdr )); memset (& arg, 0, sizeof arg ); arg. iov [0]. iov_base = ( unsigned char *)& rep ; rep. opt = htonl (( TCPOPT_NOP << 24) ( TCPOPT_NOP << 16) ( TCPOPT_CNO << 8) TCPOLEN_CNO ); arg. iov [0]. iov_len = sizeof ( rep ); /* Swap the send and the receive. */ rep.th. dest = th -> source ; rep. th. source = th ->dest ; rep. th. doff = arg. iov [0]. iov_len / 4; rep. th. seq = htonl (th ->ack ); rep. th. ack_seq = htonl (th ->seq+skb ->len -( th ->doff *4)); rep. th. ack = 1; rep. th. window = htons (th -> window ); arg. csum = csum_tcpudp_nofold (skb ->nh.iph ->daddr, skb ->nh.iph ->saddr, arg. iov [0]. iov_len, IPPROTO_TCP, 0); arg. csumoffset = offsetof ( struct tcphdr, check ) / 2; ip_send_reply ( tcp_socket ->sk, skb, &arg, arg. iov [0]. iov_len ); goto discard_it ; Listing 3.16: The ACK containing the CNO is assembled and sent. The received corrupt data segment is discarded

54 44 Implementation The changes to congestion control The changes to the congestion control mechanism were the trickiest part of the implementation. The solution presented here is not optimized at all, it is rather a dirty hack, as it is an effort to avoid entering the loss state where slow start threshold and congestion window are reduced if it is known that on the link packets are lost, but the loss was not due to congestion. It is just a simple idea, as there is no in depth research on how the congestion control mechanisms should be changed in the presence of a lossy link, and the draft does not elaborate on that, as it should be part of a separate research endeavor. int tcp_rcv_established ( struct sock *sk, struct sk_buff *skb, struct tcphdr *th, unsigned len ) {... if ( tcp_fast_parse_options (skb, th, tp) && tp -> rx_opt. saw_tstamp && tcp_paws_discard (sk, skb )) { if (!th -> rst ) { NET_INC_STATS_BH ( LINUX_MIB_PAWSESTABREJECTED ); tcp_send_dupack (sk, skb ); goto discard ;... } } void tcp_parse_options ( struct sk_buff *skb, struct tcp_options_received * opt_rx, int estab ) {... case TCPOPT_CNO : if( opsize == TCPOLEN_CNO ) { opt_rx ->cno = 1; } };... } Listing 3.17: If an ACK arrives the options are parsed, and possibly the flag in the tcp socket is set The only indications given in the draft, are how to react to an ACK which carries a CNO. The draft states that the RTT must be updated, the RTO timer must be reset, the missing segment must be retransmitted and the ACK then must be treated as normal ACK. As the ACKs in the test setups were not carrying any special information it was decided to discard the ACK segment after performing the first three actions for the sake of simplicity. Additionally it was decided to record the sequence number of the packet, such that the loss state could be avoided as long as ACKs for packets with higher sequence numbers are missing. As CDO and CNO are used only on packets within the established state of a connection [45][31], everything could be done in the function tcp rcv established of the file net/ipv4/tcp input.c as shown in listings 3.17 and It has to be specified that this implementation covers only the slow path of TCP, because ACKs with a CNO automatically leave the fast path. The idea of these changes to the congestion control

55 3.3 The implementation of the draft 45 if (tp -> rx_opt. cno ) { } /* record sequence number */ tp -> last_cno_seq =th ->seq ; /* recalculate RTT */ tcp_rcv_rtt_measure ( tp ); /* reset RTO */ icsk -> icsk_rto = min ( icsk -> icsk_rto << 1, TCP_RTO_MAX ); inet_csk_reset_xmit_timer (sk, ICSK_TIME_RETRANS, icsk ->icsk_rto, TCP_RTO_MAX ); /* retransmit the segment */ tcp_retransmit_skb (sk, skb_peek (& sk -> sk_write_queue )); /* discard the ACK */ goto discard ; Listing 3.18: If the cno option is set in the socket, the RTT is recalculated, the RTO reset, the corresponding data segment retransmitted and the sequence number recorded in the socket. mechanism is abutted on the work done in TCP-ELN, where the most important idea was to stop the reduction of the congestion window as long as non-congestion-related packet loss is known. In order to be able to record the sequence number in the socket, an additional field was added in the file include/linux/tcp.h where the TCP socket is defined. This field contains the sequence number of an ACK carrying a CNO as long as it is not overwritten by a newer ACK carrying a CNO, or an ACK which acknowledges a data segment with a equal or higher sequence number, which means that the formerly corrupt data segment is acknowledged too. A higher sequence number could occur in two cases: first, as explained, the sequence number extracted from the ACK carrying a CNO could not be correct (in the case of the test done for this thesis, it could only be smaller than the real size), so the sequence number of the correct ACK could be higher, or the corrupt segment was then acknowledged by a delayed ACK [45], which could acknowledge more than one segment at once. The recorded sequence number is automatically overwritten on reception of a new CNO ACK in the function shown in listing 3.18, while the deletion of the record (set to 0) is done in the function tcp ack in the same file (listing 3.19). Figure 3.4 shows the flow diagram of an incoming ACK segment. Listing 3.19: static int tcp_ack ( struct sock *sk, struct sk_buff *skb, int flag ) { struct inet_connection_sock * icsk = inet_csk ( sk ); struct tcp_sock * tp = tcp_sk ( sk ); u32 prior_snd_una = tp -> snd_una ; u32 ack_seq = TCP_SKB_CB ( skb )->seq ; u32 ack = TCP_SKB_CB ( skb )-> ack_seq ; u32 prior_in_flight ; s32 seq_rtt ; int prior_packets ; if ( ack_seq >= tp -> last_cno_seq ) tp -> last_cno_seq =0;... } The sequence number recorded in the socket is deleted (set to 0) if the ACK acknowledges a higher or equal one

56 46 Implementation Figure 3.4: The flow diagram of an incoming ACK segment The last changes to the TCP stack are added to the function tcp enter loss still in file net/ipv4/tcp input.c. The quick solution is not to change the slow start threshold and not to halve the congestion window while there are outstanding acknowledgments for packets with sequence numbers lower than the one recorded by the ACK carrying a CNO. This means that everything in the function is avoided in this case as shown in listing The simple comparison to 0 can be done, because the tp->last cno seq variable is set to a value different from 0 only when there are still outstanding data to acknowledge. When the correct ACK arrives the value is reset. Thus the value (the sequence number) itself is only important for incoming normal ACKs which may have to reset the value. void tcp_enter_loss ( struct sock *sk, int how ) { const struct inet_connection_sock * icsk = inet_csk (sk ); struct tcp_sock * tp = tcp_sk ( sk ); struct sk_buff int cnt = 0; * skb ; } if (! tp -> last_cno_seq ){ } /* Do everything as usual */ Listing 3.20: The enter loss function does nothing if the sequence number of the packet calling the function is lower than the recorded sequence number.

57 3.4 Summary and conclusions Summary and conclusions With the changes to the hostap driver, and with the use of the linux wireless tools, the link layer can be enabled to receive corrupt data with some tricks and to pass this data to the upper layers where the implementation of the draft may take care of that information. The implementation of the draft is adapted to the test setups shown in 4, which show some loss of generality. For different setups and tests a different implementation may be needed, or at least changes haveto be made to the code. The changes to the congestion control mechanisms are just a simple try, abutted to the work done in other research, as there are no definitions of the adaptations to be made in the draft, as already explained in chapter 2. The following chapter (chapter 4) will provide more details about the hardware used and the test setups and show the preliminary measurements done to get a first idea on what the benefits of the corruption notification might be, and if it was worth to do the full implementation. Furthermore the final results are presented which show exactly what the outcome of the idea is in a real IEEE environment.

58 48

59 Chapter 4 Measurements 4.1 Introduction This chapter presents the measurements for the TCP corruption notification. The measurements were done in two tests, once for the preliminary results and one for the final results. The preliminary measurements were done after the application of the first part of the implementation: the changes to the hostap driver and the implementation of the corruption detection option. The preliminary measurements should show if it would be reasonable to do the complete implementation of the draft and to perform the final measurements. There are already several theses written at the University of New South Wales in Sydney, which measured error behavior in wireless LANs, but as in [46], where the authors mainly focused on the kind of interfering activities and the resulting errors in wireless links, no work focused on the integrity of protocol headers. As also stated in [19], it may be deduced that a corrupt packet is likely to be followed by a complete packet loss. The final measurements are related to this problem. The measurements are done by collecting the same statistics as in the preliminary measurements, but additionally the real transfer rate per second was recorded. The resulting graphs give an overview of the performance of TCP with corruption notification in real environments. 49

60 50 Measurements 4.2 Equipment Software As already explained in chapter 1, the software needed for the thesis was a Linux running at least kernel which includes the hostap driver. The choice of the distribution was gentoo linux [47], as it was already installed on one of the notebooks needed for the tests. The kernel used was a gentoo-r7 kernel patched with gentoo patches [38]. However, these patches affect neither the TCP stack nor the hostap driver. Additional software needed were the linux wireless-tools [13] to be able to set up the wireless cards, the metalog system logger [48] for statistics collection, tcpdump [49] and wireshark [43] for packet sniffing, an ftp server [50] and client for the preliminary measurements, and the iperf tool [51] for the final measurements Hardware The system used as a sender was a Acer Ferrari 3400 Lmi notebook with integrated minipci wireless card. The card is based on a g broadcom chipset, which is not supported under linux. There exists a project which creates a driver for the broadcom chipset through reverse engineering, but it has basic functionality only and currently it does not support every chipset at full speed, although it has found its way into the kernel source tree already. There exists a possibility to use the card in linux with ndiswrapper [52], which allows to use Windows driver to operate certain WLAN cards. The notebook has a 64 Bit AMD processor and the operating system is compiled for 64 bits, but this has no importance for the changes to the kernel. These changes are platform independent. As explained a WLAN card with a prism chipset was needed; a Netgear MA b card with a prism2.5 chipset was used. As receiver a Toshiba Satellite notebook was used. This notebook did not have any integrated wireless LAN card, such that a second Netgear card, the same as for the first notebook, was used. The notebook used the same Linux operating system as the other notebook, but it has a 32bit processor and therefore used a 32 bit operating system. 4.3 Preliminary measurements In order to know whether the use of corruption detection options makes sense or not, it was researched how many of the sent packets were lost in transit, or were received with

61 4.3 Preliminary measurements 51 errors. As the errors are distributed randomly through the packets, it was checked how many of the broken packets were recoverable. The measurements shown in this section used the implementation of the corruption detection option (CDO) to generate some statistics which indicate that it is worth a try to implement the draft. These measurements are not comprehensive; only the final measurements show the real benefit of the draft. The measurements in this chapter were done with a simple FTP file download from one notebook to the other notebook. Firstly it was tested with both notebooks using a prism card, and secondly using the prism card only at the receivers side and a g device at the senders side. There were problems at the beginning to be able to receive the corrupt frames because the WLAN card had an old firmware. The card disconnected from the other station after receiving some corrupt frames. By flashing [42] the primary firmware from version to and the secondary from to 1.8.4, there was no problem with the frame reception anymore. As already mentioned in the previous chapters, no encryption could be enabled, only ad-hoc mode was used and link layer retransmission was switched off. All the measurements were done at 11 Mbit/s, this means the b devices the prism cards operated at maximum speed, while the g devices were left to automatic mode, which then resulted in a speed reduction to 11 Mbit/s. Actually, the g devices operated in b mode. The packet count was done by writing tagged messages to the kernel message buffer and by saving this special tagged messages in a log file in user space. To be able to do this, the metalog system logger [48] was used, which can be configured to filter messages by the facility which prints the message, and also by regular expressions. The configuration is very simple and the system logger can be run in synchronized mode, which means that the kernel messages are instantly written to the user space without buffering Indoor measurements Most of the measurements were made indoors, to try to find a distance from the sender to the receiver where the transfer rate slows down notably without succumbing. The first tests were made with both notebooks using a prism card. The tests showed very poor results. Only few corrupt packets with intact TCP header were caught, while hundreds of thousands of packets were sent. Different test setups were tried, as shown in the next sections. However using a g device as a sender and the prism card as a receiver yielded much better results.

62 52 Measurements Both notebooks on the same floor In a first test the measurements were carried out with both notebooks on the same floor at a distance of around 8 meters, with a brick wall and some plants in between. Next to the notebook, which acted as a server, was an AP without associated devices. This means that no other interference other than some SSID broadcasting was generated. Figure 4.1 shows the setup. Figure 4.1: The test setup for measurements with the notebooks on the same floor With this setup both notebooks used the prism card, and the result was quite poor. Table 4.1 shows the amount of transmitted, completely received, lost and corrupt packets, and corrupt packets with intact TCP header. The lost packets are calculated as the difference between the received packets and the sent packets. Packets transmitted Packets received Packets lost Packets corrupt 8 Packets corrupt - TCP header intact 2 Table 4.1: The statistics for the measurement on the same floor It can be seen that the outcome is much worse. This result shows that trying an implementation of the draft at that point may not have been worthwhile. The percentage of the usable packets in all sent packets is % which is extremely low.

63 4.3 Preliminary measurements 53 Sender on the upper floor and receiver on the lower floor The next test was done by leaving the server notebook at the same place, but by putting the receiver on the lower floor in different rooms. Again only prism cards were used. Figure 4.2 shows one of the setups. As the receiver was placed in different rooms, the signal passed not only through the floor, but passed also through walls. Figure 4.2: The test setup with one notebook at the upper floor and the other at the lower floor Almost all results were poor. No usable corrupt packets were intercepted for the first two setups and just one corrupt packet with intact TCP header was intercepted for the last setup. With these results corruption notifications seem to be absolutely useless. Figure 4.2 shows one of the setups. Tables 4.2, 4.3 and 4.4 show the results of the measurements. Packets transmitted Packets received Packets lost Packets corrupt 47 Packets corrupt - TCP header intact 0 Table 4.2: Receiver downstairs in the living room (see fig. 4.2)

64 54 Measurements Packets transmitted Packets received Packets lost Packets corrupt 18 Packets corrupt - TCP header intact 0 Table 4.3: Receiver downstairs in the kitchen (see fig. 4.2) The number of sent and received packets was similar for each measurement. From this it can be deduced that the real transfer rate was also almost constant, even if it had not been recorded. The distance between the stations was 10 meters at maximum. Trying to increase the distance resulted in an extremely poor signal and disconnection of the stations from each other. Packets transmitted Packets received Packets lost Packets corrupt 26 Packets corrupt - TCP header intact 1 Table 4.4: Receiver downstairs in front of the entrance (see fig. 4.2)

65 4.3 Preliminary measurements 55 Both notebooks on the same floor using one g card The next experiment was indoors again but in a different location. The first part of the experiment was done with the prism cards, the second part with the g card at the sender side. The setup was the same for both measurements (as shown in figure 4.3) but the results are completely different. Table 4.5 shows the results of the prism-to-prism connection; table 4.6 shows the results of the second connection. Since the useful packets on the prismto-prism connection were zero, no significant comparison for the two measurements can be done. Figure 4.3: Test setup with both notebooks on the same floor, but through two walls In the connection between the two prism cards on a good quality link (having transfer rates of 250 Kbyte/s) there was almost no packet loss, and less than 10 corrupt packets were received. However with the g card as sender, higher transfer rates were achieved but it resulted in higher packet loss. It is another important detail that with the two prism cards the transfer rate in this test setup was less than 50 Kbyte/s, but with the g card the transfer rate was about Kbyte/s. This difference was caused by the different transmission power of the cards. Packets transmitted Packets received Packets lost Packets corrupt 364 Packets corrupt - TCP header intact 0 Table 4.5: Prism-to-prism connection

66 56 Measurements It was tried to record the whole data transfer at the sender side with wireshark, to be able to detect how many packet retransmissions occurred during the file transfer, but the program always run out of memory. This means that the number of retransmissions can only be guessed by the number of lost packets, which is still higher than the number of usable packets. Compared to the sent packets in this measurement the usable packets are %. This is a much higher percentage than the percentage of the former measurement with the two prism cards. Even if the numbers are really low, it was worth a try to implement the draft and to examine the performance that can be obtained with it. Packets transmitted Packets received Packets lost Packets corrupt Packets corrupt - TCP header intact 1326 Table 4.6: g to prism connection Figure 4.4: Outdoor measurements within a line of sight

67 4.3 Preliminary measurements Outdoor measurements The outdoor measurements were done with a line of sight connection on a distance where the transfer rate for the prism-to-prism connection fell below 20 Kbyte/s (Figure 4.4). The results are slightly better than the indoor measurements on the prism-to-prism connection, which is due to the fact that it was easier to find the extreme margin of a wireless connection than indoors, where the connection instead is abruptly lost as the distance is increased. So the number of lost packets is higher than for the indoor measurements. It is important to note that the number of corrupt packets having the TCP header intact has also increased. Again the results for the g to prism connection show a huge difference. Tables 4.7 and 4.8 show the results for the outdoor measurements. It is important to look at the ratio of usable to lost packets: We get a ratio of 0.046% for the prism to prism link, and 0.037% for the link with the g card. This shows that as far as the ratio is concerned, the outcome with a g card is even worse than with the prism card. As the number of usable corrupt packets is not zero, this is again an indicator that it is worth a try to implement the draft. Packets transmitted Packets received Packets lost Packets corrupt 14 Packets corrupt - TCP header intact 12 Table 4.7: Prism-to-prism connection outdoors Packets transmitted Packets received Packets lost Packets corrupt 654 Packets corrupt - TCP header intact 153 Table 4.8: g to prism connection outdoors It is interesting to note that table 4.8 shows lots of lost packets, but less corrupt packets than the indoor g session. On the other hand, the ratio of usable and unusable corrupt

68 58 Measurements packets is much higher. As explained before this results from a better signal strength, which is due to the different power mode of the cards. 4.4 Final measurements The preliminary measurements indicated, that it was worthwhile to implement the draft. The sender just needed to use a g card instead of the prism card, so many of corrupt packets with intact header arrived. This section discusses the performance measurements over lossy links with the use of the implementation described in chapter 3. Different setups, outdoor and indoor, were tried. The setups always used a g card as a sender and a prism card as a receiver. The performance tests were done using the iperf tool, in usual surroundings with all the typical radio noise from cellphones and similar gadgets. As every measurement lasted 20 minutes (1200 seconds), the same setup could lead to different interferences and show odd results. The figures in this chapter show the transfer rate measured by iperf every second. The amplitude of the graphs shows how stable the transfer rate was. A smaller amplitude means less oscillations of the transfer rate and thus a stable throughput. The frequency of rate changes gives an idea of how good a connection worked. This means if a graph shows a long part with almost the same Kbit values above the average (marked by the green line), it can be deduced that packet loss is low and constant. The lower line in the graphs shows the signal strength of the connection in percent 1. It gives a good overview of how the transfer rate changes according to the signal strength. Every measurement was done 4 times, once without the implementation and SACK, once with SACK and once with the implementation without SACK and with SACK. In each case the linux kernel NewReno congestion control was used. Measurements with two notebooks as a sender were also tried. The proportions of sent and received packets were not different from the proportions found in the measurements with a single sender. Since a thorough study of scenarios with multiple senders is beyond the scope of this thesis, no further such results are reported here. It was also considered to do a measurement with the receiving notebook moving around, but as it would be impossible to repeat the same measurement if not using a too extensive setup, the idea was discarded too. 1 The signal strength percentage was multiplied with 100 to fit better into the graphs

69 4.4 Final measurements Indoor measurements Two indoor measurements were done just like in the preliminary measurements. The first with both notebooks on the same floor and the second with the notebooks on a different floor. As the preliminary indoor measurements id not yield good results, it was tried on a different location, with two thick stone walls in between on the same floor, which resulted in a much lower signal strength and transfer rate. Same floor Figure 4.5 shows the test setup. It shows that there might have been much interference generated by people staying in the living room with cellphones in their pockets and watching TV as the direct connection between the two notebooks traversed the TV. At the kitchens side there might have been some interference generated by humidity, as the connection traversed the refrigerator. Figure 4.5: The test setup for the measurements on the same floor The results of the measurement show some really different graphs. This measurement was the one with the weakest signal strength. It means that the approximate maximum distance of the two notebooks was reached. The transfer rate hit 0 Kbit many times, which shows that the packet loss was high. It is interesting to see that, as could be expected, in all figures the transfer rate exactly follows the signal strength. The signal strength in figures 4.6, 4.7 and 4.8 is low but remains mostly constant, while in figure 4.9 it gets worse towards the end of the measurement.

70 60 Measurements Figure 4.6: Indoor measurement with both notebooks on the same floor. Only the corruption notification option were used Packets transmitted Packets received Packets lost Packets corrupt 632 Packets corrupt - TCP header intact 16 Table 4.9: The packet statistics for measurements on the same floor with CNO enabled It can be seen on graphs 4.6 and 4.7 that the average transfer rate was much higher with CNO than without, but a look at tables 4.9 and 4.10 tells that it is improbable that the higher transfer rate derives from the use of the corruption detection option, as there were only 16 and 14 usable packets in the two measurements. It is more probable that the difference is due to the better signal strength in the two measurements.

71 4.4 Final measurements 61 Figure 4.7: Indoor measurement with both notebooks on the same floor. The corruption notification option and selective acknowledgments were used Packets transmitted Packets received Packets lost Packets corrupt 178 Packets corrupt - TCP header intact 14 Table 4.10: enabled The packet statistics for measurements on the same floor with CNO and SACK With an average transfer rate of 2500 Kbit/s a sending rate of approximately 200 full sized segments 2 per second can be achieved. If we add the packets which may be lost per seconds from a calculation of one of the tables, we arrive at additional packets lost. All in all there might be about 220 packets in flight per second. 3 Now even if 14 or 16 packets are reported as corrupt within that second, no special improvement on the transfer rate would be noticeable, as for a second with low packet loss the whole number of packets arrived properly would already mask the bad effect of the loss, 2 A full sized segment is calculated with 1440 Bytes, and the result is rounded. 3 This number is merely a rough guess based on sample observations of partial wireshark captures

72 62 Measurements Figure 4.8: Indoor measurements with both notebooks on the same floor. The standard linux kernel without selective acknowledgments was used Packets transmitted Packets received Packets lost Packets corrupt 1435 Packets corrupt - TCP header intact 0 Table 4.11: The packet statistics for measurements on the same floor with NewReno only while for a second with lots of packet loss, the high number of losses against the number of corrupt packets would mask any improvement. Tables 4.11 and 4.12 show the high packet loss on the measurements without the corruption detection option. It is interesting to note that even SACK does not increase the transfer rate at all as seen in figures 4.9 and 4.7.

73 4.4 Final measurements 63 Figure 4.9: Indoor measurement with both notebooks on the same floor. The standard linux kernel stack with selective acknowledgments was used Packets transmitted Packets received Packets lost Packets corrupt 722 Packets corrupt - TCP header intact 0 Table 4.12: The packet statistics for measurements on the same floor with NewReno and SACK

74 64 Measurements Different floors The test setup for the measurement on different floors was similar to the one in the preliminary measurements, as shown in figure The two notebooks were situated diagonally below each other, with no wall in between, just the floor. Again there could be interferences from any kind of gadget used by people around the receiver. Figure 4.10: Test setup for measurements from one floor to the other The almost straight line of the signal strength in all graphs shows that those expected interferences were not present. Figures 4.11, 4.12, 4.13 and 4.14 all show the same behavior: There is no noticeable difference among the measurements done with CNO, SACK, both or none. The average transfer rate is about 5200 Kbit/s which is at the physical limit of an b WLAN, while the theoretical limit is set to 11 Mbit/s. This result shows that it would never have been realistic to perform the tests at 1 Mbit/s, only to be able to obtain more relative performance improvements due to less lost but more corrupt packets.

75 4.4 Final measurements 65 Figure 4.11: Notebooks on different floors using the corruption notification option Packets transmitted Packets received Packets lost 5017 Packets corrupt 0 Packets corrupt - TCP header intact 0 Table 4.13: The packet statistics for measurements on different floors with corruption notification options only Like in the tests on the same floor, the packet loss does not differ significantly among the measurements, and no benefit from the corruption notification options can be inferred, which is obvious if we look at tables 4.13 and Even with the use of the normal linux kernel, there were no corrupt frames in the measurements (see tables 4.15 and 4.16).

76 66 Measurements Figure 4.12: Notebooks on different floors using the corruption notification option and selective acknowledgments Packets transmitted Packets received Packets lost 5070 Packets corrupt 0 Packets corrupt - TCP header intact 0 Table 4.14: The packet statistics for measurements on different floors with CNO and SACK This absence of corrupt packets was due to the good signal strength in all measurements. Such results were expected, as the floor is made of mainly wooden transoms mixed with iron transoms and filled with cement. An iron grill should also be present in between to hold the cement and the transoms together. This construct should usually attenuate radio waves, but apparently not that severely. The tables with the results of the measurements with plain NewReno and NewReno with SACK obviously show no corrupt packets with an intact TCP header as the corruption detection option and thus the TCP header checksum were not used. Anyway corrupt packets could be received, as this addition only affects the driver and not any TCP related code. This means that for the measurements with the standard kernel, the changed driver was used.

77 4.4 Final measurements 67 Figure 4.13: Notebooks on different floors using the standard linux kernel without selective acknowledgments Packets transmitted Packets received Packets lost 8014 Packets corrupt 0 Packets corrupt - TCP header intact 0 Table 4.15: The packet statistics for measurements on different floors with NewReno only The indoor measurements indicate that the corruption notification options are not very useful. The problem in indoor setups is that the signal is either absolutely good or disappears completely, and this happens within a distance of few steps. Obviously there are many variables which have to be taken into account for an indoor setup. Interferences may be generated by any kind of gadget which emits radio waves (indoors one finds more than one might expect), attenuations by walls and the water pipes in the walls and so on. The outdoor measurements should work better but the following section shows that these expectations were not fulfilled.

78 68 Measurements Figure 4.14: Notebooks on different floors using the standard linux kernel and selective acknowledgments Packets transmitted Packets received Packets lost 8087 Packets corrupt 0 Packets corrupt - TCP header intact 0 Table 4.16: The packet statistics for measurements on different floors with NewReno and SACK Outdoor measurements The outdoor measurements were split into two measurements. One was a complete outdoor measurement, which means that both notebooks were set up outdoor in a line of sight 4 and the other with one notebook in the building and one notebook outdoors. During the mesaurements the weather was cloudy with irregular wind which could have affected the results. 4 As there was no possibility to get on top of the buildings around the test area, a line of sight may still have some vegetation in between which could produce interferences.

79 4.4 Final measurements 69 Line of sight Figure 4.15 shows the test setup for the outdoor measurement. The sending notebook was placed on the window of the third floor while the receiving notebook was placed in a line of sight in front of a nearby house at a distance of 70 meters. The interferences should have been low because nobody could cross the connection; the biggest interference could be produced by the tree in front of the sending station. Any interferences by cell phones or other wireless LAN hot spots may not be excluded as the test was conducted within an inhabited area. Figure 4.15: The setup for the outdoor - outdoor measurements.

80 70 Measurements Figure 4.16: Outdoor measurements with only corruption notification options enabled Packets transmitted Packets received Packets lost Packets corrupt 41 Packets corrupt - TCP header intact 5 Table 4.17: The packet statistics for the outdoor measurement with CNO only As expected the outdoor to outdoor measurements show relatively good transfer rates of around 5-6 Mbit/s. Interestingly there are no great differences again whether CNOs (figure 4.16), SACK (figure 4.17), both (figure 4.18) or none (figure 4.19) were used. The signal strength was about 20% for every measurement and also the average transfer rates were almost the same each time. Even sent and received packets and packet loss were almost the same as shown in tables 4.17, 4.18, 4.19 and 4.20.

81 4.4 Final measurements 71 Figure 4.17: Outdoor measurements with standard NewReno and SACK enabled Packets transmitted Packets received Packets lost Packets corrupt 23 Packets corrupt - TCP header intact 0 Table 4.18: The packet statistics for the outdoor measurement with standard kernel and SACK Only few lost packets were recorded compared to the sent and received packets. In such a case the corruption notification should get a better chance to operate efficiently as this depends on the ratio of lost and usable corrupt packets.

82 72 Measurements Figure 4.18: Outdoor measurements with CNO and SACK enabled Packets transmitted Packets received Packets lost Packets corrupt 28 Packets corrupt - TCP header intact 4 Table 4.19: The packet statistics for the outdoor measurement with CNO and SACK The tables of this outdoor measurement show that there was no possibility to experience any improvements of the transfer rate within seconds, when only 4 or 5 corrupt packets were present. It can be seen, that the number of usable corrupt packets is very low compared to the lost packets.

83 4.4 Final measurements 73 Figure 4.19: Outdoor measurements with nothing other than standard NewReno enabled Packets transmitted Packets received Packets lost Packets corrupt 32 Packets corrupt - TCP header intact 0 Table 4.20: The packet statistics for the outdoor measurement with standard NewReno only As explained before the outdoor measurements show a similar signal for each measurement just like the indoor measurements show with the notebooks on different floors. The greatest difference is in the stability of the signal. For the indoor case the signal was extremely stable at the same level, however for the outdoor case the signal fluctuates. It is interesting to see that the transfer rate is almost constant.

84 74 Measurements Indoor to outdoor The indoor to outdoor measurements were expected to yield similar results compared to the results of the outdoor measurements, as the sending notebook was just a floor below the floor in the previous setup but was behind a veranda (figure 4.20). As the graphs show this is not the case. The signal strength was much lower than for the outdoor measurements and so was the transfer rate. It is interesting to note that the combination of CNO and SACK (figure 4.22) which was expected to perform best, performed worst in these measurements even if the signal strength was very constant compared to the signal strength in the other measurements. Figure 4.20: The setup for the indoor-to-outdoor test

85 4.4 Final measurements 75 Figure 4.21: Measurement from indoors to outdoor with CNO only Packets transmitted Packets received Packets lost Packets corrupt 1475 Packets corrupt - TCP header intact 109 Table 4.21: The packet statistics for the indoor-to-outdoor measurement with CNO Figure 4.23 shows some difficulties to recover back to the previous transfer rate after a signal gap, which is a typical behavior expected from NewReno. Figures 4.21 and 4.24 both show some starting difficulties even if the signal strength was already at the same level again as for the rest of the measurement and also show almost the same behavior of the transfer rate. Tables 4.21, 4.22, 4.23 and 4.24 show the packet statistics for the measurements.

86 76 Measurements Figure 4.22: Measurement from indoors to outdoors with CNO and SACK Packets transmitted Packets received Packets lost Packets corrupt 2081 Packets corrupt - TCP header intact 93 Table 4.22: The packet statistics for the indoor-to-outdoor measurement with CNO and SACK The bad transfer rate and the high packet loss are generated mainly by the signal gap. On such a lost signal, which may also have brought the devices to disconnect for that time period, the packet loss is immense and a congestion window reduction obviously happened many times too. Therefore it can be seen that, even if the signal eventually arrived at the level before the gap, the transfer rate had some difficulties to reach the old average. Corruption notification should improve exactly such behavior, but as the number of usable corrupt packets was low compared to the number of lost packets, it is obvious that there can t be seen any improvement within a time period of a second.

87 4.4 Final measurements 77 Figure 4.23: Measurement from indoors to outdoors with NewReno only Packets transmitted Packets received Packets lost Packets corrupt 1277 Packets corrupt - TCP header intact 0 Table 4.23: The packet statistics for the indoor-to-outdoor measurement with NewReno only The tables of the indoor-to-outdoor measurement show the highest amount of usable corrupt packets compared to all other measurements, but there was also a high packet loss, and therefore the ratio between lost and usable corrupt packets is still bad: more precisely it is 0.13%, which is way too little to be able to experience any improvement.

88 78 Measurements Figure 4.24: Measurement from indoors to outdoors with standard kernel and SACK Packets transmitted Packets received Packets lost Packets corrupt 1479 Packets corrupt - TCP header intact 0 Table 4.24: SACK The packet statistics for the indoor-to-outdoor measurement with NewReno and A cause for this difference between the outdoors and the indoor-to-outdoor measurements could be caused by more interferences generated by the TV or cellphones in the living room, where the sending notebook was placed for the indoor-to-outdoor test. Another cause could be the AC adapter of the notebook, which in the indoor-to-outdoor measurement was placed near the wireless LAN card of the sending station. Even the vegetation or a change in the air humidity might have changed the link quality. There are too many variables which may affect the quality of a wireless link, so it is difficult to exactly identify an affecting variable.