1 Simulation of TCP for Orbiting Spacecraft Through the TDRS Satellite System Marco Duarte, Ken Fisher, Abdul Kabbani Rice University {duarte, krfisher, akabbani@rice.edu Abstract In recent years NASA engineers have designed an interface system that connects the Space Shuttle and Space Station communication system to the Internet on Earth. Such a system allows for the astronauts to use services like the World Wide Web and Electronic Mail. This application provides a scenario with unique latency and throughput parameters that the standard TCP/IP protocol was not designed for. We discuss the changes to the TCP protocol for acceptable performance in this scenario. Simulations of the behavior of the modified TCP protocol over the Space Station or Space Shuttle links are performed using ns 2. This paper describes the method and results of those simulations. I. INTRODUCTION In recent years NASA engineers have found a way to connect the Space Shuttle and Space Station communication system to the Internet. This was accomplished by first designing an interface card that looks like an Ethernet adaptor to the PC, but sends data signals compatible with the existing Ku- Band communication system already present on the Shuttle and Station. These communication systems were designed from 197 s and 198 s technology and were never designed for computer networking of any kind. Since the distance through the satellite relay link is on the order of 5, miles, the TCP protocol needed to be modified, particularly in the way that the congestion control mechanisms are implemented. Once this was accomplished, the astronauts were able to use TCP to surf the Internet and send Email anywhere on the Internet. The performance of the network link will be adversely affected by the increased propagation delay. The delay will be varied depending on the position of the craft in its orbit in relation to the geostationary relay satellite. The roundtrip delays will be on the order of 1-2 seconds. The RF signal must travel from the orbiter, which is at an altitude of 1 to 2 miles, to the relay satellite (located at an altitude of about 22,3 miles) then down to the ground station in White Sands, New Mexico. From there, the signal must be relayed to the NASA mission control center in Houston. Our simulation does not include the trip from the ground station to Houston, but only from the mobile orbiter through TDRS (Tracking and Data Relay Satellite) to the ground station. In order to provide greater connectivity during one complete orbit, the communication system requires east and west satellites and the implementation of handovers between the two. Each satellite must maintain line of sight to the ground station. Therefore, they are positioned approximately 12 to 13 degrees apart in geostationary orbits around the Equator. As a result, there is also a zone of exclusion on the opposite side of the Earth, since the range of these satellites will not cover the Equator completely. In our simulations we use ns [1] to determine a maximum theoretical data rate through the link based on a data rate of 2 Mbps and a packet size of 512 bytes. We observed a variation in this rate using TCP based on the position of the orbiter around its orbit and also measured the actual latency using CBR packets. We also observe the behavior during the east to west satellite handover and going into, during, and coming out of the zone of exclusion. The system is being used currently by NASA, and we expect that the obtained simulation data will be useful for the enhancement and development of communication systems for future Spacecraft. The paper is organized as follows. Section II describes our implementation of the scenario for the simulation. Section III describes the necessary changes made to the ns 2 simulator. Section IV presents our simulation results and Section V offers conclusions.
2 Fig. 1. Graphic 3-D depiction of the simulation scenario. II. IMPLEMENTATION In order to accomplish this simulation, we first developed a program to model the position of the nodes in xyz space. This program is a modification of a graphics program that was developed for a previous project. The program allows us to set up objects such as a sphere (representing the Earth) a circle (representing the orbit), and points in space (representing the relay satellites) and then draw a graphic of them like the one shown in Figure 1. The graphic shows the Earth and the two TDRS satellites with solid rays drawn to the ground station on the Z axis (actually used as the Y axis in ns). One satellite looks like it is at a higher altitude, but that is just an illusion created by the viewing reference point. The orbiter is positioned on its initial point on the X axis. The orbit circle cannot be seen in this picture because it is so close to the Earth (2 miles) compared to the altitude of the satellites (22,3 miles). At each increment the orbiter takes around its orbit, the program determines if each satellite is in line-of-sight. This is depicted on the graphic by drawing a solid line to the satellite if it s in lineoff-sight, and drawing a dotted line if it is not in line-of-sight. The program is run off-line and generates.tcl script commands that will place the node in a specific position at a simulation time t. The program will generate a file with commands in the following format: $ns at t1 "$node (3) setdest x1 y1 766.764" ;#LOS1:1 LOS2: These can be incorporated into the main.tcl script to simulate the orbit around the earth. The off-line program also sets flags to indicate if the orbiter has line of sight to the relay satellites or not. This is printed as a comment with LOS1:1 indicating that there is line-of-sight to satellite 1 and LOS2: indicating that there is not line-ofsight to satellite 2. The number 766.764 is the velocity of the orbiter in meters per second. This corresponds to a velocity of 17,5 miles/hr, which is the approximate velocity of the obiter for a 9 minute orbit. We set the increment to be 1 minute, so we generate 9 setdest commands per orbit. We also modified the ns-2 source code to allow longer propagation delays and longer ranges. This required modifications to both the TCP protocol implementation and the physical wireless channel implementation. In our problem, we assume that the communication link is in range as long as there is line-of-sight. We also assume no interference between the relaying satellites. We then implement the positioning of objects in wireless scenarios in order to simulate the effect of the line-of-sight loss between the orbiter and the satellite due to the Earth. For our case, the obstruction object type is a sphere. We also implement the handoff between relaying satellites manually using a static routine table. The routing is changed based on the simulation time. These times are determined by the LOS1 and LOS2 flags that are calculated on the off-line program. III. SIMULATOR MODIFICATIONS In this section we describe the changes made to the ns simulator for our scenario. These changes can be grouped into changes to the routing protocol, changes to the physical layer, changes to the MAC protocol, changes to the TCP protocol A. Routing Protocol For our project, we assumed the satellites to be using DSR only for the simplicity of using DSR for implementing a static routing protocol as far as our simulations are concerned. We had the same functions inside the DSRAgent class unchanged, with the exception of the rcv function. This is the function that handles the static routing scheme that we have for our project. This was implemented
3 by introducing a couple of nested if statements that can handle the different combinations of the current node, the destination node, and the reachability status (in terms of time) when this might be an involved factor. The reachability status was implemented in terms of time only because of the path predictability for our shuttle s motion, i.e., because of its periodicity. Just before proceeding, we are going to define the node numbers that we used in our function: Node : Earth Station in White Sands, New Mexico Node; Node 1: TDRS Satellite West Node; Node 2: TDRS Satellite East; Node 3: Orbiter. We saw that nodes, 1, and 2 are supposed to be able to communicate with each other always no matter where our orbiter s position is (i.e., irrespective of time). Note that we don t need to have all what we had in our actual code below for other nonexistent paths as far as our project is concerned (for example between nodes and 2 or 1 and 2), nevertheless, we had that just for completeness. The parts in our code corresponding to what has been mentioned are shown below where iph->daddr() is the address of the destination node, and cmh->next hop() is the node to which we want or are supposed to forward to: For node if (iph->daddr() == 1){ cmh->next hop() = 1; else if(iph->daddr() == 2){ cmh->next hop() = 2; For node 1 if (iph->daddr() == ){ cmh->next hop() = ; else if(iph->daddr() == 2){ cmh->next hop() = ; For node 2 if (iph->daddr() == ){ cmh->next hop() = ; else if(iph->daddr() == 1){ cmh->next hop() = ; For node 3 // All routes are time-dependent On the other hand, and when the time factor is involved (i.e., when the path between either one of the satellites and the shuttle is involved), we had the periodicity of the connections (in terms of the handover procedures and the blockage) by taking our time modulo the orbit period (equal to 54 seconds). Whenever one node was not supposed to be in reach of another one while carrying a packet destined to it, that node would drop that packet. Using the line of sight calculations from the offline program described earlier, we found out that during the intervals [, 87] and [351, 54] the orbiter would be in line of sight with TDRS Satellite East; during the interval [171, 447] the orbiter would be in line of sight with TDRS Satellite West; and therefore during the interval [87, 171] we will have the zone of exclusion. Therefore, the interval [351,447] is the overlap period when the orbiter has simultaneous line of sight to both satellites. We define in our source code for the routing protocol the times when line of sight begins and ends: #define SAT1 START 171 #define SAT1 STOP 447 #define SAT2 START 351 #define SAT2 STOP 87 On the other hand, we can have the handover from TDRS Satellite West to Satellite East at any time during the overlap period ([351,447]). We can set up the TDRS Satellite West to acquire the signal when the orbiter comes out of the zone of exclusion at any time during the zone of exclusion period ([87, 171]). We set up the handovers to occur at the middle of each time range. This assures optimal throughput behavior as described in Section IV. The code that defines the handover time is as shown below: hand21 = ((int)sat2 STOP + (int)sat1 START) / 2; hand12 = ((int)sat1 STOP + (int)sat2 START) / 2; The source code that implements our static routing tables is shown in Figure 2 where iph->daddr() and cmh->next hop are the same as defined before. B. Physical Layer We assume that as long as line of sight exists between two nodes, the signal will be received
4 Fig. 2. Source code for the static routing table.
5 correctly; the satellites are positioned so that they always have line of sight to the earth station, while the line of sight from the orbiter to the satellites is dependent to its position in the orbit. In addition, we assume that the nodes will always send packets at sufficient high power for the destination to detect and decode the packet successfully. To account for the huge distances involved, we set the receive power threshold to zero to ensure that the packet is accepted at the receiver despite the transmission power levels used in the simulator. C. MAC Protocol In our scenario, due to its physical configuration, the connections between nodes are point-to-point, and thus no channel access arbitration is required. Thus, we used the generic MAC layer available in the simulator for our case, which passes the packets to the next layer with no processing. This allows for all senders to have continuous access to the transmission channel without any need for access control. D. TCP Protocol Based on the transmission distances involved, our round trip times vary from.52 sec to.62 sec. This special characteristic of our scenario led us to consider changes in some of the parameters of a TCP connection; specifically, the parameters relevant to the congestion control mechanism (RTT estimate, congestion window size, etc.). We expected that setting the initial RTT estimate to our projected value would improve throughput at the beginning of the simulation due to the exponential weighted averaging algorithm to estimate RTT in TCP; we found that it did not affect the performance of the scenario since the first calculation of the RTT is drawn from the actual round trip time delay for the synchronization packet sent at the initialization of the TCP connection. Another parameter that affects the throughput and is sensitive to the round trip time of the communication is the congestion window size. The optimal maximum size is equal to the amount of packets that can be in flight during a round trip time; this will allow for full utilization of the channel bandwidth while an acknowledgement is received by the sender. In the NS simulator, the original value of this parameter was very low for our scenario (4 packets). We estimate the number of packets N that can be sent during a round trip time as follows: N = RT T BW/P SIZE =.52 2 1 6 /(8 512) 254 (1) The throughput then will not be constrained by the maximum congestion window size if its value is set larger than 254; we set it to 4 for our simulations. We proceed to consider the effect of the initial congestion window size; we expected the throughput to increase if the initial congestion window size is set to the maximum instead of the default value of 1. We then considered that since the window size is doubled every round trip time, the calculated maximum value for the congestion window should be reached in approximately 8 RT T = 4.16 seconds. Thus, the throughput will improve only for a very small section of the orbit period. IV. RESULTS We perform simulations for a FTP application that originates at the orbiter and ends at the earth station. Our simulator follows all the modifications described previously; our main design space considerations are the timing of the handover between the two relaying satellites. We first evaluate the performance of the TCP protocol for this application using all the enhancements described earlier, and setting the handover times to the middle point of the permissible ranges. We evaluate the throughput of the connection in packets per second; the results are shown in Figure 3. We observe that the throughput per second increases up to 225 packets per second, which is equivalent to 9 kbps. We observe no throughput period during the zone of exclusion period (87-171 seconds), which is expected. However, we observe a limited throughput phenomenon right after the acquisition of line of sight for Satellite 1 (at 171 seconds) and after the satellite handoff (at 399 seconds). We explored several different possible reasons for this behavior. To test the correctness of the physical and link layers, we simulated a CBR connection over our same scenario to test packet reachability and propagation delay. The round trip time results are shown in Figure 4; the throughput is shown in Figure 5.
6 3 Congestion Window Size, Packets Throughput, packets per second 25 2 15 1 5 2 15 1 5 Fig. 3. 25 5 1 15 2 25 3 35 4 45 5 1 2 3 4 5 Fig. 6. Congestion window of TCP connection for standard settings. Throughput for TCP connection for standard settings..56 Round Trip.55.54.53.52.51.5.49 Fig. 4. 1 2 3 4 5 6 Round Trip Time for CBR traffic for standard settings. 45 Throughput, Packets per Second 4 35 3 25 2 15 1 5 Fig. 5. 1 2 3 4 5 Throughput for CBR traffic for standard settings. 6 These figures show an expected behavior for the latency of packet transmission, which varies according to the position of the active satellite. We see that the throughput to the satellite is stable, which shows that the characteristics of the channel are not dependent on the position of the satellite. This allows us to rule out simulation bugs in the physical and link layers. We also notice that the reduction in throughput occurs only during the period where the distance from the orbiter to the active satellite and thus, the RTT is decreasing. At the approximate point of minimum latency the expected throughput is restored, and the throughput oscillates continuously between 1 and 225 packets per second. After examining the simulator traces, we see that packet drops occur only during the zone of exclusion; thus, we assume that the reduction in rate is due to congestion window resetting due to duplicate ACKs or retransmission timeout. Figure 6 shows the size of the congestion window and Figure 7 shows the number of duplicate ACKs received by the orbiter as a function of time. From this data we see that the throughput falls repeatedly during the simulation due to the duplicate ACKs caused when a packet is sent out of order. We also see a frequent occurrence of three duplicate ACKs during the periods of low throughput. The reason behind this behavior is unknown to us at this point. We conjecture that out of order reception of packets has a large effect on the congestion control mechanism due to the large latency of a round trip time.
7 35 35 Number of consecutive duplicate ACKs 3 25 2 15 1 5 Throughput, packets per second 3 25 2 15 1 5 1 2 3 4 5 6 1 2 3 4 5 6 Fig. 7. Number of duplicate ACKs for standard settings. Fig. 9. Throughput for TCP connection for maximum handoff times. Throughput, packets per second 35 3 25 2 15 1 5 1 2 3 4 5 6 Round Trip.56.55.54.53.52.51.5.49 1 2 3 4 5 6 Fig. 8. Throughput for TCP connection for minimum handoff times. Fig. 1. Round trip time for minimum handoff times. We also evaluate the effect of the satellite handover time on the throughput. We can set one of the handovers to any time during the zone of exclusion; the second one can be set to any time during the period when the orbiter has line of sight with both relaying satellites. Figure 8 shows the throughput when the handovers times are set to the minimum possible values, while Figure 9 shows the throughput when the handover times are set to the maximum possible values. We expect both of these schemes to perform worse than the selected scheme due to the sudden changes in round trip time. The round trip time for the second scheme is shown in Figure 1; the change in round trip time during handoff is propor- tional to the change of distance between the orbiter and the satellites. The problem is critical when the round trip time increases suddenly, triggering several timeouts and retransmissions. Furthermore, a new problem arises if the handover time is set to the maximum for the second handover. The packets that are sent to the satellite by the earth station close to the end of line of sight may be received by it after the line of sight is not present, and they will thus be dropped, causing reduction in the throughput as seen in Figure 9. This leads to our reasoning of performing handover at the middle of the feasible interval, where the distance to each of the satellites from the orbiter will be the same.
8 V. CONCLUSIONS AND FUTURE WORK We have modeled the Internet connection of the Space Shuttle and Space Station using the ns 2 simulator, and studied its behavior for typical TCP applications. Some unexpected behavior was observed regarding out-of-order transmission of packets, but theoretical figures for the performance of the system were obtained. We have not been able to determine whether this behavior is inherent to the behavior of TCP for our scenario or is introduced by errors in the modified simulator. If the former is the case, we expect that a mechanism at the TCP protocol that would delay acknowledgements for some packets that are delivered out of order would allow us to reach the optimal throughput level throughout the simulation period. We have concluded that handoff should occur at the middle point of the overlap of line of sights to cause the smallest disturbance possible to the congestion control mechanism state. We have also concluded that a large part of the loss in potential performance is due to the effect of the congestion control mechanism during the zone of exclusion, in which no communication is possible. Since the orbit cycle of the orbiter is ninety minutes, applications that would require continuous connectivity for a larger interval will suffer. We expect that mechanisms similar to [2] that set the TCP connection to stand-by state during periods without connectivity, which was proposed for ad-hoc wireless networks, should improve the performance in this scenario as well. We remain interested in determining the reasons behind the erratic behavior observed in the simulations, and would welcome suggestions regarding these issues. VI. ACKNOWLEDGEMENTS We thank Shu Du and Khoa To for their invaluable suggestions regarding the implementation of the handoff schemes. REFERENCES [1] (24) The network simulator - ns-2. [Online]. Available: http://www.isi.edu/nsnam/ns/ [2] G. Holland and N. Vaidya, Analysis of TCP performance over mobile ad hoc networks, in Proceedings of the Fifth Annual International Conference on Mobile Computing and Networking (MOBICOM), Seattle, WA, August 1999.