Institutionen för datavetenskap Department of Computer and Information Science

Transcription

1 Institutionen för datavetenskap Department of Computer and Information Science Final thesis Implementation of a Delay-Tolerant Routing Protocol in the Network Simulator NS-3 by Fredrik Herbertsson LIU-IDA/LITH-EX-A--10/046--SE Linköpings universitet SE Linköping, Sweden Linköpings universitet Linköping

2

3 Linköping University Department of Computer and Information Science Final Thesis Implementation of a Delay-Tolerant Routing Protocol in the Network Simulator NS-3 by Fredrik Herbertsson LIU-IDA/LITH-EX-A--10/046--SE Supervisor: Mikael Asplund, IDA, Linköpings universitet Examiner: Simin Nadjm-Tehrani, IDA, Linköpings universitet

4

5 Avdelning, Institution Division, Department Division of Software and Systems Department of Computer and Information Sience Linköpings universitet SE Linköping, Sweden Datum Date Språk Language Svenska/Swedish Engelska/English Rapporttyp Report category Licentiatavhandling Examensarbete C-uppsats D-uppsats Övrig rapport ISBN ISRN LIU-IDA/LITH-EX-A--10/046--SE Serietitel och serienummer Title of series, numbering ISSN URL för elektronisk version Titel Title Implementation of a Delay-Tolerant Routing Protocol in the Network Simulator NS-3 Författare Author Fredrik Herbertsson Sammanfattning Abstract Small mobile devices with networking capabilities are becoming more and more readily available and used. These devices can be used to form mobile ad hoc networks to communicate, where no infrastructure for network communication exist or where it has been destroyed or is overloaded e.g. in a natural disaster such as a hurricane. Such networks are almost never fully connected, and are part of the category of delay/disruption-tolerant networks (DTN) and suffer from limited resources e.g. bandwidth, storage and limited energy supply. The Opportunistic DTN Routing With Window-aware Adaptive Replication (ORWAR) is a delaytolerant protocol intended to be used in disaster relief efforts or emergency operations were a DTN could be a fast way to establish communication. In these kinds of scenarios high success rate together with efficient usage of the networks resources are critical to the success of such operations. ORWAR has been implemented and simulated on a high-level simulator, with promising results. To make a better assessment about what ORWARs performance would be in a real world network, more realistic and detailed simulations are needed. This Master's Thesis describes the design, implementation and evaluation of ORWAR in the network simulator ns-3, which simulates networks down to physical layer. The contributions of this thesis is a extension to ns-3 giving it an framework with support for the bundle protocol and delay-tolerant routing protocols and an evaluation of the ORWAR performance using more detailed simulations. The simulations represent a city scenario in down-town Helsinki city, Finland, were pedestrians, cars and trams form a network to communicate. The simulations with a higher level of detail has added to insight about the protocol. The obtained results showed that the high-level simulation may be overly optimistic and hides implementation details. On the other hand, some assumptions were found to be too pessimistic. For example we have shown that ORWAR actually performs better than the high level simulations, with regard to partial transmissions and that the high-level simulations have rather optimistic assumptions regarding the latency. Nyckelord Keywords delay-tolerant network, DTN, routing protocol, ORWAR, network simulator, NS-3

6

7 Abstract Small mobile devices with networking capabilities are becoming more and more readily available and used. These devices can be used to form mobile ad hoc networks to communicate, where no infrastructure for network communication exist or where it has been destroyed or is overloaded e.g. in a natural disaster such as a hurricane. Such networks are almost never fully connected, and are part of the category of delay/disruption-tolerant networks (DTN) and suffer from limited resources e.g. bandwidth, storage and limited energy supply. The Opportunistic DTN Routing With Window-aware Adaptive Replication (ORWAR) is a delay-tolerant protocol intended to be used in disaster relief efforts or emergency operations were a DTN could be a fast way to establish communication. In these kinds of scenarios high success rate together with efficient usage of the networks resources are critical to the success of such operations. ORWAR has been implemented and simulated on a high-level simulator, with promising results. To make a better assessment about what ORWARs performance would be in a real world network, more realistic and detailed simulations are needed. This Master s Thesis describes the design, implementation and evaluation of ORWAR in the network simulator ns-3, which simulates networks down to physical layer. The contributions of this thesis is a extension to ns-3 giving it an framework with support for the bundle protocol and delay-tolerant routing protocols and an evaluation of the ORWAR performance using more detailed simulations. The simulations represent a city scenario in down-town Helsinki city, Finland, were pedestrians, cars and trams form a network to communicate. The simulations with a higher level of detail has added to insight about the protocol. The obtained results showed that the high-level simulation may be overly optimistic and hides implementation details. On the other hand, some assumptions were found to be too pessimistic. For example we have shown that ORWAR actually performs better than the high level simulations, with regard to partial transmissions and that the high-level simulations have rather optimistic assumptions regarding the latency. The performance studies on the protocol follow the general trends shown in the high-level simulations although with lower but more realistic results. Moreover, the high level implementation has hidden the fact that ORWARs signalling messages needs a more careful implementation. vii

8

9 Contents 1 Introduction Background Problem Description Goals Approach Limitations Contributions Document Structure Chapter Overview Reading Instructions Background Delay-Tolerant Networks Routing in Delay-Tolerant Networks Opportunistic DTN routing with Window-aware Adaptive Replication Message differentiation Contact Window Estimation Adaptive Replication The Algorithm The Delay-Tolerant Network Architecture Naming and Addressing Bundles The ns-3 network simulator Design Architecture Detailed Design Modularity and extendibility Compatibility with the Bundle Protocol specification Protocol Stack Implementation Message Helper Bundle - Variable size messages Bundle Manager General Description Registration Management ix

10 x Contents 4.4 Convergence Layer General Description Segmentation of bundles Queue management Neighbourhood Detection Agent General Description Hello Generation Link Manager General Description Link and Contact Router Design issues Router signalling Message Exchange and Replication Redundant Transfers General description Testing and Evaluation Testing Functional Tests Scalability Tests Evaluation Simulation recreation Simulation set-up Message size implication Energy implications Net workload implications Conclusions Results Lessons Learnt Future work Bibliography 53

11 List of Figures 1.1 A fully connected mobile ad hoc network in a urban scenario[6] An opportunistic delay-tolerant network in a city scenario[6] Demonstration of store carry forward[6] Illustration of the Contact Window, taken from [17, p. 6, Figure 3] Pseudo code and message exchange Illustration of the bundle layer overlay network Division of the bundle layer into its two sub layers Structure of the primary bundle block [18] Structure of extension blocks [18] ns-3 software architecture [14] ns-3 extended software architecture Architecture overview Overview class diagram of the final design Message Helper Class diagram of the Bundle implementation Implemented primary block Data Bundle Extension Block Class diagram of the bundle manager implementation Bundle Manager Class diagram of the convergence layer implementation The Convergence Layer Segment format Segment header Class diagram of the neighbourhood detection implementation The Neighbourhood Detection Agent Hello header Class diagram of the link management implementation Class diagram of the bundle routing implementation The layouts of the two types of messages used by ORWAR Bundle Type Field for router signalling and and typical message exchange The ORWAR Bundle Router Topologies used for functional testing Map of down town Helsinki, [7] xi

12 xii List of Figures 5.3 Size distrubution in standard message set S, taken from [17, p.6, Figure 4] Result for Delivery Ratio comparison Transmitted data size Results for Overhead comparison Results for Partialtransmissions comparison Results for Throughput comparison Results from Delivery Ratio comparison Results from Median Latency comparison Results from Accumulated Utility comparison

13 List of Tables 2.1 Initial message copies as an utility function Supported Functionality of the Bundle Protocol Type and response field description OrwarLink states Node types Simulation Parameters xiii

14

15 Chapter 1 Introduction This chapter describes the background to and purpose of this Master s Thesis project, which is submitted as a partial fulfilment for the degree of Master of Science in Computer Science and Engineering at Linköping University. It also gives a short overview of the report and reading instructions. 1.1 Background As Mobile devices with networking capabilities are becoming more readily available and used e.g. PDA:s, mobile phones and laptops, the concept of letting these devices form networks to communicate with each other have become a increasingly viable solution for applications where setting up a wired infrastructure might be too expensive, e.g. in remote areas or in developing countrys, or where using wired infrastructure is not a option e.g. in disaster-recovery scenarios where the existing infrastructure has been been destroyed due to a natural disaster such as earthquakes, hurricanes etc. Other possible applications could be social application deployed in urban scenarios. Such networks formed by autonomous mobile users are called mobile ad hoc network s (MANET), see Figure 1.1 for a example of such a network. Traditional MANET routing protocols are built with the assumption that the network is dense enough so that it is fully connected i.e. that there always exist a path between every node in the network or that links are down for a very short period of time. This assumptions do not hold in many MANET application, where the free movement of the nodes and the radio range and quality of the wireless links can lead to partitioning of the network, making these protocols unable to operate completely or with unacceptable performance. In addition to this, efficient usage of resources is important in wireless environment with mobile node since they typically have limitations in bandwidth, available storage memory and a limited energy supply i.e. battery capacity, making resources scarce. This adds additional problems on top of the ones introduced by partitions. These kind of networks where there most of the time no path between the sender and the destination exists are belonging to the general category of delay/disruption tolerant networks (DTN). New kinds of routing protocols are needed to be able to communicate in such networks efficiently. 1

16 2 Chapter 1. Introduction Figure 1.1: A fully connected mobile ad hoc network in a urban scenario[6] Opportunistic DTN Routing with Window-aware Adaptive Replication (OR- WAR) [17] is a resource efficient delay-tolerant routing protocol proposed by Gabriel Sandulescu and Simin Nadjm-Tehrani, for routing messages in such delay-tolerant networks. ORWAR is intended to be used in disaster relief efforts or emergency operations were a DTN could be a fast way to enable rescue workers to be able to communicate. In these kind of scenarios fast delivery of messages and high success rate together with efficient usage of the network resources are critical to the success of such operations. For ORWAR to be a good option as a routing protocol in such situations it must fulfil these criteria. 1.2 Problem Description To evaluate the performance of ORWAR and to verify that it worked as intended an implementation for the network simulator Opportunistic Networking Environment (ONE) [11, 10] was created by Gabriel. Results from simulations of the protocol showed promising results, and it was also shown that it compared well to other DTN-routing protocols [17]. The problem is that ONE simulates the network on a high level, that is on the higher layers of the OSI model and does not have support for the lower layers, most notably the physical and link layers. Hence it ignores issues such as interference between nodes and always sends with the specified transmission speed without errors. To make a better assessment of ORWARs performance, these issues and more must be taken in consideration as they will affect the

17 1.3. Goals 3 performance when it is deployed in the real world. A solution is to implement ORWAR on real devices and perform the testing in the real world. The disadvantage with this approach is that it is typically expensive and hard to do due to the mobility and its also hard to repeat the results and test different settings. An alternative solution that does not have these disadvantages is to implement ORWAR in a more detailed simulator that simulates the lower layers realistically, an approach that this thesis will use. An implementation of ORWAR will be created for the network simulator ns-3 [15]. ns-3 is an open source event driven network simulator, that provides models for all layers of the protocol stack, including models for the physical layer and simulates the network at packet level. The simulation scenarios from ONE will be recreated for ns-3 and ORWAR will be simulated again with ns-3. The result from the simulations will then be evaluated and compared with the results from simulations performed in ONE. 1.3 Goals This thesis has the following goals: Successfully implement the ORWAR routing algorithm in the network simulator ns-3. Recreate the simulation scenarios used in the ONE networks simulator in the ns-3 network simulator. Evaluate the implementation and compare the results with the simulations from the ONE network simulator. The main goal of this thesis is to evaluate and make visible the effects on the performance of the ORWAR algorithm when simulated in the more realistic environment of ns Approach The work for this thesis began with a literature study. First articles regarding MANET s[2] and DTN s[19, 9] were read to get a basic understanding of how these networks work. This was continued with studying of articles about routing in delay-tolerant networks and a detailed study of the documentation of ORWAR and its existing implementation in ONE. When a firm understanding of these concepts was formed, the documentation and code base for ns-3[13, 14, 16] was studied in detail to determined how the work of implementing ORWAR should proceed. After an initial analysis of the problem of implementing ORWAR in ns-3, it was determined that the work of implementing and simulating ORWAR could be divided into three main tasks, the adding of the functionality necessary to implement ORWAR in ns-3, setting up the simulation environment in ns-3 and evaluating the simulations results and comparing them to the results obtained from the ONE simulator.

18 4 Chapter 1. Introduction 1.5 Limitations In the original work for ORWAR[17] four different experiments where performed, only the first three experiments testing the message size implication, energy implication and network load implication has been recreated, excluding the fourth testing the performance of ORWAR with different mobility models. Most of the added modules to ns-3 are not full implementations, and only provide the necessary functionality to implement ORWAR leaving out extra functionalities that are not needed. For example the functionality for bundle layer signalling and fragmentation has been left out, subsection gives a complete list. 1.6 Contributions The major contribution of this thesis are, A extension to ns-3 giving it a framework with support for the bundle protocol and delay-tolerant routing protocols. Adding a basic module for the bundle protocol for ns-3 Adding a basic module for creating delay-tolerant routing protocols. Adding a module for neighbourhood detection. A working implementation of the ORWAR routing algorithm. An evaluation of the ORWAR routing algorithm simulated with different parameters in a city scenario. 1.7 Document Structure This section gives a overview to the chapters in this report and provides reading instructions Chapter Overview Chapter 1, Introduction This chapter introduces the background and goal of the Master s Thesis project and gives a outline of the report. Chapter 2, Background This chapter provides the necessary background information. Chapter 3, Design This chapter shows the design of the ns-3 implementation of ORWAR. Chapter 4, Implementation This chapter shows and discusses important design and implementation details. Chapter 5, Testing and Evaluation In this chapter the results from the simulation and evaluation of ORWAR is presented. Chapter 6, Conclusions In this chapter the results of the Master thesis are discussed and recommendations for future work are presented.

19 1.7. Document Structure Reading Instructions Chapter 1 gives a overview of the report and the presented work and should be read by all readers. People with knowledge of delay-tolerant networking and routing can skip most of chapter 2 since this chapter is meant to give a introduction to those topics. The most interesting part of chapter 2 is section 2.3 where the ORWAR algorithm is described. This section can be skipped if the reader already is familiar with ORWAR. Section 2.4 gives background for the bundle protocol architecture and section 2.5 describes the ns-3 network simulator. Chapter 3 shows the design of the implementation and chapter 4 describes the implementation of ORWAR and should be read by readers interested in the details of how the implementation has been structured. The results of the simulations is shown in chapter 5 and should be read by all readers. Chapter 6 discuss the obtained results and reflects upon them and suggets future work.

20 Chapter 2 Background This chapter starts with presenting the basics of mobile ad hoc networks and delay-tolerant networks and how routing is performed in these kinds of networks. It continues with presenting the ORWAR routing protocol and ends with describing the bundle architecture that facilitates the necessary functionality to operate in delay-tolerant networks and a short introduction to neighbourhood detection. 2.1 Delay-Tolerant Networks A MANET is a network composed of an autonomous collection of mobile users that communicate wirelessly, without the need to use any existing network infrastructure such as base stations, wires or fixed routers. The term ad hoc comes from that the fact that the network is formed dynamically as the need arises and mobile comes form the fact that the nodes can move freely. Since no fixed infrastructure are assumed to be used and the nodes making up the network are changing continuously as nodes enter and leave the network, MANETs need to be decentralised. All network activities are performed by the nodes themselves, e.g. detecting nodes to communicate with and routing and forwarding messages through the network to other nodes. This is a change from the more centralized and fixed composition of normal networks using infrastructure. Traditional MANETs operate under several assumptions, among those are: A path from the sender and the receiver can always be established. The end-to-end packet loss rate is very low. Propagation and end-to-end delays is low. These assumptions do not hold in many wireless applications, because the environment is uncontrolled and nodes are truly autonomous. Wireless connections between nodes are usually unstable with high error rates resulting in intermittent connectivity between nodes which in turn leads to partitioning of the network and in its turn to large delays. Examples of factors that have a role in the intermittency of contacts are, 6

21 2.1. Delay-Tolerant Networks 7 Sparseness There are to few nodes to cover the complete area of the network e.g. due to a large geographic area of the network compared to the number of nodes or their communication range. Mobility Nodes move in and out of each other s communication range when they move around or obstructions e.g. a house if in a city scenario, can come between two nodes preventing them from communicating with each other. Energy Nodes powering up and down e.g. due to energy management schemes to conserve energy making them appear and disappear in the network. Interference Interference between nodes increases the chance of packet loss and can make nodes unable to communicate with each other. Also efficient usage of resources is important in wireless environment with mobile node since they typically have limitations in bandwidth, available storage memory and a limited energy supply i.e. battery capacity making resources scarce. To cope with these kinds of conditions the assumptions for the wireless network needs to be relaxed. Networks were there most of the time no path between the sender and the destination exist are called intermittently connected mobile networks (ICMN) or intermittently connected MANET (IC-MANET). IC-MANETs belong to the general category of delay-tolerant networks (DTN). DTN s are meant to operate in stressed environments with large delays, intermittent connectivity and/or high error rates. Figure 2.1: An opportunistic delay-tolerant network in a city scenario[6] Figure 2.1 shows an example delay-tolerant network, that is partitioned due to some of the above mentioned factors. The communication range of the nodes

22 8 Chapter 2. Background are not long enough to connect all nodes to each other and some of the contact opportunities are unavailable preventing them from communication due to obstructing buildings or being powered down. A communication opportunity between two nodes are called a contact in DTN s. Since DTN represents a large group of networks several different kinds of contacts have been identified. A contact can be persistent i.e. the contact is always available, on-demand i.e. a contact can be initiated when needed, scheduled i.e. the contact and its characteristics is known in advance or opportunistic i.e. the contact is found unexpectedly. The focus here will be on DTNs with opportunistic contacts, called opportunistic DTN. 2.2 Routing in Delay-Tolerant Networks MANET routing protocols are built with the assumption that the network is dense enough so that it is fully connected i.e. that there always exists a path between every node in the network or that paths are down for a very short period of time. So traditional routing protocols for MANETs do not work well for DTNs, since they try to discover a full path before sending any data. If no path between the sender and the destination exists, these protocols will not succeed to send any data. To overcome the problem with intermittent connectivity and partitions in the network, DTN routing protocols utilize the mobility of the nodes and buffering of messages, this makes is possible for a node to carry a messages and in that way bridge partitions in the network. (a) (b) Figure 2.2: Demonstration of store carry forward[6] This model of routing is called store-carry-forward (SCF). When a message is created it gets stored at the source node, when a contact becomes available to a next-hop node the message is sent over this contact. The message gets stored at the new node until the next-hop in the path is found and so on, until

23 2.3. Opportunistic DTN routing with Window-aware Adaptive Replication 9 the destination is found. This results in a path from the source node to the destination without a guarantee for a contemporaneous path. This is illustrated in Figure 2.2, were a pedestrian forwards a message to a bus which then carries the message to its destination. Store-Carry-Forward is similar to the store-and-forward in standard Internet routers, but the difference here is that the messages is potentially stored a long period of time e.g. from several seconds, minutes or even days compared to the microseconds used to hold a message in Internet routers. Since no end-to-end paths can be assumed, the location of the destination node is unknown and contacts with other nodes are unpredictable, SCF routing is performed as a series of opportunistic local forwarding decisions, based on only the current contact information and potentially prediction on future contacts, were each such forwarding decision is made independently. Herein lies the main challenge for DTN routing schemes: which node should be chosen as the next-hop for a message to give it a better chance of delivery? Routing protocols can be differentiated by queue management, the amount of information available when making the forwarding decisions and the number of destinations a message can have. The focus here will be on unicast protocols, were a message only has a single node as it s destination. DTN routing protocols can be categorised in to two basic categories, singlecopy schemes and multi-copy schemes. The difference between the schemes is the number of copies of a message that may exist a the same time in the network. Single-copy schemes forwards a single copy of each message through the network. This is a resource efficient method, but it suffers from long delivery delays, since the probability of finding the destination node is low when only on copy exist. Multi-copy schemes forwards a copy of each message sent. This is also called replication. Several copies of the same message exists in the network, thus having a higher resource consumption compared to single-copy, but it gives a lower delivery delay, because the higher probability that one of the copies will reach its destination. Because of the higher resource consumption algorithms try to avoid replication when it does not increase the chance of delivery or otherwise limit the number of copies. Examples of existing routing protocols are direct delivery and epidemic routing. Direct delivery is the simplest single-copy routing scheme, a node holds a message until it encounters the destination node of the message, to which it deliveries the messages. Epidemic routing is the simplest multi-copy scheme, whenever a node comes in contact with another node it replicates all it s messages to that node making the messages to spread rapidly in the network. 2.3 Opportunistic DTN routing with Windowaware Adaptive Replication Opportunistic DTN routing with Window-aware Adaptive Replication (OR- WAR) is an unicast multi-copy DTN routing scheme, every message has only one destination and several copies of the same message can exist in the network. The goal of ORWAR is to have an efficient utilisation of the resources present

24 10 Chapter 2. Background in the network at a global level particularly energy and bandwidth but also storage, while still providing high delivery ratio. To achieve efficient usage of the energy and bandwidth ORWAR introduces message differentiation based on utility to be able to choose the best message for each occasion and tries to reduce the overhead in the network by avoiding partial transmissions by utilizing local connectivity information. To achieve efficient usage of the storage in the nodes a buffer strategy is adopted were the message with the least utility is removed first when the buffer becomes full and messages known to be delivered are removed. To get a high delivery ratio ORWAR uses controlled replication that distributes a fixed number of copies of each message over the network Message differentiation All messages in the system are assigned a priority by the application, this priority is used to express the delivery importance of the message for the source application. ORWAR uses 3 levels of priority, this priority gives differentiation at the application level but do not provide a good enough differentiation for the system level. The priority is combined with the size of a message to calculate the utility (u) of a message. This utility is calculated as u = priority/size and can be interpreted as the utility per bit of a message and gives the benefit of transmitting this message in comparison to other messages. This gives higher detail in the differentiation than just using a priority and can be used to optimise the usage of the resources at system level by allocating more to high utility messages Contact Window Estimation Partial transmission happen when the size of a message being sent is larger than the maximum transmittable size between two nodes. Partial transmissions happens because when nodes meet in the system they will only be in contact for a limited period of time, called the contact window (t cw ). The length of t cw is calculated from the nodes movements i.e. current position ( p) and relative velocities ( v) and their effective communication range (r),this is illustrated in Figure 2.3 Figure 2.3: Illustration of the Contact Window, taken from [17, p. 6, Figure 3]

25 2.3. Opportunistic DTN routing with Window-aware Adaptive Replication 11 The maximum number of bytes that can be sent during a contact is called the contact window size C s and is calculated from the contact window time and the nodes data rate (b) as C s = b t cw. This implies there exist a maximum limit on the size for a bundle to be transmitted successfully during a contact. Partial transmissions waste resources in form of bandwidth and energy, since the partially sent data cannot be utilised, ORWAR tries to minimise the number of partial transmissions to avoid this. By estimating C s at every meeting between two nodes before sending, C s can be used as an upper limit for the largest transmittable bundle S max between those node and by only choosing bundles that fits inside S max when sending the probability of a partial transmissions is reduced. Since the real contact window size can be different from the estimated, partial transmissions of a selected message can still occur Adaptive Replication As the name of the protocol implies it uses a window aware adaptive replication strategy. It utilizes the utility and contact window to become resource efficient. To limit the overhead from allowing multiple copies of the same message, every message is assigned a maximum number of allowed copies. This number is called the replication factor of message k (L k ). The replication factor of a message is assigned at creation and its initial value is calculated as a function of the priority of the message hence the adaptive replication part. Table 2.1 shows the calculation, L and are fixed parameters. Table 2.1: Initial message copies as an utility function Priority Class Priority L k High 3 L + Normal 2 L Low 1 L The replication of each message in ORWAR is divided into two phases, depending on the L k of the message. In the first phase when L k > 1, half of the copies of a message forwarded is to the other node and the rest are kept. When L k = 1 it enters the second phase were it stops replicating the message and instead holds the message until the destination is encountered. Which message to replicate at a contact is decided based on the utility and the contact window. It starts with the highest utility first and only chooses the from the messages that fit inside the current S max. By giving a higher number of copies to high utility messages and always starting with replicating the high utility messages, their delivery probability and latency improves compared to lower utility ones The Algorithm The message differentiation, contact window estimation and replication is combined to form the ORWAR routing algorithm. Figure shows the pseudo code for the algorithm.

26 12 Chapter 2. Background The operation of ORWAR can be divided into three parts, reception of bundles from the application, exchange of router messages, and the replication and reception of messages. The first part results in the message being inserted in the routers message queue (mq) in utility bit order. The exchange of router messages and replication has a number of phases. These are illustrated in Figure 2.4b. When two nodes meets they start by exchanging and updating their list of known delivered messages (KDM). Then all messages that appear in the new KDM list or have an expired T T L is purged from the mq. Then they continue by exchanging their contact window information (CWI) and calculate their contact window. The contact window is used to replicate the nodes messages in utility per bit order as described in subsection It stars by replicating messages that have a neighbour as its destination and then continues with replicating all other messages, that fit in the contact window. // Merge and d send kdm i to j receive kdm j remove from qu remove from qu kdmi = kdm i k remove mk from smax = compute_ while s max > 0 // final deliver for each messa if s k<smax deliver s max = sm // updat if m k tr insert remove end for each m // forward to j for each messa if mk send m k L ki= Lki/ smax = s end for each m // receiving mes if mq i is full if uk/sk > la replace la else insert m k in if destination insert m k i remove mi fr end at each meet Function compute send ( v i,ri,xi receive ( v j,rj return s max Figure (a) The ORWAR pseudo code taken from [17, p. 5, Figure 2] (b) Illustration of the 3 phases Figure 2.4: Pseudo code and message exchange

27 2.4. The Delay-Tolerant Network Architecture The Delay-Tolerant Network Architecture The Internet Research Task Force (IRTF) Delay-Tolerant Networking Research Group (DTNRG) [8] has proposed a network architecture [4] to be used in delaytolerant networks. The architecture introduces a new network layer called the bundle layer (BL) that resides between the application layer and the rest of the ordinary protocol stack. The DTNRG has also proposed an accompanying protocol called the bundle protocol [18] that implements the bundle layer. The bundle layer implements a message-oriented overlay network that provides the necessary functionality to approximate end-to-end connectivity in DTNs. It is agnostic to the underlying layer and is focused on virtual message forwarding instead of packet-switching. Figure 2.5: Illustration of the bundle layer overlay network The bundle layer is designed to handle both the disruptions in the network and the potential heterogeneity of the devices and protocols making up the network. To provide heterogeneous communication, each node implementing the bundle layer has protocol-specific convergence layer adapters (CLA) that provide the functions necessary to send messages on the specific protocol. This makes it possible to run the bundle protocol on top of any protocol stack if a conforming convergence layer exists. The bundle layer in it self has been divided into two sub layers, called bundle protocol agent (BPA) and convergence layer (CL). The bundle protocol agent offers the bundle layer services and implements the procedures of the bundle protocol and the convergence layer implements the CLAs. Figure 2.6: Division of the bundle layer into its two sub layers

28 14 Chapter 2. Background The bundle layer uses variable size messages called bundles as its protocol data unit. This is to enable the application and the bundle protocol to create bundles in the optimal size for any situation. The name bundle comes form the fact that the bundle protocol suggests that application to should bundle together both application level data and meta data when sending, to make the bundles self contained and hence help to minimize the number end-to-end transactions, since the end-to-end delay is typically large. It also employs a coarse grained quality of service to be able to give important messages higher priority. To transfer bundles reliably, the bundle layer provides the option to do custody transfer. When using custody transfer a bundle is moved between custodians in the same manner as Store-Carry-Forward, a bundle is stored in persistent memory until the new custodian has received it successfully, this guarantees that the bundle is not lost. It also supports a number of other delivery options and status messages to be used, these will be briefly described in subsection No bit-level error detection or correction is provided by the bundle protocol, this is instead left to the other layers to provide. In essence the bundle protocol uses naming, layering, encapsulation and persistent storage to communicate Naming and Addressing The bundle layer has been designed to allow nodes to have multiple identifiers and even multiple types of identifiers to support interoperability. This is achieved by employing a general naming system with node identifiers in the form of Universal Resource Identifiers (URI) [3]. Each URI is of the form <scheme>:<scheme-specific-part>, were the scheme is a string holding the name of the scheme used by the URI and the scheme-specific-part is a string holding the identifier to be interpreted. By using URIs identifiers can represent both names or addresses and be from different name spaces. The bundle layer supports direct forwarding on symbolic names, by performing late binding. Late binding means that the name to address binding, does not have to happen at the source node but can happen later in the sending process. Both late and early binding were the name to address binding is performed first is supported, depending on the URI used. The URIs used in the bundle layer are called Endpoint Identifiers (EID). A node can have several EIDs and an EID can belong to several nodes to support different kinds of delivery e.g. unicast, anycast and multicast. If an EID only belongs to one node it is called a singleton Bundles The bundle layer PDU bundle consists of one or more blocks, each containing meta data or application data or both. The bundle blocks hold the same kind of information that is found in headers in other protocols. They are called blocks instead of headers due to the fact that they are not necessarily always placed in the front of the message. Most block fields are encoded as self-delimiting numerical values (SDNV), making them variable size. This has been done to support extension in future versions of the protocol.

29 2.4. The Delay-Tolerant Network Architecture 15 The first block of all bundles is called the Primary block, illustrated in Figure 2.7, it contains the information necessary to route the bundle to its destination. The version, processing control flags, length of the block, source and destination EIDs, and fragmentation information fields are found in most protocol headers. The bundle protocol specific fields are the custodian and report-to EIDs and the creation timestamp, sequence number, life time and the dictionary. Bit Meaning 0 Bundle is a fragment 1 ADU is a status report 2 Do not fragment 3 Custody transfer is requested 4 Singleton destination 5 Application ack requested 6 Reserved 7-13 Quality of service 14 Report reception 15 Report custody acceptance 16 Report forwarding 17 Report delivery 18 Report deletion Figure 2.7: Structure of the primary bundle block [18] The two addtional EIDs custodian and report-to are used to hold the EIDs of the current custodian of the bundle and the node status report should be sent to. The report-to EID allows a node to direct status messages for its bundles to another node. The strings of the EIDs of the blocks are placed in the dictionary, the offset fields are used as pointers to the beginning of that string in the dictionary. This is done to reduce the space of the block by not storing duplicate strings. The creation timestamp time field holds the real time when the bundle was created. The lifetime field indicates the time when the bundle is no longer useful, and is represented as a positive offset from the creation timestamp time. The creation timestamp sequence number is a monotonically increasing counter that together with the creation timestamp time and the source EID forms an unique identifier of the bundle. The processing control flags can be divided into three section. General flags holding general information about the bundle, the QOS flags indicate the bundles quality of services, 1 to 3, and the status report flags indicates what status report that should be generated for this bundles. Figure 2.7 shows the meaning of the flags. Additional blocks can be added after the primary block to provide extensions to the bundle protocol e.g. adding security. They all share a common basic format, illustrated in Figure 2.8, a 1 byte block type, processing flags, block length and a number of type specific fields.

30 16 Chapter 2. Background Bit Meaning 0 Must be replicated in fragments 1 Report if block can t be processed 2 Delete bundle if block can t be processed 3 Last block 4 Discard block if it can t be processed 5 Forwarded without being processed 6 Contains an EID-reference field Figure 2.8: Structure of extension blocks [18] 2.5 The ns-3 network simulator ns-3 is a new discrete event driven network simulator aimed at research and educational use written in C++. The development of the simulator is performed as an open source project, and is actively encouraging community participation by providing open mailing list for user and developer discussions, a tracker for reporting bugs and a wiki with user contributed information. All software is available under the GPL2v license. The goal of the ns-3 simulator is to provide realistic simulation of all parts of the network down to the physical level, so that the gap from simulation to real implementation is decreased, while providing an architecture that is easily configurable and extended. ns-3 is aligned with real systems by modelling the nodes more like real computers with applications, protocol stacks, and network interfaces and support key interfaces such as the Berkeley sockets and IP/device driver interface for Linux to enable reuse of application and kernel code. It also support standard formats e.g. PCAP 1 and ns-2 mobility traces to enable software integration, with other tools. ns-3 has a powerful attribute system making all simulation parameters accessible for configuring and a callback based tracing system that decouples the generation of trace events e.g. from the logging and analysis of the data. Figure 2.9: ns-3 software architecture [14] The implementation of ns-3 uses a modular design as depicted in Figure 2.9. The different models implements: 1 Packet capture (PCAP), an application programming interface for capturing network traffic.

31 2.5. The ns-3 network simulator 17 Core Contains generic core functions of the e.g smart pointers, callbacks, logging. Simulator Contains the implementation of events, schedulers and time objects that forms the simulator. Common Library of objects that are common to all networks e.g packet and packet headers. Node Contains the fundamental simulator models e.g nodes, channels, base class for net devices and sockets. Mobility Contains different mobility models e.g static, ns-2 traces and random walk. Routing Contains different router implementations e.g AODV 2 and OLSR 3. Internet Stack Contains Internet related models, e.g models TCP, UDP and IPv4. Net Devices Different net device implementations, e.g Ethernet, Wifi, Wimax etc. Application Base classes for implementing applications. Helper Provides a simpler interface to lower-layer APIs. 2 Ad hoc On Demand Distance Vector (AODV) a MANET routing protocol, see for more information. 3 Optimized Link State Routing Protocol (OLSR) a MANET routing protocol, see for more information.

32 Chapter 3 Design In this chapter the architecture and design of the implementation of ORWAR and the bundle protocol in ns-3 is described. It starts with a description at a high-level showing the architecture and general structure of the design. It continues with a description of the final design of the system and general design decisions. 3.1 Architecture From analysing the existing ns-3 documentation and code base it was determined that no existing support for DTN-protocols were available in ns-3 when this project began. To be able to transfer the implementation of ORWAR from the ONE simulator to ns-3, a new framework similar to those already existing for TCP, UDP and IP were needed, implementing a model of the bundle layer and the bundle protocol. It was also determined that ns-3 did not have any general implementation of neighbourhood detection and instead each wireless routing protocol had included its own detection system hard coded in the implementation. Four main functionalities were identified as necessary to implement in ns-3 to be able to support ORWAR and re-evaluate ORWAR: Bundle Layer A model of the bundle layer that implements the bundle protocol giving ns-3 support for DTN and bundles. Delay-Tolerant Routing a model for delay-tolerant routing algorithms to be used to implement the ORWAR algorithm. Neighbourhood Detection A neighbourhood detection model providing ns-3 with the ability to let nodes detect other nodes in its vicinity. Recreation of Simulations Functionality to recreate the simulation scenarios in ns-3. The ns-3 architecture was extended by adding a new module containing the implementation of the bundle layer and a number of helper functions as Figure 3.1 depicts. From the analysis an architecture was created providing a high-level description of the system. Since ns-3 tries to model the real-world implementations as 18

33 3.1. Architecture 19 Figure 3.1: ns-3 extended software architecture closely as possible, the already existing specification for the bundle-layer architecture [4] and the proposed implementation of the architecture for the bundleprotocol [18] were used as a reference, to make the work more aligned with a possible real implementations, in terms of terminology and functionality. The resulting architecture for the bundle layer follows the layered architecture pattern and is internally divided into five modules. Figure 3.2 depicts the architecture, the left figure shows the protocol stack and the right shows the stack, with the bundle layers modules displayed. The arrows represent how data flows in the interface between the different modules. Figure 3.2: Box diagram of the architecture The architecture shows how the functionality has been divided into the different modules, the routing functionality has been separated from the other functionality of the bundle protocol. The implementation of the bundle protocol has been divided into the modules called bundle manager and convergence layer. The bundle manager gives applications access to the bundle protocol services and the convergence layer interfaces the bundle protocol with the lower layer, as described in section 2.4. The router logic has been divided into three parts: the bundle routing, link management and neighbourhood detection. The bundle router module implements the actual routing algorithm i.e choosing what message to send and to where, the neighbourhood detection agent and link manager addresses the

34 20 Chapter 3. Design issue of finding contact opportunities and how to manage them and keep track of the current neighbourhood. 3.2 Detailed Design The final design borrows concepts and ideas from the existing specifications of the bundle protocol [18, 4], and the architecture of ONE [11, 10]. The design follows the architecture. Each box from the architecture has been implemented as a C++ class providing the intended functionality, creating five major classes BundleManager, BundleRouter, LinkManager, ConvergenceLayerAgent and NeighbourhoodDetectionAgent. Figure 3.3 shows an overview UML class diagram of the design, showing the specific sub classes for implementing ORWAR and the relationship between the main five classes. The result is a general framework for implementing delay-tolerant routing algorithms for the bundle protocol. The main design decision for the overall design were that it should be modular and extendable and be compatible with the Bundle Protocol specification. Application Registration RegistrationManager RegistrationEndpoint BundleManager BundleRouter OrwarRouter ConvergenceLayerAgent NeighbourhoodDetectionAgent LinkManager WifiNetdevice OrwarLinkManager Figure 3.3: Overview class diagram of the final design Since no support for delay-tolerant networking exists in ns-3 each of these classes more or less needed to be designed from scratch. A base for implementing bundle protocol and bundle routing convergence layer adaptor for the bundle protocol and bundle routing and neighbourhood detection was needed. The design decisions and details for each part of the system is described in chapter Modularity and extendibility The design has been created with the intention that it should be possible to extend it with new functionality and that it should be possible to change the

35 3.2. Detailed Design 21 algorithm for routing, link management, neighbourhood detection and convergence layer so that different strategies can be tested. This is supported in the design by decoupling the five main classes, by only letting them communicate with each other through callbacks. This makes it possible to exchange a class with another that implements the needed methods. The main classes have been implemented using the non-virtual interface and template method design patterns, were the base classes implement the public interface and the basic steps of the algorithm and lets a subclass implement the specific behaviour. So to for example test a new routing algorithm only a new subclass to the bundle router would be needed, leaving the rest of the code unchanged Compatibility with the Bundle Protocol specification To reduce the complexity of the design and to save time when implementing, some of the functionality described in the DTNRG bundle protocol specification [18] has been excluded, since they were non-essential to the operation of ORWAR. Table 3.1 shows the functionality that have been supported. Functionality with an asterisk (*) has not been fully implemented or replaced with a simpler solution. The design has been structured with the intention that it should be possible to extend it to become a full implementation of the bundle protocol. The changes are discussed in more detail in chapter 4. Table 3.1: Supported Functionality of the Bundle Protocol Bundle Protocol Supported Unsupported Store-Carry-Forward X Variable size messages X Bundle Headers X* Self-Delimiting-Values X Endpoint Identifiers X* Class of Service X Fragmentation Custody Transfer X* Registration X* Bundle Signalling X X Protocol Stack The implementation has been designed to communicate directly with the maclayer running b wifi. The convergence layer and neighbourhood detection algorithm uses the MAC layer protocol directly bypassing the transport and network OSI layers. By doing this overhead from TCP/UDP and IP is avoided. TCP needs to set-up a connection before sending any data, which is not necessary since this is done by the bundle layer, so it only adds overhead, also the headers for the TCP adds to the size of the messages unnecessarily. The later is also true for UDP and IP.

36 Chapter 4 Implementation In this chapter the implementation of the bundle protocol and the ORWAR routing protocol is described. The UML class diagrams shown in this chapter have been simplified to reduce their size. Attributes and method return types and input parameters are hidden and only get methods are displayed while hiding the accompanying set methods. 4.1 Message Helper To transfer the messages used when simulating ORWAR in ONE, a helper class was created to import message trace files holding the message used by ONE. The functionality is simple, a message trace file is given to the helper class when it is created. The trace file is then read line by line when the method install is called and from each line a bundle message is created with the properties described and scheduled to be sent by the node, at the given time. Figure 4.1b shows an excerpt form a message trace file. OneTrafficHelper + OneTrafficHelper() + Install() - LayoutObjectStore() - GetNode() - StringToPriority() (a) Class diagram of the traffic helper Time (s) Command Message Id From To Size Priority 3600 C M u C M u C M u C M u3 (b) Excerpt from message trace file Figure 4.1: Message Helper 4.2 Bundle - Variable size messages The bundle protocols PDU has been implemented by the class Bundle, Figure 4.2 shows the class diagram of the implementation. A bundle have two different representations in the system, an internal representation used by the system implemented by the class Bundle and its serialized byte representation 22

37 4.2. Bundle - Variable size messages 23 used when sending and receiving. In serialized form the all fields of the bundle are SDNV encoded with variable size according to the specification. The Bundle class uses several help classes, implementing the endpoint identifiers, blocks and the different fields of its blocks. Dictionary + GetDestinationEndpoint() + GetSourceEndpoint() + GetCustodianEndpoint() + RemoveDestinationEndpoint() + RemoveSourceEndpoint() + RemoveCustodianEndpoint() + GetSerializedSize() - GetUnique() PrimaryBundleBlock + GetVersion() + GetProcessingControlFlags() + GetDestinationEndpoint() + GetSourceEndpoint() + GetCustodianEndpoint() + GetCreationTimestamp() + GetLifetime() Bundle + GetPrimaryHeader() + GetCanonicalHeaders() + GetBundleId() + RemoveRetentionConstraint() + RemoveAllRetentionConstraints() + GetPayload() + GetReceivedFrom() + HaveBeenReceivedFrom() BundleEndpointId + GetUri() + GetScheme() + GetSsp() + GetId() + GetAnyBundleEndpointId() + IsSingleton() + IsMatchingType() - GetType() CreationTimestamp + CreationTimestamp() + GetTime() + GetSeconds() + GetSequence() GlobalBundleIdentifier + GetSourceEid() + GetCreationTimestamp() ExtentionBundleBlock + GetBlockType() + GetProcessingControlFlags() Sdnv + Encode() + Decode() + EncodingLength() Figure 4.2: Class diagram of the Bundle implementation The bundle protocol blocks has been, slightly modified and redesigned in this implementation. The primary block has been modified, illustrated in Figure 4.3. An additional field containing the replication factor used by ORWAR has been added to the processing flags and the report-to-field has been removed since it is never used, as explained in section 4.3. The bundle blocks are implemented by the classes PrimaryBundleBlock and Extension-BundleBlock. Figure 4.3: Implemented primary block The bundle endpoint identifier implementation is represented by the class BundleEndpointId. It is not a true implementation of endpoints as URI:s but

38 24 Chapter 4. Implementation is instead a mockup. This is because the features given by using a URI is not needed. The endpoint identifiers are instead a integer internally. This also saves bandwidth, because the string representation of a URI is large, compared to sending a single 4 byte integer. Three types of extension blocks has been implemented: two for ORWARs signalling, described in subsection and one for the normal data bundle. Figure 4.4 illustrates the extension block for the data bundles. Value Meaning 1 Data bundle (a) The data bundle (b) Bundle Type Field Figure 4.4: Data Bundle Extension Block 4.3 Bundle Manager For an application to be able to use the bundle protocol an interface between them must exist and it must be able to create and send bundles and receive bundles. The main purpose of the bundle manager is to provide the bundle protocol services to the DTN applications running on a node and handle the communication between the application, bundle router and convergence layer General Description This interface is provided by a module called the bundle manager. The bundle manager implements the functionality of the bundle protocol agent (BPA) in the bundle protocol specification [18, 4], but without the router logic. It has been designed to act as a mediator between the application, bundle router and the convergence layer. The bundle manager provides the following functionalities: An interface to the application layer by registrations Logic for managing the registrations. Logic for creation and parsing of Bundles An interface for communication between the bundle router and the convergence layer Figure 4.5 shows a class diagram of the classes that implement the bundle manager. The bundle manager is implemented by the class BundleManager. The BPA functionality has been simplified on a number of places. The bundle signalling functionality has been omitted in the implementation since its functionality was not needed. This does not affect the operation of the bundle protocol since its main purpose is for debugging networks and would not

39 4.3. Bundle Manager 25 be used in normal operation. Also fragmentation and reassembly of bundles has been omitted since this feature was not used in the implemented version of ORWAR, when this thesis started. The registration management has also been simplified. In the specification registration should be persistent and be retained over application and system restarts. This functionality has been removed since this functionality has not been necessary. BundleManager + BundleReceivedFromApplication() + Allocate() - BundleReceivedFromConvergenceLayer() - GenerateBundle() - CancelTransmission() - TransmissionCancelled() - BundleDispatching() - LocalBundleDelivery() - ForwardUp() - BundleForwarding() - BundleDeletion() - BundleSentOk() - BundleSentFailed() - SendBundle() RegistrationEndpoint + GetBundleEndpointId() + SetBundleEndpointId() + ForwardUp() + NotifySend() * 1 RegistrationManager + LookupLocal() + Lookup() + Allocate() + DeAllocate() Registration + SetBundleProtocol() + Register() + Unregister() + Send() + Recv() # NotifyRegistrationSucceeded() # NotifyRegistrationFailed() # NotifySend() # NotifyRecv() - ForwardUp() Figure 4.5: Class diagram of the bundle manager implementation The operation of the bundle manager can be divided into three parts: registration management, bundle management and management of the communication between the convergence layer and bundle router. Figure 4.6 shows a high level description of how the bundle manager should function. It consists of a registration manager that keeps track of the registrations and logic for processing and dispatching bundles and relays calls between the bundle router and the convergence layer. Bundle Router Figure 4.6: Bundle Manager The general operation of the bundle manager is as follows: To be able to utilize the bundle protocol an application must register itself to the bundle manager. This makes the bundle manager aware of the application and can deliver received bundles to the application. When an application data unit (ADU) is received from the application, it is transformed into the bundle

40 26 Chapter 4. Implementation format by adding the bundle headers. The new bundle is then given to the bundle router for storage and further processing. When a bundle is received from another node via the convergence layer it is first parsed to check that it is valid and all its block can be processed. If valid the destination of the bundle is checked if it is to be delivered to this nodes application(s) or given to the bundle router. If it is invalid the bundle is dropped. When a call from the convergence layer or bundle router is received it is relayed to the other part e.g if the bundle router wants to replicate a bundle the call is relayed to the convergence layer and the result of the replication is relayed back to the bundle router Registration Management A registration is an inter process communication endpoint similar to an Internet socket. It is used by the application to send and receive bundles using the bundle protocol. Registrations implement an asynchronous application interface, since the environment have large and variable delays on sending and receiving. Registration has been implemented as two classes Registration and RegistrationEndpoint. The Registration class is used by applications in the same way normal sockets are used and implements a simple send and receive interface. The RegistrationEndpoint implements the actual logic for sending and receiving. The RegistrationManager class manages the registrations to the applications. It holds a list of all registration endpoints and manages the creation and deletion of registrations, and gives the functionality to check if an endpoint is registered or not when checking if a received bundle is for this node. When a Registration is created by an application a RegistrationEndpoint is allocated in the RegistrationManager and a pointer to this Registration- Endpoint is created in the Registration and is used when bundle transmissions are issued. 4.4 Convergence Layer A convergence layer adaptor is needed to connect the bundle protocol to the underlying protocol stack and to enable it to send and receive bundles. The convergence layer adaptor sends and receives bundles utilizing the services of some native protocol. The purpose of the convergence layer is to interface the bundle protocol to its underlying layers and to perform the actual sending and receiving of bundles by marshalling/unmarshalling bundles to a format suitable for the underlying layers General Description This section describes an implementation of a convergence layer adaptor that uses the b wifi protocol. Not many implementations of actual convergence layer adaptors (CLA) exist, and no CLA for the b wifi could be found, so it needed to be designed from scratch. It borrows the idea of segmentation from the TCP CLA described

41 4.4. Convergence Layer 27 in a RFC draft [12]. To make it simple it was decided it should be a simple stop-and-wait protocol i.e it sends a message and then waits for an ack, before sending the next. Stop-and-wait protocols are not very effective because of the wait for each ack, but since all the communication is one-hop this delay will be very small. No connection is established when sending, so a request for transferring a bundle starts directly. It is the job of the higher layer i.e. bundle router to make sure a transfer is possible, but a transfer is aborted if no acknowledgements are received after a number of retransmissions. The convergence layer provides the following functionalities: An interface for the bundle manager to send and receive bundles Marshalling bundles into a suitable format for the b protocol Transmission of bundles over the b wifi protocol Reception of bundles over the b wifi protocol Figure 4.7 shows a class diagram of the classes implementing the convergence layer. The convergence layer adaptor has been implemented directly by the ConvergenceLayerAgent, that implements both the send and receive logic. The class is designed to be subclassed with its functionality overridden. It is meant to provide a basic implementation of a CLA that can be used as is by the other components. Therefore the functionality is directly put into the base class, instead of making it an abstract class, with a subclass implementing the CLA behaviour. ConvergenceLayerAgent + Init() + Stop() + LinkLost() + BundleQueued() + CancelTransmission() - SendBundle() - SendSegments() - SegmentSentOk() - SegmentSentFailed() - BundleSentOk() - BundleSentFailed() - ReceiveSegment() - ReassembleBundle() - SendAck() - BundleReceived() - TransmissionCancelled() - GetSequenceNumber() - RemoveOrphanedSegments() - Retransmit() <<typedef>> <<struct>> SendQueue SendQueueElement <<typedef>> <<struct>> ReceiveQueue ReceiveQueueElement SegmentHeader + SetType() + GetStartFlag() + GetEndFlag() + SetNumberOfSegments() + SetSegmentNumber() + SetSequenceNumber() + SetResponse() Figure 4.7: Class diagram of the convergence layer implementation

42 28 Chapter 4. Implementation Figure 4.8 shows a high level description of the convergence layer. It consists of send and receive queues for sending bundles, logic for marshalling and unmarshalling bundles and logic for sending and receiving bundles. Figure 4.8: The Convergence Layer The operation of the protocol can be divided into two parts sending and receiving. The general operation on the sending side is as follows: An bundle transfer is requested by the bundle router and given to the convergence layer. The bundle is marshalled into one ore more protocol data units called segments, that are logical fragments of a bundle and put in a send queue. The segmentation is necessary because bundles can be of arbitrary size, while the b protocol has a maximal transferable unit (MTU) that limits the size of the frames it can send. Bundles need to be split into smaller parts that can be transferred. Each fragment consists of a message header followed by a byte array of data. The size of the segments are an implementation matter but must be smaller than the MTU of the b so that it can fit inside a frame. When a segment has been sent, the protocol waits for an ack, by starting a timer with the time out value T ack. If no ack is reported for the segment by the , the segment is retransmitted and the timer is restarted. When a transmission of a segment has failed rs max times, the transfer of the protocol is reported to have failed to the bundle router via the bundle manager. The number of times it can try to retransmit a segment is an configurable parameter of the protocol. The convergence layer continues to send segments until the send queue is empty. When sending the last segment a timer is started to wait for the acknowledgement to be received. If no acknowledgement is received it reports the transfer as failed. The general operation on the receiving side is as follows: When a segment is received and it is checked to see if it belongs to an ongoing bundle transmission or if it is a new one. Based on this the segment is placed in a slot together with the rest of the segments for the bundle in the receive queue or placed in a new slot. When all the segments for a bundle have been received, a convergence layer acknowledgement is sent to notify hat the bundle has been received successfully. The bundle is then reassembled and forwarded up to the bundle manager Segmentation of bundles A segment is a logical fragment of a bundle similar to a fragment, it is used by the convergence layer to be able to transfer bundles over the b wifi

43 4.4. Convergence Layer 29 protocol. The difference between a segment and a fragment of a bundle is that when fragmenting a bundle each fragment becomes a new independent bundle that can be transferred through the network to the destination, a segment on the other hand is a specialised unit that is only valid over one hop between two nodes. A segment consist of a header followed by the data, see Figure 4.9. Figure 4.9: Segment format To be able to reassemble a bundle the receiver must be able to identify which segments belong together and when all segments of a bundle have been received. This is the purpose of the segment header illustrated in Figure The segment header has been implemented by the class SegmentHeader and its class diagram can be seen in Figure 4.7. Figure 4.10: Segment header Type Indicates the type of the segment, see Table 4.1a. Response Holds the ack response, only present if the type is ack, Table 4.1b. Sequence Number Identifier for the segmented bundle Number of Segments The total number of segments Segment Number The number of this segment Table 4.1: Type and response field description (a) Type field (b) Response field Value Meaning Value Meaning 0x01 0x02 Data segment Ack segment 0x01 0x02 Succeeded Failed The sequence number acts as an identifier for a segmented bundle and is shared by all its segments. The sequence number is necessary because there is no contact established in the CL making this possible. The segment number identifies a specific segment together with the sequence number and the number of segments specifies the total number of segments this bundle has been divided in.

44 30 Chapter 4. Implementation The segment number and the number of segments are used by the receiver to know when and how to reassemble the segments into a bundle. Figure 4.7 shows the class diagram of the SegmentHeader Queue management The convergence adaptor on each node has two queues, a send queue and a receive queue. The send queue holds all bundles that should be transmitted over b, divided into their segments. The elements of the queue consist of a global bundle identifier and the list of segments for this bundle. The queue holds the elements in FIFO-order and is implemented as a C++ standard library deque holding and the elements as a struct called SendQueueElement. Elements are removed from the queue when a transmission completes or when the transmission is aborted by the bundle router. The receiver queue holds all received segments. The elements of the queue consist of an identifier for the bundle and a list of the received segments. The identifier for the bundle is composed of the sequence number and MAC address of the sending nodes to uniquely identify each bundle transmission, this is used to hold the segments of that transmission together. The MAC is needed because the sequence number is not enough on its own to uniquely identify a segment since several nodes could have assigned the same sequence number to a bundle transmission. The queue is implemented as a C++ standard library map, using sequence number, MAC address as key and the list of received segments. Elements are removed from the queue when a bundle has been completely received or when the reception of a bundle is deemed to have failed. 4.5 Neighbourhood Detection Agent For a node to be able to communicate with other nodes it is necessary for it to be able to detect them. A neighbourhood detection protocol is needed for a node to be able to do this. The purpose of the neighbourhood detection protocol is to handle the detection of contact opportunities with other nodes and making its own node detectable by other nodes General Description In this implementation the neighbourhood detection is implemented by a module called the neighbourhood detection agent (NDA). When designing the NDA it was decided to keep the solution simple. Figure 4.11 shows a class diagram of the classes that implement the NDA. The NDA is implemented directly by the class NeighbourhoodDetectionAgent. It provides a basic implementation of a neighbourhood detection algorithm that can be used as is, by the rest of the components in the system. It has been designed with a simple interface and uses a ns-3 callback to notify when a link is found. The intention is that different neighbourhood detection algorithms should be implemented by new classes implementing the same interface.

45 4.5. Neighbourhood Detection Agent 31 NeighbourhoodDetectionAgent + GetHelloIntervall() + Init() + Start() + Stop() - HelloTimerExpire() - SendHello() - HandleHello() - NotifyDiscoveredLink() HelloHeader + SetBundleEndpointId() + GetBundleEndpointId() Figure 4.11: Class diagram of the neighbourhood detection implementation The neighbourhood detection agent provides the following functionalities: Detection of nodes with the b wifi Make nodes detectable on the b wifi Figure 4.12 shows a high level description of the NDA, it consist of logic for sending and receiving hello messages. Figure 4.12: The Neighbourhood Detection Agent The general operation of the protocol is as follows: The NDA periodically broadcasts a specialised message, called a hello message, with a T hello seconds interval. The hello message contains the necessary contact information about the node. By broadcasting the hello message the node is announcing its presence to all nodes that receive the messages. When a hello is received a contact opportunity for the sender of the message is found. The message is processed and then the result is forwarded up to the bundle router, notifying that a new contact opportunity is available Hello Generation Hello messages are generated and broadcasted periodically, with the interval T hello. The size of the interval configurable parameter of the algorithm. A small interval increases the detection probability but increases the overhead. A trade off must be made between the interval and the bandwidth usage. 1 In addition to this the periodic nature of the hello generation, has the disadvantage that it increases the risk of simultaneous transmissions resulting in 1 For more information about the size of T hello and how to implement jitter see RFC5148[5]

46 32 Chapter 4. Implementation collisions. If the nodes generate hellos with the same intervals, and if some nodes generate hellos at the same time, all of those hellos will be simultaneous. This is especially a problem in a simulated environment where all the nodes have perfectly synchronised clocks. To avoid this the transmission time is randomized by introducing jitter (J). This decrease the probability of simultaneous transmissions. The jitter is introduced by reducing the T hello by a random amount. This makes the interval between two transmission equal to (T hello J). The jitter value is generated uniformly in a interval [0, J max ]. The size of J max is a implementation matter. The hello messages is an empty message with a header, called the hello header, containing the endpoint identifier of the sending node. Figure 4.13: Hello header 4.6 Link Manager To be able to utilize the contacts found by the neighbourhood detection agent the validity of those contact opportunities needs to be checked so that it is known they are it no longer valid. In this implementation this functionality is provided the module called the link manager (LM) General Description The link management has been designed, to be extendible by creating a abstract base class, that implements an interface and some base functionality that is shared by all link management algorithms. To implement the functionality needed to run ORWAR a subclass was designed that implements a link management algorithm that utilizes the ORWAR protocols contact window estimation described in subsection and the neighbourhood detection algorithm described in section 4.5. The purpose of the link manager is to keep track of the links to the current neighbourhood of nodes and check their validity to detect when a contact opportunity is no longer available and provide this information to the bundle router. The link manager uses two different concepts for representing contact opportunities, called link and contact. A link represents a possible connection between two nodes and contains the connection information needed to communicate with the remote node of the link. The contact represents a actual ongoing contact between two nodes and contains information about it e.g. possible data rate and when subclassed a contact window estimation when running ORWAR.

47 4.6. Link Manager 33 The link manager provides the following functionalities: The functionality to detect when a contact opportunity has ended Maintain a list of all available links Maintain a list of all connected links The specialised ORWAR link manager extends this functionality with: The functionality to detect when a contact opportunity has ended based on contact window Functionality to decrease the contact window over time Maintain a list of all connected and ready links Figure 4.14 shows a class diagram of the classes that implement the link manager. The link manager is implemented as a subclass OrwarLinkManager to the class LinkManager. The LinkManager implements the basic functionality and interface and OrwarLinkManager implements the full algorithm that is used with ORWAR. LinkManager + DiscoveredLink() + AddLink() + RemoveLink() + HasLink() + FindLink() + OpenLink() + CloseLink() + GetAllLinks() + GetConnectedLinks() + GetAvailableLinks() + GetUnavailableLinks() # CheckIfExpired() # NotifyLinkIsAvailable() # NotifyClosedLink() # SetupTimer() OrwarLinkManager + DiscoveredLink() + GetSetupHoldTime() + GetCwMaxHoldTime() + AddLink() + CloseLink() + GetReadyLinks() + ContactSetup() + FinishedSetup() # RecalculateContactWindows() Link + SetRemoteEndpointId() + GetRemoteEndpointId() + SetRemoteAddress() + GetRemoteAddress() + GetLastHeardFrom() + SetContact() + GetContact() + GetState() + ChangeState() + Open() + UpdateLastHeardFrom() # Close() OrwarLink + HaveCw() Contact + GetTypeId() + Contact() + ~Contact() + EnqueueBundle() + DequeueBundle() + DequeueBundle() + GetQueuedBundles() + GetNumQueuedBundles() + IncreaseRetransmissions() + ResetRetransmissions() + SetRetransmissions() + GetRetransmissions() # DoDispose() OrwarContact + GetState() + SetContactWindowDuration() + DecreaseContactWindowDuration() + GetContactWindowDuration() + GetLastUpdated() + SetDataRate() + GetDataRate() + GetCwMax() Figure 4.14: Class diagram of the link management implementation

48 34 Chapter 4. Implementation All operations on links are performed by the link manager. The bundle router operates on the links through the link managers interface e.g. closing a link, or manually adding a link. The operation of the ORWAR link manager when managing a link, is divided in two phases, with two different approaches to detect if a link contact is still valid. A link is in the first phase when it has not yet estimated the contact window, and becomes in phase two when the router has estimated it. In first phase, when no contact window estimation have been established, a simple periodic check is preformed that assumes that if a contact has been used successfully e.g a successful transmissions or reception of data, the link will be valid until the next check is used. This method closes a link if nothing has been heard from the since the last check or if the bundle router closes the link due to failed retransmissions. When a contact window has been estimated this time is assumed to be correct, and the link is closed when it ends, if not closed earlier by the bundle router. The general operation of the algorithm is as follows: When a contact opportunity is reported found from the neighbourhood detection agent, the link manager checks if this contact opportunity is already known or if this is a totally new one. If it is a known one, a time stamp t lh indicating when the link was last heard from on the link is updated, otherwise a new link and contact is created and put in the LM and the bundle router is notified that a new contact is found. A timer is started for the new link with a timeout T hold. When the timeout occurs the link is checked for validity, if the time stamp on the link indicates that data has been received from this node during the timeout interval the link is determined still be valid and the timer is reset. If no data has been received the link is determined to be no longer active and is removed and the bundle router is notified that the link is no longer valid. When a link has its contact window estimated phase two starts. The timer is stopped and the timeout value is replaced with the estimated contact window. When the timeout occurs the contact window is over and the link is removed and the router is notified. A special case can occur in phase two. The duration of a contact window can be estimated to be infinite and cannot be used as a timeout value. This happens when two nodes have the same velocity and direction when estimating the contact window e.g. if the two nodes are standing still. The link is instead handled as in phase 1 but with a timeout value of T Max. The size of T Max is an configurable parameter Link and Contact Links have been implemented by the classes Link and OrwarLink. The Link implements the interface that should be shared by all link types and the OrwarLink implements the link type used by ORWAR. A link holds the endpoint identifier and physical address of the remote node the link is connected to, and is used by the bundle router when replicating bundles. It also contains a time stamp showing the last time the link was used and a contact if it is connected. A link can be in four states, the states represents what stage the link is in, Table 4.2 shows the states with their meaning.

49 4.7. Router 35 Table 4.2: OrwarLink states State Unavailable Available Connected Meaning A link to the node is unavailable A link to the node is available Link is available and in a connection is setup Contact has been implemented by the classes Contact and OrwarContact. The Contact implements the interface that should be shared by all contact types and the OrwarContact implements the contact used type used by ORWAR. The OrwarContact contains its state, the contacts data rate and its estimated contact window. The state is used by the router to represent what state the contact is in the exchange of router messages as described in subsection Router The bundle router module provides a platform for implementing store-carryforward bundle routing algorithms. It uses the link manager and neighbourhood detection agent modules to achieves its goals. The implementation is divided into two parts, a bundle router that provides general functionality meant to be used by all router schemes and a specialisation that implements the ORWAR algorithm. The router provides the following functionalities: Provide an interface to the bundle manager, for receiving and forwarding bundles. Provide persistent storage of bundles with different queue management schemes Replication behaviour The ORWAR router extend on this functionality with: ORWAR router signalling Contact window estimation Utility based replication and queue management Figure 4.15 shows a class diagram of the classes that implement the bundle router. The bundler router is implemented as a subclass OrwarRouter to the class BundleRouter. The BundleRouter implements the basic functionality and interface and OrwarRouter implements the ORWAR routing algorithm.

50 36 Chapter 4. Implementation BundleRouter + Init() + IsRouterSpecific() + IsEmpty() + ReceiveRouterSpecific() + HasBundle() + SentRouterSpecific() + InsertBundle() + GetNBundles() + DeleteBundle() + GetNBytes() + GetBundle() + GetFreeBytes() + LinkDiscovered() + RemoveExpiredBundles() + LinkClosed() - NotifySend() + AcceptBundle() # SendBundle() + AcceptCustody() # SendRouterSpecific() + CancelTransmission() # AddRouterSpecificBundle() + CancelTransmissionTo() # RemoveRouterSpecificBundle() + TransmissionCancelled() # HasRouterSpecificBundle() + CalculateReplicationFactor() # GetRouterSpecificBundle() + BundleSent() # MakeRoomForBundle() + BundleTransmissionFailed() # TimeExpired() + BundleReceived() # GetAllDeliverableBundles() + BundleDelivered() # GetAllBundlesForLink() + IsSending() # GetAllBundlesToAllLinks() AddEntry() + HasEntry() + GetEntry() ForwardLog + GetNEntries() + GetAllEntries() + GetEntries() <<typedef>> BundleList + RemoveEntry() + RemoveExpiredEntries() + ClearLog() - DoInit() - DoBundleDelivered() - DoBundleReceived() - DoSendBundle() - DoIsRouterSpecific() - DoLinkDiscovered() - DoLinkClosed() - DoCanDeleteBundle() - DoInsert() - DoDelete() - CanMakeRoomForBundle() - DoCancelTransmission() - DoTransmissionCancelled() - DoCalculateReplicationFactor() - PauseLink() - DoBundleSent() - DoBundleTransmissionFailed() KnownDeliveredMessages + Insert() + Merge() + Has() + Get() + NEntries() + RemoveExpiredBundles() + Clear() OrwarRouter - GetAllBundlesForLink() - TryToStartSending() - FindNextToSend() - RemoveDeliveredBundles() - SendCwi() - SendKdm() - SentCwi() - SentKdm() - HandleCwi() - HandleKdm() - GenerateCwi() - GenerateKdm() - CalculateContactWindow() - UnPauseLink() ContactWindowInformation + IsInitiator() + GetPosition() + GetVelocity() + GetTransmissionRange() + GetTransmissionSpeed() + GetSerializedSize() Figure 4.15: Class diagram of the bundle routing implementation Design issues When designing the router implementation, three issues where found that where not adequately addressed in the ORWAR paper [17] or the ONE implementation of ORWAR, 1. The router signalling is not simulated 2. The order that the messages are chosen needs to take into account router messages 3. A free check for avoiding redundant transmissions These issues originate from deficiencies in the realisation of the high level of abstraction in the ONE simulator, hiding how a real implementation would need to be done. The first issue is that the router signalling is not simulated in ONE. The information is instead instantly transferred between the nodes, and no actual messages are exchanged. These signalling messages were needed to be explicitly created, and logic for how the these messages should be exchanged has been needed to be created, the solution to these two issues is described in subsection below. This leads to the second issue, the introduction of two new messages needs to be handled when the router is to choose the next message to send, this is disscussed in subsection The third issue was that in the ONE implementation, a check if a message already is held in the remote node s message queue was provided for free and used to avoid redundant transmissions. This is not possible in a more detailed implementation. An solution to this problem is presented in subsection by

51 4.7. Router 37 adding a data structure to the bundle router implementation to reduce redundant transmissions Router signalling To implement the router signalling, two new message types are needed to be created that are implicitly used in ONE. Two extension bundle blocks have been created, to implement these ORWAR routers signalling specific messages. Their bundle type field is specified in Table 4.17a. Contact window information messages (CWI) that is used to exchange the contact characteristics used to by the nodes to estimate the contact window size, shown in Figure 4.16a. Known delivered messages messages (KDM) that is used to exchange the identifiers for the messages that are known to be delivered, shown in Figure 4.16b. (a) The CWI message (b) The KDM message Figure 4.16: The layouts of the two types of messages used by ORWAR When introducing the exchange of router information as real messages a problem not visible in the ONE implementation of ORWAR were found. In the original algorithm the KDM messages are first exchanged and then the CWI messages, as described in subsection This results in that the contact window is estimated after the KDM message has been sent, since the size of the KDM message is dependent on the number of known delivered messages the size of this message can grow large and this increases the risk of a partial transmission when sending it. CWI has a fixed size and does not cost as much to send so a failed transmission is not as costly. If instead we calculate the contact window size first, the KDM messages can be compared against the Value Meaning 192 CWI message 193 KDM message (a) Bundle Type Field (b) Router signalling between two nodes Figure 4.17: Bundle Type Field for router signalling and and typical message exchange

52 38 Chapter 4. Implementation contact window and hence partial transmission of the KDM can be avoided in the same as with data bundles. This was not an issue in the ONE simulator where the exchange of router data were done for free. So in this implementation the exchange has been reversed so that the CWI message is exchanged first and then the KDM message, as illustrated in Figure 4.17b Message Exchange and Replication To support router signalling an additional, the routers message buffer as been separated into two queues. One queue holding the data messages and one queue for holding the router messages. This enables the router to separate the router specific messages from the data messages and make it easier for the router to handle them differently. The new queue is called the router message queue (mq r ). Between two nodes the data exchange would be something like that shown in Figure 4.17b, first router messages and then data. When more than two nodes are in contact and want to communicate, the order in which messages are sent has been needed to be modified to take into account the sending of router messages, since both data and router messages can be able to be sent at the same time. This was not an issue in the ONE simulator were the exchange of router data is done outside the sending of messages. In the ORWAR router implementation the CWI messages are put in the mq r, and handled separately from the KDM and data messages, due to the natural reason that no contact window is estimated. The CWI message has been given the highest priority by the ORWAR router and is always sent first, or in FIFO order if more than one exists in the mq r. The assumption is that it is better to establish a new contact and estimate its contact window, than sending a data message. The KDM messages are created with the highest priority (3) are put in the data message queue and is handled in the same way as the data messages Redundant Transfers To avoid redundant transfers between nodes a data structure has been added to the router implementation, called the forward log. The forward log is used to keep track of all successful forwards of bundles. This is done to avoid replicating the same bundle several times to the same node, wasting resources. In addition to this the log keeps track of from which nodes the bundles in the router message queue has been received from, this is done to avoid sending a message back to a node that already has it. The entries in the log is kept until the TTL of the bundle is expired. Before replicating a message, it is checked with the forward log that the message has not been forwarded to or received from the destination before. The forward log is implemented by the class ForwardLog, Figure 4.15 shows the class diagram General description Figure 4.18 shows a high level description of the ORWAR router implementation of the bundle router. It consist of four parts; a queue that stores all bundles, the link manager that keep track of the current available neighbourhood of

53 4.7. Router 39 nodes, the forward log, the receive log and the known delivered messages log. The implementation is event-driven. Different events trigger the behaviour of ORWAR, e.g. the end of TTL for a message trigger the router to remove the message, or the discovery of a new contact trigger the router to start the message signalling. An attempt to replicate messages can be triggered by the establishment of a new contact, the insertion of a new bundle or the completion of a previous transfer. When a forwarding completes successfully an entry is added to the forward log and if the forward was the final delivery for the bundle it adds a new entry in the known delivered messages for the bundle. When receiving a bundle that is delivered to the application, the router s known delivered message log is updated, otherwise it is put in the bundle queue according to its utility and the bundles list of endpoint identifiers is updated. Bundle Router ORWAR Bundle Router Bundle Reception Logs Bundle Queue Router Bundles Data Bundles Known Delivered Messages Forward Log Receive Log Link Manager Bundle Replicator Detection Agent Neighbourhood Figure 4.18: The ORWAR Bundle Router

54 Chapter 5 Testing and Evaluation This chapter starts with a description of the testing and debugging of the implementation, and then continues with describing the method to simulate and validate the implementation of the ORWAR algorithm and the result of the simulations is shown and evaluated. 5.1 Testing During the development of the implementation smaller unit and module test were performed to test that the most important parts of the implementation of each module worked as intended. The most complex part, containing most of the functionality is the bundle router, so the focus of the testing has been on the router implementation and its interaction with the link manager and neighbourhood detection agent. Before performing the final simulations, three small simulation scenarios were used to test the behaviour and functionality of the whole implementation, after this scalability tests were performed to check that the implementation worked with a larger number of nodes Functional Tests The first scenario consisted of two stationary nodes placed according to Figure 5.1a communicating with each other to test single hop behaviour. In the second scenario Figure 5.1b, one stationary and one moving node was used to test the contact window estimation. In last scenario three stationary nodes were placed according to Figure 5.1c to test the multi hop behaviour. (a) One hop topology (b) One hop topology (c) Multiy hop toplogy Figure 5.1: Topologies used for functional testing 40

55 5.2. Evaluation 41 The advantage of using these smaller scenarios is that it is possible to know what the end result of the simulation should be for it to be correct. The tests showed promising results, and only minor adjustments were needed to get the intended result. However the stationary test lead to the realisation of the special case were the contact window is estimated to be infinite, as described in section 4.6 which could lead to problems if not handled correctly Scalability Tests Following the success of the functional tests, scenarios with larger number of nodes were tested. This proved to be the most time consuming part of the debugging phase, when testing with 100 nodes using random walk mobility, memory leaks were discovered that stopped the simulations, by either consuming memory until the computer stopped by the slowdown or causing segmentation faults that aborting the simulation. Several bugs had to be fixed before the system was able to simulate several nodes and showed scalability. 5.2 Evaluation The goal of this thesis was to evaluate the performance of ORWAR routing under more realistic circumstances, by performing more detailed simulations and compare with the previous simulations done in the ONE simulator. To get directly comparable results the same simulation scenarios that were used in one has been recreated as faithfully as possible in ns Simulation recreation The recreate the simulation scenarios used when simulating ONE, the mobility model, the message traffic and the nodes needs to be transferred or recreated in ns-3. The mobility and message generation has been transferred from the ONE to ns-3 by using trace files. ONE supports generation of both mobility traces in the ns-2 mobility format and message traces. The mobility traces were directly supported by ns-3, and could be used without any extra work, to inject the messages into the simulations a new help function was created as described in section 4.1. The nodes were recreated with the existing functionality for creating nodes in ns Simulation set-up All the simulations used to evaluate ORWAR runs in a 4500m x 3500m playground, representing a city scenario in down-town Helsinki city, Finland. Figure 5.2 shows a map of the simulated area. A total of 126 nodes divided on three types populate the city, 80 pedestrians, 40 cars and 6 trams, Table 5.1 shows their different characteristics. The mobility model used is map based movements resulting in realistic movements for then nodes, cars drives on the roads, the pedestrians follow the side walks and the trams follow the tracks in the streets. This is illustrated on the map, Figure 5.2.

56 42 Chapter 5. Testing and Evaluation Figure 5.2: Map of down town Helsinki, [7] Table 5.1: Node types Node type Range Bandwidth Buffer size Speed Pedestrian 10 m 2 Mb/s 5 MB km/h Car 20 m 2 Mb/s 50 MB km/h Tram 20 m 2 Mb/s 50 MB km/h Figure 5.3: Size distrubution in standard message set S, taken from [17, p.6, Figure 4]