Masterarbeit. Implementation and Performance Evaluation of the IETF QoS NSLP Protocol

Transcription

1 Georg-August-Universität Göttingen Zentrum für Informatik ISSN Nummer GAUG-ZFI-BM Masterarbeit im Studiengang "Angewandte Informatik" Implementation and Performance Evaluation of the IETF QoS NSLP Protocol Bernd Schlör Bachelor- und Masterarbeiten des Zentrums für Informatik an der Georg-August-Universität Göttingen 30. Oktober 2007

2 Georg-August-Universität Göttingen Zentrum für Informatik Lotzestraße Göttingen Germany Tel. +49 (5 51) Fax +49 (5 51) WWW

3 Ich erkläre hiermit, dass ich die vorliegende Arbeit selbständig verfasst und keine anderen als die angegebenen Quellen und Hilfsmittel verwendet habe. Göttingen, den 30. Oktober 2007

4

5 Masterarbeit Implementation and Performance Evaluation of the IETF QoS NSLP Protocol Bernd Schlör 30. Oktober 2007 Betreut durch Prof. Dr. Fu Computer Networks Group Georg-August-Universität Göttingen

6 Acknowledgement The author would like to thank (in alphabetical order) Gerald Ash, Attila Bader, Roland Bless, Xiaoming Fu, Dieter Hogrefe, Ingo Juchem, Cornelia Kappler, Georgios Karagiannis, John Loughney, Jukka Manner, Andrew McDonald, Niklas Steinleitner, Martin Stiemerling, Hannes Tschofenig, John-Patrick Wowra and the IETF NSIS working group for discussions, feedback and code reviews. 2

7

8 Abstract Today s multimedia applications, like VoIP or video streaming, work best with an allocated bandwidth, low latency and low packet loss. However, the traditional Internet works on a best-effort basis and the available resources cannot be assigned to certain flows. Quality of Service (QoS) mechanisms have been introduced to overcome these limitations and to provide more reliability in terms of bandwidth, delay, jitter and packet loss. Since the patience of humans is finite, there is a demand for a higher quality in IP based multimedia services and thus there is a demand for deploying QoS for these services. 4

9 Contents 1. Introduction Problem Description and Motivation Thesis organisation Background Integrated Services (IntServ) Resource ReSerVation Protocol (RSVP) NSIS Framework GIST Transport Layer QoS NSLP Session Management Route Changes Retransmissions Session Teardown QSPEC QoS Model Controlled-Load Service (QOSM CLS) Implementation Software Design State Machine General Design Actions and Transitions Traffic Control Interface Implementation of the Controlled Load Service QoS Model Implementation of a Timer Library using Double Linked Lists NSIS on a Linksys WRT54GL running OpenWRT Evaluation Comparison between DLL and XORP Timer Performance Evaluation Testbed Set-up Measurement Methods and Tools Testing Method and Results

10 Contents 5. Conclusion 55 A. QoS NSLP Parameters 57 B. QSPEC Parameters 64 C. QSPEC Objects for CLS 70 D. FSM Diagrams 75 E. Installing OpenWRT on a Linksys WRT54GL 94 E.1. Router Configuration Configure Static IP of Router WLAN Configuration E.2. NSIS on OpenWRT Bibliography 99 6

11 List of Figures 2.1. RSVP Multicast Reservation NSIS Framework [1] QoS NSLP Design [2] QoS NSLP Reservation Styles [2] Aggregation [2] Local QoS Models [2] QoS NSLP Design Overview QoS NSLP FSM Design Linux Kernel Traffic Control [3] CPU usage of 100,000 DLL and XORP Timers CPU usage 5 Million DLL Timers Testbed Setup CPU consumption at QNI D.1. ST_IDLE D.2. ST_WR D.3. ST_INST

12 1. Introduction Multimedia applications, like VoIP or Video on Demand, work best with a certain amount of reserved bandwidth. The traditional Internet is based on a best-effort forwarding, i.e. no distinction is made between different priority levels of packets and no firm guarantees are given. For non-real-time traffic, like or file transfer, this has not been a problem so far. However IP-based telecommunication and video broadcasting applications are delay sensitive. According to ITU-T [4] delays in telecommunication environments should not exceed 150 ms since they are noticeable for humans. A delay of 300 ms is still acceptable while delays over 400 ms are not acceptable. In those networks QoS signalling may be deployed. The Internet Engineering Task Force (IETF) is working on standards to improve existing signalling services in the Internet that provide and maintain a flow-specific control state. The Next Step in Signaling (NSIS) working group has created the Quality of Service Signaling Layer Protocol (QoS NSLP) to meet these requirements. Together with the General Internet Signaling Transport (GIST) it provides end-to-end IP bandwidth and traffic control functionality similar to RSVP and extends it in several ways. The most remarkable differences to RSVP are transport layer security (TLS), sender and receiver initiation, support for more QoS Models than just IntServ and Diffserv and support for local domains and mobility. 8

13 1. Introduction 1.1. Problem Description and Motivation The resources within a network are limited. Often the requested bandwidth by an application can not be fulfilled. Since network resources are not infinite latency, jitter and packet loss occur across a network. QoS allocated network resources fulfil the demand for the traffic an application is going to send. On the way from the source to the destination packets traverse routers having limited capacities in terms of buffering and forwarding capability. A congestion situation occurs when packets are sent at a higher rate than a router can process these packets. Routers either buffer the packets until the congestion situation has ended or packets are dropped. The results are increased latency and packet loss. Traffic that is less intolerant to latency can be processed with a higher priority or earlier than traffic that is more tolerant to latency. Therefore different traffic classes have to be identified and associated with the existing resources. The packet classifier separates the traffic into different flows. Each flow is assigned to a certain queue for the corresponding network device. The queueing algorithm handles the rate at which packets are sent. The set of classification, queues and queueing algorithms is referred to as traffic handling mechanism. A proper configuration of the traffic handling mechanisms provides a proper end-to-end service through the network. Examples of traffic handling mechanisms include among others Differentiated Services (Diffserv), Integrated Services (IntServ), Asynchronous Transfer Mode (ATM). The IETF Next Steps in Signaling (NSIS) working group was founded in The effort of this working group is to create a new signalling framework that meets the upcoming requirements on Quality of Service in the Internet. The NSIS Transport Layer Protocol (NTLP) and the NSIS Signaling Layer Protocols (NSLPs) together form the NSIS protocol stack. 9

14 1. Introduction This thesis provides background information about RSVP and NSIS and compares them. The main focus of this thesis is an open source implementation of the NSIS QoS NSLP with the support of signalling IntServ Controlled Load Service and its evaluation. The implementation was done in a research project at the University of Goettingen. Several software design features were introduced during the implementation work which yields in a stable and timely responsive application. The design features include a state machine, a traffic control interface and a timer library which together cover the whole functionality of the implementation Thesis organisation The thesis is structured as follows. Chapter 2 provides information about the NSIS Framework and gives background information about the related protocols. Chapter 3 discusses the implementation details. In chapter 4 the results of the performance evaluation are presented. Chapter 5 concludes the thesis. 10

15 2. Background This chapter provides background information about the protocols which are used for the implementation described in this thesis and related protocols in comparison. It gives an overview about the whole protocol stack and discusses essential protocol details. Due to space and time limitations not all details are provided. Interested readers should refer to the given references Integrated Services (IntServ) IntServ is an architecture to provide QoS in IP networks. It defines two services: Guaranteed Services (GS) and Controlled Load Service (CLS). While GS provides fixed limits to packet delay, a certain level of bandwidth and no packet loss, CLS is more comparable to an unloaded network offering best-effort. CLS is used where traffic can tolerate a reasonable level of delay and packet loss. Up to now IntServ has only been signalled with the Resource ReSerVation Protocol (RSVP) which installs state on nodes within a network on a per-flow basis. RSVP is discussed more detailed in the following section Resource ReSerVation Protocol (RSVP) RSVP [5] has been designed to enable senders, receivers and intermediate nodes to set up reservation state per flow. RSVP itself is not a routing protocol but relies on underlying 11

16 2. Background routing mechanisms. It is designed to handle multicast- and unicast sessions. RSVP as a layer-3 protocol is independent of the underlying network media. RSVP messages refer to a particular session. A session consists of the IP address, transportlayer protocol type and port numbers of the destination [6]. Every message carries a unique session identifier to assign the message to the particular session. There are mainly two control messages to signal for reservations: Path and Resv. Path messages consist of the following information: Phop, which is the address of the previous RSVP hop. Every RSVP router updates this information. Sender Template, specifies the IP address and port (optionally) of the sender. Sender Tspec, defines the traffic characteristics the sender is going to send. Adspec, which is optionally included and updated at every RSVP capable node to inform the receiver about available resources along the path. A Resv message typically contains: Filterspec, is identical to the Sender Template to identify the sender. Flowspec, contains traffic specifications and is usually set to the Sendet Tspec. ResvConf object, is optionally included to request a confirmation. A Path message is issued by the sender and includes a SenderTSpec specifying the traffic an application is going to send. The RSVP AdSpec is optionally included. It probes for the available bandwidth on the data path. This reservation model is referred to as One Pass with Advertising (OPWA). When the AdSpec is omitted, the Receiver cannot determine 12

17 2. Background Figure 2.1.: RSVP Multicast Reservation the available resources for the resulting end-to-end QoS. This reservation model is referred to as One Pass. On arrival at the Receiver the FlowSpec, consisting of the TSpec, is created. The TSpec is usually derived from the SenderTSpec and if available from the AdSpec. It contains the desired QoS. The Resv message is sent upstream to the Sender. At each hop the desired resource reservation is reserved. The last node sends a ResvConf back to the Receiver to indicate that end-to-end reservation has been installed. The figure below shows the direction of RSVP messages for an example multicast session. The Sender S1 initiates a Path message downstream to discover RSVP capable routers along the path and to provide the receiver with traffic characteristics the sender is going to send. Each router on the path updates the Path message and installs reverse routing state. The Resv messages in response are issued by the receivers RCV1-RCV3 in upstream direction and carry the actual reservation request. If a node cannot satisfy the requested QoS the Resv message is dropped and a rejection message is sent back to the receiver. If all nodes along the path meet the QoS request the reservation is installed. 13

18 2. Background Figure 2.2.: NSIS Framework [1] 2.3. NSIS Framework The IETF NSIS protocol suite [7] consists of the following two layers: General Internet Signaling Transport (GIST) [8] NSIS Signaling Layer Protocols (NSLPs), such as QoS NSIS Signaling Layer Protocol (QoS NSLP) [9], Network Address Translation (NatFW) NSLP [10], and NSLP for metering [11]. The NSIS protocol suite enables signalling applications to install, modify or remove control state in a network. This control state is related to a data flow the application wants to send. 14

19 2. Background Paul P. White defines a data flow in [6] as follows: A data flow identifies the set of packets to receive special QoS. It is defined by a session comprising the IP address, transport-layer protocol type, and port number of the destination along with a list of specific senders to that session that are entitled to receive special QoS. The NSIS Transport Layer Protocol (NTLP) transports messages up- and downstream along the signalling path. One or more NSLPs run on top of the NTLP. NSLPs are distinguished according to their NSLP-ID. The two-layer architecture can be seen in figure 2.2. The QoS NSLP is not the only signalling application. Other examples are an NSLP for metering [11] and for Network Address Translation (NatFW) [10]. The NSIS Transport Layer Protocol operates between adjacent peers; the end-to-end messaging is handled at NSLP level. Each session is identified by a unique session identifier. Intermediate nodes are bypassed where the corresponding NSLP is not present, i.e. the NTLP only forwards messages to the upper layer NSLP where the NSLP-ID matches. Where the corresponding NSLP is not present, messages are forwarded. The NSIS protocol suite provides support for sender and receiver-initiated reservation, uniand bidirectional operations, as well as aggregation. Bidirectional reservations are useful where two unidirectional flows form a relationship, e.g. where QoS is reserved for VoIP calls. Thus decisions for particular sessions can be derived from the state of other sessions. In the example with the VoIP one side can tear down two unidirectional sessions which were bound together. NSIS reacts on route changes by discovering and maintaining a new route. Route changes are found in mobile and multi-homed environments where a mobile client moves from one AP to the next, but also in the fixed network. 15

20 2. Background 2.4. GIST Transport Layer GIST, as the main building block of NSIS, is responsible for the discovery of signalling peers and for the delivery of signalling messages along the data path. It reuses existing transport layer protocols, like TCP, UDP and SCTP. GIST establishes and maintains Message Routing State (MRS) per session using a three-way handshake (Query, Response and Confirm). For the discovery of the next hop, a Router Alert Option (RAO) is set in the IP header of the Query message. GIST is a soft state protocol and therefore the MRS needs to be refreshed regularly. The refreshes are sent in UDP with set RAO to discover route changes. The communication between the two protocol elements is done via an Application Programmer Interface (API). NSLP messages are transported by GIST as payload and passed through the API. The NSLP-ID in GIST messages ensure that only relevant messages are passed to the associated NSLP. The routing information, used to describe a flow, are carried in the Message Routing Information object (MRI). It carries the 5-tupel protocol, source/destination address, address prefixes and ports with some additional flags. The MRI is transferred between NTLP and NSLP over the SendMessage() and RecvMessage() API-calls QoS NSLP The design concept of the QoS NSLP together with GIST is similar to RSVP and extends it. Besides the receiver-initiated reservation, sender-initiated and bidirectional reservations are also supported. Reservations are performed hop-by-hop. Like RSVP, the QoS NSLP uses soft state as state management. Unlike RSVP, there is no support for IP multicast. QoS requests are either requested by a local application or arrive over the network by GIST 16

21 2. Background Figure 2.3.: QoS NSLP Design [2] and are passed to the QoS NSLP. A local application includes user applications, e.g. multimedia applications but also network management to handle initiating a tunnel, bypassing a node or performing aggregate reservations. Incoming messages are processed at NSLP level. For state manipulating operations or to query the available bandwidth for an interface, interaction with the Resource Management Function (RMF) is required. The RMF is responsible for managing the actual resource reservation and is discussed in more detail in section 2.6. Policy Control determines whether the reservation request is authorized. Admission Control checks whether enough resources are available for the request. The QoS NSLP supports four Message Types: RESERVE 17

22 2. Background QUERY RESPONSE NOTIFY The RESERVE message is used to create, refresh, modify or remove a QoS NSLP reservation state. This is the only message that manipulates reservation state. A QUERY message is used to collect information about available bandwidth on the reservation path. A QUERY message does not manipulate any reservation state. The RESPONSE message provides information about previously installed reservations. It is used to signal the success of an installed reservation, as a response to a QUERY message, or it can contain errorspecific information. Also, the RESPONSE message does not modify any reservation state. A NOTIFY message is sent asynchronously, i.e. not in response to a previously received message, and typically indicates error conditions. It does not refer to any particular state or any received message in general. Control information needed for state installation, removal or refreshes are carried in messagespecific objects. Examples are sequence numbers, session IDs or objects requesting a response. The QSPEC object which is described in section 2.6 consists of a number of further objects which are passed opaquely to the RMF. This object describes specific information about the actual resource reservation, like bandwidth, token bucket or Diffserv Code Point (DSCP). The Policy object is used to authorize resource reservations. Like RSVP, the QoS NSLP supports receiver-initiated reservations, where the RESERVE message is sent in the opposite direction of the data flow. An extension to RSVP is the support for sender-initiated reservations where the RESERVE message travels in the same direction as the data flow. 18

23 2. Background A QNE is an NSIS Entity (NE) that supports QoS NSLP. Intermediate nodes are referred to as QNEs but also end node are QNEs. The QNI is an end node that initiates a reservation request. The QNR is the last node on the data path and acts as the responder of a reservation request. To make a sender initiated reservation the QNI creates a RESERVE containing the required QoS parameters in the QSPEC object. The message is transported by GIST hop by hop to the QNR, the final destination of the message. Every intermediate node (the QNE) creates a new RESERVE message based on the received one. If a response has been requested the QNR creates a RESPONSE message indicating the success of the reservation. The sequence of the messages can be seen in diagram 2.4. Figure 2.4.: QoS NSLP Reservation Styles [2] 19

24 2. Background For a receiver initiated reservation, as it is done by RSVP, three messages are exchanged: QUERY, RESERVE and RESPONSE. The QNR as the sender issues a QUERY and informs the QNI as the receiver about the desired bandwidth. The requested resources are contained in QoS Desired. Resource information about the path is collected in QoS Available. The receiver copies the content of QoS Available into QoS Desired. The QNI is updated about available resources before sending the RESERVE. A RESPONSE is sent back to confirm the reservation. A bidirectional reservation is a combination of two sender-initiated reservations or a combination of a sender-initiated and a receiver-initiated reservation. This combination is done by binding the two reservations together. For this purpose a BOUND_SESSION_ID object is inserted into the message at the QNI and at the QNR. The RESERVE message may travel along different QNE on the path due to asymmetric routing. Session Binding is done here to express the unidirectional dependency relation between the two sessions set-up for forward and backward reservation. Aggregate reservations are another case where Session Binding is used for. Aggregation is done to reduce the amount of reservation state. The aggregate reservation needs an MRI describing the aggregated traffic. A different session is needed to handle this different MRI. At the edges of a local domain reservation and signalling aggregation, the local QSPEC concept or pre-emption influences the operation of the QoS NSLP and the RMF. The QNI request end-to-end resources by adding a QSPEC into the RESERVE message. At the edges of a QOSM X domain the end-to-end semantic is preserved by adding a local QSPEC. The initiator QSPEC is passed through the local domain unchanged. The RMF has direct influence on the QoS NSLP operation as it triggers messages to be sent. If a reservation fails the 20

25 2. Background Figure 2.5.: Aggregation [2] RESERVE message is not propagated further along the path, instead a RESPONSE message is created to inform the QNI about the failure Session Management The QoS NSLP supports a reduced form of refreshes to minimize the size of messages sent over the network. Reduced Refreshes use a RESERVE message containing an RSN object and the SESSION_ID and does not include a QSPEC object. If a QNE does not support reduced refreshes a RESPONSE message with a corresponding INFO_SPEC object is sent back. After a rerouting event a full RESERVE including a QSPEC is required. Summary Refreshes and Summary Tears are other options to reduce the message over- 21

26 2. Background head on the network. A refreshing RESERVE for a primary session may include a SES- SION_ID_LIST object that stores a number of Session Ids for secondary sessions to be refreshed. Additionally an RSN_LIST object containing the corresponding reservation sequence numbers has to be included. To tear down several sessions with one message the SESSION_ID object is included in the tearing RESERVE without RSN_LIST object. The QoS NSLP makes use of a number of objects. The objects and their functions are discussed here for a better understanding. A bi-level representation of the objects described in this thesis can be found in the appendix. The RII is a 32-bit random number which is used to match back a RESPONSE to a prior sent RESERVE or QUERY. This random number has to be unique within a session represented by a SESSION_ID. The RSN object consists of a 32-bit Reservation Sequence Number(RSN) and an Epoch Identifier. The RSN is incremented with every reservation state manipulating RE- SERVE message, e.g. state installation or state removal. The RSN has only significance between a QNE and its downstream peer and is incremented with each RE- SERVE message that modifies installation state. The Epoch Identifier is set at boot-up and enables a QNE to determine whether a peer has rebooted. In the REFRESH_PERIOD object the Refresh Period (R) is signalled between QNEs. In backbones, e.g., R may be larger than in the access network. The BOUND_SESSION_ID object consists of an 8-bit Binding Code and a 128-bit Session Identifier (SID). It is used when two sessions are bound together, e.g. in case of aggregation or session binding. Different flags in the PACKET_CLASSIFIER object show which information from the MRI will be used by the packet classifier. These flags include source/destination 22

27 2. Background address and prefix, source/destination port, DiffServ Code Point and some other. The INFO_SPEC object is used to indicate the success of a reservation or any error conditions. Additionally it can contain the IP address of the node where the error condition was found and one or more erroneous objects. The SESSION_ID_LIST object contains a list of Session Ids. It is used to refresh or teardown a larger number of sessions in order to reduce the message overhead on the network. Summary refreshes or teardowns can only be done in combination with an RSN_LIST object. It contains a list of the Reservation Sequence Numbers belonging to the Session ID list Route Changes The RecvMessage() API-call provides a Source Identification Information (SII)-Handle to the NSLP. The SII-Handle changes if the (downstream) signalling peer changes. The previous SII-Handle can be used to explicitly route messages to the old peer. This is used to tear down reservations on the old path which are no longer valid. A QNE can also keep the reservation on the old path by setting the REPLACE flag in the QoS NSLP common header. A mobile node, e.g. could move after a mobility event to the old attachment point (Ping Pong handovers) Retransmissions A QNI sending a RESERVE towards the QNR can request an end-to-end confirmation by including an RII object. If the QNR fails to send this response the QNI resends the RE- SERVE until the maximum number of retries has been reached. A QNE can also request a confirmation from its next peer by setting the ACK-Flag in the QOS NSLP header. If a QNE 23

28 2. Background decides to set an ACK flag for a refreshing RESERVE (it need not be set for every refreshing message), it will trigger the same retransmission algorithm which is used by QNEs that set an RII. After QOSNSLP_RETRY_MAX it will give up and send the Path truncated notification upstream Session Teardown A node that receives a tearing RESERVE interprets and compares the RSN in the message to the stored RSN. If the RSN values match, the QNE tears down the reservation and forwards the tearing RESERVE with the RSN object. If the received RSN is greater than the stored RSN, then the QNE also tears down the reservation and forwards the tearing RE- SERVE with the received greater RSN value. If the received RSN is less than the stored RSN, then the QNE silently drops the tearing RESERVE and does not tear down the reservation nor forwards the tearing RESERVE QSPEC The QSPEC is an object that carries QoS specific parameters. The QSPEC is part of the QoS NSLP signaling message. The QSPEC template [12] provides a set of parameters that is universally understood and interpreted by QoS Models (QOSMs) along the path. A QOSM enables QoS for a certain traffic flow, such as Diffserv or IntServ. The QoS NSLP operation is independent of the QOSM that is used. It describes the operation of the Resource Management Function (RMF), can trigger another session, e.g. in case of aggregation and triggers several messages that are processed by the QoS NSLP. The QOSM defines also the QSPEC parameters which are used. An abstract API for the communication between the QoS NSLP and the RMF is defined in [9]. Examples for QOSMs are: 24

29 2. Background RMD-QOSM [13] IntServ CLS QOSM[14] Y.1541 QOSM [15] The QSPEC parameters are similar to the TSpec, RSpec and AdSpec in RSVP and describe the QoS Desired, QoS Available, QoS Reserved and Minimum QoS. QoS Desired describes QoS parameters which are desired by a QNI. QoS Available describes the available resources along a path. The QSPEC in a resource Query or in a Query requesting a receiver initiated reservation collects information about the available bandwidth. QoS Reserved describes the resources which are actually reserved. QoS Reserved is included by the QNR and the information is derived from QoS Available. Minimum QoS is an optional parameter which can be included by the QNI to indicate a range of QoS which is acceptable. The range is between QoS Desired and Minimum QoS. If the available resources are less than Minimum QoS then the reservation fails. The QSPEC object is opaque to the QoS NSLP and only interpreted and modified by the RMF. A number of QSPEC parameters are defined. Please refer to the appendix for a bit level representation of the QSPEC objects: the TMOD parameter, which has to be present in every QSPEC, describes the traffic parameters. It indicates the desired, available, reserved or minimum resources along the path. constraints parameter, e.g. path latency or path jitter 25

30 2. Background Figure 2.6.: Local QoS Models [2] traffic handling directives, such as excess treatment traffic classifiers, like PHB class A local domain can define a local QSPEC which is used in this domain. As shown in figure 2.6 at the ingress to a local domain the initiator QSPEC is encapsulated in the local QSPEC. At the egress of the local domain the original QSPEC object is included into the QoS NSLP message again QoS Model Controlled-Load Service (QOSM CLS) The QOSM CLS makes use of mainly one QSPEC parameter: token bucket represented by TMOD. The TMOD parameter consists of the following elements: 26

31 2. Background TMOD Rate-1 [r] TMOD Size-1 [b] Peak Data Rate-1 [p] Minimum Policed Unit-1 [m] To signal CLS IntServ with the QoS NSLP, the SenderTspec and the AdSpec is translated to QoS Desired and QoS Available to query the network for available resources. For the RESERVE message, the FlowSpec is translated to QoS Desired and QoS Available. The QoS Available parameter is updated by all QNEs along the path. The optional RESPONSE messages carry the QoS Desired object to inform the initiator about the reserved resources. In contrast to RSVP, IntServ CLS can be signalled using sender and receiver initiation. The appendix provides bit-level representations of the QSPEC objects used for sender- and receiver initiated reservations. The message flows can be seen in figure

32 2. Background The following table shows a translation of objects and parameters between QoS NSLP and RSVP: Message Object(s) Parameter(s) RSVP Path Sender TSpec TMOD ADSpec avail. bw and MTU QoS NSLP QUERY <QoS Desired> <TMOD> (TMOD) <QoS Available> <TMOD> (bandwidth) RSVP Resv FlowSpec TMOD QoS NSLP RESERVE <QoS Desired> <TMOD> (TMOD) <QoS Available> <TMOD> (bandwidth) RSVP ResvConfirm QoS NSLP RESPONSE <QoS Reserved> <TMOD> (TMOD) In QoS NSLP, the sender populates the QoS which it desires by including a QoS Desired and optionally queries the network for the QoS that is available. In this case it carries the QoS Available parameter which is updated by all QNEs to reflect the QoS that is actually available. With the QoS Desired object, the Receiver (QNI) is informed about the requested resources. 28

33 3. Implementation In this chapter the software design of the QoS NSLP and the CLS QOSM done at the University of Goettingen is explained. The functionality and the interactions of the single software modules is presented to give an overview how the implementation works. The appendix provides state machine diagrams and a subset of the developed source code. The full source can be downloaded under [16] Software Design The implementation of the QoS NSLP as well as the GIST and NatFW code is written in C++ and the main target platform is Linux. The software design is object-oriented and each protocol layer is running as a single-threaded process. The communication between the NSLP and NTLP level is done via UNIX sockets. The structure of software modules and how they interact can be seen in figure 3.1. The following items explain the functionality of the modules one by one: GIST is responsible for transporting message from the QoS NSLP to the next node and for delivering messages coming from the network to the QoS NSLP. For delivering the messages the API-call RecvMessage() is used. To send messages from the QoS NLSP to the next node over the network the API-call SendMessage() is used. The QoS Engine listens for incoming messages from the RMF and for messages com- 29

34 3. Implementation Figure 3.1.: QoS NSLP Design Overview 30

35 3. Implementation ing over the network from GIST. Messages that come from the RMF can be generated by the RMF itself, may be triggered by the local application or come from network management. For each new SessionID (SID), a new State Machine (FSM) is created A table is searched for the given SID. If it is not found, a new FSM is created and its address together with the SID is added to the table. If the SID is found, the address of the FSM is returned. Incoming Messages are parsed into objects. The FSM is triggered with the corresponding event to the message. If, e.g. a RESERVE message has arrived the event EV_RX_RESERVE is triggered. If the type of the message is a QUERY with set R-Bit, a receiver-initiated reservation is requested and the node position is QNR; in all other cases, the node position is QNI. If the FSM returns to the QoS Engine in IDLE, it is deleted together with its table entry. The FSM performs the processing of the messages. Reservation requests or tear down requests are passed to the RMF. The RMF triggers new messages which are processed by the FSM. Messages are sent out with GIST using the SendMessage() API-call. Timers are started for messages where a response is requested or for state time-out. A detailed description of the FSM is given in section 3.2. The RMF is the location where the QOSM is implemented. A generic API is used between the FSM, RMF and the QoS Engine. A variety of QOSMs can be attached to the FSM using the API. The RMF performs resource queries for a received QUERY message. It interacts with the Traffic Control interface using the Unix system call system(). The RMF also constructs QOSM dependent QSPECs. A more detailed description of the RMF follows in section 3.4. The Timer class used in this project is an own implementation from scratch. It is based on double link lists and hash tables for the timer management. Unix signals are used for time out generation. See section 3.5 for a detailed description of how 31

36 3. Implementation they work and how they are implemented. The Local Application communicates with the RMF by using the NSLP-API call send_rmf(). The Local Application is the interface between the QoS NSLP and a local application, e.g. a multimedia application that want to reserve bandwidth for an audio stream. It also serves as an interface for network management. A network management application can create tunnels, configure a QNE as an ingress node, bind sessions together and many more State Machine A Finite State Machine (FSM) can be useful with even two states when having many events. An example for two states could be: light on and light off. Examples for events could be entering a room and leaving a room. The example state machine transitions from light off to light on when entering a room. And when leaving the room this state machine transitions from light on to light off. Using a FSM yields in a clear software structure. FSMs consist of four main elements: states, state transitions, input events and a set of rules to allow state transitions. To a given state, a certain event triggers an action: the execution of a callback function (the event handler). The initial state machine implementation was taken from the ISDN driver of the Linux Kernel. It was ported from C to C++ and, by using Templates, it can be included in any sub project, like GIST, NatFW or QoS NSLP. All possible states, events and resulting callback functions are stored in an array, the jumpmatrix. The FSM can be operated using two functions. TriggerEvent() executes to a given state and a given event to corresponding callback function. SetState() manipulates the in- 32

37 3. Implementation Figure 3.2.: QoS NSLP FSM Design ternal state of the FSM. The state machine of the QoS NSLP has around 1000 lines of code and provides almost all functionality for the processing of messages. More background information about FSMs and the FSM framework presented in this thesis can be found in [17] General Design Figure 3.2 provides an overview of the QoS NSLP FSM design. Three states are defined for the operation: ST_IDLE ST_WR 33

38 3. Implementation ST_INST The state ST_IDLE (state idle) is the initial state when the FSM is created. In this state no reservation state is installed and no timers are running. In the state ST_WR (state wait response) the FSM is waiting for a response to a previously sent message. A timer may be set in this state to retransmit the initial message, if no response comes back. In ST_WR no reservation state is installed. This is the main difference to the state ST_INST (state installed) where reservation state for a given session is installed. The events are defined as follow: EV_RX_QUERY, QUERY message received EV_RX_NOTIFY, NOTIFY message received EV_RX_RESERVE, RESERVE message received EV_RX_RESPONSE, RESPONSE message received EV_RX_RMF_MSG EV_TIMEOUT_RESPONSE, Wait-Response interval timer expiration EV_TIMEOUT_REFRESH, Refresh interval timer expiration EV_TIMEOUT_STATE_LIFETIME, State lifetime timer expiration EV_SII_CHANGED EV_NETWORK_NOTIFICATION Actions and Transitions This section describes the operation of the FSM. Only relevant functions are considered. Please refer to the state diagrams in the appendix. 34

39 3. Implementation State ST_IDLE idle rx_query This function is executed when a QUERY message has arrived over the network from GIST and the FSM is in State ST_IDLE. If a receiver initiated reservation has been requested and the node position is QNI, Direction is set to upstream and the message is passed to the RMF. Transition is made to ST_WR. In all other cases a resource query has been received, the message is passed to the RMF and transition is done to ST_WR. idle rx_notify A NOTIFY message has been received. If the node position is not the final destination for this message, it is passed further along the path. idle rx_reserve A RESERVE from the network in ST_IDLE has arrived. If the T-Flag is set, then the message is passed to the RMF to tear down the reservation. Transition is made to ST_IDLE. In the other case where a reservation request has arrived, Direction is set to UPSTREAM at QNRs. On state full nodes the state timer is started. In any case the message is passed to the RMF and transition is made to ST_WR. idle rx_response In ST_IDLE the node is not expecting a RESPONSE message. It is forwarded further to the final destination. The state ST_IDLE is kept. idle rx_rmf_msg If the RMF is triggering a RESPONSE message and the containing INFO_SPEC object indicates either a reservation failure or that the tear down was successful the message is passed further and transition is done to ST_IDLE. If a QUERY message has been triggered, it is checked whether the R-Flag is set or an RII object is present. In this case the response timer is started and transition is made to ST_WR. If a normal QUERY containing no R-Flag and no RII object, no response timer is started and transition is made to ST_IDLE. In case the RMF has triggered a RESERVE message at state full intermediate nodes the state timer is started. If it contains no RII object, then no response is triggered and the refresh timer is started 35

40 3. Implementation immediately to frequently refresh the session. If an RII object is included in the message the response timer is started instead of the refresh timer. The message is sent further along the path. At the QNI no state timer is started as this is the node which takes care of the refreshes. It does not receive any refreshes which would restart the state timer. The message is sent further along the path. The QNR is starting only the state timer. As this is the end node it does not propagate the message any further. No responses are expected and no refresh messages will be sent. Transition is made to ST_INST. State ST_WR wr rx_query A QUERY message has arrived over the network from GIST and the FSM is in State ST_WR. The message is entirely passed to the RMF using send_rmf(). Transition is made to ST_WR. wr rx_notify If the node is not the final destination of the NOTIFY message it is sent further along the path and transition is made to ST_WR. wr rx_reserve If a reserve message with set T-Flag has been received the message is passed to the RMF to tear down the reservation to to construct a new tearing reserve message. If no confirmation of state removal has been requested, i.e. no RII object is present, transition is done to ST_IDLE. QNRs and stateless nodes also go directly to ST_IDLE. If the RII object in the tearing reserve is present, state full nodes except QNRs start the state timer for one refresh period and pass the message to the RMF to remove reservation state. Transition is done to ST_WR. If the received message is requesting a reservation the state timer is started. The message is passed to the RMF to install reservation state and transition is made to ST_WR. If the node position is QNR then this is the requested RESERVE message for the Receiver Initiated Reservation. The Response Timer is stopped. 36

41 3. Implementation wr rx_response In ST_WR a RESPONSE message has arrived. A QNE_Egress forwards the response message is with the BYPASS_STATELESS_ID as nslp_id that stateless nodes do not intercept this message. If an Ingress node receives a response with the BYPASS_STATE-LESS_ID, the nslp_id is translated to the QOS_NSLP_ID and the message is forwarded up to the requesting node. Normal QNEs just forward the response message. At this point no state transition is done. The relevant checks follow in the next paragraph. If the received RII is from the local node the response timer is stopped. If there are no more outstanding RII, either from the local node or from foreign nodes transition is done to ST_IDLE otherwise ST_WR. If the received RII was from a foreign node the same transitions are done depending on the outstanding RIIs. For a stored foreign RII no response timer was started and therefore no response timer has to be stopped. wr rx_rmf_msg See the section where a RESERVE messages has been triggered. Transition is made to ST_INST. wr timeout_response A Response Timer has timed out while in ST_WR. If the maximum number of retries has been reached, transition is done to ST_IDLE if no more Response Timers are pending, otherwise transition is done to ST_WR. If the maximum number of retransmissions has not been reached, then the query message is resent. The response timer is restarted and transition is made to ST_WR. wr timeout_statelifetime If the state timer in ST_WR times out the confirmation message for state removal has not been sent. In this case no retransmissions of the message is done and transition is done to ST_IDLE after stopping all timers. wr rx_rmf_msg In ST_WR any triggered message from the RMF is forwarded and the same state is kept. 37

42 3. Implementation State ST_INST inst rx_query See wr rx_query. Transition is made to ST_INST. inst rx_notify Reduced Refreshes are the default for refreshing RESERVEs. Notify messages are checked whether the next peer accepts reduced refreshes or not. The state ST_INST is kept. inst rx_reserve All incoming RESERVEs are checked for their RSN value. if it is smaller than the stored peer RSN the message is rejected. If the value is the same to the stored one the reservation is refreshed. Larger RSN value modify the reservation state. A tearing reserve is passed to the rmf to remove the existing reservation state. If no confirmation is requested, i.e. no RII object is present, transition is done to ST_IDLE. If a confirmation is requested, the state time is started for one refresh interval and transition is done to ST_WR. If no T_Flag is set the message is passed to the RMF to install or modify the reservation if the RSN value is larger than the stored peer RSN. The state timer is restarted for equal and larger RSN values. The state ST_INST is kept. inst rx_response In ST_INST a RESPONSE message has arrived. If the node position is QNE and no Scoping Flag is set, then the RESPONSE message is forwarded. If the RII object is stored to match back a response, then the Response Timer is stopped. If the received Error Code of the INFO object is SUCCESS, then the Response Timer is started. Transition is done to ST_INST. If the Error Code is FAILURE, then the reservation state is deleted and pending timer are stopped. Transition is made to ST_IDLE. inst timeout_response A Response Timer has timed out while in ST_INST. If the maximum number of retries has been reached, then all pending timer are stopped and the RMF is informed to remove existing reservation state. Transition is made 38

43 3. Implementation to ST_IDLE. If maximum numbers of retransmissions has not been reached, then previously sent message is resent. The Response Timer is restarted and the state ST_INST is kept. inst timeout_refresh A refreshing RESERVE message is created. If Reduced Refreshes are accepted, no QSPEC object is added. The refresh timer is being restarted. Transition is made to ST_INST. inst timeout_statelifetime The state life timer timed out in ST_INST. All active timer are stopped and existing reservations are removed. The RMF is informed to tear down reservation state and to trigger a tear down message. Transition is done to ST_IDLE. inst sii_changed A different SII-Handle indicates that route change has occurred. A full reserve including QSPEC is sent on the new path. The reservation on the old path is torn down using the old SII-Handle. The state ST_INST is kept. inst network_notification A network notification from GIST is indicating a route change has occurred. A full reserve including QSPEC is sent on the new path. The state ST_INST is kept. inst rx_rmf_msg See idle rx_rmf_msg for details. Processing of Messages in the RMF Message Type Reserve At the QNI if the message has set the T-Flag the reservation is torn down and a new reserve message is triggered using the API-call recv_rmf(). If no T-Flag was set in the reserve message, reservation state is installed. If successful a new reserve message is triggered to be sent further along the path. 39

44 3. Implementation At QNEs the (tearing) reserve is sent further if the scoping flag is not set. Otherwise this node triggers a response if an RII object is present. If a reservation request fails a response is triggered to inform the initiator about the failure. If reduced refreshes are supported a notify message is sent back to the previous peer. Ingress nodes trigger two new (tearing) reserve message, one with the old nslp_id and one with the BYPASS_STATELESS_ID to bypass stateless nodes. No special treatment of the messages is done at interior nodes. The egress node triggers a response message, if a response was requested by including an RII object and if the nslp_id of the received message is QOS_NSLP_ID. If a requested reservation fails, a response is sent back in any case. Notify message for reduced refreshes are sent with the corresponding nslp_id. The QNR as the destination of the message echoes a response if the reservation fails or if a response was requested by including an RII object. Also here a Notify message is sent back to the previous node, if reduced refreshes are accepted. Message Type QUERY If a QUERY message with set R-Flag arrives at the QNI, the RMF triggers a new RESERVE message to be sent towards the QNR. Resource query messages are sent further after updating the QSPEC. The QNR triggers a response message containing the QSPEC Traffic Control Interface Linux Kernel from version 2.3 on is able to control the way packets are transmitted and received on a router. The Traffic Control (TC) interface determines the order and the rate 40

45 3. Implementation at which packets are transmitted. Traffic conditioning is usually performed on the outgoing interface. Some configurations may also need traffic conditioning on the incoming interface. Queuing mechanisms with different priority levels control the traffic of different network services. SSH, e.g., requires low bandwidth but short response times, where VoIP demands a constant bit rate. File transfer traffic is usually insensitive to changes in bandwidth and delay and works well with a lower assigned priority. Outgoing packets are first queued before transmission. If the queue is full, packets may be dropped, shaped or remarked. A traffic shaper delays metered traffic. Dropping out of bound traffic simply discards packets exceeding bandwidth limitations. Remarking enables out of bound traffic to use unallocated bandwidth resources. A queuing algorithm controls the way in which packets are en-queued and de-queued. The QoS NSLP implementation makes use of the Hierarchical Token Bucket (HTB). The HTB is based on the token bucket mechanism. A bucket is assigned to each class of the outgoing traffic. The bucket is filled periodically with tokens. If there are tokens in the bucket, data can be dequeued from the class. If there is no data in the class to be sent the token can be saved for later use. The number of tokens are limited. The maximum number of tokens is equivalent to the maximum burst size. The HTB can handle multiple token bucket classes in a tree. Tokens which are not used by one class can be borrowed by another class. Different priority levels can be assigned to the classes within the tree. A higher priority yields in a shorter queueing delay. Figure 3.3 shows that incoming packets are first demultiplexed. Packets that are not for the local machine and the packets from the local machine that are being transmitted are sent into the forwarding queue. Packets on the outgoing interface are queued in a buffer and sent out to the network. Linux TC affects packets on the Input Demultiplexing and on the output queue. Filters are used to selectively drop, buffer or forward packets. 41

46 3. Implementation Figure 3.3.: Linux Kernel Traffic Control [3] 3.4. Implementation of the Controlled Load Service QoS Model The RMF is responsible for the reservation on the network device. The complete message that comes from the QoS NSLP is passed to the RMF. There are four main functions used within the RMF: create_qspec(), constructs the QOSM-dependent QSPEC. query_rmf() for available bandwidth reserve_qos() to perform the actual reservation using TC and iptables to mark packets free_qos() to release the reservation again. 42

47 3. Implementation The RMF determines and triggers the message which is sent in response to the received one. For a received message one or more than one messages can follow in response. If a reservation request is successful, a new reserve is created and passed to the FSM to send it further towards QNR. If the reservation fails, a response message is generated to send it back to the originator of the reservation. On the edges of a stateless domain, e.g. in case of RMD, the RMF triggers two new reserve messages. According to [18] the token bucket specification includes the token rate r bucket depth b peak rate p minimum policed unit m In TC the following parameters can be used for the token bucket (taken from the command line help): rate, rate allocated to this class (class can still borrow) burst, max bytes burst which can be accumulated during idle period computed mpu, minimum packet size used in rate computations ceil, definite upper class rate (no borrows) {rate} To use the TMOD parameter with TC, a translation was done from TMOD-Rate to HTB- Rate, from TMOD-Size to HTB-Burst, from Peak Data Rate to HTB-Ceil and from MPU to HTB-MPU. 43

48 3. Implementation The following commands are used to communicate with the Traffic Control Interface: Add root qdisc (HTB - Hierarchical Token Bucket): tc qdisc add dev eth0 root handle 1: htb default 2 Add root class tc class add dev eth0 parent 1: classid 1:1 htb rate r ceil c burst b Construct HTB class string: tc class delete/add dev eth0 parent 1:1 classid 1:n htb rate r burst b ceil c mpu m 3.5. Implementation of a Timer Library using Double Linked Lists In the early stage of the NSIS protocol suite implementation the timer and socket functions provided by XORP [19] library was used. XORP is an open source implementation of a router platform. Internal simulation results have shown that the performance of the XORP timer library had a great impact on the overall performance. NSIS is a soft state protocol and uses a couple of timers per sessions. For example the QoS NSLP implementation uses a response timer to retransmit a message if a response has not arrived within a certain amount of time. A state times out and is deleted if it is not refreshed within a given period. The XORP timer library organises all active timers in a heap and reorganises the timer list regularly. To improve the overall performance it was necessary to develop a new timer library with a more efficient data structure. The TimerNode list is organized as a double linked list. A TimerNode is a structure that stores pointers to the previous and to the next TimerNode element. New timers are added at the end of the list. When a timer is deleted, the previous pointer is set to previous timer(from the deleted timer point of view) in the list and vice versa. There is no need to 44

49 3. Implementation reorganize or resort the list. When a time-out occurs all timers from the beginning to the end of a chosen TimerNode list are executed. The state of the corresponding timer is stored in an 8-bit variable. Possible states are whether this timer is marked to be deleted or, if it is a one-off timer, whether it is active or not. The type of the timer (one-off or periodic) is also stored in this variable. The TimerNode lists are organized in another double linked list, in the BucketNode list. A BucketNode structure holds pointers to the first(head) and to the last(tail) TimerNode element. Head points to the first element in the list and tail to the last one. New TimerNodes are added at the end of this list, so no time is needed to search for the right position. A Bucket Node also stores pointers to the previous and to the next instance and holds information about the interval and the offset. The Buckets are organised in a hashtable to reduce the lookup time when adding a new timer. The processing of the lists is done in TimerBase.cpp. One time-out is generated every second. The system call setitimer() is used to set an interval timer with the given timer resolution. Setitimer() uses internally signals. Signals are asynchronous events which are not predictable during the execution of a process. Signals are software interrupts. Examples are pressing CTRL-C on a keyboard to interrupt a running process (SIGABRT), a segmentation fault (SIGSEGV) or the time-out of an expired timer (SIGALRM). The latter is generated by setitimer() on expiration. Before a new timer can be registered, an instance of TimerBase needs to be created. The constructor of TimerBase generates a time stamp(tv_offset) of the current system time. This time stamp is later used to calculate the offset value of each timer. The signal handler function is initialized and the interval timer is started. When a new instance of TimerNode is added to the TimerBase, a new time stamp is calculated and stored in the TimerNode structure. A BucketNode with the corresponding interval/offset combination is searched where the TimerNode is added. When such a BucketNode was 45

50 3. Implementation not found, a new one is added at the end of the BucketNode list. Internal software tests have shown that the network buffer of the Linux Kernel when sending refreshes was the bottleneck of the implementation. While the timer function executes all pending refreshes one-by-one, incoming messages were not processed due to the singlethreaded software design. The network buffer could reach its capacity limit when serving around 30,000 sessions. A separate timer thread was no choice as synchronizing the mainand the timer-thread would have been too much work. A solution that allows a quasi concurrent processing of timer callbacks and incoming messages where no synchronization is needed was quite simple to implement. After the execution of 50 timer callbacks the timer handler returns to the socket handler. When the socket handler has processed 50 incoming messages, a signal is raised to get back to the timer processing. These steps are repeated until all timer callbacks are executed NSIS on a Linksys WRT54GL running OpenWRT OpenWRT is a Linux distribution for embedded devices. It provides a writable filesystem and comes with a package manager to install or remove software. Instead of having a static firmware the device it runs on can be customised. The distribution counts more than 100 software packages. Some example are ssh server, firewalling based on iptables, PPPoE, VLANS or cronjobs. The OpenWRT community provides many add-on packages. New software packages can be made with an SDK. OpenWRT exists for a number of hardware devices. The appendix provides a tutorial how to install OpenWRT on a Linksys WRT54GL and how to run NSIS for OpenWRT on it. A virtual LAN(VLAN) is a group of physical interfaces on a switch. The physical interfaces 46

51 3. Implementation can be separated logically as if they were standalone switches. One port of the switch is configured as the Trunk Port and has connectivity to all VLANs or a subset of the VLANs. VLANs can be used to separate traffic between groups of devices. Every packet is tagged to indicate the source or destination VLAN. A bridge is used to connect two ethernet devices to the same LAN. A wired interface, e.g. can be bridged together with a wireless interface. With an assigned IP address the bridge behaves like a normal interface. The bridge members cannot be configured with an IP address. The diagram below shows the internal architecture of a Linksys WRT54G/GS. VLAN and Bridging 47

52 3. Implementation The WAN port is the outer port connected to the Internet and tagged as VLAN1. The internal Ports 1-4 are configured as VLAN0. The internal port 5 is linked as the Trunk Port to eth0. eth1 is connected to the wireless interface. The bridging is done between the interfaces eth0 and eth1. Further information about the VLAN and bridging architecture can be accessed under [20]. This link provides also examples of how to configure the interfaces. 48

53 4. Evaluation The prototype implementation of the QoS NSLP needed to be tested to confirm the usability of the software design. This chapter provides performance results gathered on standard PCs as well as scenarios where the described protocol implementation runs on embedded devices. The results also show that the chosen software design features provide a stable and efficient framework for the main implementation Comparison between DLL and XORP Timer As shown in figure 4.1 the XORP timers need 54 seconds to build a list containing timers. The set up time for the same number of Double Linked List (DLL) timers is below one second. The DLL timers allocate 20 bytes per instance. The XORP timers consume twice as much memory. To set up a list containing 5 million timers took 14 seconds at a CPU usage of 100% (figure 4.2. The execution of the callback functions took 14 seconds at a CPU usage of 20%. The callback function just incremented a counter and removed the timer from the list. The memory usage here was 218 MB Performance Evaluation In an early stage the maximum number of sessions a router running GIST and QoS NSLP can maintain was identified. During these tests many bugs in the implementation and in 49

54 4. Evaluation Figure 4.1.: CPU usage of 100,000 DLL and XORP Timers Figure 4.2.: CPU usage 5 Million DLL Timers 50

55 4. Evaluation the software design were found. A lot of bugs did not appear when running a few sessions. Those came out when reaching 20,000 to 30,000 sessions. In a later stage the interesting question was how many sessions a router can achieve and maintain for a longer period of time Testbed Set-up The structure can be seen in figure 4.3. Testing was done on standard PCs with the following hard-/software configuration: Machine A Pentium III Coppermine 1000 Mhz 256 MB RAM 100 MBit Network Interface Card (NIC) Operating System: Fedora Core 7 Machine B AMD Athlon MB RAM 100 MBit Network Interface Card (NIC) Operating System: Knoppix Measurement Methods and Tools During the development of the QoS NSLP implementation the tool Valgrind [21] was used to identify memory leaks. Memory leaks occur where allocated memory is not freed after it is not needed any more. This is a common programming mistake and the result is 51

56 4. Evaluation Figure 4.3.: Testbed Setup a huge memory consumption. Valgrind is a free memory debugging and profiling tool. Besides the detection of memory leaks, it can also detect illegal access to unallocated or freed memory when reading or writing to it. Illegal memory accesses are often undetected during runtime of the application and can result in undefined behaviour. For the QoS NSLP, there exist unit tests checking the object structure, the functionality of the state machine and the traffic control interface. The CPPUNIT framework was used to create the test cases and integration tests. Unit tests are used to check the functionality of functions or small software modules. Software tests minimize the risk of remaining code errors. Ideally software tests are created before the actual code is written. With existing software tests it is easier to detect programming mistakes that occur by changing the source code. As an example a unit test can check the result of a functions that returns the square of the input parameter. Integration tests check the functionality of a larger block of source code that itself calls several functions during processing. An example of an integration test is the testing of the state machine described in this thesis. By using Valgrind and the CPPUNIT framework a lot of bugs were discovered that could hardly be detected by just running the application and checking the behaviour or the debug 52