NetIDE Manual. Document History: Version Date Issued by October-2016 Roberto Doriguzzi Corin

Transcription

1 Ref. Ares(2017) /08/2017 Document: CNET-ICT /D 2.7 Document Properties Document Number: D 2.7 Document Title: Manual Document Responsible: Document Editor: Authors: Target Dissemination Level: Status of the Document: Version: 1.0 CREATE-NET (CN) Roberto Doriguzzi Corin (CN) Roberto Doriguzzi Corin (CN), Pedro A. Aranda (TID), Elisa Rojas (TELCA), Arne Schwabe (UPB), Christian Stritzke (IEM), Kévin Phemius (THALES), Alec Leckey (INTEL), Carmen Guerrero (IMDEA), Sergio Tamurejo (IMDEA), Antonio Marsico (CN) PU Final Production Properties: Reviewers: Holger Karl (UPB), Pedro A. Aranda (TID) Document History: Version Date Issued by October-2016 Roberto Doriguzzi Corin Disclaimer: This document has been produced in the context of the Project. The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7/ ) under grant agreement n All information in this document is provided as is and no guarantee or warranty is given that the information is t for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission has no liability in respect of this document, which is merely representing the authors view. i

2 Document: CNET-ICT /D 2.7 Abstract: The Framework delivers an integrated environment for SDN development that unies dierent controller technologies and allows developers to write, test and deploy modular network applications that are independent from the underlying SDN technology. In this deliverable we illustrate the most relevant architectural and implementation concepts developed during the project, as well as we provide manuals, explaining how end users (software developers, network administrators, network operators) can install, congure and run the Framework. Keywords: Software Dened Networking (SDN), Manual, Framework, Integrated Development Environment, SDN Application Re-Usability, SDN Tools ii

3 Document: CNET-ICT /D 2.7 Contents List of Figures List of Acronyms v viii Executive Summary 1 1 Introduction 2 2 The Framework The Network Engine The Core Message format Intermediate Protocol Shim Backend The Integrated Development Environment Network Engine manuals General requirements The Core Installing Running ONOS Shim Installing Running OpenDaylight Shim Installing and running Ryu Shim Installing Running Testing a Shim Ryu Backend Installing Running Floodlight Backend Installing Running Testing Backends Testing IDE manual Installation Prerequisites Installation via the Eclipse Marketplace iii

4 Document: CNET-ICT /D Installation using the update site Installation by building from source Usage Application format management Specifying and using parameter schemas Conguring Network Topologies Testing network applications Monitoring the network application Global Conguration Application Loader Tools manuals Logger Installing Running Debugger Installing Running Proler Installing Running Trac Emulator Installation Running Memory Management System Installation Running framework at work Demo scenario Advanced architectural concepts Denitions Types of composition Single module without composition Multiple modules without composition Multiple modules with harmonizing Parallel composition Serial composition Approximate serial composition Composition and order of middle boxes Composition with OpenFlow Denitions Multiple modules Achieving consistent run-to-completion Parallel composition Serial composition Beyond the end of Eclipse Marketplace iv

5 Document: CNET-ICT /D OpenDaylight The Network Modelling (NEMO) language Evolution during the lifetime of () Beyond the lifetime of ONF Englewood Beyond the lifetime of Conclusions 69 A Protocol Specication 70 A.1 The protocol header A.2 Module announcement A.3 Heartbeat A.4 Handshake A.5 The fence mechanism A.6 The OpenFlow protocol A.6.1 Properly handling reply messages A.7 Other SBI protocols B XML Schema for Composition specication 77 Bibliography 84 v

6 Document: CNET-ICT /D 2.7 List of Figures 1.1 The Framework Network Engine architecture Core Bundle Overview Location of the LogPub within the Core The inner working of the LogPub in PUB mode The inner working of the LogPub in SUB mode Architecture of a Shim Architecture of ONOS Shim Architecture of ODL Shim within Beryllium release Architecture of the Ryu Shim Architecture of a Backend The Backend within the Ryu architecture The Floodlight Client Controller Backend Developer Toolkit Architecture 's Network Engine repository at GitHub [1] Engine Demo v0.3 (2016) at YouTube [2] The application format project tree Parameter congurator Topology Editor SDN network application testing dashboard Logpub interface and view Global conguration view Logger trace Debugger's output with OpenFlow Debugger's output with OpenFlow Wireshark dissector Application Proler GUI Demonstration workow Demonstration scenario Using a virtual overlay network for composition of module A and B Typical order of IDS and Firewall and NAT load balancer middle boxes in a traditional network Parallel composition of IDS, FW and NAT load-balancer modules Simplied diagram of the Englewood architecture Integration of Englewood with the Network Engine A.1 Fence mechanism workow. Both nxid and module_id refer to the header elds vi

7 Document: CNET-ICT /D 2.7 A.2 Request/reply message handling. xid refers to the OpenFlow header eld vii

8 Document: CNET-ICT /D 2.7 List of Acronyms AMQP API CLI FIB FOSS GUI IDE IETF JAXB LAN LLDP MSM MMS NBI NEMO Advanced Message Queueing Protocols Application Programming Interface Command Line Interface Forwarding Information Base Free and Open-Source Software Graphical User Interface Integrated Development Environment Internet Engineering Task Force Java Architecture for XML Binding Local Area Network Link Layer Discovery Protocol Most Specic Match Memory Management System Northbound Interface Network Modelling NETCONF Network Conguration Protocol NFV ODL OF ONF ONOS OO OSGi OVSDB REST Network Functions Virtualisation OpenDaylight OpenFlow Open Networking Foundation Open Network Operating System object-oriented Open Source Gateway initiative Open vswitch Database Management Protocol Representational State Transfer viii

9 Document: CNET-ICT /D 2.7 RFC SBI SDN SoC T-API TID UA UC VNF VNFD Request For Comment Southbound Interface Software Dened Networking Separation of Concerns Transport API Telefónica, I+D Union of Actions Use Case Virtual Network Function Virtual Network Function (VNF) Descriptor XML extensible Markup Language [3] ix

10 Document: CNET-ICT /D 2.7 Executive Summary This nal WP2 deliverable comprises the results achieved during the whole project, in two words: the framework. The Engine (and its accompanying tools) and the Integrated Development Environment (IDE) are the main components of the Framework. Its architecture and implementation have been developed in work packages 2, 3 and 4. As a manual, this document includes implementation details as well as administrator and end-user documentation to guide installation and usage of the Framework. It also provides guidelines for developers willing to enhance the framework with new components for SDN controller platforms or new control protocols not considered during the project. 1

11 Document: CNET-ICT /D Introduction The Software Dened Networking (SDN) paradigm allows networking hardware to be made malleable and remotely manageable by the so-called SDN controllers. However, the current SDN landscape is extremely fragmented. Dierent open and closed-source controller frameworks such as ONOS [4], OpenDaylight [5], Ryu [6], or Floodlight [7] exist. Porting SDN applications from one such platform to another is practically impossible and SDN users like network operators hence face a situation where they are either conned to applications working for the platform of their choice or are forced to re-implement their solutions every time they encounter a new platform. The goal of the Framework is to deliver an integrated environment for SDN development that unies dierent controller technologies and allows developers to write, test and deploy modular network applications that are independent from the underlying SDN technology. In this regard, it re-denes the concept of Network Application as a set of SDN software modules, possibly written for dierent SDN controller platforms, that are congured to cooperate on controlling the network. A Network Application hence allows to run such a set of modules as a single entity. As shown in Figure 1.1, the Framework is an integrated environment that comprises three main components: an Integrated Development Environment (IDE), a multi-controller SDN platform called Network Engine, and a set of Tools for SDN developers. Figure 1.1: The Framework. The IDE is built as a set of plugins for the Eclipse framework [8]. It provides editors that support network programming languages, a graphical editor to specify network topologies and the interfaces for tools to debug network applications and monitor the network. Such an IDE includes a graphical interface to create complex network applications as composition of modules that are independent from the SDN platform. The composite network application is deployed onto the so-called Network Engine, the second key element of the Framework. The Network Engine is a multi-layer SDN platform which orchestrates the execution of multiple SDN modules to control the network cooperatively as a single network application. The Network Engine comprises (1) a client controller layer that executes the 2

12 Document: CNET-ICT /D 2.7 SDN modules constituting the network application and (2) a server SDN controller layer that drives the underlying infrastructure. Between them, we have introduced an intermediate layer called Core: it hosts all logic and data structures that are independent from the particular controller frameworks and communicates with client and server controllers using the Intermediate Protocol. Both Network Engine and IDE can be used to test the network applications before the actual deployment onto an operational network. To this purpose, the IDE implements an automatic procedure that generates a virtual machine where network applications can be executed over a precongured Network Engine and a network emulator. For the testing phase, many tools are provided to validate the network applications, such as the Logger, the Debugger, the Vericator, the Proler and the Trac Emulator. Other tools, like the Memory Management System are meant to improve the performance and the resiliency of the network especially in environments where multiple SDN modules cooperate on controlling the same network. The remainder of this document starts at Chapter 2 with architectural and implementation aspects of the two main building blocks of the Framework: the Network Engine and the IDE. We defer a comprehensive description of the tools to Deliverable D4.3 of the project ( App Engine nal release and evaluation). Chapters 3, 4 and 5 provide installation and running manuals for developers and end-users for all the components of the framework, tools included. We conclude the actual manual of the Framework in Chapter 6 by presenting a concrete use case where the framework is not only used to design, implement, and test network applications, but also to support the network operator in providing new functions to operational SDN networks. Chapter 7 discusses the most challenging aspects of the design of the Network Engine: trying to dene a proper mechanism to compose SDN modules written for dierent platforms into a single network application. In Chapter 8, we explore the technical concepts that we have developed in and how they will be maintained beyond the lifetime of the project. For the sake of completeness, in Annex A we provide the latest specication of the protocol. The nal version of the Network Engine uses it to communicate between Core, Backends and Shim. 3

13 Document: CNET-ICT /D The Framework has developed a framework that fully covers the design and execution cycle for Network Applications and introduces Software Development paradigms into the SDN world. The framework includes a multi-platform SDN application execution environment (the Network Engine) and an Integrated Development Environment to guide Network Application developers through the whole development cycle. In addition, we initially envisaged the possibility of implementing a Network Application repository. During the lifetime of, we have interacted with dierent research projects that were developing application repositories with the intent of integrating support for Applications where suitable. At the end, we decided to concentrate on the Network Engine and the IDE to provide a viable environment and downgraded the activities around the Network Application repository. In the course of, several other projects, including T-NOVA [9] in the FP7 Framework programme developed concepts for repositories that can host our Network Applications and thus nicely complement the ecosystem. In this chapter, we provide a full description of the nal Network Engine architecture and the Integrated Development Environment as well as how they related to one another. 2.1 The Network Engine Controlling an SDN network with multiple, independent SDN applications is challenging for two main reasons: (i) the Northbound Interface (NBI) and East-West Interface (EWI) of SDN controllers are still not standardized, therefore each SDN platform implements its own interfaces making it hard for SDN applications written for dierent controllers to coexist in the same network, and (ii) SDN controllers only provide basic mechanisms to coordinate independent SDN modules, basically individual SDN modules can transfer network events to other modules, creating simple chains of independent modules that handle an event. The Network Engine (Figure 2.1) has been designed and implemented to tackle these challenges. The Network Engine is an SDN component that provides to SDN applications (called Modules in the gure) the runtime environment they expect and makes them cooperating with each other, eectively building a single Network Application. The Network Engine generalises the notion of chaining provided by some controllers by (a) supporting explicit and exible composition semantics, which (b) the Network Engine can execute even across multiple controllers. Our Network Engine follows the layered SDN controller approach proposed by the Open Networking Foundation in their SDN Architecture Technical Reference TR-502 [10]. It comprises a client controller layer that executes the modules that form the Network Application and a server SDN controller layer that drives the underlying infrastructure. The challenge is to integrate client and server controllers. A rst idea is to connect the Southbound Interface (SBI) for the (possibly multiple) client controllers to the Northbound Interface (NBI) of the (single) server controller. But as these interfaces do normally not match, adaptation is necessary. This adaptation has to cater for the idiosyncrasies of the dierent controller frameworks we intend to support and has to be implemented for each of them. Conceivably, we could translate each supported controller's SBI into the NBI of each any other support controller, but that quickly gets out of hand and also voids the opportunity to coordinate execution between multiple client controllers. We hence impose a protocol between server controller's NBI and client controller's 4

14 Document: CNET-ICT /D 2.7 Figure 2.1: Network Engine architecture. SBI, the Intermediate Protocol (cf. Annex A). To maximize reuse of adaptation code from the specic controller to the Intermediate Protocol, we use separate adaptors for the SBI, called Backend, and for the NBI, called Shim. In our rst implementations [11], we have proved that the Shim/Backend structure connected by an intermediate protocol is feasible and sensible. However, they still left the fundamental component in these modules: the composition logic. This implied that it needed to be reimplemented for each controller framework we wanted to support. To overcome this shortcoming, we introduce an intermediate layer, the Core (cf. Section 2.1.1): it hosts all logic and data structures that are independent of the particular controller frameworks and communicates with both Shim and Backend using the same Intermediate Protocol. The Core makes both Shim and Backend light-weight and easier to implement for new controllers. We implemented the Network Engine based on SDN controller platforms available as Free and Open-Source Software (FOSS). Since we provide the functional specications and platformindependent libraries [12], it should be easy to implement any Network Engine component for a closed-source commercial SDN environment as well. However, due to potential restrictions on publishing results, we deemed them as impractical to work with within a research project and concentrated on FOSS implementations. In consequence, we adopted OpenDaylight (ODL) and ONOS as server controllers, Floodlight as client controller, and Ryu as either one. ODL and ONOS are carrier-grade controllers targeted to service providers, enterprises and mainstream deployments. Being able to scale to a large number of network elements, we recognized both ODL and ONOS as the most suitable SDN platforms for playing the server controller's role. On the other hand, Ryu and Floodlight are aimed at facilitating the rapid prototyping of SDN applications. We were able to isolate their SBI drivers to add our Backend alongside and to hook into their event dispatching mechanism to drive the execution logic (e.g. see Section ). In the rest of this section, we provide architectural details of the components of our Network Engine. With respect to previous deliverables, we also include some useful hints to help developers to extend the Network Engine to other platforms. We also describe the composition mechanism implemented in the Core and how to extend it to support other control/management protocols. 5

15 Document: CNET-ICT /D 2.7 Moreover, we present the Intermediate Protocol as a means for the Core to drive the execution ow of the whole Engine The Core The Core contains several components that connect to the Shim and Backends, provide interfaces for the tools, and handle composition and conict resolution. In this section, we focus on the architecture of the Core; we defer the discussion on composition and conict resolution to Chapter 7. The Core is implemented as a modular Java-based application on top of the Apache Karaf platform. Apache Karaf is an Open Source Gateway initiative (OSGi) runtime distribution for highly dynamic applications and supports features such as runtime reconguration, service discovery and modularization [13, 14]. Since it uses the same platform, the Core can be integrated into the OpenDaylight or ONOS platforms, opening up synergies. Core Dependencies core.api bundle core.connectivity bundle core.caos bundle core.management bundle core.routing bundle eu.netip.library bundle Core Interfaces Shim Connection Specification Parser Management Connection OpenFlow Request Message Routing Message Parser Shared Classes Backend Connection Composition Configuration Change Handling Message Builder Conflict Resolution Karaf CLI Commandhandlers Abstraction Classes Figure 2.2: Core Bundle Overview. As an OSGi-based application that adheres to the patterns of modular applications and serviceoriented architectures, the Core is built out of several bundles, each implementing a particular functionality of the Core. An overview of the Core bundles can be found in Figure 2.2. The bundles were chosen to get a high reusability as the bundle are almost self-contained. For example putting the composition engine itself in its own bundle allows to reuse the composition engine in other contexts, e.g. in a SDN controller. Or that a bundle can be simply exchanged with another functionally similar bundle. Similarly, the communication that is handled by ZeroMQ [15] is implemented in the core.connectivity bundle. If, for example, the communication to Backends and Shim should be done via HTTP (not a good idea, just as an example), a core.httpconnectivity could replace it. The individual bundles are explained in detail in the following sections. The individual bundles provide services to and consume services of other bundles. The service references are managed by the Apache Aries Blueprint Dependency Injection module [16]. It is integrated in Karaf and was therefore selected as the dependency injection framework for the prototype. Service exports and imports are described in a blueprint.xml le that is part of each bundle The core.api bundle The core.api bundle contains shared classes and interfaces for the Core bundles. It does not contain any logic itself and does not provide or consume services. It is referenced as a dependency by the other bundles and provided at runtime in Karaf as part of the Core The lib.netip bundle (dependency) With the Intermediate Protocol also came the need for libraries that make handling the protocol's messages easy. This bundle contains a Java implementation of the Intermediate 6

16 Document: CNET-ICT /D 2.7 Protocol, i.e., it contains classes for parsing and creating messages. It is not directly part of the Core project but is implemented separately as a Java library so that other Java-based projects of (e.g., our OpenDaylight Shim, ONOS Shim and the Floodlight Backend) can utilize it. Therefore, it is added as a dependency and deployed in Karaf as part of the Core The core.connectivity bundle This bundle contains all classes necessary for the connection to the Shim and the Backends. As mentioned earlier, these classes utilize the ZeroMQ library for connections. In particular, it uses the JeroMQ library [17], an open-source pure-java implementation of ZeroMQ, which is referenced as a compiled dependency. The connections are handled by so-called managers that run tasks like message parsing and calling events on registered listeners, following common object-oriented (OO) and Java patterns. There are dedicated managers for Shim and Backend connections. The managers do not handle network connections directly; instead, they use a connector that is responsible for receiving and sending data. This is valuable since it allowed a fast migration from TCP sockets to ZeroMQ-based communication, having to rewrite only the connector and none of the managers, and also follows the Separation of Concerns (SoC) principle The core.caos bundle The core.caos (Composition and Orchestration) bundle contains the most important logic of the Core, namely the composition and conict resolution mechanisms. Since it is responsible for handling the composition as specied in the composition specication, the bundle also includes abstraction classes that enable the automatic deserialization of a provided extensible Markup Language (XML)-specication (Section ). The deserialization mechanism uses the Java Architecture for XML Binding (JAXB) reference implementation [18], guided by annotations in the abstraction classes. JAXB is also used to generate the XML schema for composition specications out of these annotated classes The core.routing bundle This bundle manages the mapping of requests and answers in order to give only the right Backend/- module the response to a request and not to broadcast the response to all Backends, risking that a Backend gets confused by a response that it did not request. See Annex A.6.1 for more details The core.management bundle This bundle provides a management interface for the Core using ZeroMQ and Management messages. This interface allows external tools to change the conguration of the Core at runtime (i.e. by installing a new composition specication). Runtime re-conguration is achieved by using the Apache Aries Blueprint Conguration service [16], which is managed by Karaf's CongAdmin service. The bundle denes conguration properties in their blueprint.xml conguration le whose values are resolved at runtime and injected into the class instance. The Blueprint conguration system also supports dynamic conguration changes either by reloading the aected bundle or by calling the specic method for the changed property. When a conguration change request is received, the change is reected in Blueprint conguration space, which takes care of reloading the aected bundles. The Core bundles make use of Blueprint conguration for several conguration properties such as connection ports. An overview of the available conguration properties can be found in Table 2.1; guidance on how to use them can be found in the ocial Karaf documentation. 7

17 Document: CNET-ICT /D 2.7 Table 2.1: Core conguration properties Bundle Key Value Description core.connectivity port The port to listen for incoming Shim and Backend connections. core.caos maxwaittime Integer (>= 0) The maximum time in seconds to wait for required modules to connect before aborting. 0 means wait indenitely. bypass Unsupported- Boolean If set to true, unsupported messages that arrive from the Shim will be relayed to the targeted Messages Backend, if possible. If set to false, composition- Specication XML composition specication string such messages will be discarded. The composition specication that should be used. If the string is not a valid composition specication, the change will be rejected and the former specication will be used. core.management port The port to listen for management connections Extending the Core to non-openflow protocols Almost all modules of the Core are written in a protocol-agnostic way. The only modules that know about and handle the content of messages with OpenFlow payload are core.caos and core.routing. core.routing serves a very special purpose of mapping/remapping OpenFlow XIDs and is therefore OpenFlow specic. Most of the core.caos module is also protocol-agnostic. It will do sequential or parallel composition by instancing the ConflictResolvers class, which in turn calls the specic conict resolver depending on the message content. At the moment this is only implemented for OpenFlow in the form of the DefaultOFConflictResolver class. Extending caos for other protocols simply needs implementing resolver classes for the other protocols Composition specication The specication language has to be able to express all the semantics that specify how to compose SDN application modules running in the SDN controllers in the Network Engine to a Network Application. We used an XML-based composition specication since that allows us to ooad checking the validity of the le to an XML validation tool and enables us to use well established XML tools and libraries. The application is composed in the Modules element (line 2). This information is used to start modules and client controllers (if necessary) when the new application is loaded. Additionally, lters can be applied to modules or invocations to only apply them to selected events. The Composition element (line 11) denes the execution ow on network events. An example for such a specication is given in Listing 2. It uses an XML-based notation. For the underlying XML schema, refer to Annex B. 8

18 Document: CNET-ICT /D 2.7 Listing 2.1: Composition Specication Example. 1 <CompositionSpecification xmlns=" 2 <Modules> 3 <Module id="fw" loaderidentification="ryu-fw.py"/> 4 <Module id="appa" loaderidentification="appa.py"/> 5 <Module id="appb" loaderidentification="appb.py"/> 6 <Module id="lb" loaderidentification="loadbalancer.jar"> 7 <CallCondition events="packetin" datapaths=" "/> 8 </Module> 9 <Module id="log" loaderidentification="logger.py"/> 10 </Modules> 11 <Composition> 12 <ModuleCall module="fw" /> 13 <ParallelCall resolutionpolicy="priority"> 14 <ModuleCall module="appa" priority="1"/> 15 <ModuleCall module="appb" priority="2"/> 16 </ParallelCall> 17 <Branch> 18 <BranchCondition events="flowmod"/> 19 <If> 20 <ModuleCall module="log"/> 21 </If> 22 <Else> 23 <ModuleCall module="lb"/> 24 </Else> 25 </Branch> 26 </Composition> 27 </CompositionSpecification> In the following section we will describe the most important elements of the composition XML format in greater detail Conict resolution policies Currently, the following conict resolution policies are dened: Ignore A baseline policy, simply ignores all potential conicts and returns the union of the commands. Pass If two commands conict, discard them both. Also a baseline policy; currently not implemented (but trivial to do). Priority Assuming that priorities have been assigned to the applications in the composition specication (even if only implicitly by the order in which they appear in the specication le), then pick the command resulting from the application with the higher priority (and ignore the other commands). Auto The auto policy tries to determine a smart resolution. It applies only to commands with match conict but no action conicts. First, it generates the Most Specic Match (MSM) out of the two: for each match eld that has wildcards in both commands, keep the wildcard; if there is a wildcard in one command and a specic value in the other, keep the specic value; if there is the same specic value in both commands, keep that value; otherwise, the auto policy fails. 9

19 Document: CNET-ICT /D 2.7 As candidate set of commands for this MSM, we consider the Union of Actions (UA) of C 1 and C 2. By assumption these commands have no action conict, hence can be combined. The actions for this match are dened as follows: If the MSM equals the matches of both commands, replace C 1 and C 2 by the new rule with the MSM and the UA. If the MSM equals the match of C 1 and UA equals the actions of C 1, then keep both C 1 and C 2 and ignore the MSM (the new rules would not add anything, it is equivalent to C 1 but does not contradict C 2 ). Similarly for C 2. Otherwise, add a new command with the MSM and the UA. If this policy fails for two conicting C 1 and C 2, fall back to a default policy. Other merge policies are easy to add to the Core if needed, for example automatic rewriting of rules by priorities The LogPub module The LogPub is one of the modules of the Core. It acts as a unique interface between the tools and the other modules of the Core. With this architecture the LogPub is seen by the rest of the Core as another Backend and allows an abstraction of the tools. The LogPub tasks can be summarized as: 1. converting messages from to ZeroMQ format (and the other way around); 2. transmitting messages originated by the Shim/Backend(s) to the tools; 3. receiving messages from the tools and sends them to the Shim/Backend(s). The LogPub uses a PUB/SUB paradigm to facilitate the handling of messages in both directions and thus exposes two queues to the tools. The PUB queue is where the LogPub publishes messages and the tools are subscribed to it; the SUB queue is where the tools push messages and the LogPub is subscribed to it. Figure 2.3: Location of the LogPub within the Core. The LogPub module receives the messages from the Shim and Backend(s) using the Java interface core_connector that is used by the core.routing module to send messages to the LogPub. The main receiving loop of the LogPub runs in a Thread (to allow concurrent access between the dierent 10

20 Document: CNET-ICT /D 2.7 queues); hence, there is an initial process to deposit the message in the ZeroMQ in_process queue (as depicted in Fig. 2.4). 1 This prevents the Thread from hanging as ZeroMQ handles all communication from/to the process. This functionality is integral for ZeroMQ to handle concurrency. The messages are then classied by the Identifier; its role is to assign them a topic; when the messages are dispatched in the PUB queue, the tools will only receive the messages they are subscribed to (see Fig. 2.4). For example the Logger is supposed to get all messages and will subscribe to all topics (represented by the * mnemonic), while the IDE will subscribe to the topology update topic. Figure 2.4: The inner working of the LogPub in PUB mode. In SUB mode (see Fig. 2.5), the LogPub has an open queue which accepts every message sent from the tools and transmits messages either to the core.routing module which, as its name suggests, handles the routing of synchronous OpenFlow requests messages between the LogPub, the Shim and the Backends (e.g. STATISTICS_REQUESTS) or directly to the Shim/Backend for asynchronous messages (e.g. FLOW_MOD). Figure 2.5: The inner working of the LogPub in SUB mode Message format Messages exchanged between the LogPub and the tools must use the format inside a ZeroMQ multipart message shown in Listing 2.2. The location eld contains a string identifying the destination or source of the message (Shim or Backend) and the message eld contains a netide_message: In the Tool LogPub direction, the location eld indicates whether the message is intended for the Shim or a Backend (eg. {shim,message}). 1 We introduced the LogPub architecture in Deliverable D2.5 [19], while in Deliverable D2.3 [20] described the two Advanced Message Queueing Protocols (AMQP) options we evaluated to implement it: RabbitMQ and ZeroMQ. 11

21 Document: CNET-ICT /D 2.7 In the LogPub Tool direction, the location eld indicates whether the message originates from the Shim or a Backend. As mentioned earlier, the message will also have a topic (eg. {topic,shim,message}). struct logpub_message{ struct zeromq_message location; struct zeromq_message message; }; Listing 2.2: MQ multi-part message header Interface with the IDE The Core connects with the IDE via two separate channels. The testing dashboard uses SSH to start and stop the Core as well as load a composition specication. SSH is also used to automatically install the Core, the Shim and the Backends. The output and prompt of the Core can also be displayed in the IDE via an SSH connection. The connection to the LogPub interface (Section ) is established via a MQ connection. The IDE listens to the LogPub interface, deserializes the messages and uses them for visualization and monitoring. A detailed explanation of the connections from the IDE to a engine setup can be found in Section Intermediate Protocol The Intermediate protocol implements the following functions: (i) it carries management messages between the Network Engine's layers (Core, Shim and Backends), e.g., to exchange information on the supported SBI protocols, to provide unique identiers for application modules, or to implement the fence mechanism (Section A.5 in Annex A or Section in Deliverable D2.5 [19]). (ii) It carries event and action messages between Shim, Core, and Backend, properly demultiplexing such messages to the right module based on identiers. (iii) It encapsulates messages specic to a particular SBI protocol version (e.g., OpenFlow 1.X, NetConf, etc.) with proper information to recognize these messages as such. The protocol covers the communication between the dierent components of our architecture: (i) Module advertisement using the NETIDE_MODULE_ANN and NETIDE_MODULE_ACK messages by client controllers (ii) Transaction control with the NETIDE_FENCE message, and (iii) NETIDE_OPENFLOW, NETIDE_NETCONF or others, to exchange SBI specic messages like OpenFlow packets, etc. The protocol provides a 64-bit datapath_id eld to uniquely identify network elements when SBI messages are carried. It also provides a 32-bit module_id to uniquely identify application modules running on top of each client controller. The composition mechanism in the Core leverages on this eld to implement the execution ow of these modules. Finally, the nxid eld is used in the context of the fence mechanism (Section Section 2.1.1). The full protocol specication is provided in Annex A. 12

22 Document: CNET-ICT /D Shim The Shim is a platform-specic component that translates the NBI of the server controller to the Application Programming Interface (API), thereby exposing it to the other components of the Network Engine. Figure 2.6: Architecture of a Shim. Figure 2.6 shows the internal architecture of the Shim integrated with a server controller platform. It includes hooks placed inside SBIs (small grey boxes in the gure) to route all messages from the network to the Shim itself, overriding the server controller's processing logic. This prevents other application modules running on top of the server controller from interacting directly with the network (i.e. by-passing the Core's logic). Its implementation depends on the server controller platform and it does not necessarily necessitate the modication of the Southbound interfaces, as represented in Figure 2.6, but it can involve a platform's core components instead (such as the application manager in Ryu, cf. Section ). The Core Connection component handles the communication with the Core through the Intermediate Protocol. The API Translator converts between the protocol and platformspecic NBI messages. Please note that, in this case, NBI means a bi-directional and event-driven server controller's northbound interface, like the Java interface available in ONOS and OpenDaylight or the Python interface in Ryu. In fact, the Core, as well as the client controllers on top of it, needs to monitor and handle the events that are asynchronously generated by the network, such as new ows, port status changes, etc. This operation would be possible but not ecient with Request/Response interfaces like Representational State Transfer (REST), also available in many SDN controllers. The Topology module allows the IDE, via a REST interface (see Section Section 2.2), to retrieve the topology underneath the Network Engine. Topology discovery is achieved by interacting with the network through the server controller's SBI or by retrieving the topology information collected by specic core components available in advanced controller platforms (e.g. ONOS and OpenDaylight). Topology details include datapath_ids, number of supported ow tables, (administrative and operational) port information and other properties of the network elements. We implemented the Shim for ODL and ONOS as OSGi bundles, leveraging their respective Java NBIs to interact with the network elements. Both implementations support OpenFlow versions 1.0 and 1.3 and make use of serialization methods to convert messages from the controller's NBI format into OpenFlow byte arrays before encapsulating them with the header. Vice-versa, deserializers are used to convert byte arrays contained in the payload of messages (OpenFlow commands) into NBI Java classes. While for ONOS we could leverage the implementation of serializer/deserializer methods provided by the OpenFlowJ-Loxi library [21], in ODL (which uses its own library OpenowJava [22]) we had to implement the serializer from scratch and to extend the existing deserializer in order to cover all OpenFlow message types. We also implemented the Shim for Ryu, based on Ryu's Python NBI. It includes serializer and deserializer methods based on Python's struct module [23]. 13

23 Document: CNET-ICT /D ONOS Shim The ONOS Shim is implemented as an OSGi bundle and it is based on the OpenFlowController API [24] (Figure 2.7). This API provides the complete management of the OpenFlow protocol and uses the OpenFlowJ-Loxi library to serialize and de-serialize OpenFlow messages. This strategy allows the Shim to handle OpenFlow messages without any translation into/from the abstracted ONOS NBI. This avoids an additional computational burden due to the translation and any loss of information this translation may cause. For example, when a PACKET_IN arrives at the ONOS OpenFlow southbound, the ONOS OpenFlow provider converts the packet to a high-level representation, called the PacketContext [25]. This interface does not maintain all the information carried by a PACKET_IN, such as the buffer_id, that is often relevant for SDN forwarding applications. In short, the ONOS Shim simply encapsulates the OpenFlow message received by the OpenFlow- Controller API in the Intermediate Protocol and then it relays the encapsulated message to the Core. Figure 2.7: Architecture of ONOS Shim. Although the ONOS Shim may appear bound to the OpenFlow protocol (specically, it currently supports OpenFlow v1.0 and v1.3), actually it can potentially handle several SBIs at the same time, such as OpenFlow, NETCONF and OVSDB. The ONOS Shim is implemented to import any new SBI through the available Java APIs, without requiring any modication to its architecture. For example, one can add NETCONF support to the ONOS Shim by simply importing the NetConfController interface [26] (Listing 2.3). Listing 2.3: Example of Java import for NETCONF in the ONOS Shim. import org.onosproject.netconf.netconfcontroller After that, the ONOS Shim has to be registered as a message listener to the NETCONF southbound in order to receive messages from the NETCONF devices OpenDaylight Shim The ODL Shim has been implemented as an OSGI bundle. OpenDaylight uses Apache Karaf (an OSGI container) to manage the dierent bundles that make up the project, with each oering their own service interface as a Karaf Feature. The feature provides the required functionality to manage the OpenFlow protocol with each connected switch. Unlike the ONOS Shim and the Core which use the NetIP library to generate Intermediate Protocol messages (Section ), the ODL Shim had to use a modied version of this library. The NetIP library uses OpenFlowJ- Loxi to serialize Openow messages, also used by Floodlight. However, OpenDaylight uses its own 14

24 Document: CNET-ICT /D 2.7 internal project OpenowJava to serialize Openow messages. As such, we needed to create an alternative NetIP library that uses OpenowJava for serialization. This ODL-NetIP library resides in the OpenDaylight GIT repository [27] and is built as part of the nightly build process. Unfortunately, the OpenowJava project only provides serialization functions in one direction, e.g. serializing a Packet_Out java class to a byte array for sending to a switch. As the OpenFlow protocol dictates a Packet_Out will never arrive from a switch, there is no corresponding function to deserialize a byte array back to a Packet_Out java class. This is true for all messages for a given direction, eg, Packet_In arrive from switches so they do not require serialization to byte arrays. However, as the Shim passes these serialized objects to the Core for routing to the Backends, the project required these additional serialization functions. These functions are specic to just OpenFlow and were coded and submitted as patches to the OpenowJava serialization library of OpenDaylight. These patches have been accepted ensuring these functions are now maintained going forward. The ODL Shim uses the ZeroMQ library (a message queue wrapper library) to place messages received from the switches onto the Message Queue that is managed by the Core. In addition, the Shim subscribes to this same message queue so that it can receive the messages coming from connected Backends. These outgoing messages are then sent to the relevant switch based on the datapath ID that is part of the Intermediate Protocol. Figure 2.8: Architecture of ODL Shim within Beryllium release Ryu Shim The Ryu Shim is implemented as an application for the Ryu controller [6] based the Python Open- Flow libraries included in Ryu's source code [28]. Fig. 2.9 shows the Shim in the context of Ryu's architecture when Ryu is deployed as server controller. In this gure, modules besides the Shim are SDN applications that run on Ryu and that are part of the composite network application along with the modules running on client controllers. The Shim Layer is a single-threaded entity that interacts with Ryu's internal OpenFlow controller to communicate with the underlying OpenFlow devices. The nal version of the Ryu Shim works with any version of the OpenFlow protocol ( at the time of writing) and is compatible with the latest version of Ryu (v4.6 at the time of writing). The Ryu Shim converts messages from the controller's NBI format into OpenFlow byte arrays before encapsulating them with the header. Vice-versa, it converts byte arrays contained in the payload of messages (OpenFlow commands) into NBI Python structures. Such format 15

25 Document: CNET-ICT /D 2.7 Figure 2.9: Architecture of the Ryu Shim. translations are based on Python's struct module [23]. The latest version of the Ryu Shim implements the Shim architecture version 2.0, as presented in Deliverable 2.6 and depicted in Figure 2.6. To route all the network events to the Shim, we modied send_event_to_observers in app_manager.py, which is the method that distributes the events to the registered application modules (called observers in the Ryu controller framework). As a consequence, only the observer with name RyuShim gets the events from the OpenFlow SBI (Listing 2.4). The others, i.e. the SDNs modules running on Ryu, only receive the events via a Backend SBI, which is driven by the Core with intermediate protocol. Listing 2.4: Augmented send_event_to_observers for Ryu. def send_event_to_observers(self, ev, state=none): for observer in self.get_observers(ev, state): if observer is "RyuShim": self.send_event(observer, ev, state) Backend The Backend is designed as an additional southbound interface for the client controller that interacts with the underlying layers of the Network Engine. Figure 2.10 shows the internal architecture of a Backend. At boot time, the Backend starts the discovery of the application modules running on top of the client controller (Module Discovery) and registers them to the Core (Announcement Handler). The Core, in turn, assigns a specic identier (module_id) for each registered module. As part of its initialization process, the Backend also queries the Shim for the physical topology. Backends store the topology information as instances of the network elements (BackendDPs in Figure 2.10), which are exposed to the application modules. Further details on Backend initialization are provided as part of the protocol specication in Annex A, Sections A.2 and A.4. At runtime, Backends use the module_id in the protocol header to tell the Core which module is sending the message. On the other side, the Core uses the module_id based on predened policies to indicate which module handles the event. The Event Handler steers the event distribution inside the Backend, ensuring events are sent to the correct module Ryu Backend The Ryu Backend represents an SBI for the Ryu controller [6] and it is implemented as an independent module (its implementation does not impact Ryu's core modules) that can be loaded at 16

26 Document: CNET-ICT /D 2.7 Figure 2.10: Architecture of a Backend. boot time along with all the user's applications. The Backend is a modied version of Ryu's internal OpenFlow protocol SBI which (i) uses the protocol to communicate with the Core, (ii) implements the fence mechanism to synchronize with the Core (cf. Section A.5) and (iii) maintains internal representations of the network elements that are exposed to the applications. The nal version of the Ryu Backend has been tested with OpenFlow v1.0 and v1.3, although it is implemented to work with any version of the protocol ( at the time of writing), and it is compatible with the latest release of Ryu (v4.6 at the time of writing). Figure 2.11: The Backend within the Ryu architecture. Like the Shim, the Ryu Backend converts messages from the controller's NBI format into Open- Flow byte arrays before encapsulating them with the header. Vice-versa, byte arrays contained in the payload of messages (OpenFlow commands) are converted into NBI Python structures which are relayed to the running SDN modules. In Figure 2.11, the general Backend architecture is integrated with the rest of the Ryu architecture. The Event handler implements the fence mechanism which executes the event handlers implemented in the SDN modules and it takes care of intercepting the exit points of such handlers that trigger the NETIDE_FENCE message (cf. Section A.5). Listing 2.5: Implementation of the fence mechanism in the Ryu Backend. 17

27 Document: CNET-ICT /D if value == module_id and key in event_observers: 2 module_brick = ryu.base.app_manager.lookup_service_brick(key) 3 module_brick_handlers = module_brick.get_handlers(ev) 4 for handler in module_brick_handlers: 5 handler(ev) 6 break 7 8 msg = Ops.netIDE_encode('NETIDE_FENCE',self.netide_xid,module_id,0,"") 9 self.channel.socket.send(msg) Listing 2.5 provides an excerpt of Ryu Backend's code. In line 1, the Backend checks whether the SDN module with the module_id (specied by the Core in the header) implements the handler for the event. In line 5, the Backend passes the event to the module by executing its handler. Please note that, since the SDN module receives the events from the Backend, it uses the send_msg method implemented in the Backend to respond. In such a method, the Backend intercepts the message (e.g. a FLOW_MOD or a PACKET_OUT) and encapsulates it with the header before sending it to the Core. In lines 89, a fence message is sent to the Core indicating that the event has been processed, no matter if the module has produced zero, one, or many control messages to handle the event Floodlight Backend The Floodlight Backend was developed as a Floodlight Module that is loaded when the Floodlight controller starts. By accessing the native Floodlight application interface (API), this gives the module access to additional events and functions that are not available from the RESTful interface. While Floodlight REST applications can start up and connect over HTTP whenever they want, Floodlight Modules are required to register with the controller and specify where in the order of other Modules they start. Figure 2.12: The Floodlight Client Controller Backend. By default, Floodlight listens on port 6633 for all connecting OpenFlow switches. By changing 18

28 Document: CNET-ICT /D 2.7 this to a dierent port 6634, no physical/virtual switch will now connect to Floodlight. For each Switch connected event that is received by the module from the Core, a new Dummy Switch class is created in the backend that connects to Floodlight on this separate port, 6634, representing the actual switch. The Dummy Switch replays and responds to OpenFlow protocol messages ( hello, echo_reply, config_reply, stats_reply, etc). Floodlight will now add this switch to its list of Managed Switches and trigger the addedswitch event to other IOFSwitchListener listeners. This is where SDN applications developers traditionally push flow_mod messages proactively to the switch. As SDN Applications and other Floodlight modules send OpenFlow messages to switches, each Dummy Switch will now receive this message. After inspecting the message, it then creates the relevant Intermediate Protocol message and sends it to the Core. When the response is received from the Core, the equivalent OpenFlow Reply message is created and the relevant properties are set. The message is then sent back to Floodlight through the correct Dummy Switch. Similarly, all switch-initiated messages (eg, packet_in) are sent by the Shim Layer (through the core) to the Module of Floodlight, which in turn creates the relevant OpenFlow response message and sends it out through the correct Dummy Switch. The latest version of Floodlight now supports the proposed fence mechanism to notify completion of message processing. As each Backend completes the processing of an inbound message, the associated fence message is now sent. This ensures that the Backend supports the latest version of the Intermediate Protocol. To manage connected switches, Floodlight maintains a separate TCP channel for each connected switch. It is therefore important that each response from the Core is relayed through the correct Dummy Switch's TCP connection. This is tracked through the Datapath Identier (DPID) and provides uniqueness. It is possible to view the list of switches that Floodlight is managing by using the Floodlight Management Web interface available at What is novel here is that none of these physical switches are actually connected to Floodlight, but are in fact managed by the Server Controller (Shim component) and Core. But Floodlight will continue to operate and generate topology maps based on information provided to it by the Backend component. 2.2 The Integrated Development Environment This chapter provides a short overview of the architecture of the developer toolkit and its interactions with the Engine. Please refer to WP3 Deliverables [29] for further details. We describe the architecture of the IDE in the context of running controller machines and the engine architecture. Figure 2.13 shows a component structure of the IDE and their connections to other elements of the project. Test UI The test UI in the IDE is used to deploy and run the Network Engine for testing purposes. This requires a connection to the test machine via SSH as well as a connection to the virtual machine manager Vagrant [30] in case a test VM has to be created on the developers machine. Copying SDN modules and topology congurations from the IDE to a remote machine is done via the Remote Machine Manager which uses SCP. The App Manager starts and stops SDN modules on remote machines and VMs using SSH. All these management GUIs build upon the GUI framework SWT. Monitoring UI The Monitoring UI processes information from the LogPub interface and makes it human-readable via graphical visualization. 19

29 Document: CNET-ICT /D 2.7 The necessary data is retrieved by a ZeroMQ listener which operates inside the IDE and listens to LogPub messages. These messages are parsed and can be passed on to other components. A Visualizer creates graphical representations of the message log and displays it to the user. It is implemented upon the Eclipse-internal data visualization framework BIRT. App Format Tools The App Format Tools mainly consist of editors and conguration assistants which do not interact with the Network Engine or remote machines. These editors are used to edit the parameter schema and the system requirement specication les, and test topologies (Topology Editor). The rst two editors use the Xtext framework and the latter uses the graphical editor framework Sirius. The Packager creates a gzipped package from a project. This package can be used by the package Loader which provides a headless way to start SDN apps on the Engine. The Topology Importer is an application built on top of the OpenDaylight (ODL) SDN Controller [31]. It produces notications when changes on the topology controller by ODL occur. An update would be expected in the following cases: (i) a switch is connected to ODL; (ii) a switch is disconnected from ODL; (iii) a new host is connected to a switch controlled by ODL; (iv) a host is disconnected from a switch controlled by ODL; (v) a connection is created between two switches. 20

30 Document: CNET-ICT /D 2.7 Figure 2.13: Developer Toolkit Architecture. 21

31 Document: CNET-ICT /D Network Engine manuals In this chapter we describe how to install and run the dierent components of the Network Engine. As described in Section 2.1, the Network Engine is dened by three types of components: the Core (unique piece of code in the Network Engine), the Shim (only one instance can run at a time, although more than one Shim is implemented) and the Backends (one or more instances can run at a time). All the code of the Network Engine can be found at Engine, where the master branch is the most updated and stable version of it. Figure 3.1: 's Network Engine repository at GitHub [1]. Following Fig. 3.1, the Core component is contained in the core folder, three implementations of the Shim for ODL, Open Network Operating System (ONOS) and Ryu are contained in the 22

32 Document: CNET-ICT /D 2.7 odl-shim, onos-shim and ryu-shim folders respectively, and two implementation of the Backend for Floodlight and Ryu are in floodlight-backend and ryu-backend. We describe how to install all of these components in the next sections. Additionally, the Network Engine repository contains ve more folders. The libraries/netip folder has the libraries implementing the Protocol for both Java and Python, which are common for the Core, Shims and Backends (because all of these components of the Network Engine are either developed in Java or Python). The loader folder is an application loader for the Network Engine, directly related with the IDE and the deployment process. The odl/topology-update implements a topology loader for the ODL Shim, used in runtime by the IDE. The tests folder contains several examples of composition to be executed in the Network Engine, including the module applications and the composition specications les to be loaded. Finally the utils folder has a mix of dierent utils, including a Vagrant le and scripts to generate a working instance of the Network Engine automatically, instead of installing components and dependencies one by one. Finally, the YouTube channel [2] contains several demos of the Network Engine (as illustrated in Fig. 3.2), in which any user can follow how to run the Network with an initial schema that shows the deployment and step-by-step commands later on. Figure 3.2: Engine Demo v0.3 (2016) at YouTube [2]. 3.1 General requirements The common requirements for all components of the Network Engine are: Git version control system. No particular version required, but we recommend to use version and beyond. Java 8 JDK, or Python 2.7/3.5 for Python-based platforms. Maven [32]. As of writing, the approved maven version is v A Linux-based operating system. Even though most components are developed and tested on dierent platforms as well, the setup on Linux has been tested most thoroughly. 23

33 Document: CNET-ICT /D The Core Installing 1. Clone the central Engine source repository git clone netide-engine git checkout master 2. The following commands will build the bundles needed by the Core and install them to your local Maven repository: cd $HOME/netide-engine/core mvn clean install -Dgpg.skip=true Gpg is skipped since normally parts of the builds are signed with gpg to able to publish them. 3. Download Apache Karaf from in version and extract it to $HOME/netide/apache-karaf Install the branding osgi packet to the lib directory of karaf cp core.branding/target/core.branding snapshot.jar $HOME/netide/apachekaraf-3.0.5/lib/ 5. Start Karaf by running $HOME/netide/apache-karaf-3.0.5/bin/karaf 6. Install the netide-core feature: feature:repo-add mvn:eu.netide.core/core.features/1.1.0-snapshot/xml/features feature:install core 7. Load a composition specication with netide:loadcomposition, for example the Minimal Specication: netide:loadcomposition /Users/arne/netide/netide-engine/core/specification/ OneSimpleSwitch.xml 24

34 Document: CNET-ICT /D 2.7 Warning: the karaf shell does not expand variables and placeholder characters like or $HOME, so the path needs to be an absolute path like in the example. 8. Run now Mininet, the Shim and nally the Backend (Note: If you are using the example composition le, you should use the Ryu simple_switch.py application) Running The Core provides several commands to perform debugging and simple status checks. Karafs commands all start with log:, e.g. to have a tail -f style log viewing, use: log:tail Additionally specic status commands are available: Show connected Backends/modules karaf@root()> netide:listmodules Id Name Backend Last message (s ago) 797 SimpleSwitch backend-ryu backend-ryu backend-ryu ,66 - shim - 0,32 The commands show all known Backends, the applications running on the Backend and the Shim and also the last time the Core received a messages from the Backends and applications. Show OpenFlow requests from Backends netide-core>netide:listroutingrequests Shim xid Backend B xid #Resp Type Last message (s ago) backend-ryu STATS_REQUEST 75,31s backend-ryu STATS_REQUEST 4,27s backend-ryu STATS_REQUEST 4,26s This command lists all symmetric requests between SDN modules and the network that are handled by the Core. Refer to Annex A.6.1 for more detail about this mechanism. Show installed composed ow mods As another debugging feature, the Core allows to list all Flows (OpenFlow 1.3 only) that have been installed by the composition mechanism: netide-core>netide:listflowmods DefaultFlowEntry{rule=DefaultFlowEntry{id=0, deviceid=of: , priority=10, selector=[in_port:2, ETH_DST:72:A3:E9:28:94:71], treatment= DefaultTrafficTreatment{immediate=[OUTPUT:1], deferred=[], transition=none, cleared=false, metadata=null}, tableid=0, created= , payload=null}, state=added} 25

35 Document: CNET-ICT /D 2.7 DefaultFlowEntry{rule=DefaultFlowEntry{id=0, deviceid=of: , priority=10, selector=[in_port:1, ETH_DST:5A:4F:00:AF:E8:56], treatment= DefaultTrafficTreatment{immediate=[OUTPUT:2], deferred=[], transition=none, cleared=false, metadata=null}, tableid=0, created= , payload=null}, state=added} [...] 3.3 ONOS Shim Installing The ONOS Shim is developed for ONOS version ONOS and Shim have been tested on Ubuntu LTS, with the supporting software as detailed in Section 3.1. To perform the ONOS and Shim installations, the following steps are required: Download ONOS: git clone git checkout v1.5.1 Set-up the environment and build the ONOS source code. Please refer to this guide: https: //wiki.onosproject.org/display/onos/installing+and+running+onos To simplify the execution of ONOS, the best way is to create a ONOS cell. This guide explains the creation and usage of cells: Scratch#ONOSfromScratch-4.Createacustomcelldefinition Download and build the ONOS Shim application: git clone cd onos-shim && mvn clean install -DskipTests Running Once ONOS and Shim are compiled, they are ready to be executed. First, start ONOS by using this command: ok clean and the system prompt should look like this: onos> Now ONOS is running but the conguration of FlowRuleManager must be modied to allow the persistence of ow rules that are not installed directly by ONOS. To this end, the following commands must be issued in the Karaf shell of ONOS: onos:cfg set org.onosproject.net.flow.impl.flowrulemanager allowextraneousrules true Open a new BASH shell and install the ONOS Shim: onos-app $OC1 install target/onos-app-shim snapshot.oar The ONOS Shim is now installed but not running. The next step is the conguration of the connection with the Core. This step is done via the ONOS shell by entering the following commands: 26

36 Document: CNET-ICT /D 2.7 config:edit eu.netide.shim.shimlayer property-set coreaddress {CORE_IP} property-set coreport {CORE PORT} config:update Where CORE_IP is the IP address of the machine where the Core is running (by default localhost) and CORE_PORT is the TCP port being listened by the Core (by default 5555). Now the ONOS Shim can be activated by using the following command in the ONOS shell: app activate eu.netide.shim 3.4 OpenDaylight Shim Installing and running OpenDaylight can be downloaded from the OpenDaylight Software Downloads Page: opendaylight.org/software/downloads The Karaf distribution has no features enabled by default. However, all of the features are available to be installed. The only system requirement is the Java 8 runtime: Extract OpenDaylight from the archive. 1. Unzip the zip le 2. Navigate to the directory where you decompressed the ODL Karaf distribution 3. run./bin/karaf 4. install the netide feature [~/distribution-karaf]$./bin/karaf... \ \ \ \.. / _ / \\ \_/ \ / \ \\ \< / \ \ \ / \ _> > / \ ` \/ \\ _ / /_/ > Y \ \ / / \ > / ( / / \ / / \/ \/ \/ \/ \/\/ / / \/ Hit '<tab>' for a list of available commands and '[cmd] --help' for help on a specific command. Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown OpenDaylight. opendaylight-user@root> Install using the karaf console. opendaylight-user@root>feature:install odl-netide-rest Conguration of the ODL-shim component is through an xml le located in the./configuration directory. Default attributes are pre-set, but these can be overridden if required <!-- CONNECTION INFO FOR SWITCHES --> <address> </address> <port>6644</port> <transport-protocol>tcp</transport-protocol> 27

37 Document: CNET-ICT /D 2.7 <switch-idle-timeout>15000</switch-idle-timeout> <!-- CONNECTION INFO FOR THE CORE --> <core-address> </core-address> <core-port>5555</core-port> 3.5 Ryu Shim The Ryu Shim is implemented as application for the Ryu controller [6] by using the Python Open- Flow libraries included in the Ryu's source code [28, 33]. The current implementation supports the Intermediate protocol v1.4.1 (cf. Annex A) and works with the latest version of Ryu (v4.6 at the time of writing) Installing First, clone the Ryu code from [28] on a Linux-based local machine (let say, in your home folder $HOME). Copy the python folder from into $HOME/ryu/ryu and rename it as netide. After that, install Ryu by running the command from $HOME/ryu: cd $HOME/ryu python./setup.py install Then, add installation path for Ryu to your PYTHONPATH variable in your prole (i.e. $HOME/.profile or $HOME/.bashrc) by adding the following line: either export PYTHONPATH=/usr/local/lib/python2.7/dist-packages/ryu The path may depend on the Linux distribution and on the version of Python used for the installation. Additional Python packages may be required to successfully complete the installation procedure. On a Ubuntu Linux OS the following Debian and Python packages must be installed (using sudo to gain root privileges for installation): apt-get install python-pip python-dev python-repoze.lru apt-get install libxml2-dev libxslt1-dev zlib1g-dev python-zmq pip install ecdsa stevedore greenlet oslo.config eventlet WebOb Routes Running In general, the Shim is a client for the Core. Therefore, always start the Core rst (cf. Section 3.2), then run ryu-manager ryu-shim.py This command will start the Shim along with the rest of the Ryu platform. The Shim will try to connect to a running Core which must be already running and listening on TCP port Note: The Ryu controller platform listens for connections from the switches on the port Therefore, when using Mininet for testing purposes, start Mininet by specifying the controller information as follows: sudo mn --topo linear,4 --controller=remote,ip=ip_addr,port=6633 The IP address information is not needed when Ryu and Mininet are running on the same machine. 28

38 Document: CNET-ICT /D Testing a Shim To test on of the Shims presented above, it is necessary to run the Core and at least one of the Backends described in the following sections. In this Section we provide a testing procedure based on the Ryu Backend. First, start the Core by following the procedure described in the accompanying README or in Section Section 3.2 of this document and then, run either the ONOS, Opendaylight or the Ryu Shim. A network emulator such as Mininet can be used to test the software. In folder ryu-backend/tests of the Github repository, one can nd script netide-topo.py that automatically congures Mininet with a 4 switches and 4 hosts topology. sudo mn --custom netide-topo.py --topo mytopo --controller=remote,ip=ip_addr,port=6633 Where IP_ADDR is the IP address of the machine where the Network Engine is running. It is not needed when the Network Engine and Mininet are running on the same machine. To use OpenFlow v1.3 and test applications, add switch ovs,protocols=openflow13 to the options. This script congures the following topology: alice alice-eth0:s22-eth1 bob bob-eth0:s22-eth2 charlie charlie-eth0:s11-eth1 www www-eth0:s23-eth1 s11 lo: s11-eth1:charlie-eth0 s11-eth2:s21-eth1 s21 lo: s21-eth1:s11-eth2 s21-eth2:s22-eth3 s21-eth3:s23-eth2 s22 lo: s22-eth1:alice-eth0 s22-eth2:bob-eth0 s22-eth3:s21-eth2 s23 lo: s23-eth1:www-eth0 s23-eth2:s21-eth3 Where alice, bob and www belong to a hypothetical LAN protected by a rewall (switch s11), while charlie is outside the LAN. Once both Core and the Shim are running, the Ryu Backend and OpenFlow 1.0 network applications can be started with: ryu-manager --ofp-tcp-listen-port 7733 ryu-backend.py \ tests/simple_switch.py tests/firewall.py or ryu-manager --ofp-tcp-listen-port 7733 ryu-backend.py \ tests/simple_switch_13.py tests/firewall_13.py if you want to test the OpenFlow 1.3 applications. For instance, the composition conguration for the Core may assign switches S21, S22 and S23 to the simple_switch application, while the S11 to the firewall application. As an alternative, one may want to test dierent applications running on dierent instances of the client controller. In this case, just open two terminals and run the following commands (one for each terminal): ryu-manager --ofp-tcp-listen-port 7733 ryu-backend.py tests/simple_switch.py ryu-manager --ofp-tcp-listen-port 7734 ryu-backend.py tests/firewall.py Within the Mininet CLI, a pingall command demonstrates what trac is allowed and what is not, depending on the rules installed by the rewall application. 29

39 Document: CNET-ICT /D Ryu Backend The Ryu Backend is implemented as an additional southbound interface for the Ryu controller [6] by using the Python OpenFlow libraries that are part of the Ryu source code [28, 33]. It is an adaptation of the original controller.py which implements the OpenFlow SBI in Ryu. The current implementation supports the Intermediate protocol v1.4.1 (cf. Annex A) and works with the latest version of Ryu (v4.6 at the time of writing) Installing The Ryu Backend is provided as module for the Ryu controller. procedure: To load it, use the following 1. Clone the Ryu code from [28] on a Linux-based local machine. 2. Copy the python folder from into $HOME/ryu/ryu 3. Rename it as netide. 4. Install Ryu by running the command python./setup.py install from $HOME/ryu. 5. Then, add the installation path of Ryu to your PYTHONPATH variable in your $HOME/.profile or $HOME/.bashrc by adding the following line: export PYTHONPATH=/usr/local/lib/python2.7/dist-packages/ryu The path may depend on the Linux distribution and on the version of Python used for the installation. Additional system and Python packages may be required to successfully complete the installation procedure. The following commands ensure that all required packages are installed on a Linux Ubuntu distribution: apt install python-pip python-dev python-repoze.lru apt install libxml2-dev libxslt1-dev zlib1g-dev python-zmq pip install ecdsa stevedore greenlet oslo.config eventlet WebOb Routes Running Backends are clients for the Core. Therefore, always start the Core rst (cf. Section 3.2). The command below starts the Ryu Backend along with the rest of the Ryu platform. The Backend will try to connect to a running Core which must be already running and listening on TCP port Recall that, Ryu is not built using containers, such as Apache Karaf, that allows modules and applications to be loaded and removed dynamically. Thus, all modules and SDN applications must be included in the start up command. In the example below, we show how to run Ryu plus the Backend and two applications: a L2 forwarding (simple_switch.py) and a rewall (firewall.py). Both applications are available in the Ryu Backend folder on github. Other applications can be used as well. A few sample applications are available in the Ryu source tree in /home/foo/ryu/ryu/app. ryu-manager --ofp-tcp-listen-port 7733 ryu-backend.py simple_switch.py firewall.py The ofp-tcp-listen-port 7733 is used to avoid possible conicts that may happen when two dierent controller platforms are running on the same machine. In our case, we could have Ryu hosting the Backend and, e.g., ONOS running the Shim. By default, they both bind the TCP port 6633 to accept OpenFlow connections from the network elements. Such a parameter forces Ryu to listen on a dierent port (in this case 7733), which is anyhow never used, as the only active southbound interface on client controllers is the Backend. 30

40 Document: CNET-ICT /D Floodlight Backend Installing When developing a module for Floodlight, you must download the Floodlight source code rst and add your module as a new namespace. In particular, follow these steps: Clone locally Floodlight ( and then switch to v1.1 version: git checkout v1.1 Build oodlight using make make Copy the package net.floodlight.interceptor in the oodlight source folder. (Copy floodlight-backend/v1.2/src/main/java/net/floodlightcontroller/interceptor folder to floodlight/src/main/java/net/floodlightcontroller). Copy the jar les under lib folder in oodlight lib folder (except floodlight.jar used only during development for compilation). Replace /floodlightv1.1/src/main/resources/floodlightdefault.properties with the given one that can be found at: /floodlight-backend/v1.2/src/main/resources/floodlightdefault.properties. Add to build.xml of floodlightv1.1 the following lines inside <patternset id="lib"> tag: <include name="jeromq jar"/> <include name="javatuples-1.2.jar"/> <include name="netip snapshot.jar"/> (or directly replace the le with the one provided in manual/build.xml). Add to /floodlightv1.1/src/main/resources/meta-inf/services/net.floodlightcontroller. core.module.ifloodlightmodule the following line: net.floodlightcontroller.interceptor.netidemodule (or manually replace the le with the one provided in manual/ifloodlightmodule). Compile the project with make command Running Before starting the Floodlight Backend, there is an assumption that the Core is already running and listening for connections Use the script provided in the oodlight folder floodlight.sh to start the controller. If the script returns an error related to java options used to start oodlight, replace the 14 line of the script with the following: JVM_OPTS="$JVM_OPTS -XX:CompileThreshold=1500" #-XX:PreBlockSpin=8" Similarly for Ryu, if Floodlight and another controller are running on the same machine, you may receive a Manager address already in use error when starting Floodlight. In this case, manually change the manager port used by Floodlight in Class net.floodlightcontroller.core.internal. Controller, line 136, from 6653 to (or manually replace the le with the one provided in manual/controller.java and recompile the project). 31

41 Document: CNET-ICT /D Testing Backends Testing To test a Backend it is necessary to run the Core and at least one of the Shims described in the previous sections (either OpenDaylight, ONOS or Ryu), plus the Mininet network emulator. For the Ryu Backend just follows the example in Section 3.6. For the Floodlight Backend, use the instructions in Section

42 Document: CNET-ICT /D IDE manual 4.1 Installation This chapter explains the installation of the Eclipse-based IDE. It lists the prerequisites and afterwards shows dierent ways to install the IDE Prerequisites In order to use properly, you need to have an installation of Vagrant (>v1.6), VirtualBox, and SSH on your machine. Windows users, make sure that the Vagrant and the SSH executable are referenced in your PATH variable. works on Eclipse Luna (4.4), Mars (4.5), and Neon (4.6). We recommend using Eclipse Neon Installation via the Eclipse Marketplace We provide an Eclipse Marketplace listing of the latest stable version to provide an easy and fast installation of the developer toolkit. The marketplace entry can be found at http: //marketplace.eclipse.org/content/netide. The necessary steps to install from the Marketplace are as follows. 1. Open your Eclipse installation and navigate to Help->Eclipse Marketplace... in the Menubar. 2. Search for, click Install and follow the instructions. After restarting Eclipse, is ready to run Installation using the update site The plugins are also hosted at an update site which can be found at netide.eu. The update site hosts the stable version as well as nightly builds. The necessary steps to install from an update site are as follows: 1. Open your Eclipse installation and navigate to Help->Install New Software Add the repository 3. Enter for a stable installation or netide.eu/nightly for nightly builds. 4. Select, click Next and follow the instructions. After restarting Eclipse, is ready to run Installation by building from source can also be compiled from source using the build automation system Maven[32]. compilation and installation consists of the following steps: 1. Check out the master branch on The 33

43 Document: CNET-ICT /D Navigate to the folder releng/eu.netide.dependencies and type mvn p2:site. This will build an Eclipse repository for external dependencies at releng/eu.netide/dependencies/ target/repository. 3. Navigate to the folder releng/eu.netide.parent and type MAVEN_OPTS="-Xmx1024M" Maven package. The build process can take some time. 4. Once it has nished, you'll nd the contents of an Eclipse update site as a zip le in releng/eu.netide.product/target. 5. Open your Eclipse installation. 6. Go to Help->Install New Software->Add->Local and specify the location of the dependency repository. 7. Go to Help->Install New Software->Add->Archive and specify the location of the zip le. 8. Select from the list and follow the instructions. After restarting Eclipse, is ready to run. Cool. 4.2 Usage consists of various tools to assist the development, composition, deployment and testing of SDN network applications. Namely, there are the following features: (i) managing the application format for a package via various helper functions, (ii) network topologies can be created and edited using the graphical topology editor, (iii) network applications can be annotated with congurable parameters and nally run and tested out of the IDE on remote and virtual machines. We are going to explain each of these features in the following subsections Application format management provides a set of tools to manage the SDN modules composing the network application in the application format (see Chapter 3 in Deliverable D2.4 [34]). The functions necessary for that are activated in the perspective. The perspective can be opened via Window->Perspectives->Other->. A new Project providing a skeleton of the application format can be created using the new project wizard. Select File->New-> Project and follow the instruction of the wizard. This creates a new project in the Eclipse workspace with the initial les to compose a app. New SDN modules can be added via the app wizard. Open the app wizard by right-clicking the project folder and selecting New->Other->-> App. Enter a name and a controller for the module. This creates a folder containing a system requirements specication and a parameter schema (see Subsection 4.2.2) Specifying and using parameter schemas app developers can specify the congurable parameters for SDN modules. Every app created with the app wizard contains an empty parameter schema le. This le can be edited in the parameter schema editor which is also a part of. Listing 4.1 shows a simple parameter schema le which denes the parameters of a rewall SDN modules. Each parameter in the parameters section is dened by a name, a type and a default value. The complex type IP is dened in the types section as a string matching a certain regular expression. 34

44 Document: CNET-ICT /D 2.7 Figure 4.1: The application format project tree. { } Listing 4.1: Parameter Schema parameters: { stateful_idle_timeout: "Integer = 5", port_web: "Integer = 80", host_web: "IP = ' '", port_dns: "Integer = 53" }, types: { IP: { type: "String", regex: "\"\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b\"" } } This le can now be used for easy, platform-independent app conguration. The graphical form for parameter conguration can be invoked by right-clicking the project folder and selecting ->Configure Parameters. This brings up a GUI in which values can be assigned to parameters. The congured values for all congured apps are stored in a parameters.json le. Figure 4.2 shows a conguration GUI generated from the parameters le in Listing 4.1 and a short parameter le for a simple MAC learner. The form entries are already lled with the default values. Mapping of the values in parameters.json to concrete conguration les can be achieved by creating conguration le templates. The following steps are needed to create a template for a conguration le. 35

45 Document: CNET-ICT /D 2.7 Figure 4.2: Parameter congurator. 1. Select a le form an app and select ->Copy to Templates. A copy of the template will appear in the templates folder. 2. Edit the le in the templates folder and enrich it with template tags. The tags must match the parameter names in the parameter schema les as they will be substituted by the parameter values later on. The syntax of the tags has to comply to the handlebars specication [35]. With the templates and the parameters.json le at hand, can generate platform-specic congurations for the SDN module in the package. The les can be rendered by right-clicking the project folder and selecting ->Render Templates Conguring Network Topologies provides tools to test composed network applications on virtual machines and physical machines optionally with a virtual network. This section explains the editors needed to congure test topologies and environments. Setting up virtual topologies is supported by the Topology Editor. This editor supports the graphical creation of SDN-enabled network topologies. Figure 4.3 shows a screenshot of the editor. There is a canvas on the left-hand side and a palette on the right-hand side. Network models can be created by clicking on an element in the palette and clicking on the canvas. The rst step towards creating a network model is to add a network (white rectangle). A network contains switches (light blue rectangles) and hosts which can be connected via ports and connectors. Choosing a connector from the palette and subsequently clicking on two switches or switch and a host creates a port on each switch and connects them. In order to create a controller to manage the switches, a controller needs to be added to the canvas (not a network) and connected to the switches it is supposed to manage. Each element has properties such as a name, an IP address, or a datapath ID. These properties can be edited by right-clicking on an element and selecting Properties. This opens a view in which all properties of an element can be edited. The tab Semantics shows the aforementioned semantic properties. 36

46 Document: CNET-ICT /D 2.7 Figure 4.3: Topology Editor. The topology model is stored as an EMF [36] compliant model. In order to create a runnable conguration for the Mininet network emulator for testing, expand the topology model in the project tree, right-click on the Network Environment element and select Generate Mininet Configuration. This stores a python-based Mininet conguration in the folder gen/mininet. The virtual network can be set up by running the python script <projectname>_run.py Testing network applications Network applications can be executed directly from the developer toolkit on top of the Engine using a dashboard (cf. Figure 4.4). The dashboard can be shown by opening the le <projectname>.wb in the project tree. Alternatively, such a le can also be created using the respective wizard under New->Other->-> Configuration Manager. Figure 4.4 shows the user interface of the dashboard. The elements of this interface are described below: 1. The upper section of the UI shows the settings and controls to manage a connection to a machine on which the apps are executed. Users can create establish an SSH connection to a remote machine or set up a Vagrant-powered virtual machine. 2. The Network Engine's Core can be started by pressing the On button in the respective section. The Off button switches it o. Starting the Core opens an integrated terminal in the IDE. If the terminal closes, it can be displayed again using the Reattach button. 3. The Core's behaviour can be specied via a composition specication le(section ). This le can be loaded inside of the IDE by selecting a composition specication le and loading it into the Core by pushing the Load button. 4. Dierent Shims can be started up via the IDE. Pushing the button Import Topology fetches the underlying topology of the Shim if the specic Shim supports it. The imported topology is stored as imported.topology in the current project root. 37

47 Document: CNET-ICT /D 2.7 Figure 4.4: SDN network application testing dashboard. 5. Mininet can be started, stopped and reattached to a terminal by pushing the On button in the Mininet section. 6. The topology section can be used to specify the topology used by Mininet. The Generate button generates a Mininet conguration from the topology specication. 7. The On button starts the debugger which logs the Core messages in a human-readable format. 8. The Apps section contains a list of SDN modules to run on the engine. New modules can be added to the composed netowkr application via the Add button. This opens a dialog in which the controller platform, the entry point, and the OpenFlow port can be congured, as well as whether it should run on the engine. The Start, Stop, Reload, and Reattach buttons are used to start, stop, and restart an app, as well as reattach a terminal to a running app Monitoring the network application The developer toolkit provides a ZeroMQ [15] interface to the Core which listens to messages sent by the LogPub module. A basic list-based visualization of the messages can 38

48 Document: CNET-ICT /D 2.7 be invoked by the ZMQ Listeners view. This view can be opened by selecting Window->Show View->Other->->ZMQ Listeners. This opens up a list of ZeroMQ listeners. Right-clicking on a listener in the list and selecting Toggle activates the listener and double-clicking a listener opens up a log window containing the logpub messages. Figure 4.5: Logpub interface and view Global Conguration There are several global options to congure the developer toolkit. explains the available parameters. The following list Figure 4.6: Global conguration view. 1. The executable paths for Vagrant, SSH, and SCP are specied in the rst section. These parameters must have a value since the testing feature heavily uses these three tools. Windows 39

49 Document: CNET-ICT /D 2.7 users have to specify the paths manually, OSX and Linux users have default values for these parameters which can be changed if necessary. 2. This section enables users to dene a proxy server for their test VMs. This server conguration is written into the shell prole as well as into the package manager conguration of the VM during the VM provisioning phase. 3. Custom boxes can be used to skip the provisioning process by providing an already provisioned VM image. This way, an image can be created once and reused among App testers. A provisioned VM can be exported to an image by navigating into the folder containing the Vagrantle and typing vagrant package name <name> where <name> denotes the given name of the box. The box can be copied on another machine added via vagrant box add <filename>. The given name of the box can then be entered into the line in this section of the conguration GUI. 4. The section on the bottom of the GUI contains pre-congured ZeroMQ listeners. All listeners are dened with the format <name>,<address> Application Loader This tool validates and runs deployed packages on client controllers. It starts all controllers and apps as well as the Core in a detachable tmux session Setup Use the script loader/setup.sh in the Engine repository for bootstrapping. Cleanest way to do the setup is to either put it inside a docker container or a virtual machine. The scripts assume a recent 64-Bit Ubuntu as the OS Install Package Before starting an package, it needs to be installed rst. To do so run the loader/startloader.sh script with the install command which will install all the prerequisites to make the contained network application runnable. Example usage:./startloader.sh install /path/to/package. The package can either be a folder or an archive in the tar.gz format. If an archive is used it will be extracted to /tmp/netide/ Available Commands Every command is used through the startloader.sh. Currently there are the following commands available: install /path/to/package/ used to setup the chosen package for running. run /path/to/package/ --server --param runs the given package. Server is optional, choose from ODL and ryu. Choose ryu for ryu, else ODL will be used. param is optional, a parameter le can be specied with a conguration used for this run. extractionpath /path/to/extract denes a path to which packages should be extracted. extractarchive /path/to/archive used to extract the given archive to the extraction path. Default is /tmp/netide 40

50 Document: CNET-ICT /D 2.7 generate /path/to/package --param generates the param conguration specied in the le given with param. list lists all running SDN controllers. stop stops all running SDN controllers. attach the packages are started in tmux sessions. If there are any detached sessions available this command attaches them. 41

51 Document: CNET-ICT /D Tools manuals In this chapter, we describe how to install and execute the dierent tools that project oers. The main purpose of the tools is to diagnose, debug and troubleshoot the data and control planes of the SDN network. Engine is connected to these tools thanks to the LogPub, an interface which is in charge of capturing the messages exchanged among the client controllers and the server controller, and pushing them into a queue. The tools will retrieve the enqueued messages and will carry out their task. The LogPub provides the capability to connect new tools to the Engine and it oers a complete independence of the SDN technology used. Although, most of the tools perform very dierent tasks (Proler whose aim is to show relevant information of the control and data planes, the Trac Emulator which simulates several trac patterns or the Memory Management System that optimizes the memory consumption of the devices), there are also tools which are tightly related as they share part of the code: the Logger, the Debugger and the Vericator. The le debugger.py is in charge of obtaining all the distributed messages enqueued by the LogPub. Although called debugger.py, it serves the three tools at the same time, let us explain how: It prints all the messages received on runtime and generates a plan text le for the Logger: results.txt It creates a PCAP [37] le for the Debugger: results.pcap It produces a postcard le for the Vericator: results.card Later on, such tools can be used to analyse the output les. All the code of the tools can be found at where the master branch is the most updated and stable version of it. 5.1 Logger The Logger is in charge of retrieving all the messages that are exchanged between the dierent modules of the Engine and display them into a terminal. This tool lets the user know what is happening in the network at runtime. It also saves all logs in a plain text le results.txt for oine checking Installing For this purpose, it is necessary that the Logger has access to the queue where the messages are kept. The queue mechanism is provided by ZeroMQ [15], which therefore must be installed. The ocial site of ZeroMQ provides a guide which explains how to install it. Here we summarize how to install it in Ubuntu 14.04: Make sure that libtool, pkg-config, build-essential, autoconf, and automake are installed. Check whether uuid-dev package, uuid/e2fsprogs RPM or equivalent on your system is installed. 42

52 Document: CNET-ICT /D 2.7 Download one of the version of ZeroMQ: wget Unpack the source archive: tar -xvf zeromq tar.gz Install ZeroMQ: cd zeromq /autogen.sh./configure && make check sudo make install sudo ldconfig Running The whole logic of the tool is implemented in one python script, therefore executing such script is enough to activate the Logger. The following command is used to run the Logger and the Debugger, as both tools are implemented with the same script: python debugger.py The user will be able to visualize the messages exchanged between the network and the SDN module composing the network application. The green messages belong to the data plane and their destination is the controller, while the blue ones are sent from the control plane to the datapath elements. Figure 5.1: Logger trace. 5.2 Debugger The Debugger saves all captured logs from the LogPub, and thus trac exchanged in the Network Engine, in a le called results.pcap. As mentioned before, it shares the runtime code debugger.py with the Logger and the Vericator. The dierence is the resulting le for later analysis with Wireshark [38]: results.pcap. Additionally, we have developed a Wireshark dissector, a plugin for Wireshark to analyse the messages encapsulated with the header (see Annex A), either at runtime or for oine analysis of the debugger's output results.pcap. 43

53 Document: CNET-ICT /D Installing For the installation, we will distinguish between two components: the Debugger and the Wireshark dissector. For both cases, we need the source code of Wireshark, because we are going add new dissectors to it. Therefore, rst we install Wireshark's sources and we check that everything works correctly. Typical dependencies on a Ubuntu Linux OS to be installed before the compilation are: git, autoconf, automake, libtool, bison, flex, qt-sdk, libgtk-3-dev, libpcap-dev. However, please refer to the Wireshark documentation [38] for more detailed information. Then do as follows: Go to the source directory and congure your source so it will build correctly for your version of UNIX:./autogen./configure Build the sources: make Install the software in its nal destination: make install Finally, if you have the following error when running Wireshark: wireshark: error while loading shared libraries: libwiretap.so.4: cannot open shared object file: No such file or directory then run the following command: sudo ldconfig Debugger As the Logger, it requires the installation of ZeroMQ (follow the guide in 5.1) to run debugger.py. After this, we need to copy some les from debugger/core and follow the steps below: Copy packet-user_encap.c in the epan/dissectors/ folder of the Wireshark sources. Then recompile Wireshark with the following three commands (where the rst one species the location of the compiled Wireshark binary):./configure --prefix=<location-of-your-wireshark-installation>/wireshark make make install Copy the les netide.so and netide.la in the lib folder of your Wireshark installation. Normally the path of the plugin library for most Wireshark installations from source code is /usr/local/lib/wireshark/plugins/x.y.z (where x.y.z is the specic version you installed). 44

54 Document: CNET-ICT /D 2.7 Currently, the packet-user_encap.c automatically associates the DLT_USER0 link-layer header type with the protocol. If for some reason this does not work, the rst time you run Wireshark you must do it manually. Normally, Wireshark tells you to do so at start-up, but if not, you can follow the instructions in: to tell Wireshark how to decode our user link-layer header type, which consists of going to: Edit Preferences Protocols>DLT_USER Edit Encapsulations Table and adding an entry for, according to: chunked/chuserdltssection.html Wireshark dissector In the case of the simple dissector, the installation is much easier. We need to copy two les from that contain the dissector and do as follows: Copy the les netide.so and netide.la in the lib folder of your Wireshark installation. Normally the path of the plugin library for most Wireshark installations from source code is /usr/local/lib/wireshark/plugins/x.y.z (where x.y.z is the specic version you installed) Running Although the installation of the Debugger and the Wireshark dissector contains some steps in common (as they both share the use of Wireshark libraries), we use dierent approaches for running them. The Debugger needs the LogPub to attach to the Network Engine, while the Wireshark dissector captures the trac on any network interface of the machine where it is running, included the messages exchanged between the components but without requiring the LogPub interface Debugger To run the Debugger, you need to execute the following command once the Engine is running: python debugger.py Once we stop the execution, a results.pcap le will be generated, which can be opened with the Wireshark installation we prepared in the previous sections. Figures 5.2 and 5.3 show examples of the results.pcap inspected with Wireshark and the dissector Wireshark dissector Open Wireshark and capture any trac in the network. Usually the components of the Network Engine communicate via the localhost interface using the intermediate protocol when all its components are running on the same machine. The capture will show something like Fig. 5.4, where is the IP address of the localhost interface. The advantage of using the Wireshark dissector is that the installation is much simpler and it does not require to directly interface with the Network Engine. However, the drawback is that it indirectly requires the ZeroMQ dissector, which produces errors when a ZeroMQ message contains more than one message. 45

55 Document: CNET-ICT /D 2.7 Figure 5.2: Debugger's output with OpenFlow 1.0. Figure 5.3: Debugger's output with OpenFlow 1.3. Figure 5.4: Wireshark dissector. 5.3 Proler In software engineering the term proling is known as the performance analysis of a program. Its objective is to study the behavior of a software in order to detect failure points or areas where a performance optimization is needed. In this project, the concept of proling is applied to SDN networks, implementing an SDN Application Proler. Such a tool is focused on collecting relevant statistics from the control plane, as the execution time of the applications executed on top of the controller platforms. 46

56 Document: CNET-ICT /D Installing The Application Proler, is based on the proling concept of software engineering. The goal of this approach is to obtain the execution time of the methods executed in a controller and in their applications. Firstly, it is necessary to install the R software [39], a software environment for statistical computing. In this section, we explain how to install the R software in Ubuntu LTS, for other operating systems please refer to the project's web page. As a rst step, add repository deb trusty/ to the list of software channels in /etc/apt/- sources.list le. Secondly, execute the next commands in a terminal: sudo apt-get update sudo apt-get install r-base This will install the latest version of R (otherwise, some packages might be missing and the visualization of data of the proler might not work correctly). Note: If you obtain a GPG error when trying to update apt-get, you might want to check this link: how-do-i-fix-the-gpg-error-no-pubkey which basically solves it by installing y-ppa-manager and then executing Advanced->Try to import all missing GPG keys. Additionally, users who need to compile R packages from source [e.g. package maintainers, or anyone installing packages with install.packages()] should also install the r-base-dev package: sudo apt-get install r-base-dev Installation and compilation of R or some of its packages may require Ubuntu packages from the backports repositories. Therefore, it is suggested to activate the backports repositories with an entry like: deb trusty-backports main restricted universe. Finally, install RStudio (a GUI for R) [40]. This is optional, but eases for the visualization of the results Running Here we show how to prole the Ryu controller and its SDN modules when running in the context of the Network Engine. Compared to Section 3.7, the command for executing the Ryu controller, Backend and SDN modules includes additional parameters and becomes as follows: python -m cprofile -o statistics ryu/bin/ryu-manager --ofp-tcp-listen-port 7733 $NETIDE/Engine/ryu-backend/ryu-backend.py \ $NETIDE/Engine/ryu-backend/tests/simple_switch.py The output of this command is a text le called statistics.txt which contains the execution time of all the functions under proling. Statistics can be analysed with R by running command profiler_graphics.r or with the RStudio Graphical User Interface (GUI) by double clicking on it (Figure 5.5). 5.4 Trac Emulator This tool is composed of applications which can be used to generate trac for the Emulator. The current applications available are: http: a client-server application which simulate HTTP trac from a host to a server. The trac patterns are similar to what can be found in this work [41]. pcap: an application which takes.pcap les like those who can be found in.pcap repositories [42] [43] or using an external tool like DCT2Gen [44], and create traces playable by a tool like tcpreplay [45]. 47

57 Document: CNET-ICT /D 2.7 Figure 5.5: Application Proler GUI. tftp: a client-server application which simulate TFTP trac. iperfp: a application which uses iperf to simulate common ow patterns (VoIP, video,...) in terms of packet size and bandwidth Installation To get the Trac Emulator application, clone the Tools repository. git clone cd traffem The following packages are used as system requirements for the dierent applications: http pcap tftp : wget, python : tcpreplay, tshark, wireshark : tftp, python iperfp : iperf They can all be installed with sudo apt-get install [name_of_package] Running http This application is composed of two scripts, one to run on server(s) and another one on client(s). On the server(s) run:./httpserver.sh 48

58 Document: CNET-ICT /D 2.7 you can quit by either pressing Enter of Ctrl+c. The server will create multiple les in a folder called HTTP_les. This folder will be deleted upon quitting the application. On the client(s) run:./httpclient.sh <serverip> you can quit by either pressing Enter of Ctrl+c. The client will connect to the provided IP and simulate the trac from a user. The user will browse the internet and send/receive (mandatory patterns). It will also randomly use a media pattern between listening to a web radio, stream music or watch a video. It will also randomly use a le synchronization service. The patterns available are: Mandatory pattern(s): Internet: Simulate the loading of a web page (size 2MB) and wait for 5-60s to simulate reading Simulate sending and receiving an (75KB) continuously Random pattern(s): Radio: Simulate a web radio. This pattern will download a file at a low rate (20KB/s) constantly. Music: Simulate a streaming service. This patten will download a "song" (3-5MB file) and wait before loading another "song" (3-5mn) Sync: Simulate the synchronization of data Video: Simulate the browsing and streaming of a video all files created on the host will be deleted upon closing the application pcap You can use the le test.pcap and run./producepcap -t to produce three pcap les. Otherwise to use the application run: Usage./producePCAP -p <.pcap> -i <source IP> -m <source MAC> -f <host(s) file> <.pcap>: the.pcap file <source IP>: the IP of the desired host from the.pcap <source MAC>: the MAC of the desired host from the.pcap <host(s) file>: a file with the output host in this format: HOSTNAME HOST_IP HOST_MAC Example:./producePCAP -f hosts.txt -p test.pcap -i m f8:1e:df:e5:84:3a 49

59 Document: CNET-ICT /D 2.7 With host.txt containing: H :08:15:16:23:42 H :01:46:56:62:96 H :09:53:59:84:87 When the.pcap les are ready, use tcpreplay on the hosts command line. For example, to replay the.pcap once out of eth1: tcpreplay -i eth1 -K test.pcap To replay the.pcap 5000 times 1 at line rate: tcpreplay -i eth1 -t -K --loop 5000 test.pcap To replay the.pcap 5000 times and limiting the rate at 50 Mbps: tcpreplay -i eth1 --mbps=50 -K --loop 5000 test.pcap See the tcpreplay help for more options tftp This two part application is composed of two scripts, one to run on server(s) and another one on client(s). On the server run:./tftpserver.sh On the client(s) run:./tftpclient.sh <serverip> [test small medium big] By default the test le is downloaded. The other available les are small (1MB), medium (10MB) and big (100MB) iperfp This two part application is composed of two scripts, one to run on server(s) and another one on client(s). On the server run:./iperfp.sh --server On the client(s) run:./iperfp.sh <serverip> [--imix --voip --video] Options: --server: run in server mode --imix: play the Internet Mix pattern. 7 packets of 40B 4 packets of 576B 1 packet of 1500B --voip: play a VoIP pattern (G.726 audio codec) - 130B 55KB/s --video: play a video straming pattern (H.264 video codec) B 471KB/s 1 tcpreplay does not replay by duration so the total time would be the duration of the.pcap times the loop argument. 50

60 Document: CNET-ICT /D Memory Management System The Memory Management System (MMS) is a tool for SDN environments that aims at improving the usage of the memory of the network devices and, as a consequence, the robustness and the performance of the network. The MMS automatically removes the ow entries installed by applications that are no longer running (we called this feature memory de-allocation) and automatically moving the less used wildcard entries to a slower memory when the devices' TCAM is full (memory swapping). The MMS has been implemented as a bundle for the ONOS platform. However, as explained in [46], its architecture is generic enough to be implemented for other platforms such as OpenDaylight and Ryu. Thus, the MMS has been designed to provide memory management techniques to the Network Engine as a tool executed in the server controller Installation The MMS is implemented as a bundle for ONOS version First, the following general requirements are necessary to run ONOS and the MMS on Ubuntu LTS: Java 8 JDK Apache Maven git In order to perform the installation of ONOS and MMS, the following steps are required: Download ONOS: git clone git checkout v1.5.1 In order to setup the environment and build the ONOS source code, please refer to this guide: To simplify the execution of ONOS, the best way is to create a ONOS cell. This guide explains the creation and usage of cells: Scratch#ONOSfromScratch-4.Createacustomcelldefinition Download and build the MMS: git clone cd memory-management-system/onos && mvn clean install Install the MMS: onos-app $OC1 install target/onos-app-mms snapshot.oar Running Once ONOS and the MMS are compiled, they are ready to be executed. In order to start ONOS, run the following command from a terminal: ok clean and the Command Line Interface (CLI) should look like this: 51

61 Document: CNET-ICT /D 2.7 onos> Now ONOS is running, so the MMS can be installed and activated from another terminal: onos-app $OC1 install! target/onos-app-mms snapshot.oar At this point, the MMS is running. The memory swapping is automatically performed when the SDN device noties a table full message. To get information about the number of ow rules swapped out from a SDN device, this command can be used in the ONOS CLI: onos> mms:swapsize {device_uri} The user can tune the conguration of swapping threshold, which is the amount of ow rules deleted by the swapping out operation. The following command can be used: onos> cfg set eu.netide.mms.mmsmanager flowdeletionthreshold {value} For example, to swap the 20% of ow rules from a SDN device, value is 0.2. The memory deallocation is not automatically activated at the MMS start-up. The network administrator can activate it for particular SDN applications running on ONOS. This command registers an application to MMS deallocation: onos> mms:addtomms {application_uri} In order to unregister an application use this command: onos> mms:removeapplication {application_uri} 52

62 Document: CNET-ICT /D framework at work We have presented the framework at work in dierent conferences. In this chapter we describe the setup and considerations for the demonstration we did at the NetSoft'16 [47] conference. To provide a comprehensive example of how the engine can be used, we expand it to cover additional component(s) that are not treated in depth in the demonstration paper. We defer an in-depth discussion of how to implement the project use cases to the nal WP5 deliverable. With, we have envisaged a network application development workow centred on (i) the IDE, a tool for developers to implement, deploy and test new SDN applications, and (ii) the Network Engine, where SDN applications can be tested on an emulated environment or executed on a real SDN network. Below, we describe the scenario where a developer leverages to implement, congure and test a network application in an emulated environment before the actual deployment on a production SDN network. In this scenario, we also show how IDE and Network Engine support the dynamic reconguration of the Network Application to provide new network services. 1. We rst use the Integrated Development Environment to design the network topology for the network emulator and to build a network application by means of the implementation and composition of two or more modules. 2. Once the application is ready, we start the testing phase. Here, the Network Engine is congured to meet the network application requirements in terms of client controllers and composition specication. In this phase, Mininet loads the conguration le produced with the IDE which contains the topology details dened by the developer (Section 4.2.3). The network application can be tested with any topology conguration to verify its behaviour in dierent conditions. 3. We show how dierent modules, written for dierent controller frameworks, cooperate as a single network application in controlling the network. 4. We modify the composition specication by adding or removing modules from it, then we reload the composition specication, and then see how the corresponding modules are started/stopped and the network behaviour changes accordingly. The process, up to this point, Development Environment Topology Design Code Editor PY/Java Composition Specification <Module LearningSwitch/> <Module Firewall/> <Module Monitor/> <Module LoadBalancer/> <Module DOSProtection/> Engine Configuration Server Controller ONOS Network Application Learning Switch Floodlight Firewall Ryu Monitor Floodlight Load Balancer Ryu DOS Protection Ryu Deployment Runtime Virtual Machine Network Application LS FW Monitor Load Balancer DOS prot. Network Engine FloodLight Ryu Core ONOS Mininet Composition Spec. < /> Figure 6.1: Demonstration workow. was demonstrated at Netsoft'16 and its rationale is shown in Figure

63 Document: CNET-ICT /D Demo scenario To demonstrate the lifecycle described above, we will use the setup we demostrated at Netsoft'16 (Figure 6.2). The initial conguration includes a Local Area Network (LAN) controlled by a Layer- 2 forwarding module and protected against unauthorized access by a network element acting as a Firewall. Trac injected into the LAN is monitored by an SDN module which intercepts relevant control messages and displays them on a terminal. Figure 6.2: Demonstration scenario. Design and Development: The aforementioned scenario is prepared from the Development Environment (white elements on left side of Figure 6.1), where we design the topology, select/modify a set of SDN application modules (Learning Switch, Firewall and Monitor in the gure), and nally we prepare the conguration for the Network Engine. Application modules are written either in Java or in Python and implemented for dierent controller platforms (in this demo we use Floodlight and Ryu as client controllers). Such modules will cooperatively control the network as a single network application thanks to the Composition Specication, a set of XML-encoded policies that determine how the Core coordinates the modules (Section ). The output of this phase consists of: (i) a conguration le for Mininet, (ii) a conguration le for the Network Engine (the Composition Specication in Listing 6.1), and (iii) the network application as a set of modules written for Ryu and Floodlight. Deployment: The output of the previous phase is deployed onto a pre-congured Virtual Machine where all the required components are congured and started. As represented by white elements on the right side of Figure 6.1, Mininet loads the topology model produced by the IDE and the Network Engine is assembled with the required controller frameworks, i.e., ONOS as server controller and Ryu and Floodlight as client controllers along with the required SDN modules. As soon as the Core loads the Composition Specication, the Network Engine starts controlling the network through the network application based on the policies dened in the specication. The Composition Specication used for this specic use case is shown in Listing 6.1, where switches with datapath_ids 0x21, 0x22, 0x23 and 0x24 within LAN and DMZ are controlled by Floodlight via the LearningSwitch module. Switch 0x11 acts as a rewall and is controlled by Ryu by means 54

64 Document: CNET-ICT /D 2.7 of the Firewall module. Trac injected in switch 0x22 is also monitored via the Monitor module for Ryu which is composed in parallel with LearningSwitch. With this conguration, hosts inside the LAN can communicate with each other, the LAN is protected from undesired trac and it is also monitored to discover possible security breaches coming from trusted users. Listing 6.1: Initial conguration. 1 <?xml version="1.0" encoding="utf-8"?> 2 <CompositionSpecification xmlns=" 3 <Modules> 4 <Module id="simpleswitch" loaderidentification="learningswitch.jar"> 5 <CallCondition events="packetin" datapaths="0x21 0x22 0x23 0x24"/> 6 </Module> 7 <Module id="monitor" loaderidentification="monitor.py"> 8 <CallCondition events="packetin" datapaths="0x22"/> 9 </Module> 10 <Module id="firewall" loaderidentification="firewall.py"> 11 <CallCondition events="packetin" datapaths="0x11"/> 12 </Module> 13 </Modules> 14 <Composition> 15 <ParallelCall resolutionpolicy="priority"> 16 <ModuleCall module="simpleswitch" priority="1"/> 17 <ModuleCall module="monitor" priority="2"/> 18 </ParallelCall> 19 </Composition> 20 </CompositionSpecification> Runtime: The framework can be also used to activate new network services at runtime to improve, for instance, performance and security of a production network. Referring to the red elements in Figures 6.1 and 6.2, the IDE can be used to re-congure the Network Engine which is currently controlling the network to satisfy new requirements without the need of restarting the Network Engine. As an example, we add a Load Balancer module to spread the users' requests over dierent web servers, we install another module that cooperates with the Firewall on protecting the network from Denial of Service (DoS) attacks, and we move the monitoring module from one switch to another. Listing 6.2: Updated composition specication. 1 <?xml version="1.0" encoding="utf-8"?> 2 <CompositionSpecification xmlns=" 3 <Modules> 4 <Module id="simpleswitch" loaderidentification="learningswitch.jar"> 5 <CallCondition events="packetin" datapaths="0x21 0x22 0x23"/> 6 </Module> 7 <Module id="simpleswitch" loaderidentification="loadbalancer.jar"> 8 <CallCondition events="packetin" datapaths="0x24"/> 9 </Module> 10 <Module id="monitor" loaderidentification="monitor.py"> 11 <CallCondition events="packetin" datapaths="0x23"/> 12 </Module> 13 <Module id="firewall" loaderidentification="firewall.py"> 14 <CallCondition events="packetin" datapaths="0x11"/> 15 </Module> 16 <Module id="dosprotection" loaderidentification="dos.py"> 17 <CallCondition events="packetin" datapaths="0x11"/> 18 </Module> 55

65 Document: CNET-ICT /D </Modules> 20 <Composition> 21 <ParallelCall resolutionpolicy="priority"> 22 <ModuleCall module="simpleswitch" priority="1"/> 23 <ModuleCall module="monitor" priority="2"/> 24 <ModuleCall module="dosprotection" priority="1"/> 25 <ModuleCall module="firewall" priority="2"/> 26 </ParallelCall> 27 </Composition> 28 </CompositionSpecification> Listing 6.2 shows the nal Composition Specication. At runtime, it has been updated (either in one single step or with multiple incremental changes) by (i) assigning the control of switch 0x24 to the LoadBalancer module written for FloodLight, (ii) moving the Monitor module to switch 0x23 and composing in parallel DosProtection written for Ryu with Firewall on switch 0x11. 56

66 Document: CNET-ICT /D Advanced architectural concepts Composition of SDN applications has been one of the main topics of. At the end of the project, we step back for a moment and look at the problem from a more formal and generalised perspective. 7.1 Denitions We can see a software-dened network as a collection of interconnected nodes (switches) forming a graph G = V (V, E), where E describes the connectivity between the switches. We dene the state N v of a node as the state of its Forwarding Information Base (FIB). A FIB entry is dened as tuple (p, m, i) specifying a priority, the packets to match and a list of instructions i. 1 The network state N is then dened as the collection of all node states N = {N v v V } The network state can be changed by a command C. A command is a sequence of basic commands C = [c 1, c 2,...] with c j {F inst, F del } that modify the FIB of a switch: F inst = (v, p, m, i): Install a FIB entry on node v with priority p, match m and list of instructions i. F del = (v, p, m, i): Remove the FIB entry that was installed by F inst (v, p, m, i) We dene the function a: N C N as the function that applies a network command to a network state and produces the new network state: a(n, [c1,..., c k ]) = a(a(n, c 1 ), [c 2,..., c k ]) { Add (p,m,i) at v if cj = F inst a(n, c j ) = Remove (p,m,i) at v if c j = F del for a basic command c j = (p, m, i, v). In a network, these commands are generated by a control application or module M j. We dene a network module as a state-based function M j that reacts to an event ev with a network command (and possibly modies its internal state): M j : ev C Examples for network events are the arrival of certain packet types (like ARP request) or the arrival of a new ow currently not handled by the FIB. 1 A typical FIB entry used in IP forwarding is (100, if {ingress_port==*, dst_ip { /24}}, set {src_mac 00:00:00:ab:cd:ef, dst_mac aa:bb:cc:00:11:22, egress_port 3}). 57

67 Document: CNET-ICT /D Types of composition The goal of composition is to run multiple modules on the same physical network and incorporate all their network commands into the network state. We distinguish dierent ways of composing applications Single module without composition We start with the simplest case of a single module M 1. All commands are simply forwarded and applied to the network and the resulting network state N is: N = a(n, M 1 (ev)) This is the case in most toy applications and demonstration scenarios of SDN controller frameworks nowadays Multiple modules without composition A typical SDN controller runs multiple SDN modules. Commonly, all outputs of these modules are applied to the network without any explicit form of composition. In this case, each command is applied to the network as it happens, just like in the single module scenario. The resulting network state N (k) for multiple applications looks like this: N = a(n, M 1 (ev)) N = a(n, M 2 (ev)) N (k) = a(n (k 1), M k (ev)) The simplicity of this approach is also its biggest problem. Since every network command is applied when it happens, the resulting network state depends on the order of transactions applied: a(a(n, M 1 (ev)), M 2 (ev) is not necessarily the same as a(a(n, M 2 (ev), M 1 (ev)). The result can even be non-deterministic, i.e. for multiple executions in the same network, dierences in execution speed of modules M 1 and M 2 can result in dierent sequences and thus results even when applying the same modules to the same network state and the same event. Another problem are transient states. In the time after the rst module has answered but not the second, the transient network state N is active. This transient is problematic since it only reects the output of rst module but not the others. These ill-specied, non-deterministic transient networks states are usually undesirable and constitute the main reason to explicitly dene a composition logic that is able to deal with the transient state Multiple modules with harmonizing The output of the multiple modules might contain conicting or overlapping commands. For example, two modules might instruct one switch to deal with the same packet by either forwarding or dropping it, at the same priority. To deal with such conicts, a stateful function h: command command 58

68 Document: CNET-ICT /D 2.7 to modify the commands can be used; the goal is to harmonize the outputs. The network state after executing modules and harmonizing their outputs can be expressed as: N = a(n, h(m 1 (ev))). N (k) = a(n (k 1), h(m k (ev))) When the harmonizing function h is the identity function, we get the same result as in the previous subsection. This harmonizing function can deal with some problems, but intermediate, hard-to-predict network states still exist. If one of the modules a (slowly reacting) rewall its rule get applied, unwanted trac may pass in a transitional state. It is hence not a satisfying solution for all problems Parallel composition To overcome the problem with transient state and varying order of applied results, parallel composition collects all commands for all composed modules and then resolves all conicts betweens these commands, composing the results into a single command to be applied to the network. This is done by a special resolving function r that gets all command outputs and generates a conict-free version that can be applied to the network. r : command... command command The new network state N can then be expressed as: N = a(n, r(m 1 (ev), M 2 (ev),..., M k (ev))) This composition requires all command outputs of an event to be available; it must also be possible to tie a command output of a module to a specic event (necessary when multiple events are passed to a module before commands have been produced, compare challenges of OpenFlow, Section 7.3). A big dierence between the parallel composition and the harmonizing composition is that the parallel composition is reactive, i.e. it depends on the fact that the network commands generated by the modules are a response to a network event. The harmonizing composition works without this assumption and can also be applied to network commands that are sent proactively without an event (C = M j ( )) Serial composition For the serial composition, one module is fed the output of a previous module. The desired network state of the rst module only exists as an input to the second module. To really support this behavior, we need to change the signature of the modules. They must accept a command as an additional input: M : event command command With that, we can dene a new function M 12 : ev (M 1 M 2 )(ev, ) and use that in place of a normal module function (in the harmonizing composition or parallel composition). For example, if only the serially composed function is used, the new network state 59

69 Document: CNET-ICT /D 2.7 will be: N = a(n, M 12 (ev)) = a(n, (M 1 M 2 )(ev, ) = a(n, M 2 (ev, M 1 (ev, ))) Chaining more than two network modules, e.g. M 1 M 2 M 3, is dened by obvious induction. Network programming languages like Pyretic [48] also dene their function signature to have symmetrical input and output: f : policy policy. This is the key insight to give them a well dened semantics! The main distinction of the serial composition to composition strategies discussed so far, is that the last module in the composition chain can provide consistent network commands. It also allows a module to incorporate the decisions of a previous module into its own decision. The downside of the serial composition is, however, that modules need to be explicitly designed and programmed to be used in this way. We will take a look how useful serial composition is with existing modules in the next sub section (7.2.6) Approximate serial composition As most network modules are not designed for serial composition (i.e., they do not accept a command as an input), we dene an approximate way to do serial composition with existing network modules. In this scenario, we need to incorporate as much as possible from the network command into the input event of the following module by a function α: Ñ command event where Ñ is the approximated network state resulting from applying the output of the rst function to the current network state. This approximated state is a representation of the state in the controller; its manipulation does not involve manipulation of actual state in network devices. What can be incorporated into the new event is often very limited as we will see in Section 7.3. The new network state using this function can be expressed as: N = a(n, (M 1 M 2 )(ev)) = a(n, M 2 (α(ñ, M 1(ev)))) Similarly, chaining three modules in an approximate serial composition works as well: N = a(n, (M 1 M 2 M 3 )(ev) = a(n, M 3 (α(ñ, α(ñ, M 1(ev))))) where Ñ is the approximated network state resulting from applying M 1 (ev) to Ñ. Using overlays for approximate serial composition A conceivable variant to implement this approximation of the network state and the function α is to use an overlay of virtual switches to a real network as shown in Figure 7.1: For each physical switch, a number of virtual switches corresponding to the number of modules is emulated. Each module is assigned to one virtual switch. The approximated network states are the state of the virtual switches and the function α would process the packet that the module sends to its virtual switch output ports as event for the next module. 60

70 Document: CNET-ICT /D 2.7 Figure 7.1: Using a virtual overlay network for composition of module A and B. This semantic is not as useful as it seems. In the pure form of this approach the modules are seeing a fragmented network of switches in which every neighboring switch is controlled by a dierent module. Normal routing and Link Layer Discovery Protocol (LLDP) neighbor discovery will fail or produce unreasonable results. Adding special logical to alleviate these problem gives up the advantage of a clear simple semantic Composition and order of middle boxes A common misconception is that the placement of middleboxes (a networking device that transforms, inspects, lters, or otherwise manipulates forwarded trac) always carries over to the composition order. If multiple middleboxes, for example a rewall and a monitoring/ids system should act on all the same trac, these boxes are set up in sequence to pass trac to box after another. Figure 7.2 shows an example of a traditional middle box setup with an IDS, a rewall and a load balancer. Monitoring Server Backend Servers External Network IDS Firewall NAT Figure 7.2: Typical order of IDS and Firewall and NAT load balancer middle boxes in a traditional network. In stark contrast, for a composition the modules would be typically setup in a parallel composition to allow all modules to base their decisions on the original input packets. The merging process of all the outputs will then give an equivalent solution to the middle box solution. Figure 7.3 shows the setup of Figure 7.2 implemented with parallel composition. 7.3 Composition with OpenFlow OpenFlow is the most commonly used protocol used in real-world deployments and a lot of existing application logic is implemented using OpenFlow protocols. This makes OpenFlow desirable as 61