Using and Programming in Maestro

Using and Programming in Maestro Zheng Cai Rice University Maestro is an operating system for orchestrating network control applications. Maestro provides interfaces for implementing modular network control applications to access and modify state of the network. Maestro is a platform for achieving automatic and programmatic network control functions using these modularized applications. In other words, although this document focuses on building an OpenFlow [1] controller using Maestro, Maestro is not only limited to OpenFlow networks. Moreover, Maestro tries to exploit parallelism within a single machine to improve the system s throughput performance. The fundamental feature of an OpenFlow network is that the controller is responsible for every flow s initial establishment by contacting related switches. Thus the performance of the controller could be the bottleneck. In designing Maestro we try to require as little effort from programmers as possible to manage the parallelization. Instead Maestro handles most of the tedious and complicated job of managing work load distribution and worker threads scheduling. Maestro currently provides the control components for realizing either a learning switch network, or a routed network using OpenFlow switches. Some components such as the command line console, etc, are still not full-fledged. We plan to enrich and enhance the functionalities of Maestro in future releases. In this document we are going to show how Maestro s components can be programmed to compose applications and application DAGs (Directed Acyclic Graph, explained in Section 4) to realize OpenFlow control functionalities. 1 View & Event Many network control components use some subset of the network state as input and modify some subset of the network state to realize their control decisions. Thus, Maestro must provide ways to access the network state. Since Maestro manages the network state, providing access is not hard. The key question is at what granularity should such access to network state be supported. The decision should be guided by Maestro s goals, namely to enable modularized network control components to co-exist and interact in a consistent way. At one extreme, we can simply present the whole network state as one monolithic datum to the control components. Such coarse-grained access obviously creates unnecessary state access conflicts between different control components running concurrently and thus is not suitable for concurrent execution. At the other extreme, we can provide a very fine-grained attribute-value lookup and insertion interface for representing network state. Maestro strikes a balance between the two extremes. We observe that network state usually falls into natural subsets, based on what type of state it is and what control objective the state achieves. For example, one common type of state is a routing table, which determines how data packets are forwarded in the network. Routing table state naturally is in a different subset than the information about all connected OpenFlow switches in the network. Generalizing this observation, Maestro provides the view abstraction for grouping related network state into a subset, and for accessing the state in that subset. Each view is a Java class that can be dynamically defined and created in Maestro by programmers. A view can contain any arbitrary data structure to represent a particular subset of network state. For example, we can create a view which is a hash table structure that holds all pair 1

shortest path routing information for the network. The view is the minimal granularity at which Maestro synchronizes control components concurrent execution. We will provide more details in Section 4. Views are highly related to events in Maestro. An event is the basic data unit to exchange information between the underlying network and the control components in Maestro. Events can be generated from the switches in the underlying network. For example, the PacketInEvent corresponds to the struct ofp packet in in the OpenFlow protocol specification. Usually it is a packet that does not match any flow entries in the switch, thus be bounced to the controller for decision. Events can also be generated by the control components, to affect the behavior of certain switches in the network. For example, the FlowModEvent corresponds to the struct ofp flow mod in the OpenFlow protocol specification. It is used to add, update or delete one flow entry in a switch. All events should extern the base abstract class Event, and there is one abstract method that they need to override: int converttobytes(byte[] buf, int index). In this method, the writer of this event is responsible for converting all data in this event class structure into a byte array which conforms to the OpenFlow protocol (that is, in the same format as those struct ofp xxx c/c++ structures). The return value is the total byte length of the generated byte array. Of course, if this event is never going to be sent from Maestro to switches, such as the PacketInEvent, this method does not need to have a real body, just a return 0; will be enough. Please check inline comments in all the event Java files for detailed explanation about each of the events. Because usually an event is also a basic data structure for holding one type of data to be used by control components in Maestro, as a result, views are usually compound data structure that contains all related events. For example, the PacketsInView contains pending PacketInEvent generated by switches, in a linked list. All views should extend from the base abstract class View, and override these following methods: boolean whetherinterested(event e), this method returns whether this view is interested in and capable of processing one particular kind of event. This method is called once during the initialization phase when Maestro creates the event-to-view registration relations, to verify that the view one event is registered with is actually interested in processing the event. Otherwise the registration is invalid, so the initialization will stop and the user will be notified. boolean processevent(event e), this method will get called when an event which has been registered with view is posted. Usually the processing of events involves updating the data structure if necessary. The return value is whether the data structure in this view is updated, which will be used later to decide whether we should start executing a DAG to handle the view update. void commit(driver driver), this defines the actions to take when a view is committed with its changes after a DAG finishes. For views that are going to affect the behavior of the underlying network (in the current version of Maestro, they are FlowConfigView and PacketsOutView ), they need to call the interfaces provided by the driver to send necessary packets to the underlying network. For other views that do not need to send out packets to the underlying network, an empty body for this function is enough. The driver is an indirection layer between Maestro and the underlying network, and more details will be discussed in Section 3. void print(), this method is used to format and print out information contained in this view, when in the Maestro command line console the users type print to print this view. All views for the OpenFlow networks should be put in the views/openflow directory, and all events for the OpenFlow networks should be put in the events/openflow directory. Views and events for other kinds of network should be put in the directory with a different name. For example, 2

if we create views for an OSPF controlled network, the views should be put in the views/ospf directory. This gives Maestro the flexibility to work with different kinds of network by only changing the configuration file, which will be discussed in Section 6. 2 Application Each network control component is represented as one application in Maestro, which is a Java class that contains the code for the control function. All applications must extend the base abstract class App, and interact with Maestro via a simple and straightforward API, by overriding the entry point method ViewsIOBucket process(viewsiobucket input). Maestro executes the code of an application by invoking this method. Both the input argument and the return value are ViewsIOBucket, which is a map data structure that can hold multiple views. The programmer can call addview(int pos, View view) to add a view into a ViewsIOBucket at a certain position pos, or call View getview(int pos) to get a view at a certain position. Such positioning is the way to ensure that an application gets the right types of input views, and generates the right types of output views in the right places. Like the printf function with arbitrary numbers of arguments, in which the types of the arguments should be consistent with these declarations contained in the proceeding string, these positions should also be consistent with the input/output specifications in the DAG configuration which will be discussed in Section 6. An application is not allowed to interact with Maestro or other applications via other interfaces. By doing this, Maestro can enforce explicit control over the interactions among applications, thus it avoids any implicit dependence between applications on network state that is external to Maestro. When accessing the data within one view in an application, the programmer needs to call the acquire() and release() methods of the view if you expect concurrent accesses to this view. 3 Drivers The driver is for implementing the low level functions to synchronize events with the underlying distributed network devices (routers/switches), for a particular type of network. The drivers are usually provided in Maestro to hide the low level details of the underlying networks, thus they serves as a flexible way for Maestro to be able to control different kinds of networks. So for an OpenFlow network, we use the OpenFlow driver. When there are new packets coming from the network devices, the driver needs to translate the received packets into corresponding events, and post these events. The current version of the OpenFlow driver will post SwitchJoinEvent, SwitchLeaveEvent, PacketInEvent and LLDPPacketInEvent. Each driver must override the boolean commitevent(linkedlist<event> events) method, to provide the interfaces for views to commit events. Currently the OpenFlow driver only supports two types of events, PacketOutEvent and FlowModEvent. These two events int converttobytes(byte[] buf, int index) method will be called by the driver to generate packets that conform to the OpenFlow protocol, to send to related network devices. All other events and views are not related to the driver, and are used as intermediate state for applications. 4 DAG The DAG abstraction is Maestro s solution to enable explicit control over interactions among applications. Figure 1 shows examples of application DAGs. An application DAG is a Directed Acyclic Graph that specifies the composition of applications. It defines the execution sequence of applications (black arrows). The execution flow in a DAG may branch, like App 2 and App5 shown in DAG 3 in the figure. All branches may run concurrently. Applications (round-corner boxes) are inter-connected in a DAG together with their input and output views specified. By specifying input and output views, applications can communicate by sharing views with each other. For example, in 3

A A C C Driver App 1 App 2 Commit B D E DAG 1 -- Read: A, B, D Write: C, E A Driver App 1 B B DAG 2 -- Read: A, B F Commit Write: F D Timer App 4 G C H C D H I App 2 App 5 E J Commit DAG 3 -- Read: D, G, I Write: C, H, E, J Figure 1: DAG examples. DAG 1, the output view of App 1 will be the input of App 2, thus there is an explicit communication relation between the two applications. This is the only way two applications are allowed to communicate. A DAG is either triggered by the driver which posting events from the network and updating the registered view, as in DAG 1 and DAG 2, or triggered by a timer, as shown in DAG 3. When a DAG finishes, all the output views generated by applications in this DAG will be committed to update Maestro s global environment (explained in Section 5), and some views will modify the corresponding network state through the driver by sending out packets. Maestro synchronizes the concurrent execution of DAGs at the granularity of view. Maestro knows all the views that one DAG is going to read (union of all input views of applications), and all the views that one DAG is going to write (union of all output views of applications). For example, again in Figure 1, we show that views that each of the three DAGs will read and write. Before starting one DAG to execute, Maestro checks whether this DAG has read/write or write/write conflict with current running DAGs in the system. If such conflict exists, this new DAG is queued, and needs to wait for the running DAG which it has conflict with to finish before it can start execute. For example, DAG 1 and DAG 2 have no conflict, so they can run concurrently, while the executions of DAG 1 and DAG 3 have to be serialized. 5 Environments Several applications in one DAG could use the same input view. As DAG 3 again in Figure 1, both App 4 and the App 2 uses View C as input. If during the execution of this DAG, after App 4 finishes and before App2 starts, View C is changed because there is a link failure in the network, then the two applications will generate inconsistent results because they are using inconsistent View C. For example, packet filters could be installed at a wrong location. We have to make sure that all applications should base their output on consistent input views, even if such input views are outdated. Next time the DAG can run again to accommodate the latest changes, and it is important to ensure all computations are based on the consistent input views. People may ask that why not just stop the currently running DAG and immediately run a new one for the new changes. We argue that network state could change so frequently that not a single DAG can finish execution before the new changes come. In this case if we allow the current DAG to be preempted, then Maestro will never be able to perform any reactions. Instead, if we do not allow preemption, at least Maestro can react to changes as often as possible. To fulfill such requirement, we propose the abstraction of view environments. 4

In Maestro there is the global environment which contains all of the up-to-date views available in the system. These views are accessible by all application DAGs(with the right permission). When one DAG starts execution, Maestro creates a local environment for this DAG by taking a snapshot of the current global environment. The local environment will remain unchanged throughout the execution of this DAG, unless modified by applications within this DAG. This is to ensure that applications within this DAG must base their computation on a consistent set of input views derived from the global environment. When an application is invoked, the input views specified in the application are taken from the local environment of the DAG and passed to the application. After this application instance finishes, its output views are put back to the local environment. By doing this, Maestro realizes the communication among applications within DAG through the local environment. Finally when a DAG finishes, all the output views in the local environment will be committed to update the global environment. 6 Start From Scratch Example: Learning Switches In this section, we are going to show you how to start from scratch to implement a basic controller for realizing learning switches. Hopefully after going through this example, you can be more familiar with all the components and APIs in Maestro, so that you can start building your own functionalities. Remember that, all views mentioned below can be found in the.java files located in the views/openflow directory, all events in events/openflow, and all applications in apps/openflow. Whenever a new switch is connected to Maestro, the driver for OpenFlow provided by Maestro will exchange hello messages with the switch for handshaking. After this is done, the driver will send switch feature request to the switch, and expect the switch to send back switch feature reply. After this reply is received, a SwitchJoinEvent is created and posted, which includes the information contained in the feature reply packet. So to manage such information of all joined switches, we write a view called JoinedSwitchesView, and register the SwitchJoinEvent with this view in the DAG configuration file which will be shown later. So whenever this event is posted, the processevent method of JoinedSwitchesView is called, to put the information about that switch in a HashMap data structure, indexed by that switch s DPID. When a switch is disconnected, the driver will post a SwitchLeaveEvent, and this event is also registered with JoinedSwitchesView, so that the left switch will get removed from the HashMap data structure. Because this controller for learning switches is simplified, just for giving a first example how to build a simple functionality, no other application is involved in handling and managing switches joining or leaving the network. Next, after switches are all discovered by Maestro, we will start letting end hosts talk to each other. Then, Maestro will get packet in from switches because the data packets sent by end hosts do not match any flow entry in the switch. The driver will parse the header of this packet, and post a PacketInEvent. We write a view called PacketsInView which the event is registered with, to batch the events dynamically to enhance the throughput performance (detailed discussion can be found in related technical papers). The overall idea is that, if the packet incoming rate is low, then Maestro will trigger the DAG to process the event as soon as it arrives; if the rate is very high, to achieve better performance, a certain number of events are batched to be processed by one DAG together. We also create another view called LearnedMACsView, which is a two dimensional HashMap that keeps track of all MAC addresses that have been learned on each of the ports of all switches. This is the core data structure of the learning switches functionality, and will be used by the LearningSwitchesApp application. Although it can be maintained locally within the application itself, we generally suggest that all the data structures and state used by applications are better to be built as views, and declared to Maestro so that they can be managed by Maestro. By doing this, if these data structures and state are shared among applications and DAGs, the nice features such as the local and global environments provided by Maestro can take effect, to control the 5

sharing behavior. public class LearningSwitchesApp extends App {... public ViewsIOBucket process(viewsiobucket input) { PacketsInView pis = (PacketsInView)input.getView(0); JoinedSwitchesView sws = (JoinedSwitchesView)input.getView(1); LearnedMACsView macs = (LearnedMACsView)input.getView(2);... for (PacketInEvent pi : work) { macs.acquirewrite(); try { macs.addmaclocation(pi.dpid, pi.flow.dlsrc, pi.inport);... } Integer outport = macs.getmaclocation(pi.dpid, pi.flow.dldst); if (null!= outport) { FlowModEvent fm = null;... } } if (null == outport OFPConstants.OP_UNBUFFERED_BUFFER_ID == pi.bufferid) {... po.actions = new PacketOutEvent.Action[1]; po.actions[0] = PacketOutEvent.makeOutputAction(null==outPort? OFPConstants.OfpPort.OFPP_FLOOD:outPort.intValue()); po.actionslen = po.actions[0].len; pkts.addpacketoutevent(po); } } ViewsIOBucket output = new ViewsIOBucket(); output.addview(0, config); output.addview(1, pkts); output.addview(2, macs); return output; Next we will show you how to build the LearningSwitchesApp application. In this application, it takes three views as input: PacketsInView at position 0 in the input view bucket, JoinedSwitchesView at position 1, and LearnedMACsView at position 2. Because the current version is simplified, the JoinedSwitchesView is not actually used in the code. However, we expect to expand the functionality of this application in the future, and it could potentially use this view. Notice that when configuring this application in the DAG configuration file, the positions of these input views should be configured correspondingly (that is, the first one should be PacketsInView, the second one should be JoinedSwitchesView, and the third one should be LearnedMACsView). We are not going to dive into the details of the code. In general, first we update the LearnedMACsView using the source MAC address, inport and the DPID of the origin switch. Then, we use the LearnedMACsView to find which port should we send this packet to based on its destination MAC address. If we do find the output port, we will create a FlowModEvent to configure a 6

# Which set of applications, events, views and driver should Maestro load? Package openflow # Declare all view instances that will be in the global environment Views JoinedSwitchesView switches PacketsInView packets in LearnedMACsView macs End Views # Describe event-view relation Events SwitchJoinEvent by switches SwitchLeaveEvent by switches PacketInEvent by packets in End Events # Define all DAGs to run in the system DAGs Begin DAG Node a Activation Input Input PacketsInView packets in Node n1 LearningSwitchesApp Input PacketsInView packets in Input LearnedMACsView macs Output FlowConfigView flow config Output LearnedMACsView macs Node t Terminal Output FlowConfigView flow config Output LearnedMACsView macs Edge a->n1 Edge n1->t End DAG End DAGs Figure 2: The learningswitch.dag DAG configuraiton file for OpenFlow flow entry on that switch. If we cannot find the output port, it means that we haven t learned this MAC address yet. In this case we will flood this packet in the spanning tree maintained by the OpenFlow switches. In addition, if this packet is not buffered in the origin switch, we also create a PacketOutEvent to send this packet back to the origin switch (about buffered VS unbuffered packets in the switches, please check the OpenFlow design documents [2] for more information). At last, we put three views in the output view bucket: FlowConfigView which contains all FlowModEvents, PacketsOutView which contains all PacketOutEvents, and the updated LearnedMACsView. Finally, we need to configure the DAG, in the DAG configuration file. The DAG configuration file specifies what kind of network Maestro is supposed to work with (in which directories Maestro should locate the source files of views, events, applications and the driver), what are the views in the global environment, what views should be registered with what events, and finally what DAGs to run when certain views get updated. This is done in the DAG configuration file shown in Figure 2. In this DAG configuration file, first of all we need to specify what package Maestro should use. By package we mean which set of applications, events, views and driver to load. Currently because we use Maestro as an OpenFlow controller, we write Package openflow in the first line. In this case, Maestro will try to load applications in the apps/openflow directory, load views in the views/openflow directory, load events in the events/openflow directory, and load the driver of 7

drivers/openflow.java. We expect that Maestro can be used for controlling other kinds of network, and to do this we can flexibly load another package to support that kind of network. Second, we need to specify all the view instances that are going to be loaded by Maestro and appear in the global environment. All views are dynamically loaded and created by Maestro when initiating, so there is no hard-coded view class initialization. We also need to give each of the views a unique name, to identify each of them. This is done between the Views and End Views section in the DAG configuration file. Third, we need to describe all the event-to-view registration. If one view instance is registered with one kind of event, whenever a component (usually the driver) post this kind of event, the registered view s processevent will be called to process this event. Upon return, the view will tell Maestro whether this view has been updated to let Maestro decide whether a DAG should be triggered to handle the view update. During this registration phase, Maestro will call each view s whetherinterested(event e) method to guard against any configuration error. Furthermore, one view can be registered with multiple events if it needs to, but one type of event can only be registered with one view instance name. All of these are done in the section between Events and End Events. In the last section, the DAG section, we define all DAGs to run in Maestro, and we specify what view changes will trigger which DAG to run. When defining one DAG, first we create the activation node which specifies the triggering view update. So when that view gets updated, Maestro will start this DAG to handle the change. Then, we create application nodes that are going to be executed to realize the functionality of this DAG. In this example for learning switches, there is only one DAG configured. The PacketsInView is the activation view, and the LearningSwitchesApp s input and output views are configured at the right positions. After that we create the Terminal node which specifies which of these views in the local environment need to be committed to the global environment, to update the underlying network (switches) through the driver if necessary. Then, we define the edges to connect all nodes, to specify the execution order of these nodes. After creating all necessary events, views, applications, and writing the DAG configuration file, we now have a working learning switches controller for OpenFlow! Of course in reality you can always take advantages of existing events views and applications so you do not have to write everything from scratch, to implement your own functionality. We hope this section can prepare you with the basic knowledge about Maestro. In the next section, we are going to discuss how to build the more complicated routing functionality, together with some additional features provided by Maestro to realize parallelization. 7 The Routing Functionality, and Additional Features The routing functionality involves more events, more views, more applications and more DAGs. Because the configuration is very long, it is divided into four parts as shown in Figure 3, Figure 4, Figure 5 and Figure 6. In addition to the views JoinedSwitchesView and PacketsInView, there are several other views in the global environment. ConnectivityLocalView contains all links topology information about the network. RoutingIntraView contains all pair shortest path routing information for all switches in the network. ReceivedLLDPPacketsView contains all pending LLDP packets received by Maestro, used by the DiscoverApp to discover links in the network, to update the ConnectivityLocalView accordingly. RegisteredHostsView remembers which host (based on its MAC address) is registered at which location (switch-port combination) in the network, and it is used to find the destination location for a destination MAC address. You may find out that we have a keyword Concurrent following PacketsInView packets in. This tells Maestro that this view, which we use as the input queue for incoming packet in events, should be parallelized. Then upon initialization Maestro will create multiple instances of this view, depending on the numthreads configured in the parameters configuration file. This number basically decides how many parts should we divide the incoming packet in events into, and how many 8

# Which set of applications, events, views and driver should Maestro load? Package openflow # Declare all view instances that will be in the global environment Views JoinedSwitchesView switches ConnectivityLocalView conn local RoutingIntraView routing intra PacketsInView packets in Concurrent PacketsOutView packets out ReceivedLLDPPacketsView lldps RegisteredHostsView hosts End Views # Describe event-view relation Events SwitchJoinEvent by switches SwitchLeaveEvent by switches PacketInEvent by packets in LLDPPacketInEvent by lldps End Events Figure 3: The routing.dag DAG configuraiton file for OpenFlow, part 1 concurrent work threads are going to process these parts. This is all done automatically by Maestro, so that programmers do not need to manage the concurrent queues and threads themselves. You may also find out that there is one additional kind of event, LLDPPacketInEvent, which contains one received LLDP packet from the network. It is registered with the ReceivedLLDPPacketsView. In the DAG section. First we create the DAG which contains the DiscoveryApp and the IntraRoutingApp. The DiscoveryApp basically processes received LLDP packets to update the topology view ConnectivityLocalView. Then the IntraRoutingApp computes the all pair shortest path routing information view RoutingIntraView Second, we create the DAG in which the PacketsInView first will be processed by the LocationManagementApp to update the location (on which port of which switch it is attached to) of the source end host, and find out the location of the destination end host. The FlowsInView will be generated to include such location information. Then the FlowsInView will be processed by the RouteFlowApp to setup a path for the flows, based on the RoutingIntraView which contains all pair shortest path routing table for all switches in the network. You may also notice that we flag this DAG also with Concurrent. This tells Maestro that this DAG should be duplicated to create multiple instances to run concurrently, to parallelize the computation to improve the throughput. Maestro will decide how many copies to create dynamically, depending on numthreads. Because we declare earlier that PacketsInView packets in Concurrent, all appearances of packets in Concurrent in this DAG will be replaced with its corresponding specific instance name accordingly by Maestro, during initialization. All these steps are automatically done by Maestro. At last, we create another DAG which contains the ProbeApp that periodically sends out LLDP packets to all ports of all switches. Notice that this DAG is not triggered by a view change, but instead by a Timer of 2000ms. These LLDP packets will be received by switches on the other side of the links, and bounced back to Maestro, to enable the first DAG to discover the topology of the network. 8 Possible Example for Extending the Functionality In this section, let s assume that we would like to introduce one additional functionality to the routing.dag. This functionality is for security, and what it does is that it maintains a set of filter rules based on the flow information of the packets. For example, these rules can tell whether a flow should 9

# Define all DAGs to run in the system DAGs Begin DAG Node a Activation Input ReceivedLLDPPacketsView lldps Node n1 DiscoveryApp Input ReceivedLLDPPacketsView lldps Input ConnectivityLocalView conn local Output JoinedSwitchesView switches Output ConnectivityLocalView conn local Node n2 IntraRoutingApp Input ConnectivityLocalView conn local Input RoutingIntraView routing intra Output RoutingIntraView routing intra Node t Terminal Output JoinedSwitchesView switches Output ConnectivityLocalView conn local Output RoutingIntraView routing intra Edge a->n1 Edge n1->n2 Edge n2->t End DAG Figure 4: The routing.dag DAG configuraiton file for OpenFlow, part 2 be allowed in the network, based on its source/destination addresses (MAC, IP) and other information (ports, protocols type, etc). Then, whenever a new packet in event comes in, this application will first check the rules to tell whether it should be allowed. Then it is the LocationManagementApp and the RouteFlowApp as in the routing package. To realize such functionality, first we need to create a view in views/openflow for holding these filter rules, say FilterRulesView. We can use whatever data structures that are suitable and efficient in implementing such a feature. For example, regular hash map will work, but maybe others such as HyperCuts [3] decision tree could be more efficient. Then this view could probably be loaded from a rule file during initialization of Maestro. The CmdConsole could also be extended in a way that we can dynamically add more rules to this view. However, in the current version of Maestro we have not supported this yet. We plan to support this feature in the future. Then, we create an application in apps/openflow for filtering matched packets, say FilteringApp. In this application, we override the public ViewsIOBucket process(viewsiobucket input) method of its parent class App. In this method, first we get two input views from the input view bucket, FilterRulesView and PacketsInView. Then we check each PacketInEvent in PacketsInView against all the filter rules, to see whether they should be allowed. If not, we will remove it from PacketsInView. Finally, we will put PacketsInView in the output bucket, and return. At last, we need to modify the routing.dag configuration file. First we need to add a FilterRulesView in the view declaration section, and give this view a name, say FilterRulesView filters. Then, in the DAGs section, we modify the second DAG as shown in Figure 7: 10

Begin DAG Concurrent Node a Activation Input PacketsInView packets in Node n1 LocationManagementApp Input PacketsInView packets in Input RegisteredHostsView hosts Output FlowsInView flows in Output RegisteredHostsView hosts Node n2 RouteFlowApp Input RoutingIntraView routing intra Input FlowsInView flows in Output FlowConfigView flow config Node t Terminal Output FlowConfigView flow config Output RegisteredHostsView hosts Edge a->n1 Edge n1->n2 Edge n2->t End DAG Figure 5: The routing.dag DAG configuraiton file for OpenFlow, part 3 9 Parameter Configuration File In addition to the DAG configuration file, there is another configuration file which we call the parameter configuration file. In this file, we need to configure several performance/functionality parameters for Maestro. verbose: this parameter controls whether the system should print out extra information to stderr for debugging purpose. If the value is set to 1, then all calls to Utilities.printlnDebug() and Utilities.printDebug() will be enabled. If 0, these calls will take no effect. batchinputnum: this parameter controls how many pending PacketInEvents should the PacketsInView batch before the view triggers a new DAG to process this batch of events, by returning true in its processevent method. However, this batching behavior is also dynamically adjusted according to the current load of the system, so when the load is light, the DAG will get triggered as soon as a new event is posted. batchoutput: this parameter controls whether outbound packets for the same destination switch are batched and sent out all together by calling send only once. Set the value to 1 to enable output batching. numthreads: this parameter controls how many parts should we divide the incoming PacketInEvents into, and how many concurrent work threads are spawned to process these parts. threadbind: this parameter controls whether the whole process of handling one PacketInEvent should be bound to one particular worker thread. This feature is for eliminating the cross-core overhead if we allow such process to be migrated to different cores before it finishes. Set the value to 1 to enable this thread binding feature. 11

Begin DAG Node a Activation Timer 2000 Node n1 ProbeApp Node t Terminal Edge a->n1 Edge n1->t End DAG End DAGs Figure 6: The routing.dag DAG configuraiton file for OpenFlow, part 4 countdone: this parameter controls the granularity at which we measure the throughput of the system. For example, if the value is 500,000, then we will measure the aggregated average throughput of Maestro after it has processed 500,000 PacketInEvents. This parameter is for benchmarking the system, so for normal usage it can be ignored. queueupperbound: this parameter controls the maximum number of pending raw packets allowed in the driver s buffer. If this value is set too large, then Maestro is going to use more memory, and the Java memory system s performance will become worse. If this value is set too small, then it is going to affect TCP s performance and prevent the system from achieving its best throughput. Currently we suggest using the default value. outputlog: this parameter is a string value for the file name of the log file. Programmers can choose to write to Utilities.log to keep logging information in this log file. port: this parameter specifies what is the port number that the driver should be listening on. References [1] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and J. Turner, Openflow: enabling innovation in campus networks, ACM Computer Communication Review, vol. 38, pp. 69 74, April 2009. [2] OpenFlow Switch Specification, Version 1.0.0. http://www.openflowswitch.org/documents/openflow-spec-v1.0.0.pdf. [3] S. Singh, F. Baboescu, G. Varghese, and J. Wang, Packet classification using multidimensional cutting, in Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications (SIGCOMM), 2003. 12

Begin DAG Concurrent Node a Activation Input PacketsInView packets in Node n1 FilteringApp Input FilterRulesView filters Input PacketsInView packets in Output PacketsInView packets in Node n2 LocationManagementApp Input PacketsInView packets in Input RegisteredHostsView hosts Output FlowsInView flows in Output RegisteredHostsView hosts Node n3 RouteFlowApp Input RoutingIntraView routing intra Input FlowsInView flows in Output FlowConfigView flow config Node t Terminal Output FlowConfigView flow config Output RegisteredHostsView hosts Edge a->n1 Edge n1->n2 Edge n2->n3 Edge n3->t End DAG End DAGs Figure 7: The modified routing.dag DAG configuraiton file for security functionality 13