Aruba VAN SDN Controller 2.8 Administrator Guide

Transcription

1 Aruba VAN SDN Controller 2.8 Administrator Guide Part Number: Published: March 2017 Edition: 1

2 2017 Hewlett Packard Enterprise Development LP Notices The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein. Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying. Consistent with FAR and , Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no control over and is not responsible for information outside the Hewlett Packard Enterprise website. Aruba VAN SDN Controller license text The Aruba VAN SDN Controller license text is in /opt/sdn/legal/eula.txt. The Aruba VAN SDN Controller incorporates materials from several Open Source software projects. Therefore, the use of these materials by the Aruba VAN SDN Controller is governed by different Open Source licenses. Refer to /opt/sdn/legal/hp-sdn- CONTROLLER-OPENSOURCE-LIST.pdf for a complete list of the materials used. Open Source Software For information on licenses for the open source software used by the Aruba VAN SDN Controller, see the Aruba VAN SDN Controller Open Source and Third-Party Software License Agreements. This product includes code licensed under the GNU General Public License, the GNU Lesser General Public License, and/or certain other open source licenses. A complete machine-readable copy of the source code corresponding to such code is available upon request. This offer is valid to anyone in receipt of this information and shall expire three years following the date of the final distribution of this product version by Hewlett Packard Enterprise. For information about acquiring the open source code for the Aruba VAN SDN Controller, contact Hewlett Packard Enterprise Support, listing the product name and version information for which the source code is being requested. See Support and other resources for information about contacting Support. Because such information can become outdated quickly, Hewlett Packard Enterprise does not publish mailing addresses and telephone numbers for open source queries. Available source code distribution methods include network transmission of the source code and sending the source code on physical media to a mailing address. Physical media distribution might require a fee to cover the media and mailing costs. The Aruba VAN SDN Controller includes both proprietary software that is closed source in addition to the open source software listed in the Aruba VAN SDN Controller Open Source and Third-Party Software License Agreements. In response to queries to Hewlett Packard Enterprise for source code on the Aruba VAN SDN Controller, Hewlett Packard Enterprise distributes the source code for open source software only. Hewlett Packard Enterprise does not distribute source code for closed source software. Acknowledgments Intel, Itanium, Pentium, Intel Inside, and the Intel Inside logo are trademarks of Intel Corporation in the United States and other countries.

3 Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Adobe and Acrobat are trademarks of Adobe Systems Incorporated. Java and Oracle are registered trademarks of Oracle and/or its affiliates. UNIX is a registered trademark of The Open Group.

4 Contents Chapter 1 Introduction...8 About the Aruba VAN SDN Controller...8 The Aruba SDN ecosystem... 8 SDN Controller applications and the App Store...10 Hewlett Packard Enterprise SDN information library Switch and OpenFlow requirements Chapter 2 Understanding the controller architecture...12 SDN controller architecture...12 List of controller embedded applications...13 OpenFlow Link Discovery OpenFlow Node Discovery Path diagnostics...15 Topology Manager Topology Viewer...15 Path Daemon Chapter 3 Using the SDN controller UI...18 Starting the SDN controller console UI Changing the language displayed in the user interface The SDN controller user interface...20 Changing column widths in the UI SDN User window...21 Navigation menu Alerts...25 Applications...30 Understanding application states and OSGi artifacts Configuration components Audit log Licenses...48 Team Support logs...50 Packet listeners...53 OpenFlow Monitor...54 OpenFlow topology OpenFlow Trace log...67 OpenFlow Classes...74 Chapter 4 Hybrid mode for controlling packet forwarding Overview Learning more about hybrid mode Viewing and changing the hybrid mode configuration Coordinating controller hybrid mode and OpenFlow switch settings Controller packet-forwarding when hybrid mode is disabled Controller packet forwarding when hybrid mode is enabled Contents 4

5 Chapter 5 License Registration and Activation...85 Overview of the license registration and activation process License types, usage, and expiration...85 Preparing for license registration Registering and activating a license Registering your license and obtaining a license key Activating a license on the controller Managing licenses Using evaluation licenses Chapter 6 Configuring for High Availability High Availability best practices About teaming for High Availability Requirements for controller teams Requirements for configuration, licensing, or application changes to controller teams Team status Controller status Manually synchronizing Cassandra database notes using the nodetool repair utility Configuring controllers to use the same local NTP servers Viewing your team configuration using the UI Methods for configuring HA teaming Defining inputs for teaming in a configuration file Using a Python script from a controller to configure a team Chapter 7 Security SDN Controller authentication Changing the default controller keystore and truststore to use CA signed certificates SDN Controller keystore and truststore locations and passwords Encryption Built-in OpenFlow controller REST authentication Controller code verification Revoking Trust SDN administrative REST API Virgo admin UI access via localhost only Virgo console access disabled by default JMX console enabled for local access only Creating the Cassandra keystore and truststore Cassandra keystore and truststore locations and passwords Security procedure Security best practices Chapter 8 Configuring OpenFlow instances Configuring OpenFlow Instances with Multiple VLANs Configuring OpenFlow Instances with Single VLAN Identifier Configuring OpenFlow instances to enable MAC group matching Chapter 9 Backing up and restoring Backing up and restoring Best Practices Backing up a controller Contents 5

6 Restoring a controller from a backup Distributed (team) backing up and restoring Backing up and restoring the Keystone configuration and database Chapter 10 Metrics Viewing metric data Viewing all controller JVM metrics Viewing current metric data using a JMX client Generating a controller support report Chapter 11 Troubleshooting REST API request returns HTTP code Controller not listening on port TCP/ Packets not received at the end point Session expired message in the UI Error running the config_sdn.py script with date/time/ntp option Licensing Applications that use the Cassandra database are experiencing failures Controller support log fills disk space, contains multiple Too many open files messages Application management errors OpenFlow errors Troubleshooting teamed environments Chapter 12 Websites Chapter 13 Support and other resources Accessing Hewlett Packard Enterprise Support Accessing updates Customer self repair Remote support Warranty information Regulatory information Documentation feedback curl commands About the curl commands Getting an authorization token using a curl command Export audit log data as a CSV file using curl commands Licensing actions using curl commands Application manager actions using curl commands Viewing metric data using curl commands Managing SNMP keys Managing NETCONF keys Team configuration using curl commands Scripts Restoring a controller Backing up a controller team Restoring a controller team Aruba VAN SDN Controller 2.8 Administrator Guide

7 Using an external policy manager Performance testing Examples of Metrics Contents 7

8 Chapter 1 Introduction This document describes the configuration and management of the Aruba VAN SDN Controller in standalone and team modes. About the Aruba VAN SDN Controller The Aruba VAN SDN Controller provides a unified control point in an OpenFlow-enabled network, simplifying management, provisioning, and orchestration and enabling delivery of a new generation of application-based network services. In the Aruba Software Defined Networking (SDN) architecture, the control and data planes of the network are decoupled from each other, centralizing network intelligence and abstracting the underlying network infrastructure from applications. Controller software manages forwarding behavior for physical and virtual switches under its control via the industry-standard OpenFlow protocol. Network ports, links, and topologies are all directly visible, enabling centralized policy administration and more effective path selection based on a dynamic, global view of the network. This dramatically simplifies the orchestration of multi-tenant environments and the enforcement of network policy for both mobile clients and servers. The Aruba VAN SDN Controller is designed to operate in a variety of computing environments, including campus, data center, service provider, private cloud, and public cloud. The Aruba VAN SDN Controller features: An enterprise-class platform for the delivery of a broad range of network innovations An extensible, scalable, and resilient controller architecture Compliance with OpenFlow 1.0 and 1.3 protocols Support for Hewlett Packard Enterprise and H3C OpenFlow-enabled switches Secure authentication using a local or remote Keystone server Controller teaming for distributed platform High Availability (HA) Embedded applications that provide common network services Open APIs enable SDN application developers to deliver innovative solutions that dynamically link business requirements to network infrastructure using either custom Java programs or general-purpose RESTful control interfaces, including functions to extend the controller REST API and UI. Integration with HPE Intelligent Management Center (IMC). HPE IMC provides full controller application life cycle management and monitoring, enhanced reporting and SDN network visualization. The Aruba SDN ecosystem SDN architecture separates the network control plane from the forwarding hardware on network devices. Control can then be centralized, while forwarding remains distributed. SDN is based on OpenFlow, which is a standardsbased protocol allowing for a centralized-control plane in a separate device (the controller). OpenFlow is managed by the Open Networking Foundation (ONF). By separating the control plane from the forwarding plane, SDN makes it possible for the network status and capabilities to be exposed directly to the business service layer, so that business systems can request services from the network directly. SDN applications thus provide higher level application direction to the SDN controller. And freed from the control function, the forwarding plane can then provide optimized packet processing at very high speeds. The Aruba VAN SDN Controller is the central building block of the Aruba SDN ecosystem and creates a platform for application development. 8 Aruba VAN SDN Controller 2.8 Administrator Guide

9 The Aruba SDN ecosystem includes the following: Infrastructure. The infrastructure layer is made up of network devices, typically but not exclusively routers and switches. The devices are OpenFlow-enabled. An OpenFlow switch consists of one or more flow tables and a group table, which perform packet lookups and forwarding and provide an OpenFlow channel to the Aruba VAN SDN Controller. The switch communicates with the controller and the controller manages the switch via the OpenFlow protocol. Hewlett Packard Enterprise has more than 50 switch models that are OpenFlow-enabled. Control. Aruba VAN SDN Controller provides centralized control and automation for an SDN network. The controller controls policy and forwarding decisions, which are communicated to the OpenFlow-enabled switches in the data center or campus network. A variety of Hewlett Packard Enterprise and third-party SDN applications can leverage the controller to automatically deliver the necessary business and network service levels. Applications. Hewlett Packard Enterprise and third-party SDN applications provide a true end-to-end service level for network performance, quality of service, and security, which can be tuned to an applications needs. For example, SDN applications can inspect flows, or perform other network control functions via the Aruba VAN SDN Controller. Aruba SDN applications include: Aruba Network Protector SDN Application, Aruba Network Optimizer SDN Application and Aruba Network Visualizer SDN Application.The extensibility and open APIs of the Aruba VAN SDN Controller allows new applications to be created that make requests of the underlying network, without the need to physically uproot or re-configure the underlying infrastructure. Northbound APIs utilize the REST architecture and provide easy access to applications that are integrated directly in the controller or off the controller. Native APIs, provided in Java, deliver support to Network Control applications that are integrated directly in the controller. Management. The HPE Intelligent Management Center (IMC) VAN SDN Manager software integrates with HPE IMC to provide administrators with a single interface to manage both the traditional network and the SDN. The IMC VAN SDN Manager Software monitors and manages all three layers of the SDN architecture: infrastructure, control, and application, providing comprehensive management including fault, configuration, accounting, monitoring, and security for the controller and OpenFlow infrastructure. IMC provides full controller application life cycle management and monitoring, reporting of network service status and OpenFlow-related information, and SDN network visualization.in addition, the Aruba VAN SDN Controller provides REST and Java APIs that enable applications to interact with the controller to receive alerts, to get information about the network, devices, and controller, and to perform various network management tasks. Chapter 1 Introduction 9

10 SDN Controller applications and the App Store The Aruba VAN SDN Controller includes a default set of core network service applications that are installed as modules with the controller. These embedded applications provide services such as authentication, data persistence, logging and alerts. For details, see Understanding the controller architecture on page 12. The Aruba VAN SDN Controller also provides a platform for developing and deploying SDN applications. Several applications have been developed by Aruba including Aruba Network Protector SDN Application, Aruba Network Optimizer SDN Application, and Aruba Network Visualizer SDN Application. There are also SDN applications developed by third-party partners. In addition, you can develop your own SDN applications. Aruba s SDN applications as well as third-party SDN applications are available through the Hewlett Packard Enterprise SDN App Store. Access the App Store at The Aruba VAN SDN Controller includes an SDK providing the tools needed to develop applications for the controller. The SDK includes documentation for both the Java and REST APIs as well as all of the jar files necessary during compilation. A sample application is also included along with API specifications. For details on how to develop applications for the controller, see the Aruba VAN SDN Controller Programming Guide. Hewlett Packard Enterprise offers an SDN developer community, as well as forums, events, and other services, to help developers and partners build and sell SDN applications. Hewlett Packard Enterprise SDN information library The following information is provided for the Aruba VAN SDN Controller: Aruba VAN SDN Controller Release Notes Aruba VAN SDN Controller Installation Guide Aruba VAN SDN Controller Administrator Guide Aruba VAN SDN Controller and Applications Support Matrix Aruba VAN SDN Controller Programming Guide Aruba VAN SDN Controller REST API Reference Aruba VAN SDN Controller Troubleshooting Guide Aruba VAN SDN Controller Open Source and Third-Party Software License Agreements The most recent versions of these documents are in the Hewlett Packard Enterprise SDN information library at the following website: Switch and OpenFlow requirements Switch and OpenFlow requirements for use with the controller: The controller must be connected to a network that includes one or more switches configured to run OpenFlow. Plan and implement the switch OpenFlow configurations before connecting the controller to the network. OpenFlow switches in the network must be configured to allow control by the Aruba VAN SDN Controller. In a controller domain, including a switch that does not support OpenFlow or allow control by another Aruba VAN SDN Controller creates separate clusters of OpenFlow networks. Create a separate VLAN for an OpenFlow control plane. Running the OpenFlow control mode on a specified switch VLAN disrupts the traffic on that VLAN until the controller configures the required flow rules in the switch using the OpenFlow controller API. For information on configuring OpenFlow, see the latest OpenFlow Administration Guide for your switch. Do not connect OpenFlow switches in a controller domain in a loop topology with switches outside the domain. 10 Aruba VAN SDN Controller 2.8 Administrator Guide

11 Allowing such connections can create broadcast loops inside the OpenFlow network. For more on packet-forwarding decisions, see Hybrid mode for controlling packet forwarding on page 78. Set the controller to hybrid mode true (the default) in order to support IPv6 traffic running in the data plane of an OpenFlow network. In this state the controller is not aware of the IPv6 traffic. However, with hybrid mode set to false (all packets sent to the controller), the controller drops IPv6 packets, and they do not reach their destinations. For information about supported network switches, OpenFlow versions, and switch configuration requirements, see the Aruba VAN SDN Controller and Applications Support Matrix. Chapter 1 Introduction 11

12 Chapter 2 Understanding the controller architecture SDN controller architecture The Aruba VAN SDN Controller software is built upon a Linux OS, Java 1.8, and OSGI (Virgo stack and Equinox framework) and uses an Apache Cassandra distributed post-relational database. Keystone is an external service that provides authentication and high level authorization services. It supports token-based authentication REST API and GUI framework are used by SDN application developers for building applications (RESTful web services and web based UIs). Figure 1: Aruba VAN SDN Controller software stack The following gives a short description of the controller components: The controller Application Manager enables installing, upgrading, enabling (starting), disabling (stopping), and uninstalling SDN applications on the controller. The Audit Log records events related to activities, operations, and configuration changes initiated by an authorized user. The Audit Log is managed by the controller Audit Log service. The Alert Log records information about events that affect controller operation, and in some cases indicate that some action is needed to correct a condition. Alerts are managed by the controller Alert service. Client Mapper Service combines information known about a network client by the controller, such as host IP address, host MAC addresses, and the connected datapath and port, with information about the network client known by an outside policy manager, such as the Aruba ClearPass policy manager, to provide information about network clients, including user information, device information, and location information. This information is available via the REST API only. The controller uses the embedded applications Topology Manager and Topology Viewer to collect and display information about the OpenFlow network. The controller provides a framework to back up and restore controller and application state in a backup file. The backup file can be copied and stored for later use. The stored backup file can be uploaded to the controller. The Distributed Coordination Framework is one of the high-availability features of the controller. It provides the infrastructure for controller-to-controller communication and coordination of state information for controllers in a controller team. 12 Aruba VAN SDN Controller 2.8 Administrator Guide

13 The controller can be configured in a team. The teaming services of the controller keep the runtime state of each controller in the team (active, unreachable, or suspended) up to date and is used by other parts of the controller for functions related to high-availability. The Device Drivers model the capabilities of the devices and provide APIs for interacting with different device types. The controller uses the embedded applications OpenFlow Link Discovery and OpenFlow Node Discovery to discover information about the OpenFlow network. The OpenFlow controller (also called the core controller) handles the connections from OpenFlow devices and provides the means for upper layers of software to interact with those devices. The Aruba VAN SDN Controller includes a default set of core network service applications that are installed with the controller (see List of controller embedded applications. List of controller embedded applications The Aruba VAN SDN Controller includes a default set of core network service applications that are installed as modules on the controller. The following applications are embedded in the controller and are installed when you install the controller: OpenFlow Link Discovery OpenFlow Node Discovery Path Daemon Path Diagnostics Topology Manager Topology Viewer OpenFlow Link Discovery The OpenFlow Link Discovery application is the default OpenFlow link supplier application that is installed with the controller. This application implements the com.hp.sdn.supplier.linksuppliersbroker interface and uses LinkSupplierService and LinkService APIs to create and maintain link information for OpenFlow datapaths that register with the controller. The OpenFlow Link Discovery application does the following: Discovers the following types of links: direct links multi-hop links Listens to device and interface events and registers with the ControllerService API to send OpenFlow packets to datapaths. If the OpenFlowLinkDiscoveryComponent configuration has age.multihop.links=true, the OpenFlow Link Discovery application periodically injects link-discovery packets into the controlled network to refresh the multihop links. Any multihop links that are not refreshed at the interval configured for the multihop.poll.interval key are considered to be invalid and are removed from the link table. Additionally, network events such as a port going down or a device status change causes relevant links to be removed from the link table, and causes discovery packets to be sent to all datapaths that are in a ready state. How the OpenFlow Link Discovery application determines the link type 1. The application injects two controller-generated link-discovery packets to each port in an OpenFlow instance. These packets have the same Ethernet type (0x8999), but are sent to different destination MAC addresses: The link-local MAC address to discover direct links:01:08:c2:00:00:0e The multicast MAC address to discover multihop links:01:1b:78:e9:7b:cd 2. The application evaluates the hybrid mode configuration of the controller: Chapter 2 Understanding the controller architecture 13

14 If the ControllerManager configuration has hybrid.mode=true, the application installs a flow rule on every OpenFlow device to steal controller-generated link discovery packets. Packets that match this flow rule are forwarded to the controller from the OpenFlow instance and port on which they were received. If the ControllerManager configuration has hybrid.mode=false, all packets are forwarded to the controller by default. Therefore the application does not install flow rules on the OpenFlow devices. 3. The application listens for link-discovery packets.the application determines the link type by examining the link-discovery packets sent to the controller:the link type is derived from the destination MAC address of the packet (direct or multi-hop). If a link is direct, it will be discovered as both direct and multi-hop from the reporting OpenFlow instance, but the type direct has precedence over the type multi-hop, so the link is recorded as direct. Characteristics of a controller-generated link-discovery packet A controller-generated link-discovery packet: Uses a non-standard protocol, BDDP, which uses a payload format similar to LLDP. Is sent to either a link-local MAC address (to discover direct links) or a multicast MAC address (to discover multihop links). Contains the source device and port that introduced the packet to the controlled network. Has the source device ID embedded in its payload. The destination device can be derived from the PACKET_IN message. This design enables the OpenFlow Link Discovery application to populate the link table with information it learns from received packets. Is used only for discovering links, so it is received from the device by the controller via a PACKET_IN message, but is not returned to the device via a PACKET_OUT message. OpenFlow Node Discovery The OpenFlow Node Discovery application is the default OpenFlow node supplier application that is installed with the controller. This application implements the com.hp.sdn.supplier.nodesuppliersbroker interface and uses NodeSupplierService and NodeService APIs to create and maintain node information for OpenFlow datapaths that register with the controller. The OpenFlow Node Discovery application uses the following process to create and maintain node information in the node table: 1. The application evaluates the hybrid mode configuration of the controller: If the ControllerManager configuration has hybrid.mode=true the OpenFlow Node Discovery application installs flow rules that instruct OpenFlow datapaths to copy ARP packets or DHCP packets to the controller.by default in hybrid mode, IP packets are not sent to controller. The OpenFlow Node Discovery application does not install flow rules that instruct OpenFlow datapaths to send IP packets because doing so would drastically reduce network performance by overwhelming the control plane. However, the OpenFlow Node Discovery application can listen for IP packets that other applications have instructed OpenFlow datapaths to send to the controller. If the ControllerManager configuration has hybrid.mode=false, all packets are forwarded to the controller by default. Therefore the application does not install flow rules on the OpenFlow devices. 2. The application ignores PACKET_IN messages from ports that identified by the Topology Manager as being part of the infrastructure, and listens for PACKET_IN messages that contain the ARP or DHCP protocols. If the OfIpDiscoveryComponent configuration has learn.ip=true, the application also listens for PACKET_IN messages that contain the IP protocol.because these PACKET_IN messages are copies of packets that have already been forwarded to the controller, the controller does not send corresponding PACKET_OUT messages for these PACKET_IN messages. 14 Aruba VAN SDN Controller 2.8 Administrator Guide

15 3. If learn.ip=true, based upon the information supplied by these copied ARP, DHCP, and IP packets, the OpenFlow Node Discovery application registers as a node supplier and supplies updates to the node table. 4. The controller administrator can configure the timeout value for nodes discovered by each protocol by setting the value of the age key of the configurable component for that protocol. When the timeout value is reached, OpenFlow Node Discovery application removes the node from the node table. Path diagnostics The Path Diagnostics application determines and verifies the path taken by trace packets from a source host to a destination host. The application finds an existing flow that matches the description of the trace packet, clones it with higher priority, and adds an action to instruct the selected datapath to send this packet back to the controller for status tally. The Path Diagnostics application is available when the Controller Manager configuration has hybrid.mode=false only. Topology Manager The Topology Manager provides topology information about the control domain and facilitates shortest path traversals through the control domain. The Topology Manager: Indicates whether a connection point is part of Infrastructure or is connected to an end host Indicates whether ingress broadcast traffic can be allowed through a specified connection point Determines if a path exists between two datapaths Identifies the shortest path between two datapaths based on hop count or link edge weight Provides enumeration of the grouping of datapaths into clusters of strongly connected nodes For a given datapath, provides information about the cluster to which the datapath belongs Provides information about number of datapaths, number of links, and number of clusters in the current topology Creates the clusters and broadcast tree to avoid loops and broadcast storms. Provides notifications to subscribed applications about changes in its broadcast tree and cluster. Applications that subscribe to these notifications can use the information to respond to changes in topology. Topology Viewer The Topology Viewer application creates and updates a network graph for visualizing the network the controller discovers. This graph is displayed on the OpenFlow Topology screen. The Topology Viewer uses the services of the Topology Manager and the Link Manager. Path Daemon Path Daemon is a path-paving application that listens for all ARP and IPv4 PACKET_IN messages flow misses that arrive at the controller and attempts to install or modify flows on datapaths along the forwarding path to ensure that such packets get forwarded at line-rate. Path Daemon operates only when the entire network is controlled by the controller (ControllerManager configuration has hybrid.mode=false and there are no uncontrolled devices). By default, Path Daemon is responsible for Layer-2 forwarding only. Each PACKET_IN message processed by Path Daemon results in a PACKET_OUT message and possibly a flowmod message getting pushed to one or more controlled devices. By default, the Path Daemon application installs flows that attempt to forward traffic using the following: Chapter 2 Understanding the controller architecture 15

16 MAC address and incoming port for ARP PACKET_IN messages IPv4 address and incoming port for IPv4 PACKET_IN messages Specifically, the flows will match all packets that enter a specific switch on a specific port and they will match only packets with the source MAC or IPv4 address and destination MAC or IPv4 address from the PACKET_IN message. Any packets that match the flow will be forwarded by the datapath to the most optimal destination port determined by Path Daemon for the packet to reach its intended destination. Operational notes The Path Daemon: Does not handle multicast or broadcast traffic Does not configure the reverse path along with the forward path Drops packets from sources that the controller has not learned Floods packets when their destinations are not known Does not support fast-failover Performance is topology-dependent and can degrade when the network contains more than approximately 200 nodes. Match fields used by Path Daemon The Path Daemon uses the following match fields for flow-mod message. These match fields have been chosen so that the flow entries are added to hardware tables in ArubaOS-based, ProVision-based, and Comware-based switches. Ether type: OFPXMT_OFB_ETH_TYPE Source MAC or IP address: OFPXMT_OFB_ETH_SRC or OFPXMT_OFB_IPV4_SRC Destination MAC or IP address: OFPXMT_OFB_ETH_DST or OFPXMT_OFB_IPV4_DST Input port: OFPXMT_OFB_IN_PORT Detailed operation The Path Daemon does the following: 1. Registers with the controller as a Director. Directors are allowed to send a packet out. 2. Registers for ARP packets and IPv4 packets. 3. Registers for Port Status Down messages. When such messages are received, Path Daemon removes all flows configured for the impacted port, thereby causing the PACKET_IN messages to again come to the controller. 4. Uses the Node Manager to get the end hosts corresponding to the source and destination MAC addresses and the datapaths to which these hosts are connected. 16 Aruba VAN SDN Controller 2.8 Administrator Guide

17 5. Uses the Path Selection manager to get the end-to-end shortest path between the source and destination hosts. 6. Uses the controller to push flow-mod messages to the datapaths. The flowchart below provides more details of its operation. Figure 2: Path Daemon flowchart Chapter 2 Understanding the controller architecture 17

18 Chapter 3 Using the SDN controller UI The SDN controller provides a console UI you can use as follows: View information such as alerts and logs and view OpenFlow information such as data flow details, topology of discovered switches and end nodes including shortest path and view OpenFlow classes that applications have registered. Perform actions such as acknowledging an alert, adding or enabling an application, exporting log data and entering licensing information. Configure SDN controller components such as setting key values for alert policies. The SDN controller also provides REST APIs you can use to program or configure the controller and develop applications to run on the controller. For details on how to use the REST APIs and how to develop applications, see the Aruba VAN SDN Controller Programming Guide and Aruba VAN SDN Controller REST API Reference. This chapter includes details on the following: Starting the SDN controller console UI Licenses on page 48 The SDN controller user interface Team SDN User window on page 21 Support logs on page 50 Navigation menu on page 23 Packet listeners on page 53 Alerts on page 25 OpenFlow Monitor on page 54 Applications on page 30 OpenFlow topology on page 58 Configuration components on page 37 OpenFlow Trace log on page 67 Audit log on page 45 OpenFlow Classes on page 74 Changing the language displayed in the user interface on page 19 Starting the SDN controller console UI Access the SDN controller from a Chrome or Firefox supported browser. A message will be displayed indicating if you are using an unsupported browser such as Internet Explorer. Procedure 1. Using a supported browser, access the controller UI: where <SDN_Controller_Address> is the IP address for your controller. The URI is case sensitive. For example: 2. Enter the User Name and Password credentials, then select Login. For example: Default user name: sdn 18 Aruba VAN SDN Controller 2.8 Administrator Guide

19 3. Once you log in, the main controller screen is displayed. For more information about the controller console UI, see The SDN controller user interface. The Keystone default timeout is 1 hour. If it is more than 1 hour since you logged in a message indicating that the session has expired is displayed. You must reload the page and log in again. For details on changing the Keystone timeout value, see Session expired message in the UI. Default domain name, user name, and password Default domain name: sdn Default user name: sdn Default password: skyline Changing the language displayed in the user interface How the controller determines what language to display To determine what language to display, the controller uses the value of the LANG environment variable in the Ubuntu /etc/default/locale file at the time the controller starts. Applications installed on the controller might not include multiple language support, so user interface components provided by applications might not be in the same language as the controller components. Supported languages Language English Chinese Japanese LANG entry in the /etc/default/locale file LANG=en_US.UTF-8 LANG=zh_CN.UTF-8 LANG=ja_JP.UTF-8 Default language The default language is English (en_us.utf-8). The controller user interface displays the default language in the following circumstances: There is no /etc/default/locale file. The LANG environment variable is set to a value that the controller does not support. Chapter 3 Using the SDN controller UI 19

20 Changing the LANG environment variable To change the locale: Procedure 1. Log in to Ubuntu on the system on which the controller is installed. 2. Open the /etc/default/locale file for editing: ~$ sudo vi /etc/default/locale 3. Change the entry for LANG to one of the supported values, then save and close the file. 4. Enter the following command to ensure the setting is applied to the user that logs into the controller: $ su username - For example, for the sdn user, enter the following command: $ su sdn - 5. Verify that the locale has changed by entering the following command to display the current locale: $ locale 6. Restart the controller: a. Close any instance of the web interface in which the controller might be running. b. Restart the controller: ~$ sudo service sdnc restart The SDN controller user interface Descriptions for common areas, icons, and controls on the UI screen are listed after the image. Figure 3: Screen areas and menus Banner: Identifies the user interface. Contains the alert notification counter and links to the navigation menu, alert information, and the SDN User window. Alert notification counter: Displays the current number of active alerts. Clicking this icon displays the Alerts as of Today dialog box. SDN User window: Enables you to log out of the controller, link to external websites, change the theme for the controller, and identify the version of controller software currently in use. Navigation menu: The primary menu for navigating to controller and application resources. Contains the controller navigation tree, labeled General, and can contain additional navigation trees for installed 20 Aruba VAN SDN Controller 2.8 Administrator Guide

21 applications that integrate with the controller UI. Can be displayed as a pane (as shown) or as a window that overlays the controller screen (see Expanding or collapsing the navigation menu). Navigation tree: Used to select the controller or application screen to display in the details pane. General is the controller navigation tree. Navigation trees for installed applications are displayed below or to the right of the General navigation tree. Details pane: Displays the detailed interface for the controller or application resource selected in the Navigation menu. Pagination control: Can appear on screens that have lists of items. Use these controls to view the listings page by page. Listing control: Can appear on screens that have lists of items. Use these controls to select the number of items to display in a single view. The Auto option displays all items in a single screen. For listings exceeding the length of the screen, you can use the scroll bar on the right side of the screen. UI top banner components Screen component SDN Controller menu expansion icon critical alert sdn user window icon Description Expands or collapses the navigation menu as an overlay window. Expands or collapses the controller Alerts as of today window. The number next to the icon is the alert notification counter, which provides a count of the current active alerts. Expands or collapses the UserSDN window. Changing column widths in the UI To change the column widths, drag the column head borders. For example: Procedure 1. To narrow the Severity column width, click the border to the left of Date/Time and drag it to the left. 2. To change the width of the navigation menu pane, click and drag the divider between the menu pane and the details pane. SDN User window The SDN User window displays as an overlay on the controller screen. Chapter 3 Using the SDN controller UI 21

22 User window screen details Figure 4: SDN user window Screen component Log out Change SDN User Password Links: Set Theme: Description Logs the user out of the controller. Change the SDN user password. Links to websites outside of the controller:sdn Information LibraryLinks to the information library on the Hewlett Packard Enterprise Software-Defined Networking website. The Hewlett Packard Enterprise Information Library for SDN provides links to the technical documentation for the Aruba VAN SDN Controller and the HP SDN applications. The Hewlett Packard Enterprise Software-Defined Networking website provides fact sheets, case studies, white papers, product summaries, technical and business documentation, and other information to help you identify SDN solutions for your business needs.sdn CommunityLinks to the Hewlett Packard Enterprise SDN community discussion forum website within the HP Enterprise Business Community. This site offers resources such as: SDN discussion boards SDN development information An SDN knowledge base Changes the theme for the controller UI:DayWhen selected, plain text is black and the background is white.nightwhen selected, plain text is white and the background is black. Table Continued 22 Aruba VAN SDN Controller 2.8 Administrator Guide

23 Screen component SDN Controller Version: collapse icon Description Displays the version of the controller software that is running on this system. Collapses the window. Expanding or collapsing the SDN User window To expand the SDN User window, from the top banner, click sdn user window icon. To collapse the SDN User window, do one of the following: Procedure 1. In the SDN User window, click the collapse icon. 2. From the top banner, click the sdn user window icon. Changing the SDN user password Procedure 1. Expand the SDN User window. 2. Select Change SDN User Password. The SDN user password you can change on this screen is the Keystone user password not the HPE Linux operating system password. 3. In the Change SDN User Password dialog box, enter the Old Password, New Password, and Re-enter New Password and click Apply. Or click Cancel to exit without changing the password. Changing the background and text colors The background and text colors are part of the theme of the controller UI. Procedure 1. Expand the SDN User window. 2. In Set Theme:, select one of the following options: a. Day b. Night Logging out of the controller Procedure 1. From the SDN User window, select Log out. Navigation menu Chapter 3 Using the SDN controller UI 23

24 About the navigation menu The navigation menu is the primary menu for navigating to controller resources. The resources included with the controller are described in this document. Applications installed on controller might add resources to this menu. Displays as a pane or an overlay window You can display the navigation menu in the following ways: As a pane on the left side of the controller browser window. As a window that overlays part of the main screen of the controller browser window. Contains one or more navigation trees The navigation menu contains the General controller navigation tree and can contain additional navigation trees for installed applications that integrate with the controller UI. Expanding or collapsing the navigation menu The navigation menu is displayed as a navigation pane by default. You can display the navigation menu as a pane on the controller screen or as a window that overlays the controller screen. Expanding or collapsing the navigation menu as an overlay window To display the navigation menu as an overlay window, from the top banner of the controller screen, click nav menu. To collapse the navigation window, do one of the following: Procedure 1. In the window, click the collapse icon. 2. From the top banner, click Aruba VAN SDN Controller. Expanding or collapsing the navigation menu as a window pane To expand or collapse the navigation menu as a window pane, click the following icon: When the navigation menu is expanded as a window pane, the icon is located on the right side of the menu. When the navigation menu is collapsed, the icon is located in the left margin of the controller screen. Navigation menu screen details Screen component General Alerts Description Displays the navigation tree for the resources that are provided with the controller. By default, the General controller navigation tree is expanded and the Alerts screen is selected and displayed. To display the screen for another resource, select the resource in the navigation tree. Displays the Alerts screen. This screen provides details on alerts and allows you to acknowledge alerts and unacknowledge alerts back to an active state. For more information, see Alerts on page 25. Table Continued 24 Aruba VAN SDN Controller 2.8 Administrator Guide

25 Screen component Applications Configuration Audit Log Licenses Team Support Logs OpenFlow Monitor OpenFlow Topology OpenFlow Trace OpenFlow Classes Packet Listeners Other navigation menu items Description Displays the Applications screen. This screen provides details on your controller applications and allows you to add, upgrade, uninstall, enable and disable these applications. For more information, see Applications on page 30. Displays the Configurations screen. This screen lists the configurable components of the controller and allows you to modify key values. For more information, see Configuration components on page 37. Displays the Audit Log screen. This screen displays audit log records related to activities, operations and configuration changes initiated by an authorized user, such as, installing an application. For more information, see Audit log on page 45. Displays the Licenses screen. This screen provides details on licenses and allows you to enter a license. For more information, see Licenses on page 48. Displays the Team screen. For more information, see Team. Displays the Support Logs screen. This screen displays support log records of internal controller operations that can be used by a support engineer for troubleshooting an SDN installation. For more information, see Support logs on page 50. Displays the OpenFlow Monitor screen. This screen lists the Data Path IDs and descriptive information for the active switches. For more information, see OpenFlow Monitor on page 54. Displays the OpenFlow Topology. Displays a topology of discovered switches and end nodes in the controller domain. For more information, see OpenFlow topology on page 58. Displays the OpenFlow Trace screen. OpenFlow conversations are captured in messages to and from the controller and the OpenFlow devices it manages and displayed on this screen. For more information, see OpenFlow Trace log on page 67. Displays the OpenFlow Classes screen. This screen shows the OpenFlow classes that applications have registered with the controller. For more information, see OpenFlow Classes on page 74. Displays the Packet Listeners screen. This screen displays details on the packet listeners that are currently running on the controllers. For more information, see Packet listeners on page 53. May include additional navigation trees for installed applications that integrate with the controller UI. Alerts Alerts give notification of events that affect controller operation, and in some cases indicate that some action is needed to correct a condition. When controllers are operating in a team, alerts generated by any team member are visible in the Alerts screen for all active team members. Chapter 3 Using the SDN controller UI 25

26 By default, alerts are in an unacknowledged, active state. An alert must be in an active state to appear in the following places: The alert notification counter The Alerts as of today window Alerts screen details Figure 5: Display the learn.ip option Screen component Refresh Acknowledge UnAcknowledge Alert text color Severity Description Updates the alerts displayed on the screen. The controller does not update the display as new alerts are generated. Use this action to refresh the display. Changes the selected alert to an acknowledged state. The controller displays the alert in gray text. Use this action to indicate that you have read the alert. Changes the selected alert to an active, unacknowledged state. Indicates the state of the alert: The controller displays active, unacknowledged alerts the alert in the text color corresponding to the controller theme. For example, when the controller theme is daylight, the active alerts appear in black text. The controller displays the selected alert in blue text. Click an alert to select it. The controller displays acknowledged alerts in gray text. Indicates the severity of the alert. informational icon warning icon critical icon Date/Time Indicates the date and time the alert was generated. Description Describes the alert in human readable text. Table Continued 26 Aruba VAN SDN Controller 2.8 Administrator Guide

27 Screen component Origin Topic Controller ID Description Indicates which component or application generated the alert. Indicates the category for this alert. Multiple origins can contribute alerts to the same topic. Identifies the controller that generated the alert. The controller is represented as a hexadecimal number. When you use controller teaming, this ID enables you to identify which controller in the team generated the alert. Alert notification counter The alert notification counter is displayed in the top banner and appears on all controller screens. Figure 6: Alert notification counter This counter indicates the number of active alerts: The controller increments this counter when each new alert occurs. The controller decrements this counter when you acknowledge an alert or when the controller deletes an alert according to the alert policies set for aging out alerts. Viewing the ten most severe recent active alerts The Alerts as of today window shows up to 10 alerts ranked by severity (highest to lowest) and then by date and time (newest to oldest). Procedure 1. In the top banner, click the critical icon. Chapter 3 Using the SDN controller UI 27

28 The Alerts as of today window is displayed: 2. To close the window, do one of the following: a. To close the window and display the Alerts screen, click All. b. At the bottom of the window, click the collapse icon. c. In the top banner, click either the alert counter number or critical icon. Acknowledging an alert To acknowledge an alert from the Alerts as of today window: Procedure 1. Click the alert to select it. 2. Click Acknowledge. The result is that the alert is removed from the Alerts as of today window, the alert is displayed in gray text on the Alerts screen, and the alert notification counter is decremented by one. To acknowledge an alert from the Alerts screen: 1. Click the alert to select it. 2. Click Acknowledge. The result is that the alert is displayed in gray text on the Alerts screen, and the alert notification counter is decremented by one. Deleting an alert You can acknowledge an individual alert, but you cannot clear or delete the alert. 28 Aruba VAN SDN Controller 2.8 Administrator Guide

29 The controller deletes alerts according to the configured alert age-out policy. To configure the age-out policy, see Configuring how alerts age out. Configuring how alerts age out Procedure 1. On the Configurations screen in the System tab, select the com.hp.sdn.adm.alert.impl.alertmanager component.: Figure 7: Select the AlertManager component 2. Click Modify. The Modify System Configuration dialog box is displayed for the com.hp.sdn.adm.alert.impl.alertmanager component: 3. Change the values for the keys (see Key values that control how alerts age out on page 29). 4. Click Apply. Key values that control how alerts age out You can configure the following key values for alerts to control how alert data ages out. Chapter 3 Using the SDN controller UI 29

30 Key trim.alert.age Description Specifies the number of days an alert remains in persistent storage and remains displayed on the Alerts screen. Data type A number from 1 through 31 Default value 14 trim.enabled When true, specifies that the controller deletes alerts that have exceeded the trim.alert.age limit. Default value true trim.frequency Specifies how often, in hours, the controller is to delete alerts that have exceeded the trim.alert.age limit. Data type A number from 8 through 168 Default value 24 Example Enter 8 to specify that the controller delete aged-out alerts every eight hours. Applications About the application manager The Application Manager is a component on the controller that supports default and add-on network services, and enables installing, upgrading, enabling (starting), disabling (stopping), and uninstalling SDN applications. When controllers are operating in a team, actions performed on one controller are propagated to the other controllers in the team. Actions you select in the Applications screen for one controller, such as Install, Enable, and Disable, are propagated to the other controllers. See also: Obtaining applications from the Hewlett Packard Enterprise SDN App Store on page 32 Adding or upgrading an application on page 32 Disabling (stopping) or enabling (starting) an application Uninstalling an application on page 34 Understanding application states and OSGi artifacts Prerequisites for installing an application Any application to be installed on the controller must meet the following requirements: It must be in a zip format. The zip file must be accessible from the browser UI's file manager (or downloadable from the App Store). It must contain an application descriptor file with key value pairs of the attributes associated with the application, including all mandatory attributes. If jar signing checking is turned on, the application zip files must be signed as well. 30 Aruba VAN SDN Controller 2.8 Administrator Guide

31 Applications you purchase from Hewlett Packard Enterprise or the Hewlett Packard Enterprise SDN App Store meet these requirements. For information about developing applications that meet these requirements, see the Aruba VAN SDN Controller Programming Guide Applications screen details Figure 8: Applications screen details Screen component Refresh New Upgrade Uninstall Enable Disable Launch Network Protector Description Reloads the view. Installs an application on the controller. Installs an upgrade to an application that has already been installed on the controller. Removes an application from the controller. Starts or allows an application to continue operations on the controller. Stops or prevents an application from operating on the controller. If you have the Aruba Network Protector SDN Application installed, this button will be enabled to allow you to launch the Aruba Network Protector application. Table Continued Chapter 3 Using the SDN controller UI 31

32 Screen component Name Version State AppStore Purchased Applications Launch AppStore Description The name of the application.the following core embedded applications that come with the controller are listed by default on the Applications screen: Path Diagnostics OpenFlow Link Discovery OpenFlow Node Discovery Path Daemon These are the only embedded applications you are allowed to manage using the UI. Other embedded applications are not listed because they should not be disabled or uninstalled. For information about embedded applications, see Understanding the controller architecture on page 12.If you have installed any of Aruba s SDN applications, such as Aruba Network Protector, or any thirdparty SDN applications these applications will also be listed. The version number of the application. The most common states are active, staged and disabled. The name and version number of SDN applications purchase from the Hewlett Packard Enterprise SDN App Store. Launches the Hewlett Packard Enterprise SDN App Store website. Obtaining applications from the Hewlett Packard Enterprise SDN App Store From the App Store, you can purchase and download applications for your controller. In the case of a web proxy, you need a proxy configuration to connect to the App Store portal. To set the proxy, in the /etc/init/sdnc.conf file, replace c1, c2, c3, and team with the controller IP addresses and the team IP address (when applicable): env JAVA_OPTS="-Xms512m -Xmx4096m -XX:MaxPermSize=512m -Dhttps.proxyHost=my-webproxy -Dhttps.proxyPort=my-web-proxy-port -Dhttp.nonProxyHosts= localhost c1 c2 c3 team" Command Example env JAVA_OPTS="-Xms512m -Xmx4096m -XX:MaxPermSize=512m -Dhttps.proxyHost=webproxy.test.com -Dhttps.proxyPort=8088 -Dhttp.nonProxyHosts= localhost " You must update the /etc/init/sdnc.conf file on each controller and then restart the controllers for these changes to take effect. If you are downloading a signed application from the App Store, the JAR signing requires a trusted certificate in the sdnjar_trust.jks file even if the certificate is trusted in the JAVA cacerts keystore. For details, see Adding certificates to the jar-signing truststore on page 125 Adding or upgrading an application Any application in the proper format can be added to the controller (see About the application manager on page 30). To use curl commands and the REST APIs to complete this task, see curl commands on page Aruba VAN SDN Controller 2.8 Administrator Guide

33 Use this procedure to install either a new application or a new version of an existing application on the controller using the UI. Procedure 1. Do one of the following: To install a new application, click New. To upgrade to a new version of an existing application, select the application from the Name list and click Upgrade. 2. Click Browse to navigate to the location of the application zip file and select the file. 3. Click Upload to upload the file. 4. Wait for Completed to appear. For example: 5. Click Deploy. The new application then appears by name on the Applications screen as ACTIVE. After you complete this procedure: 1. The application is started and in an active state. 2. If the controller is in a controller team, the controller propagates the application to all the controllers in the team automatically. Disabling (stopping) or enabling (starting) an application This procedure temporarily stops an active application from servicing requests, but retains the application on the system. The application remains present on the system and can be restarted when needed. (The application does not automatically restart when the controller restarts.) To use curl commands and the REST APIs to complete this task, see curl commands on page 182. To disable an application using the UI: Procedure 1. In the Applications screen, select the application you want to stop. 2. Click Disable to display the Disable Application dialog box. Chapter 3 Using the SDN controller UI 33

34 3. In the Disable Application dialog box, click Disable. The Disable Application dialog box closes and the application state is changed to DISABLED. To enable an application using the UI: 1. In the Applications screen, select the application you want to enable. 2. Click Enable to display the Enable Application dialog box. 3. Click the Enable button to activate the application. The application starts or resumes operation and the application state is changed to ACTIVE. Uninstalling an application This procedure completely removes an application from the controller. To later restore the removed application, see Adding or upgrading an application on page 32. To use curl commands and the REST APIs to complete this task, see curl commands on page 182. Procedure 1. Use the following procedure to uninstall an application using the UI. 2. In the Applications screen, select the application you want to uninstall. 3. Click Uninstall. 34 Aruba VAN SDN Controller 2.8 Administrator Guide

35 4. Click the Uninstall button to remove (delete) the application. Understanding application states and OSGi artifacts In the default state, or when an application has been started, it is in the ACTIVE state and is servicing requests. Application states include the following: Table 1: Application States State ACTIVE STAGED UPGRADE_STA GED INSTALLING UPGRADING CANCELING DISABLING DISABLED ENABLING UNINSTALLING RESOLVED Description The application is running and servicing requests. A new application has been downloaded to the controller and is ready to be installed. A new version of an existing running application has been downloaded to the controller and the new version is ready to be installed (upgrade/downgrade). A transitive state indicating a new application is in the process of being installed. A transitive state indicating the existing application is being stopped and a new version of the application is being installed. A transitive state indicating a non-installed version of an application is being deleted from the controller. A transitive state indicating the application is in the process of being disabled (stopping). The application is disabled (stopped). A disabled application is not automatically started when the controller restarted. A transitive state indicating the application is being started. A transitive state indication an application is being stopped and completely removed from the controller. The application is stopped and not servicing requests. An application can only be in this state when it is stopped externally to the SDN Controller (e.g. the virgo console). Chapter 3 Using the SDN controller UI 35

36 Table 2: Error condition management State NEW > STAGED NEW > UPGRADE-STAGED STAGED > ACTIVE UPGRADE-STAGED > ACTIVE ANY STATE > UNINSTALLED ANY STATE > DISABLED DISABLED > ENABLED Description If an error condition occurs when staging the application, then it actually does not exist. (Error conditions in this stage clean up after themselves.) If an OSGi deployment exception is encountered, the application is moved to DISABLED if it fails to deploy as it is. If a File I/O or URI exception is encountered, the application remains in the installing state. If an exception is encountered (OSGi deployment, File I/O, or URI), rollback attempt is made, as listed below. (Depending on the original exception, not all options may be possible). 1. Calls AppStore.deleteStore on the upgraded version of the application. 2. Attempts to redeploy the original version of the application. If any exception is encountered, the application remains in UNINSTALLING state If an exception is encountered, remains in DISABLING state. If an OSGi deployment exception is encountered, the application is moved to the DISABLED state if it fails to deploy as it is. If any other exception is encountered (file I/O or URI), the application remains in the ENABLING state. To access the link to the OSGi artifacts for an application, click on the bullet for the application in the web GUI. For example, clicking on the bullet for the Path Diagnostics application displays the link to identity of the associated OSGi artifacts: Figure 9: Links to OSGi artifacts associated with individual applications 36 Aruba VAN SDN Controller 2.8 Administrator Guide

37 Configuration components The Configurations screen enables access to the configurable components in the controller which are used to manage the controller and application features. Some examples of when you might want to make configuration changes include: Specify an NTP server or date and time on the controller system using the NTP component or specify a static IP address using the Network component. Specify hybrid mode for controlling packet forwarding by configuring the ControllerManager component. Define how long to keep alerts and how to age out alerts by configuring the AlertManager component. Define how audit log data ages out by configuring the AuditLogManager component. Adding or removing an SDN application might add or remove additional configuration components. However, direct addition or removal of configuration components is not supported. When controllers are operating in a team, configuration changes on one active controller propagate to the other active controllers in the team. See also: Using configuration component keys on page 37 Modifying a component configuration Modifying NTP server or date and time Modifying Network settings Modifying Logger settings Using configuration component keys Each configuration component contains one or more component keys, each of which identify a configurable property of the component. Information about each component key includes the current value, the default value, and a brief description. Where applicable, the range of suggested values is also included. You can find information about each component key on the Configurations screen of the controller UI. The controller Configs REST API is also available for configuring components, excluding Network and NTP components, and the REST API for Logger configuration can only be done for each individual module (such as hp.sdn.event) not groups of modules. Inappropriate changes to key values can result in severely degraded system performance. For this reason, Hewlett Packard Enterprise strongly recommends that managing the default key values be done only by experienced network administrators and programmers who have a strong understanding of SDN controller systems. Configurations screen details On the Configurations screen, the configuration components are accessed from the following four tabs: Basic provides access to tracing, topology discovery and flow priority configuration (see Basic Configurations view). Advanced provides access to timing, performance tuning and debugging configuration (see Advanced Configurations view). System provides access to platform specific configuration (see System Configurations view). Apps provides access to configuration components for installed SDN applications (see Apps Configurations view). The controls on these screens are the same. Chapter 3 Using the SDN controller UI 37

38 Screen component Modify expand icon collapse icon Component Description Select a component and then click Modify to open the Modify Configuration dialog box for the selected component. Click to display a list of the configurable keys for the component. The display for each key includes the current value, the default value, and a brief description. Where applicable, the range of suggested values is also included. Click to hide the key and value information for the component. Lists the basic configuration components. The components are described in the following sections. Basic Configurations view Figure 10: Basic Configurations view Components in the Basic configurations tab: com.hp.sdn.ctl.of.impl.controllermanager The ControllerManager component provides parameters used in the implementation of the OpenFlow protocol. You can configure parameters such as hybrid.mode, keystore, keystore.password, truststore, truststore.password. com.hp.sdn.ctl.of.impl.tracemanager The TraceManager controls OpenFlow trace duration. Use the record.duration key to specify how long a trace is to run after it starts. com.hp.sdn.ctl.path.impl.pathdaemon The PathDaemon component provides parameters used to perform L2 path calculations based on IPv4 addresses for IPv4 packets or MAC addresses for ARP packets. You can set the following flow timeout parameters: Use the idle.timeout key (default 60 seconds) to configure the idle timeout value for each flow-mod. The idle timeout value specifies how long the flow-mod will remain in the device if the flow-mod is not actively being used. Use the hard.timeout key (default 0, which implies infinite timeout) to configure the hard timeout value for each flow-mod. The hard timeout value specifies how long the flow-mod will remain in the datapath (regardless of usage). com.hp.sdn.disco.of.node.impl.ofarpdiscoverycomponent 38 Aruba VAN SDN Controller 2.8 Administrator Guide

39 The OpenFlow ARP discovery component of the OpenFlow Node Discovery application is used for topology host discovery via ARP protocol. Use the arp.age key to configure the node timeout values. The listener.altitude key changes the altitude of the OfArpDiscoveryComponent component. For more information, see Packet listeners on page 53. com.hp.sdn.disco.of.node.impl.ofdhcpdiscoverycomponent The OpenFlow DHCP discovery component of the OpenFlow Node Discovery application is used for topology host discovery via DHCP protocol. Use the dhcp.age key to configure the node timeout values. The listener.altitude key changes the altitude of the OfDhcpDiscoveryComponent component. For more information, see Packet listeners on page 53. com.hp.sdn.disco.of.node.impl.ofipdiscoverycomponent The OpenFlow IP discovery component of the OpenFlow Node Discovery application is used for topology host discovery via IP Protocol. Use the ip.age key to configure the node timeout values. The learn.ip key indicates whether the controller should discover nodes from all IP packets it receives. The listener.altitude key changes the altitude of the OfIpDiscoveryComponent component. See Packet listeners on page 53. Advanced Configurations view Figure 11: Advanced Configurations view Components in the Advanced Configurations view. com.hp.sdn.adm.mgr.impl.hpws.hpwsinstallmanager The HpwsInstallManager component provides a service for installing applications from the Hewlett Packard Enterprise SDN App Store, a remote web service. com.hp.sdn.api.impl.alertpostmanager The AlertPostManager component uses the HTTP(s) protocol to send alert data as a JSON string to registered alert topic listeners. com.hp.sdn.cms.impl.clientmapperserviceprovider The ClienMapperServiceProvider component provides information about a client by combining information from Aruba ClearPass log on and log off events and location information from the controller about the switch and its port connected to the client. com.hp.sdn.ctl.diag.impl.pathdiagnosticcomponent Chapter 3 Using the SDN controller UI 39

40 The PathDiagnosticComponent provides the ability to send out a diagnostic packet on one switch and receive it on the next. You can use it to trace a path for debugging link failures in your network. com.hp.sdn.disco.of.link.impl.openflowlinkdiscoverycomponent The OpenflowLinkDiscoveryComponent transmits link discovery packets to the attached Openflow devices, listens to the responses, and populates the Link Service cache with the results. Use the age.multihop.links key to configure the OpenFlow Link Discovery application to remove multihop links from the link table if the link is not re-discovered in two poll intervals. Use the multihop.poll.interval key to configure the polling interval, in seconds, for multihop links. com.hp.sdn.misc.adminrestcomponent The AdminRestComponent provides parameters for internal communication between SDN components and the Admin REST API of the controller. com.hp.sdn.misc.servicerestcomponent The ServiceRestComponent provides parameters for internal communication between SDN components and the SDN controller Northbound REST API. com.hp.sdn.node.impl.nodecachecomponent The NodeCacheComponent component serves as an in memory cache of the nodes known to the controller. It provides add, update, remove, and get methods for its nodes. The cache.size key allows you to specify a maximum number of nodes that can be stored by the NodeManager. The default value is 20,000. com.hp.sdn.rs.restperfprovider The RestPerfProvider component reports performance data for the REST API. You can configure the perf.profile key. System Configurations view Figure 12: System Configurations view Components in the System Configurations view. NTP Configure NTP server or set a specific date and time for the controller system. For details, see Modifying NTP server or date and time. Network 40 Aruba VAN SDN Controller 2.8 Administrator Guide

41 Configure networking (Static IP address or DHCP) for the controller system. For details, see Modifying Network settings for the eth0 interface. Loggers Configure logging levels (ALL, TRACE, DEBUG, INFO, WARN, ERROR, OFF). For details, see Modifying Logger settings. com.hp.sdn.adm.alert.impl.alertmanager The AlertManager component controls the quantity of alert data present on the system by periodically checking for alert data to be deleted based on the configured age-out policy. For more information about alert log policies, see Configuring how alerts age out. com.hp.sdn.adm.auditlog.impl.auditlogmanager The AuditLogManager component controls the quantity of audit log data present on the system by periodically checking for audit log data to be deleted based on the configured age-out policy. For more information about audit log policies, see Configuring how audit log data ages out. com.hp.sdn.adm.auth.impl.authenticationmanager The AuthenticationManager component provides for the authentication of external users to the controller and between the controller and the Keystone server. com.hp.sdn.adm.health.impl.healthmanager The HealthManager component is the Application/Component Health Monitor parameters.you must configure the autoshutdown.properties exactly as it is done in the sample file. The possible health status are critical, unhealthy, healthy, or hung. com.hp.sdn.adm.log.impl.logmanager The LogManager component controls the number of log message rows displayed in the Support Logs display. For more information on support log queue size, see Configuring the support log queue size. com.hp.sdn.adm.metric.impl.metricmanagercomponent The MetricManagerComponent determines how measurement data is maintained by the controller. The controller includes a metering framework that internal components and installed applications can use to collect various types of data. (Data can be persisted on the controller from sources external to the controller.) Any metric created with the framework might optionally be persisted over time or directed to the controller JMX facility for viewing. Data persisted over time can be viewed using the controller REST API, while data sent to JMX can be viewed using JConsole or another JMX client. The MetricManagerComponent permits changing default values for certain aspects of the metering framework operation, such as how long the controller should retain persisted data, at what time of day persisted data that is too old should be trimmed, and how often persisted metric values should be saved to disk. (This value can be overridden for any metric when the metric is created). com.hp.sdn.dvc.impl.devicemanager The DeviceManager component serves as an in memory cache for the persistent devices known to the controller. It holds information about those devices and whether they are currently connected to the controller. It provides add, update, remove, and get methods for its devices. com.hp.sdn.link.impl.linkservicecomponent The LinkServiceComponent controls the Link Manager service, which serves as an in memory cache of the links known to the controller. It provides add, update, remove, and get methods for its links. The cache.size configurable parameter allows you to specify a maximum number of links that can be stored by the Link Manager. Default is 20,000. com.hp.teaming.imple.cassandraprocessmanager The CassandraProcessManager component controls configuration parameters of the Cassandra database. com.hp.sdn.teaming.impl.teamconfigurationmanager The TeamConfigurationManager component manages the configuration of team communication. When one of the components s keys is modified, the administrator must wait for the new value to be forwarded to the other Chapter 3 Using the SDN controller UI 41

42 members of the team (this can be confirmed by making sure the change appears in the UI of each controller), and then ALL controllers must be restarted. Apps Configurations view Figure 13: Apps Configurations view If you have other SDN applications installed, configurable components for these applications are listed in the Apps Configurations view. For example in the screen shown above the com.hp.mvisor.adm.topo.impl.networkvisualizertopologymanager component for the Aruba Network Visualizer SDN Application is listed in the Apps Configurations view. For details on configuring these SDN application specific components see the documentation for the SDN application. Modifying a component configuration Procedure 1. On the Configurations screen, select the tab that contains the component you want to modify (Basic, Advanced, System, or Apps). 2. Select the component you want to modify. 3. Click Modify. A Modify Configuration dialog box is displayed for the component you selected. For example: 4. Enter new values for each of the keys you want to modify. 5. Do one of the following: 42 Aruba VAN SDN Controller 2.8 Administrator Guide

43 a. To save your changes and close the dialog box, click Apply. b. To close the dialog box without saving changes, click Cancel. Modifying NTP server or date and time Best practices HPE recommends that you use an NTP server rather than setting date and time because if you change network settings, the date/time will be reset to current date/time. If the controller is in a team and you want to change the NTP server or date and time, the other two controllers in the team must have the same NTP server or date and time. You can make the change on an individual controller but will see a message reminding you to check that the NTP server or date and time is the same on the other controllers in the team. Modify NTP server or date and time Procedure 1. On the Configurations screen in the System tab, select the NTP component. You can use the expand icon to view the NTP information currently configured 2. Click Modify. The Modify System Configuration dialog box is displayed for the NTP component. For example: 3. Select either NTP Server or Date/Time and make the following configuration changes: a. Select NTP Server to configure an NTP server for use by the controller system. Enter either the server IP address or server name. You may only enter one server. b. Select Date/Time to configure the date and time to set for the controller system and click in the Select a Date field. The calendar view is displayed. You can select Now to use the current date and time or you can select a date on the calendar and enter the time in hours and minutes. Then click Done. c. Click Apply to save your changes. d. To close the dialog box without saving changes, click Cancel. Chapter 3 Using the SDN controller UI 43

44 If you clicked Apply, a confirmation window is displayed showing a message saying that you will be logged out of the UI and will need to log back in for authentication. If the controller is in a team, the message will also remind you to check that all controllers in the team have the same NTP server or date and time. Click Yes in the confirmation window to save your changes. Modifying Network settings for the eth0 interface Prerequisites If the controller is in a team you must first disband the team before modifying the network settings. You can configure Hostname, IP Address and Type (Static or DHCP) of network connection. The configuration is for the eth0 interface only. Modify network settings Procedure 1. On the Configurations screen in the System tab, select Network. You can use the expand icon to view the network information currently configured. 2. Click Modify. The Modify System Configuration dialog box is displayed for the Network component. For example: 3. Enter new values for Host Name, IP Address, Type, and other fields as required. No spaces are allowed in the Host Name field. If the controller is in a team, you must first disband the team before modifying the network settings. If you are configuring a static IP address, you must enter values for Gateway, Netmask, and Primary DNS fields; the Secondary DNS field is optional. 4. Click Apply to save your changes. To close the dialog box without saving changes, click Cancel. If you clicked Apply, a confirmation window is displayed showing a message that the controller automatically reboots. 5. Click Yes in the confirmation window to save your changes. 44 Aruba VAN SDN Controller 2.8 Administrator Guide

45 After applying the change, you will be disconnected from the UI and will need to wait for the controller to restart before logging back in. Modifying Logger settings For troubleshooting you may want to increase the logging level to generate more information in the log file for use in debugging a problem. Procedure 1. On the Configurations screen in the System tab, select Loggers. You can use the expand icon to view the logging levels currently configured. 2. Click Modify. The Modify System Configuration dialog box is displayed for the Loggers component. For configuration purposes the loggers are grouped into categories listed as keys you can modify. For example: You can configure the logging level for each of the logger keys listed. The log levels from most verbose to least verbose are: ALL, TRACE, DEBUG, INFO, WARN, ERROR, OFF. If the controller is restarted or if the virtual machine is rebooted, the log levels for all loggers revert back to INFO. Setting all loggers to a high verbose level of logging is not recommended because it can lead to a shortage of system storage space very quickly. 3. Click Apply to save your changes. To close the dialog box without saving changes, click Cancel. Audit log Chapter 3 Using the SDN controller UI 45

46 About the audit log The audit log is available through both the controller GUI and the REST API, and records events related to activities, operations, and configuration changes initiated by an authorized user. This includes activities such as: Installing an application (or starting, stopping, uninstalling an application) Modifying the configuration of a controller component Installing a license Forming a controller team When controllers are operating in a team, the audit log shows events for all controllers in the team. See also: Deleting an audit log entry on page 47 Configuring how audit log data ages out Exporting and archiving audit log data on page 48 Audit log screen details Figure 14: Viewing the Audit Log Screen component Refresh User Occurred Activity Description Updates the log entries displayed on the screen. The controller does not update the display as new entries are generated. Use this action to refresh the display. The user that performed the operation that triggered the log entry A time stamp (in UTC format) indicating when the controller created the log entry. The type of activity that triggered the creation of the log entry. Data Detailed information about the log entry. Table Continued 46 Aruba VAN SDN Controller 2.8 Administrator Guide

47 Screen component Origin Controller ID Description The application or controller component that generated the log entry. A hexadecimal number that identifies controller that generated the log entry. When you use controller teaming, this ID enables you to identify which controller in the team generated the alert. Deleting an audit log entry You cannot delete or modify a log entry. The controller deletes entries according to the configured audit log policies. To configure the audit log policies, see Configuring how audit log data ages out. Configuring how audit log data ages out You can configure the following key values for the audit log to control how audit log data ages out. To set these key values, configure the com.hp.sdn.adm.auditlog.impl.auditlogmanager component using the Configurations screen. Key Default Value Description trim.auditlog.age 365 Specifies the number of days to retain a log entry. Use this key to implement your record retention policy. Data type A number from 31 through trim.enabled true true Specifies that the controller deletes log entries that have exceeded the trim.auditlog.age limit. false Specifies that the controller does not delete log entries that have exceeded the trim.auditlog.age limit. trim.frequency 24 Specifies how often, in hours, the controller is to delete log entries that have exceeded the trim.alert.age limit. Data type A number from 8 through 168 Example Enter 24 to specify that the controller delete aged-out log entries every 24 hours (once per day). To configure how audit log data ages out: Procedure 1. On the Configurations screen in the System tab, select the com.hp.sdn.adm.auditlog.impl.auditlogmanager component. 2. Click Modify. The Modify System Configuration dialog box is displayed for the com.hp.sdn.adm.auditlog.impl.auditlogmanager component. Chapter 3 Using the SDN controller UI 47

48 3. Change the values for the keys (these keys are described in the table above). 4. Click Apply. Figure 15: The AuditLogManager Configuration Component Controls Audit Log Policy Exporting and archiving audit log data To retain log records for longer than the trim.auditlog.age limit, you must export the audit log from the controller to a file before the trim.auditlog.age limit is reached. Exporting audit log data does not remove it from persistent storage. To export the audit log, you must use the REST APIs since this action cannot be performed in the UI. For example, you can use the curl command at Export audit log data as a CSV file using curl commands on page 183. Licenses A license is required for the controller. In addition, SDN applications can require licenses that are separate from the license for the controller. For information on installing, activating, uninstalling or transferring licenses, see License Registration and Activation on page 85. Licenses screen details The Licenses screen displays the controller Install ID, and is used to activate new licenses, and deactivate installed licenses (for transfer to another installation). Licenses screen example: 48 Aruba VAN SDN Controller 2.8 Administrator Guide

49 Screen component Refresh Add Deactivate Copy Uninstall Key Install ID Serial# Product Licensed For Qty Type Status Expire By Uninstall Key Description Updates the screen with the latest license information. Adds and activates the specified license key on this controller. Deactivates the selected license. When a license is deactivated, an uninstall key is assigned for license transfer purposes and you can copy this uninstall key by selecting this button, see Transferring licenses on page 94. Contains the installation identifier for this controller. A sequence/serial/record number given for that license across all licenses generated for that install ID. The HPE My Networking Portal assigns the serial number while generating license records. Name of the application or product for which the license is generated. License metric name. For example, nodes and HA nodes. Quantity of the Licensed For metric based license. Type of license. For example, PRODUCTION, DEMO, or EVAL. ACTIVE, EXPIRED, or DEACTIVATED Date and time when the license Licensed For expires. When a license is deactivated, an uninstall key is assigned for license transfer purposes, see Transferring licenses on page 94. Installing, activating, uninstalling, or transferring licenses For information about installing, activating, uninstalling, and transferring licenses, see License Registration and Activation on page 85. Team The Team screen displays team and region configuration information including: Team status (top banner) Team configuration and controller status (top section) Region configuration (middle section) Device owners (bottom section) Chapter 3 Using the SDN controller UI 49

50 For details on viewing information on the Team screen, see Viewing your team configuration using the UI. For details on configuring High Availability (HA) and teaming, see Configuring for High Availability on page 101. Figure 16: Team Screen Support logs About support logs The support logs maintain an internal record of events of interest from the operations of an active SDN controller. This information is the type of data a support engineer would request when troubleshooting an SDN installation. The log file is configured to have a maximum size of 10 MB and to keep 4 previous versions. The log.log file is the primary log for controller information. With 4 full previous versions and the one active log, the core controller logging will consume at most 50 MB of disk space. The controller allows up to five support logs; one active and four in storage: Support logs are stored in the controller /var/log/sdn/virgo/logs directory. When the current log reaches 10 MB, the controller copies the log to storage and starts a new log. When the log file has rolled over four times, the controller purges the oldest log file when it needs to roll over again. The core controller has at most 5 log files. Support logs can be exported to a file. In a controller team environment: Each controller maintains its own support logs. Changing the support log queue size on any controller propagates to all active controllers in the team. The Export action gathers the set of support log file data from all active controllers in the team, and stores the data as a single compressed archive. See also: Configuring the support log queue size Exporting the support logs on page Aruba VAN SDN Controller 2.8 Administrator Guide

51 Support logs screen details Figure 17: Selecting the Support Logs screen Screen component Refresh Export Description Displays a listing of the most recent log messages, as determined by the currently configured queue size. For example, with a queue size of 100, Refresh lists the 100 most recent log messages. Gathers the set of support log file data from the standalone controller or all active controllers in the team, and stores the data as a single compressed archive. Chapter 3 Using the SDN controller UI 51

52 Screen component Level Description The severity level for the entry. The logging levels are hierarchical. Messages are logged with the lowest logging level and above. The lowest level is TRACE, which results in all messages being logged for the selected logger when the TRACE logging level is specified. INFO is the logging level In the default configuration. Severity levels are: Value ERROR WARN INFO DEBUG TRACE Description Indicates a problem to investigate. The problem could cause functional or performance issues with application. Indicates a problem that you might want to investigate. The problem could be an early indication of issues that could later cause an error. Indicates a normal operational event that requires no action. Indicates an informational event that is most useful for debugging applications. Indicates an informational event that is most useful for debugging applications. Often used to show program execution details when DEBUG-level events do not provide enough information Using the Virgo Administrator console, you can dynamically change the logging level for a component that is writing to the support log. For example, you can enable the DEBUG level logging for just the NodeManager configuration component.you can also dynamically change the logging level by using the REST API. See Aruba VAN SDN Controller REST API Reference Logger Thread Message Data Controller ID The module or feature that triggered the logging condition. The thread that caused the logging condition to occur. Describes the details of the logging condition. Detailed information about the log entry. A hexadecimal number that identifies controller that generated the log entry. When you use controller teaming, this ID enables you to identify which controller in the team generated the alert. Configuring the support log queue size The default queue size is 100 lines. Procedure 1. On the Configurations screen in the System tab, select the com.hp.sdn.adm.log.impl.logmanager component. 2. Click Modify. 52 Aruba VAN SDN Controller 2.8 Administrator Guide

53 The Modify System Configuration dialog box is displayed for the com.hp.sdn.adm.log.impl.logmanager component. 3. Change the value for the max.display.rows key. 4. Click Apply. Figure 18: Configurations screen with LogManager component keys Exporting the support logs The Export operation: 1. Gathers the set of support log file data from the controller, or in a team environment, all active controllers in the team, and stores the data as a single compressed archive file:sdn-all-logs.zip 2. Downloads the archive file from the controller to the default download directory specified by your browser. For example, in Ubuntu installations, this is usually the Downloads directory. 3. Click Export. The following menu appears in the lower-left corner of the controller console: Figure 19: Completion of the export operation 4. When the download completes, you can either resume interaction with the controller or examine the log by selecting an item from the menu, such as: Open a window showing the new log zip file. Set the default operation to always open the directory containing the log zip file. Show the log zip file in the default directory for receiving downloads. The actions resulting from these choices depend on the browser and operating system, not on the controller. Packet listeners The controller applications (and SDN applications) register packet listeners with the controller. The order of processing an incoming packet is determined by the roles (Advisor, then Director, then Observer), and then altitudes within a role (in decreasing value, with 0 the lowest altitude). An incoming packet (PacketListenerRole) is Chapter 3 Using the SDN controller UI 53

54 wrapped in a Message Context (which also holds a Packet-Out reply) which is passed to each packet listener in turn. Packet listeners display details The packet listeners screen displays the packet listeners that are currently running on the controller. Figure 20: Selecting the Packet listeners screen Screen component Refresh PacketListener Role Altitude Average (ms) Description Refreshes the information on the screen. The PacketListener Role is one of the following:advisorexamines the incoming packet. Might add processing hints to the message context, but does not modify the packet out message.directorprocesses the packet. Might add actions or instructions to the packet-out message. Can instruct the controller to block the packet, or to send the packet out.observera passive observer who might examine the incoming packet and any packet-out response.packets are given to packet listeners with role of ADVISOR first, DIRECTOR second, and OBSERVER third. Every packet listener is guaranteed to see the packet-in message. Depending on the action taken by higher altitude Directors, a lower altitude Director might be too late to influence the packet processing. The weight or priority this packet listener should have relative to other packet listeners that have the same role. The controller gives packet listeners with higher numbers priority over packet listeners with lower numbers. The average time, in milliseconds, that the packet listener spent processing a packet. # Samples The number of packets processed by that packet listener since the packet listener registered. OpenFlow Monitor The SDN controller UI includes several screens providing information on OpenFlow enabled switches: OpenFlow Monitor on page 54 OpenFlow topology on page 58 OpenFlow Trace log on page 67 OpenFlow Classes on page Aruba VAN SDN Controller 2.8 Administrator Guide

55 When the controller is active in an OpenFlow domain, the OpenFlow Monitor enables tracking of switch traffic summaries, packet traffic per port, and applied flow rules for switches detected in the controller domain. For a graphical view of Data Path ID assignments to individual OpenFlow switches, see OpenFlow topology on page 58. OpenFlow Monitor screen details The main display lists the Data Path IDs and descriptive information for the active switches and the options for viewing traffic information. To view information about a specific device, click the Data Path ID for that device and then select one of these tabs for the view you want to display: Summary, Ports, Flows, Groups. Click Refresh to update the display for Topology changes, such as a newly discovered OpenFlow device or the loss of a device that has been disconnected. Figure 21: The Main OpenFlow Monitor screen Screen component Refresh Summary tab Ports tab Flows tab Groups tab Data Path ID Address Negotiated Version Description Updates the information displayed on the screen. Displays the Summary for data path view for the selected data path. Displays the Ports for data path view for the selected data path. Displays the Flows for data path view for the selected data path. Displays the Groups for data path view for the selected data path. Identifies a detected OpenFlow switch. The OpenFlow data path identification for each detected OpenFlow switch. This ID also appears in the representation of the switch in the OpenFlow Topology screen. Identifies the IP address associated with an OpenFlow data path instance. The version of OpenFlow in use with the corresponding data path. Table Continued Chapter 3 Using the SDN controller UI 55

56 Screen component Manufacturer H/W Version S/W Version Serial Number Description Manufacturer of the device. Hardware version of the device. Software version on the device. Serial number on the device. Summary for data path view Figure 22: Summary view for a specific OpenFlow device The OpenFlow Monitor > Summary view includes the following details related to the selected device: Manufacturer Hardware and software version Serial number and device description of the device Device identification (Data Path ID) and IP address TCP port on the device Negotiated OpenFlow version (latest OpenFlow version common to both the controller and the switch) OpenFlow table and buffer information OpenFlow capabilities on the device Ports for data path view The OpenFlow Monitor > Ports view includes information on the ports used for OpenFlow traffic on the selected device. Figure 23: Ports view for a specific OpenFlow device 56 Aruba VAN SDN Controller 2.8 Administrator Guide

57 Flows for data path view The OpenFlow Monitor > Flows view shows current flows on the selected OpenFlow device. For a given flow, traffic meeting the requirements specified in the "Matches" field is directed as specified in the corresponding "Actions/Instructions" field. Beginning with version 2.8 of the Aruba VAN SDN Controller, the VAN OpenFlow flow table presentation has been enhanced to group a datapath's flows into expandable rows for each flow table ID. Each flow table ID row displays a Flow Count and Table Name, and can be expanded to view detailed information for the flows installed to that table ID. Additionally, each individual flow can be further expanded to view detailed information for that flow. The OpenFlow flow table for a datapath can be viewed by navigating to OpenFlow Monitor, selecting a specific datapath, then selecting Flows. Figure 24: Flows view for a specific OpenFlow device The Table ID applies to OpenFlow 1.3 and later, but not to OpenFlow 1.0. Chapter 3 Using the SDN controller UI 57

58 Groups for data path view The OpenFlow Monitor > Groups view provides information on group actions, if any, defined for the device. The group actions can assign more specific forwarding actions. Figure 25: Groups view for a specific OpenFlow device OpenFlow topology The OpenFlow Topology screen displays a topology of discovered switches and end nodes in the controller domain. You can view and change the graphical view of the network, as well as compute the broadcast tree to avoid loops and broadcast storms. The shortest path is computed using a Dijkstra graph search algorithm. The OpenFlow topology screen: Displays a topology of discovered switches and end nodes. Identifies the ports discovered on a given switch. Interface name and OpenFlow numbers are displayed Identifies the shortest path between two nodes. Provides node identification options (such as MAC or IP address label). Provides a view of switch port identifiers, active flow rules, and a tool for testing flow rule options. Beginning with version 2.8 of the Aruba VAN SDN Controller, the VAN OpenFlow Topology page defaults to "Collapse All", which means that end-host nodes will be "collapsed" into the OpenFlow device to which they are attached. "Collapse All" can be toggled using the "Collapse All" option on the View pull-down menu (located at the top of the OpenFlow Topology page), or using the 'c' hotkey. In previous VAN releases, the OpenFlow Topology page defaulted to "non-collapsed". Also beginning with version 2.8 of the Aruba VAN SDN Controller, the VAN OpenFlow Topology page auto-refresh timer has been extended to 5 minutes. Auto-refresh can be enabled or disabled using the Auto Refresh option on the View pull-down menu (located at the top of the OpenFlow Topology page). Regardless of the auto-refresh setting, the topology can always be refreshed using the Reload button. In a topology where two or more controlled switches connect to the same uncontrolled switch, the controller will not learn the location of hosts directly connected to the uncontrolled switch. See also: Do not configure a looped topology in the network between the OpenFlow and non-openflow portions of your network unless you enable Spanning Tree Protocol on the non OpenFlow devices operating in the network. 58 Aruba VAN SDN Controller 2.8 Administrator Guide

59 Displaying the network Topology on page 59 Using keyboard shortcuts to change the display on page 59 Changing the topology display using the View menu Viewing the shortest path between two nodes Viewing flow details for selected nodes on page 66 Viewing details on packet selection criteria for a data flow on page 67 Displaying the network Topology The OpenFlow Topology screen includes the switches and end-nodes in the controller domain. Figure 26: Topology viewer The topology legend is show in the top right corner: Switch shown in light green Collapsed Switch shown in dark green End Host shown in orange Using keyboard shortcuts to change the display Use the question mark Chapter 3 Using the SDN controller UI 59

60 To use the keyboard shortcut keys you must first click somewhere in the topology view to bring it into focus and then you can select a shortcut key. An outline around the topology indicates it is in focus. Figure 27: Keyboard shortcuts See also: Changing switch and host node labeling Using the mouse to change the topology display on page 61 Viewing node tooltips Changing switch and host node labeling You can change how nodes are labeled in the topology using keyboard shortcuts. To turn on or off ALL node labels, enter the keyboard shortcut L. To change the host node labeling in the topology, enter the keyboard shortcut H and the display will cycle through the different node labels each time you enter H. Host end-nodes can be labelled with one of the following: IP Address (default) MAC Address No Label 60 Aruba VAN SDN Controller 2.8 Administrator Guide

61 To change the switch node labeling in the topology, enter the keyboard shortcut N and the display will cycle through the different switch labels each time you enter N. Switches can be labelled with one of the following: System name (default, if the switch does not contain a system name then IP address is shown instead) IP address DatapathId No label For example, to change the default display showing System name labels to show the IP addresses of the switch nodes, click anywhere in the topology display, then press N. The switch IP addresses appear as labels in the topology diagram: Figure 28: Switch IP address labeling Press N again to display the switch datapath IDs as labels in the topology diagram: Figure 29: Switch datapath IDs as labels Press N again to display the unlabeled switch view. And press N again to return to the System Name switch labels. Using the mouse to change the topology display Zoom in or out in the topology by using the scroll wheel on the mouse. To drag the topology to a desired location, place the cursor in the topology and hold the mouse button down while dragging to move the topology. Select or deselect a switch or end-node host by clicking the node. Chapter 3 Using the SDN controller UI 61

62 Viewing node tooltips You can view node tooltips by hovering the mouse-over a node in the topology. Or you can press O to toggle on and off tooltips. Mouse-over the switch to display datapath information. Mouse over the host to display end-node information. 62 Aruba VAN SDN Controller 2.8 Administrator Guide

63 Changing the topology display using the View menu You can use the View menu to change the topology display. Figure 30: Topology View Menu See also: Using Search Viewing port labels on switches Viewing details on page 65 Using tools on page 65 Using pin, Collapse All, Auto Refresh and Reload on page 65 Using Search You can search the topology based on various criteria by using one of the following methods: Search using View > Search. Or press the F shortcut key to open the Search dialog box. This search is based on any one of the criteria Switch IP, Datapath ID, Host IP or Host MAC. Enter the search criteria in the Search (regex) box and click Search. This search is across all of the text including Switch IP address, Datapath ID, System name, End host IP address and MAC address. Search using the Search menu: Procedure 1. Select View > Search. Or press the F shortcut key to open the Search dialog box. The Search dialog box is displayed. When the Search dialog box is opened, if one or more nodes are collapsed or highlight path is enabled, all will be cleared during the search. After the search dialog is closed, the state of collapse and highlighting will be returned. 2. From the drop down list, select one of the search criteria Switch IP, Datapath ID, Host IP, or Host MAC, and then enter the search value. Chapter 3 Using the SDN controller UI 63

64 3. Click Search. If any match is found, the border changes to green. If no match is found, the border changes to red. 4. To close the dialog box, click Close or click Reset to clear the search value and reset the topology view. Search using Search (regex): 1. Enter the value you want to search on in the Search(regex) field located in the top right of the topology view. You can enter a regular expression for more complex searches. For an exact match, $ should be appended at the end. For example, if there are IP addresses like , , , , etc. and if you want to search for only , you should put the search string as $. 2. Click Search or press Enter. Viewing port labels on switches You can view port labels on the links between switches and between switches and end nodes. Port labels can be interface name or OpenFlow numbers. Select View > Ports to display port labels on switches. Press the P shortcut key to toggle between displaying OpenFlow port number or port name. Toggling only works when the Ports menu option is selected. 64 Aruba VAN SDN Controller 2.8 Administrator Guide

65 Viewing details You can view details for a switch by selecting View > Details. For more information, see Viewing flow details for selected nodes on page 66. Using tools After specifying a source and destination data flow you can view details on the packet selection criteria by selecting View > Tools. For more information, see Viewing details on packet selection criteria for a data flow on page 67. Using pin, Collapse All, Auto Refresh and Reload Pin To pin or unpin the switches and end nodes, press X or select View > Pin All. When you enable Pin All nodes and Auto Refresh, if any topology updates occur (such as an end host moved or was removed), then Pin All is automatically removed to update the topology and then once the topology is updated, the topology is pinned back. However if you had any customized view then that may be changed during that time. If the number of nodes increases by more than 500 (both switch and end hosts), Pin All will be enabled and disabled automatically. Once node count comes down to less than 500, Pin All will be re-enabled. Collapse All Collapse the topology display to show only the number of end nodes connected to each switch, instead of showing all end nodes (the default) which can present a cluttered display where a large number of end nodes are connected to the OpenFlow switches. To collapse or expand end-nodes for a particular switch, double-click the selected switch. To collapse all end nodes, select View > Collapse All. Auto Refresh To automatically refresh the topology, select View > Auto Refresh. Reload To reload the whole topology, click the Reload button in the top right of the topology view. When the topology is reloaded, Highlight, collapse, Collapse All, selection, and node labels will be reset. Viewing the shortest path between two nodes You can view the shortest path between two nodes as follows: Procedure 1. Select the source node and click Src or press S. 2. Select the destination node and click Dst or press D. The controller displays the path between the two nodes as a line, see Viewing the shortest path between two nodes). Chapter 3 Using the SDN controller UI 65

66 Features like Collapse all, collapse a single node and highlight a particular node using Ctrl click are not allowed when a path is selected. Figure 31: Locating the shortest path between two nodes To exchange source and destination nodes, press A. To clear the source and destination flags as well as clearing the path, press Z. Follow Flow The Follow Flow option is enabled only when the controller is in pure OpenFow mode, where hybrid.mode is set to false. The Follow Flow option is disabled when the controller is in hybrid mode. When the controller is in pure OpenFlow mode, select Path > Follow Flow. Highlight flow The Highlight option is enabled only when a path is selected (either Shortest Path or in Follow Flow mode). Highlight path is cleared when you toggle between Shortest Path or Follow Flow. For example, with Shortest Path is enabled you select Highlight path, then you select Follow Flow, the Highlight path will be cleared and you have to select Highlight path again for Follow Flow. Select Path > Highlight. Viewing flow details for selected nodes The Switch Details window displays flow details. 66 Aruba VAN SDN Controller 2.8 Administrator Guide

67 Select a switch node and then select View > Details or press I to display the Switch Details screen. Figure 32: Flow details for the selected source-destination end nodes Viewing details on packet selection criteria for a data flow For a source-destination data flow you can view details on the packet selection criteria used. Select View > Tools to display the Packet Selection dialog box or press T. The display is read only. The Abstract Packet window displays selection criteria for packets moving between the Source-Destination node pair. MAC addresses and IP addresses are shown based on the source and destination nodes selected. Figure 33: Searching for flows for specific packet types OpenFlow Trace log This troubleshooting tool logs OpenFlow conversations captured in messages to and from the controller and the OpenFlow devices it manages. You can export the captured messages in the trace log to a CSV (Comma-Separated Values) file that can be opened by applications such as Excel that are designed to accommodate this file type. This enables you to create a filter to display only the messages from the specific data paths you want to examine. Chapter 3 Using the SDN controller UI 67

68 About the OpenFlow Trace log The number of events that can be held in the trace log is limited by system memory. For this reason, Hewlett Packard Enterprise recommends that you export to a remote storage location any trace log content you want to retain, and to clear the controller trace log whenever its content is not needed on the controller itself. See also: Starting, stopping, or clearing OpenFlow trace on page 69 Displaying trace event details Exporting the OpenFlow Trace log on page 70 Filtering the OpenFlow trace log in a CSV file Changing the OpenFlow trace interval OpenFlow Trace screen details Figure 34: Example of OpenFlow Trace Default Display Screen component Start trace icon Stop trace icon Clear trace screen icon magnifying glass icon Description Starts trace logging. In the default configuration, the trace stops after ten seconds have passed. (To change the trace interval, see Changing the OpenFlow trace interval.) Stops trace logging before the end of the configured trace interval.trace logging stops automatically at the end of the configured trace interval.multiple consecutive traces can be held in the trace log. To add additional trace results, start another trace. Clears (resets) the current trace log. To preserve the contents of the trace log before clearing it, see Exporting the OpenFlow Trace log on page 70. Displays details of the selected trace event. Table Continued 68 Aruba VAN SDN Controller 2.8 Administrator Guide

69 Screen component Export Time Event Description Copies the trace log into a CSV (comma-separated values) file. See see Exporting the OpenFlow Trace log on page 70. The time the message event was generated. The event type. For example: CkPt Indicates a check point in the trace log, such as the starting or stopping of a trace operation. Rx Tx Indicates an OpenFlow message received by the controller (from a datapath). Indicates an OpenFlow message sent from the controller (to a datapath). Data Path ID Message The Data Path ID of the data path associated with the event. The trace message. Starting, stopping, or clearing OpenFlow trace Use the buttons above the Time field to control trace operations (see OpenFlow Trace screen details). Displaying trace event details Procedure 1. Select the event you want to examine. Figure 35: Selecting an event in the OpenFlow Trace log 2. Click Chapter 3 Using the SDN controller UI 69

70 The Event Detail dialog box is displayed Figure 36: Displaying event details 3. To close the Event Detail window, click Close. Exporting the OpenFlow Trace log Exporting an OpenFlow Trace Log places the trace content in a CSV file that is stored in the default downloads folder specified in your web browser settings. For more information about CSV files, see RFC This section shows how to export and access OpenFlow Trace Log files using Google Chrome. You might experience different results than shown here, depending on your web browser and its configuration. Procedure 1. Click Export. This action places the trace log contents into a CSV file in the default downloads folder in the system on which the controller is running. Check your web browser for an indication that the file has been created. 2. To display and filter the CSV file content, see Filtering the OpenFlow trace log in a CSV file. 70 Aruba VAN SDN Controller 2.8 Administrator Guide

71 Filtering the OpenFlow trace log in a CSV file Procedure 1. Open the CSV file in the default folder. For example, using Google Chrome, open the menu adjacent to the file name (of-trace.csv) and select Show in folder. Figure 37: Accessing the stored CSV file 2. In the resulting folder listing, locate the of-trace.csv file and open it using an application, such as Microsoft Excel, that enables you to read the log messages and configure a filter. For example, to investigate the messages collected for data path : 3. Select the DPID (Data Path ID) column. Figure 38: DPID column Chapter 3 Using the SDN controller UI 71

72 4. Set the filter. Figure 39: Setting the filter 5. Apply the filter by checking the box for data path Figure 40: Applying the filter 72 Aruba VAN SDN Controller 2.8 Administrator Guide

73 In the resulting display, only the data filtered to data path 00:00:00:00:00:00:00:02 appears. Figure 41: Filtered trace log Changing the OpenFlow trace interval The default trace interval is ten seconds. Procedure 1. From the navigation menu, select Configurations. Then select the Basic tab. 2. Select the com.hp.sdn.ctl.of.impl.tracemanager component. 3. Click Modify. The Modify Basic Configuration dialog box is displayed for the com.hp.sdn.ctl.of.impl.tracemanager component. 4. Change the value for the record.duration key. 5. Click Apply to set the new time span for active trace recording, and return to the OpenFlow Trace screen. Chapter 3 Using the SDN controller UI 73

74 OpenFlow Classes The OpenFlow Classes screen shows the OpenFlow classes that applications have registered with the controller. About OpenFlow classes When multiple applications share the same resource the flow tables of OpenFlow switches how can their priorities relative to each other be determined and how can their actions be coordinated? If flow table modification priorities are directly coded into each application, applications can end up directly competing with other applications for the highest priorities, which can result in conflicts in general network traffic control and unintended results when you implement a solution that has multiple SDN applications attempting to act on the same packets. In addition, many environments make it difficult to trace the origin of flow modification requests installed in switches. The Aruba VAN SDN Controller uses OpenFlow classes to dynamically manage the priorities of the OpenFlow rules being deployed to the network, thus enabling applications to execute their business logic in a more orderly fashion. 1. For each class of flow modification message the application can send, the application must register an OpenFlow class with the controller. The OpenFlow class must specify the types of match fields, types of actions, and (optionally) the relative position (higher than or lower than) for this class with respect to other flow classes. 2. The controller adds a unique base cookie to be used with each future flow modification to be validated against this OpenFlow class, and assigns an actual priority for the OpenFlow class. This actual priority is based on the logical priorities of all of the OpenFlow classes of all the applications that are registered with the controller. 3. When the application sends a flow modification message, it must set the match and action to be the same fields as specified in the OpenFlow class and, instead of providing an actual priority, the application sets the logical priority as assigned by the flow class, and a cookie that is derived from the base cookie of the OpenFlow class. 4. Before sending the flow table modification message to the switch, the controller evaluates the requested flow modification against the registered OpenFlow classes and replaces the logical priority provided by the application with an actual priority. In addition to enabling the controller to manage priorities for multiple applications, OpenFlow classes enable the controller to validate flow modifications an application makes against a set of expected flow modification requests. This capability means that the behavior of an application must match the intent that the application disclosed when it registered with the controller: The flow match must contain exactly the fields and field types that were disclosed when the application registered with the controller. The controller validates field types but not field value. The action or instruction must fall into the category that was disclosed during registration.an action is classified into one of the following categories: FORWARD DROP PROCESS STEAL COPY The upper 16 bits of the flow modification cookie must match the upper 16 bits of the base cookie that was issued during registration. 74 Aruba VAN SDN Controller 2.8 Administrator Guide

75 OpenFlow classes screen details The OpenFlow Classes screen displays the OpenFlow classes that are currently registered with the controller. Figure 42: Example of OpenFlow classes screen Screen component Refresh Flow Class ID Priority Cookie Match Fields Actions Description Description Refreshes the list. The symbolic name for the flow class. The prefix identifies the application that registered the class; the suffix uniquely identifies the class. The actual priority the controller assigns to flows of this class. The base value of the cookie assigned to this OpenFlow class. The application that registered this class must use this base cookie when constructing flows that belong to this class. The types of match fields that are expected to be specified in flows that belong to this class. The general category of the action or instruction a flow that belongs to this class is expected to include. For a list of categories, see About OpenFlow classes on page 74. Short description of what the OpenFlow class does. The application describes the OpenFlow class when it registers the class with the controller. Controller enforcement levels for OpenFlow classes The following table lists the enforcement levels that the controller can use for applications that send flows to switches. Chapter 3 Using the SDN controller UI 75

76 Enforcement level none weak strict Description The controller does not manage flow modification priorities or validate flow modification requests: Applications that do not register OpenFlow classes with the controller are permitted to send flow modifications to switches. The controller does not validate flow modifications, even for applications that register OpenFlow classes with the controller. The controller does not replace logical priorities with actual priorities for flow modification requests from any applications. (Default) The controller manages flow modification priorities and validates flow modification requests for applications that register OpenFlow classes: Applications that do not register OpenFlow classes with the controller are permitted to send flow modifications to switches. The controller validates flow modifications from registered applications against the OpenFlow classes that are registered. The controller replaces logical priorities with actual priorities for registered applications only. The controller manages all flow modification priorities and validates all flow modification requests: Applications that do not register OpenFlow classes with the controller are not permitted to send flow modifications to switches. The controller validates all flow modifications against the OpenFlow classes that are registered. The controller replaces logical priorities with actual priorities for all applications. Changing the enforcement levels for OpenFlow classes To change the enforcement level the controller applies to applications sending flows to switches, change the value for the flow.mod.enforcement key of the com.hp.sdn.ctl.of.impl.controllermanager component. Procedure 1. From the navigation menu, select Configurations. Then select the Basic tab. 2. Select the com.hp.sdn.ctl.of.impl.controllermanager component. 3. Click Modify. 4. The Modify Basic Configuration dialog box is displayed for the com.hp.sdn.ctl.of.impl.controllermanager component. 5. Change the value for the flow.mod.enforcement key. 6. Click Apply. 76 Aruba VAN SDN Controller 2.8 Administrator Guide

77 For information about the enforcement levels the controller can apply, see Controller enforcement levels for OpenFlow classes on page 75. Figure 43: Configurations screen with Controller Manager component keys Chapter 3 Using the SDN controller UI 77

78 Chapter 4 Hybrid mode for controlling packet forwarding Overview The hybrid mode setting determines which packet-forwarding decisions are made by controlled OpenFlow switches and which of these decisions are made by the controller itself. If hybrid mode is enabled (the default setting), the controller delegates normal packet forwarding to the controlled switches, but overrides these switches for non-standard packet-forwarding decisions required by installed applications for specific packet types. In this mode the controller relies on the controlled switches to resolve loops and determine forwarding paths by using traditional networking mechanisms (such as STP). If hybrid mode is disabled, the controller makes the forwarding decisions for all packets in the OpenFlowcontrolled network. In this state, the controller resolves network loops and determines forwarding paths. Managing hybrid mode includes the following: Viewing and changing the hybrid mode configuration Coordinating controller hybrid mode and OpenFlow switch settings on page 80 In all cases, the controller only monitors or directs packets within OpenFlow instances. The controller cannot direct or monitor packets outside of OpenFlow instances. For information on supported network switches, see the Aruba VAN SDN Controller and Applications Support Matrix. Learning more about hybrid mode For more on hybrid mode as it relates to OpenFlow, see the latest OpenFlow Switch Specification on the Open Networking Foundation website. For a list of Hewlett Packard Enterprise switches that support OpenFlow operation, see the latest edition of the Aruba VAN SDN Controller and Applications Support Matrix. Viewing and changing the hybrid mode configuration To view or change the hybrid mode setting: 78 Aruba VAN SDN Controller 2.8 Administrator Guide

79 Procedure 1. In the Controller UI, from the navigation menu, select Configurations. Then select the Basic tab. 2. Select the com.hp.sdn.ctl.of.impl.controllermanager component. Click to show the configurable keys for this component and view the current value for hybrid.mode. Figure 44: Open the Controller Manager component 3. Continue with the following steps if you want to change the setting. 4. Click Modify. The Modify Basic Configuration dialog box is displayed for the com.hp.sdn.ctl.of.impl.controllermanager component. Figure 45: Select the hybrid.mode field Chapter 4 Hybrid mode for controlling packet forwarding 79

80 5. Set hybrid.mode to one of the following: a. true: (the default): enables hybrid mode. The controller makes packet-forwarding decisions only as required by installed applications. b. false: disables hybrid mode. The controller makes all forwarding decisions. (Release 2.0 of the Aruba VAN SDN Controller operates only in this mode pure OpenFlow mode). 6. Restart the controller. In a controller team environment, restart all controllers in the team. In a controller team environment, a configuration change on one controller typically propagates to the other controllers on the team. However, to implement a hybrid mode configuration change, restart all the controllers in the team. Make certain that the change has propagated to all members in the team before restarting the controllers: a. Close any instance of the web interface in which the controller might be running. b. At the Linux command prompt (sudo), restart the controller: ~$ sudo service sdnc restart You can also use the REST API to set or reset hybrid mode. See the "configs REST API" section in the Aruba VAN SDN Controller REST API Reference. Coordinating controller hybrid mode and OpenFlow switch settings Supporting hybrid mode on OpenFlow switches The OpenFlow configuration on individual Hewlett Packard Enterprise switches must support the controller hybrid mode setting. The following table shows the correspondence between the hybrid mode configuration on the controller and the per-instance passive/active configuration on Hewlett Packard Enterprise OpenFlow switches. Table 3: Hybrid mode support on ProVision switches Hybrid Mode Settings Enabled (true) Disabled (false) ProVision OpenFlow Instance Configuration passive active For more information on the specific switch, how to configure passive/active mode, and how these switches behave if they lose their control-plane connection to the controller, see the OpenFlow documentation. For a list of switches that are supported in Hybrid and pure OpenFlow mode, see Aruba VAN SDN Controller and Applications Support Matrix. Configuring controller settings to support hybrid mode Network-related settings on the controller must agree with the controlled switches. Failure to achieve agreement between the controller s network-related settings and the settings in the controlled switches may result in unpredictable network behavior. The following table lists the specific network-related controller settings that should agree with managed switches. 80 Aruba VAN SDN Controller 2.8 Administrator Guide

81 For information on limitations in OpenFlow table support, see the Aruba VAN SDN Controller and Applications Support Matrix. Table 4: Controller settings to support hybrid mode Controller Configurations Component Key Comments com.hp.sdn.ctl.of.impl.controllermanager hybrid.mode Set to true or false. com.hp.sdn.disco.of.link.impl.openflowlinkdiscove rycomponent com.hp.sdn.disco.of.node.impl.ofarpdiscoverycompo nent com.hp.sdn.disco.of.node.impl.ofdhcpdiscoverycomp onent age.multihop.l inks arp.age dhcp.age Set this value to the refresh rate for multihop links. A faster refresh rate will introduce more linkdiscovery packets into the network. A slower refresh rate will respond more slowly to a topology change. To support ARPbased host discovery, change this setting in the controller to be greater than or equal to the ip arp-age setting on controlled switches. To support DHCPbased host discovery, change this setting in the controller to be greater than or equal to the timeout value of the DHCP server(s) on your network. To view or reconfigure any of the above controller configuration components: Procedure 1. In the controller UI, select Configurations. a. Select the Basic tab to view or modify the following components: com.hp.sdn.ctl.of.impl.controllermanager com.hp.sdn.disco.of.node.impl.ofarpdiscoverycomponent com.hp.sdn.disco.of.node.impl.ofdhcpdiscoverycomponent b. Select the Advanced tab to view or modify the following component: com.hp.sdn.disco.of.link.impl.openflowlinkdiscoverycomponent 2. Click the Modify button. In the Modify Configuration dialog box you can view the current setting for each key for the component and make changes. 3. Click Apply to save the changes. For more information on using the Configurations screen, see Configurations screen details. Chapter 4 Hybrid mode for controlling packet forwarding 81

82 Limitations For information on limitations in OpenFlow table support, see the Aruba VAN SDN Controller and Applications Support Matrix. OpenFlow 1.0 is the default version of OpenFlow for Hewlett Packard Enterprise ProVision switches. OpenFlow does not allow the controller to optimize flow location in hardware tables. For concerns about line-rate data plane performance, configure all managed switches to use OpenFlow 1.3. Failure to properly configure the switch in this way may cause packet loss or other problems associated with high switch CPU utilization. Uncontrolled switches in an OpenFlow Hybrid network are not visible to or controlled by the Aruba VAN SDN Controller. Uncontrolled switches are either controlled by another controller (outside the team) or not controlled at all (traditional networking). Traffic by such switches is independently managed. The Aruba VAN SDN Controller Path Diagnostic Tool is useful only when hybrid mode is disabled. When hybrid mode is enabled, the controller does not monitor or direct all flows in the network. As a result, the path diagnostic tool (PathDiagnosticManager) does not have visibility into all flows on the network, and should not be used. Controller packet-forwarding when hybrid mode is disabled Figure 46: Controller operation with hybrid mode disabled When hybrid mode is disabled (set to "false"), the controller examines and directs the packets in all flows for the given OpenFlow instance. The controller forwarding decisions for flows in a given instance are based on the requirements of the installed applications. The forwarding decision is communicated to controlled switches through OpenFlow. In instances where the controller has not provided the switch with a rule for how to forward a packet type, the switch sends the packet to the controller and waits for the controller to provide forwarding instructions. 82 Aruba VAN SDN Controller 2.8 Administrator Guide

83 Hybrid mode is commonly disabled in networks that are either used for experimental OpenFlow work (such as developing a controller application) or for networks that are completely new and designed to be fully controlled by OpenFlow. Controller packet forwarding when hybrid mode is enabled Figure 47: Controller operation with hybrid mode enabled When hybrid mode is enabled (the default), the specific packet types for which the controller monitors and overrides switch forwarding rules depends on the applications installed and running in the controller. That is, the controller overrides normal packet forwarding rules in the OpenFlow switch with application-specific forwarding rules, such as: copying ARP request/reply and DHCP offer/ack packets to the controller so that it can discover end-hosts stealing BDDP packets to the controller so that it can discover inter-switch links changing the priority on Microsoft Lync packets to improve instant messaging speed monitoring DNS requests to detect dangerous end-host behavior Packets in flows that the controller does not examine or direct are forwarded through normal switching operations without controller intervention. Chapter 4 Hybrid mode for controlling packet forwarding 83

84 Hewlett Packard Enterprise recommends that hybrid mode be enabled when controlling traditional, established networks where applications-related traffic is responsible for only a subset of the overall traffic load on the network. Hybrid mode is commonly enabled in established networks where new applications are installed and running on the controller, creating a need to override normal switching behavior for specific flows. 84 Aruba VAN SDN Controller 2.8 Administrator Guide

85 Chapter 5 License Registration and Activation A license is required for the controller. In addition, SDN applications can require licenses that are separate from the license for the controller. Typically, you must have both a license for the controller and a license for each application. For Aruba SDN applications, you register the license, obtain the license key, and activate the license on the controller using the same methods you use to register and activate controller licenses. For information about obtaining license keys for an application, see the Administrator Guide for the application. For information on the different types of licenses and how to maintain license registration and activation after a controller software update, see License types, usage, and expiration. Overview of the license registration and activation process After you have downloaded and installed the controller software, as described in the Aruba VAN SDN Controller Installation Guide, you can begin the license registration and activation process. Evaluation licenses are available, for details see Using evaluation licenses.. The basic steps are: Procedure 1. Preparing for license registration on page 86: a. Prerequisites for license registration on page 86 b. Identifying the Install ID displayed in the controller UI on page Registering and activating a license on page 87 a. Registering your license and obtaining a license key using the My Networking portal b. Activating a license on the controller 3. Managing licenses on page Transferring licenses on page 94 a. Deactivating licenses to prepare for transfer b. Transferring licenses to a new platform c. Use new license keys to activate the licenses on the target controller. License types, usage, and expiration The following licenses are available for the Aruba VAN SDN Controller: Aruba VAN SDN Ctrl Base SW w/ 50 node E-LTU The base license for the controller. Aruba VAN SDN Ctrl 50 node E-LTU Provides an additional 50 node license. Aruba VAN SDN Ctrl HA E-LTU Enables the controller to form a team for high availability. The following guidelines apply: The number of team members for an Aruba VAN SDN Controller team is three. When forming a team, only one Aruba VAN SDN Controller base license is required, along with at least two High Availability licenses, all on the same Master controller. Once a team is formed, Add Nodes licenses can be added to the team leader for increased support. In addition, you must: Chapter 5 License Registration and Activation 85

86 Use non-previously licensed controller installations to form the team. Use a new hardware platform (or Virtual Machine) with a new installation of the Aruba VAN SDN Controller. Run the same software version on all controllers. Application Licenses Licenses for SDN applications. For more information, refer to the administrator guide for the specific application. Preparing for license registration Prerequisites for license registration Before beginning the license registration and activation process, you must do the following: Procedure 1. Obtain a Hewlett Packard Enterprise My Networking portal user account. 2. Obtain the order number or product registration ID, and address from your Aruba VAN SDN Controller license order confirmation. 3. Install the Aruba VAN SDN Controller software and have the controller running, as described in the Aruba VAN SDN Controller Installation Guide. Identifying the Install ID displayed in the controller UI Each controller installation generates a unique Install ID that is used for licensing activities. To view the Install ID using the UI, select Licences from the navigation menu. In the Licenses screen, the Install ID appears before the list licenses. To use curl commands and the REST APIs to complete this task, see curl commands on page 182. Figure 48: Default License GUI 86 Aruba VAN SDN Controller 2.8 Administrator Guide

87 Registering and activating a license Using your Install ID, you must now register your license on the My Networking portal. Doing this results in a license key, which enables you to activate the license on the controller. If you are registering licenses in addition to the base controller license, Hewlett Packard Enterprise recommends you do so in the following order: 1. Register the base controller license. 2. Register any Add Nodes licenses, and then activate the last license key generated. 3. Register any High Availability licenses, and then activate the last license key generated. 4. Register any application licenses you have acquired. Registering your license and obtaining a license key Prerequisites To register your license and obtain a license key: Procedure 1. Log on to the My Networking portal at 2. Select My Licenses. 3. In the Order number or Registration ID field, enter your order number or registration ID and then click Next. a. If you enter a registration ID, go to step 5. b. If you enter an order number, the field appears. 4. In the field, enter either the Ship to or Sold to address listed in your sales order confirmation, and then click Next. A license selection screen appears, as shown below. Figure 49: Selecting licenses 5. Select the license type, enter the quantity to be registered to your Install ID, and then click Next. Chapter 5 License Registration and Activation 87

88 For an Aruba VAN SDN Ctrl Base SW w/ 50 node E-LTU license, the quantity must be 1. For Aruba VAN SDN Ctrl 50 node E-LTU or Aruba VAN SDN Ctrl HA E-LTU licenses, quantity is the number of licenses to be installed with a single Install ID. For information on using this process for an application license, see the administrator guide for that application. The registration details screen appears, as shown below. Figure 50: Entering the install ID 6. In the Install ID field, enter your Install ID number. (See Identifying the Install ID displayed in the controller UI on page Optional: Enter a Friendly name and Customer notes for this license. 8. Click Next. The end user software license agreement screen appears. 9. To continue after reading the license agreement, select I accept all of the above terms, and then click Finish. 88 Aruba VAN SDN Controller 2.8 Administrator Guide

89 The confirmation screen appears, as shown below. Figure 51: Reviewing your registration 10. Review your license registration details, and record the License key listed. 11. Optional: To download the license key file, click Save as, and then save it to your local hard drive. 12. Optional: To the registration details: a. Enter one or more addresses, separated by a comma or semi-colon in the field provided. b. Optional: Enter Comments about this license. c. Click Send Optional: If you want to register additional licenses for this order: a. Click Register more for this order to return to the license selection screen shown in Transferring licenses to a new platform. b. Repeat steps 5 through 13 until you have registered all licenses. Viewing your license information Procedure 1. Log on to the My Networking portal at 2. Select My Licenses. 3. Click View Licenses to see a screen similar to the following: Chapter 5 License Registration and Activation 89

90 4. To view the information for the license you just loaded, click the Select button for that license. 90 Aruba VAN SDN Controller 2.8 Administrator Guide

91 You will then see a screen similar to the following: Figure 52: Viewing your license and other information 5. Record the license key in the above screen for use when you activate the license on the controller. Activating a license on the controller Using your license key, you must now activate a license on the controller, completing the license registration and activation process. Procedure 1. If your previous curl session has closed or timed out, re-enter the authentication command to obtain a new token. 2. Activate the license on the controller: curl [options] -H "X-Auth-Token:token" \ -d license_key \ Chapter 5 License Registration and Activation 91

92 a. Replace token with the token you obtained using the authentication command. b. Replace license_key with the key obtained in Registering your license and obtaining a license key. You can view the key by logging on to the My Network portal and selecting My Licenses, as shown in Viewing your license and other information. c. Replace controller_ip with your controller IP address. If you are installing a High Availability license, enter the IP address of the lead controller. The installed license information appears in JSON format, as shown below. See: example Installed license output { } "license" : { "install_id" : , "serial_no" : 13, "license_metric" : "HA Controller", "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 500, "license_type" : "PRODUCTION", "base_license" : false, "creation_date" : " T00:26: ", "activated_date" : " T00:26: ", "expiry_date" : " T00:26: ", "license_status" : "ACTIVE" } Adding and activating a license using the controller UI Use the following procedure to add and activate a license using the controller UI. Procedure 1. In the controller UI, from the navigation menu, select Licenses. 2. On the Licenses screen, enter the license key you acquired in Registering your license and obtaining a license key in the text box next to the Add button. 92 Aruba VAN SDN Controller 2.8 Administrator Guide

93 3. Enter the key in the field to enable the Add button. Figure 53: Enter the License Key 4. To activate the license, click the Add button. The active license is displayed in the table, below the Install ID, and the Add button is no longer available. Figure 54: Active License Displayed on License screen Activating a license using a script As an alternative to using the controller UI to activate the license, you can use a post install configuration script run locally on the controller. For information on other post install configuration options using the script, see the Aruba VAN SDN Controller Installation Guide chapter on post install configuration. The post install configuration Python script is run on the local controller machine. The script is /opt/sdn/ scripts/postinstall/config_sdn.py. You can run it interactively or you can use a configuration file to enter the inputs for activating the license. The default configuration file is /opt/sdn/scripts/ansible/ config_sdn.conf or you can create a custom configuration file for use with the script. To run the config_sdn.py script to activate the controller license: Procedure 1. To use the script, first ssh to the controller system. For example, ssh sdn@ , and then enter the ssh password. 2. You can run the script either interactively or with a configuration file as follows: a. If you are running the script interactively without a configuration file, enter the option for add license on the command line: Chapter 5 License Registration and Activation 93

94 python config_sdn.py -L You are prompted to enter the license key. b. If you are using the configuration file to enter inputs for the script, edit the config_sdn.conf file or create a custom configuration file. c. In the [general] section enter the controller IP, user name and (optionally) the password. If you do not enter a password in the configuration file, you are prompted to enter the it when you run the script. d. In the [scripts] section, set addlicense=true. e. In the [addlicense] section, enter the license key, and then run the script with the command: python config_sdn.py -f../ansible/config_sdn.conf (or whatever the custom configuration file name is). 3. Respond to any prompts as the script runs. Managing licenses Transferring licenses You can transfer a license from one controller to another. To do so, you must first deactivate all licenses from the controller. Keeping a license on one controller while transferring one or more other licenses from the same controller to another controller is not permitted. When upgrading, no special effort is required to preserve the licenses. Note that the license transfer mechanism is only required when you want to switch the controller currently running hardware. You must install the controller on the new hardware and transfer the licenses to that new hardware before retiring the old hardware. Prerequisites for transferring licenses Before you transfer licenses, you must first: Procedure 1. Deactivate all licenses, as described in Deactivating licenses to prepare for transfer. 2. Obtain an Install ID for each destination controller, as described in Identifying the Install ID displayed in the controller UI. Deactivating licenses to prepare for transfer When you deactivate a license to prepare for transfer, the controller generates an Uninstall Key for that license, which you will need when you transfer the license. Be prepared to record the Uninstall Key for each license you deactivate. The Uninstall Key is a long text string. For example: AE2RCLT7CJMDI-MAGAQHS2NBTOB-6VM4QKEQ4HAEZ-3AY4QELRPG4AA-3EMHQELRPGAYQ 94 Aruba VAN SDN Controller 2.8 Administrator Guide

95 Procedure 1. In the controller UI, from the navigation menu, select Licenses. 2. Select the license to deactivate to prepare for transfer. Figure 55: License screen with Deactivate button highlighted 3. Click Deactivate. 4. Click OK when the deactivation prompt appears: You will see an Uninstall key displayed for that license. Copy the Uninstall key for that license to the clipboard by clicking Copy Uninstall Key. Figure 56: License Deactivation Prompt 5. Repeat the preceding steps for each of the remaining licenses on the controller. Transferring licenses to a new platform After you have deactivated all of the licenses for a controller, you can transfer them to another controller. Procedure 1. Log on to the My Networking portal at 2. From the My Licenses section, select Transfer licenses to a new platform. 3. In the Search field, enter the Install ID for the controller from which you deactivated the license, and then click Search. Chapter 5 License Registration and Activation 95

96 The transfer license screen displays a list of associated licenses, as shown below. Figure 57: Selecting licenses to transfer 4. Click the Select icon next to the license to be transferred. 96 Aruba VAN SDN Controller 2.8 Administrator Guide

97 The license details screen appears, as shown below. Figure 58: Reviewing details before transfer 5. Verify that this is the license you want to transfer, and then click Next. The target Install ID screen is displayed as shown below. Figure 59: Target Install ID screen Chapter 5 License Registration and Activation 97

98 In the Target Install ID field, enter the Install ID of the controller to which you want to transfer the license and then click Next. Figure 60: Entering target install and uninstall IDs 6. In the screen above, do the following: a. In each Uninstall field, enter a license uninstall key. (For more on acquiring uninstall keys, see Deactivating licenses to prepare for transfer.) For the transfer process to succeed, enter an Uninstall value for every registered license. b. Click the Transfer button in the lower-right corner of the screen. 98 Aruba VAN SDN Controller 2.8 Administrator Guide

99 New license registration information displays on the license transfer confirmation screen and license details screen, as shown below. Figure 61: Viewing license transfer confirmation and details screens 7. Review the confirmation screen details. 8. For each license you are transferring, record the new license key so that it will be available when you add and activate the license on the new controller. 9. Optional: To transferred license details: a. Enter one or more addresses, separated by a comma or semi-colon in the field provided. b. Optional: Enter Comments about this license transfer. c. Click Send . Chapter 5 License Registration and Activation 99

100 The license screen displays the status of the original licenses as Transferred, and the new Install IDs as Active, as shown below. Figure 62: Review transferred license status screens To register the transferred licenses on the new controller, see Activating a license on the controller. Using evaluation licenses Procedure 1. Install the Aruba VAN SDN Controller and install all the Aruba SDN applications you would like to evaluate. If you are using the Hewlett Packard Enterprise SDN App Store, install the Trial Mode SDN applications. 2. Go to the My Networking portal (MNP) at 3. Under Licenses, select Register License. 4. From the menu on the right of the screen, select SDN Evaluation Licenses. 5. Enter the Aruba VAN SDN Controller installation identifier (install-id). The My Networking portal generates every evaluation license possible for that install-id. 6. Apply the relevant evaluation licenses to the controller and applications. 100 Aruba VAN SDN Controller 2.8 Administrator Guide

101 Chapter 6 Configuring for High Availability Standalone controller operation provides management for the OpenFlow switches in a network. However, it does not provide high availability (HA), with the result that a controller failure leaves the network in an unmanaged state. Configuring a team of controllers and one or more corresponding controller regions creates a high availability network with failover capability, resulting in a continuously managed network in the event that a controller in the team goes down. You can view your team configuration using the UI, see Viewing your team configuration using the UI. High Availability best practices Ensure the team and region configuration meets all of the requirements for teaming. For details, see Requirements for controller teams on page 102. Ensure that IP routing configuration in the controller domain enables the controller team IP address to be reached from all areas of the domain. If any application installed on the controller uses the Cassandra database, run the Cassandra nodetool repair command every 10 days. For details, see Manually synchronizing Cassandra database nodes using nodetool repair utility. Before you create the team, configure NTP such that all the controllers that will be in the team use the same local NTP server. Using the same NTP server for all controllers helps to ensure that the controller clocks remain synchronized. Keeping the system clocks synchronized is especially important for applications that use the Cassandra database. For details, see Configuring controllers to use the same local NTP servers. Either use the same Keystone server for authentication for all three controllers in the team, or use a local Keystone server for each controller in the team. For details on security, see Security on page 113. To use TLS connections for communications between the switch and the built-in OpenFlow controller module of the Aruba VAN SDN Controller, Hewlett Packard Enterprise recommends that all controller and device certificates be signed by the same CA. For information about configuring TLS on a switch, see the documentation for the switch. If you are configuring web proxy server settings, ensure that you include the team IP address and the IP addresses for all three controllers in the team in the /etc/init/sdnc.conf file. After editing the sdnc.conf file all three controllers will need to be restarted. For details, see Obtaining applications from the Hewlett Packard Enterprise SDN App Store on page 32. About teaming for High Availability Each controller belonging to a team is a team member. To centralize team management and control, one controller is elected by the team as the team leader. Teaming is configured on one controller and is automatically propagated to the other controllers in the team, regardless of which controller becomes the team leader. After a team is configured, any configuration changes will propagate to each controller. If the team leader goes down, another active controller becomes the team leader. If a team leader that failed recovers and rejoins the team, it rejoins the team as a team member and does not resume team leadership. Each controller in the team has its own IP address, which is the IP address of the machine on which the controller is installed. In addition, the administrator configures a separate IP address called the team IP address to represent the team as a whole. The team IP address is active on the team leader. If the current team leader goes down, the failover process includes keeping the team IP address active on the new team leader. For the controllers in a team to remain active, they must be part of the team quorum. To be part of a team quorum, a controller must be connected to at least one other team member that has a status of active or Chapter 6 Configuring for High Availability 101

102 initializing. If one controller in the team goes offline, controller operations can continue. However, if two controllers in a team fail, the third controller does not operate as a standalone controller. Instead, the third controller loses its membership in the team quorum, and the controller status is changed to suspended. A region groups devices together with their controllers. A region must have three controllers which must be specified in priority order for all devices within the region (master, primary slave, secondary slave). Putting the region configuration in place for a controller team ensures seamless failover and failback among the configured controllers for the specified network devices in a region. When a controller experiences a fault, the region configuration ensures that a slave controller immediately assumes the master role over the groups of network devices for which the failed controller was master. Once the failed controller recovers and rejoins the team, the rejoining controller takes back the role for which it was configured with respect to the network devices. For details on failover and failback, see Failover behavior within a region on page 205 and Failback behavior within a region on page 207. Devices in a region can be expressed as a list of individual IPv4 addresses, a list of IPv4 ranges, or a combination of both. Devices included in a region can connect to the region s controllers. OpenFlow 1.3 devices must be configured with the IPs of all three controllers in a team. This allows one of those controllers to assert itself as the master of a given device. The device then automatically assigns a role of slave to the other two configured controllers. This ensures the master knows of all the events happening on the device while the slaves are kept up to date on a subset of events. Applications are stopped when there is a change in the teaming status for a given controller. For example: When the controller transitions from stand-alone to a member of a team, all applications are stopped prior to the creation of the team, and then restarted after the team is formed. When a teamed controller detects that it is no longer part of the quorum, all applications are stopped until the controller detects it has rejoined the quorum. The team status can be: active, unreachable, or unknown (for details, see Team status on page 103). You can view team status from the Team screen in the controller UI, see Viewing team status on page 106. The controller status can be: initializing, active, suspended, or unreachable (for details, see Controller status on page 103. You can view controller status from the Team screen in the controller UI, see Viewing team configuration and controller status. You can view region configuration from the Team screen in the controller UI, see Viewing region configuration. Requirements for controller teams Controller team operation requires the High Availability Add Controller license (HP VAN SDN Ctrl HA E-LTU). A team must consist of three controllers. A controller can be part of one team only. Each controller must be able to communicate with all controllers on TCP ports 5700, 7001, and All controllers in a team must be running the same software version. The administrator must create the teams and regions. When controllers are in a team, the use of regions is required. A team requires one IP address for each controller, plus one IP address that represents the team as a whole. Only OpenFlow 1.3 network devices are supported: Each network device must be configured to connect to the IP addresses of all the controllers in the team, but not to the team IP address. For the controller to recognize the device, when you configure regions on the controller, you must also explicitly add the network device to a region. After a team is created, controllers adopt the data of one of the members of the team. Hewlett Packard Enterprise recommends that you minimize the amount of data to be copied by configuring teams soon after the controllers are installed and started. 102 Aruba VAN SDN Controller 2.8 Administrator Guide

103 Requirements for configuration, licensing, or application changes to controller teams All controllers in a teamed environment must be active before you can make configuration, licensing, or application changes, or changes to regions. Changes attempted when a team member is initializing or disconnected are not guaranteed to be consistent. If you attempt to make a change while a controller is not in an active state: If you are using the user interface, the change is blocked. If you are using using the REST API, the change is blocked and the controller returns a Forbidden response (error code 403) and an UnsafeConfigurationException exception. Team status You can view your team status from the top banner of the controller UI, see Viewing team status on page 106. The team states are: activeall 3 controllers are actively operating. A prerequisite for this state is that all controllers are able to communicate with each other. unreachableany single controller of a team is not able to communicate with the rest of the team members. This status occurs because of either of the following possible situations: The sdnc service stopped working or a controller is in the process of rebooting. A network partition occurred and a controller in a team has become separated from the other team members. unknowna team status cannot be determined because of either of the following possible situations: A communication failure with a REST service component occurred. A network failure caused the controller to be suspended. Controller status Aruba VAN SDN Controller groups components into categories and those component groups are initialized in order as the startup sequencer moves through the stages in the startup sequence. During the operational phase, the OpenFlow port is opened. Core services are the first to be initialized and they are always active. System Information Service is part of the core services and thus the controller status cannot be determined or reported until the core services complete the initialization phase. For more information, see Error log for team configuration on page 200, Team alias node on page 202, Failover behavior within a region on page 205, Failback behavior within a region on page 207. The controller states are: initializing The sequencer completed the startup sequence through the team stage and the controller is part of the team quorum (connected to at least one other active controller in the team), but has not yet deployed and initialized the operational group services. At this point, the OpenFlow port is not open. active The sequencer completed all stages of the startup sequence and the OpenFlow port is open. If the controller is a member of a team, it is part of the team quorum (connected to at least one other team member that has started its teaming services). suspended The sequencer completed all stages in the suspend sequence. The sequencer initiates the suspend sequence when a monitored core service reports an unhealthy status or a teamed controller loses its membership in the Chapter 6 Configuring for High Availability 103

104 team quorum. Core services are started. Teaming services are started but are waiting until the controller can become a member of a team quorum. The OpenFlow port is closed. unreachable A controller sees a remote controller as unreachable if the connection to the remote controller is broken. A controller never sees itself as unreachable. If an application reports an unhealthy status, an alert is generated but the controller remains in the active state. If two controllers in a team fail, the third controller does not operate as a standalone controller. Instead, the third controller loses its membership in the team quorum, and the sequencer initiates the suspend sequence. You can view your controller status from the top section of the Team screen in the UI, see Viewing team configuration and controller status. Manually synchronizing Cassandra database notes using the nodetool repair utility The Cassandra nodetool repair utility corrects inconsistencies among instances of the Cassandra database such that all nodes have the same and current data. It is recommended that the nodetool repair utility be run on all cluster nodes periodically once a week. Beginning with version 2.8 of the Aruba VAN SDN Controller, this has been automated. As part of HP VAN SDN Controller install, a script (/opt/sdn/admin/casmaint.sh) is installed and automatically scheduled to execute every Sunday at 2:00 AM by default. The script works only on the current Team Leader of a cluster, although it is executed on all members of the cluster as part of the cron job. The script runs nodetool repair on all active Cassandra nodes that are part of the cluster. The script logs the results of it's execution in a log file located at /var/log/sdn/cassandra/maint.log. Changing the default script execution time The Cassandra maintenance script runs every Sunday at 2:00 AM by default. The script is scheduled via cron. An example of crontab format with commented fields is shown next. This example shows the Cassandra maintenance script scheduled at 2:00 AM every Sunday: # Minut e # (0-59 ) Hou r (0-2 3) Day of Month Month (1-31) (1-12 or Jan-Dec) Day of Week (0-6 or Sun-Sat) Command 0 2 * * 0 /opt/sdn/admin/casmaint.sh -r To change the default schedule, log in to the controllers as user sdn and execute the following commands: Save the current cron tab settings to a temp file: crontab -u sdn -l > /tmp/cron.tmp Edit the saved temp file (/tmp/cron.tmp) and change the schedule by editing the line that has casmaint.sh Update the crontab by running: crontab -u sdn /tmp/cron.tmp Guidelines for running the nodetool repair utility Run the utility on each server in the controller team. Schedule regular repair operations for one server in the controller team at a time. Schedule regular repair operations once every 10 days. Disk activity increases during repair operations, so schedule repair operations during low-usage hours. 104 Aruba VAN SDN Controller 2.8 Administrator Guide

105 Running the Cassandra nodetool repair command Prerequisites The commands in this procedure are run from the command prompt on the Linux system on which the controller is installed. Procedure 1. Confirm that the Cassandra database is online: ~$ /opt/sdn/cassandra/bin/nodetool status If you see the following message, the Cassandra database is not online: Failed to connect to ' :7199': Connection refused 2. If the Cassandra database is not online, you must restart the controller to restart the Cassandra instance on that controller: a. Close any instance of the web interface connected to the controller to be restarted. b. Restart the controller: ~$ sudo service sdnc restart 3. Repair and synchronize the database by entering the following command on each controller: ~$ /opt/sdn/cassandra/bin/nodetool repair You can perform this step while the system, controller, and applications are running. Configuring controllers to use the same local NTP servers Using the same NTP servers for all controllers you plan to include in a team helps to ensure that the controller clocks remain synchronized. Keeping the system clocks synchronized is especially important for applications that use the Cassandra database. The controllers in the team must be configured with one or more NTP servers. Before you create the controller team, configure all the controllers you plan to include in a team to use the same local NTP servers. You can specify an NTP server for each controller using the controller UI (see Modifying NTP server or date and time) or using the manual steps below. Obtain the IP addresses of the local NTP servers for your site and ensure that these local NTP servers are the only NTP servers configured for each controller you plan to include in the team. Procedure 1. Log in to the Linux system on which the controller is installed. 2. Verify that NTP is configured on the system: ~$ ntpdc -c peers If a list of servers is displayed, one or more NTP servers are configured on the system. 3. Edit the /etc/ntp.conf file to remove the entries for servers other than the local NTP server and replace those entries with an entry for the local NTP server: a. Remove (or enclose in comments by prepending the # character) all lines that start with the word server. b. Add the following line for each local NTP server, where local-ntp-ip is the IP address of that local NTP server: server local-ntp-ip iburst c. Save and close the file. 4. Restart the NTP service: ~$ sudo service ntp restart 5. Verify that the local NTP servers are the only NTP servers configured on the system: Chapter 6 Configuring for High Availability 105

106 ~$ ntpdc -c peers NTP is configured correctly if the IP addresses for the local NTP servers are the only entries displayed. Viewing your team configuration using the UI You can view your team and region configuration from the SDN Controller UI s Team screen. To access the Team screen, click Team in the controller UI navigation pane. The Team screen is read-only and includes: Team status (top banner) Team configuration and controller status (top section) Region configuration (middle section) Device Owners (bottom section) Figure 63: Viewing your team status and configuration Viewing team status You can view your team status from the top banner of the controller UIscreen. The team status indicator refreshes dynamically to immediately notify you of important team status changes, such as when a 3-node team changes to a 2-node team. The team status banner displays one of the following team status messages: ACTIVE All 3 controllers are actively operating (for example, all controllers are able to communicate with each other) so a healthy team status message is displayed: UNREACHABLE Any single controller of a team is not able to communicate with the rest of the team members so a degraded status massage is displayed: occurs for either of the following reasons: Unreachable status 106 Aruba VAN SDN Controller 2.8 Administrator Guide

107 The sdnc service stopped working or a controller has been rebooted. A network partition occurred and a controller in a team has become separated from the other team members. UNKNOWN A team status cannot be determined and an unknown team status message is displayed: A communication failure with a REST service component occurred. A network failure caused the controller to be suspended. Viewing team configuration and controller status Unknown status occurs for either of the following reasons: From the Team screen, the team configuration and controller status (top section) displays the following fields: IP: IP address of the controller Role: Member or leader. A team can only have one leader and at most three controllers. Status: The status for each controller in the team, which can be one of the following (for details see Controller status on page 103): initializing active suspended unreachable Version: The build version of the controller software running on the controller. CDV: Core data version. This field is incremented every time the controller experiences a change in configuration. This field is used to determine which controller to synchronize with when a controller joins a cluster. CDV Timestamp: Date and time at which the controller experienced its last change in configuration. Viewing region configuration From the Team screen, the region configuration (middle section) displays the following fields: Name: Name of the region. Controller by Priority: First controller (in bold) is the master controller for the region. The master controller handles the flows and packet-ins. The following controllers (grayed out) are slaves. If the highest priority controller is unavailable, HA fails-over to the next highest priority device in the region configuration for that device. Region UID: Identifies a region; used by certain REST API commands. To view a region s details, select the desired region. In this example, Region123 is selected: The region s details include the following fields: Ranges: The configured ranges (IP ranges). : Devices: List of IP addresses one-by-one. One of these must be present, or both may be used. Chapter 6 Configuring for High Availability 107

108 Viewing devices, datapaths, and debug logs The Device Owners portion of the Team screen (bottom section) displays the following fields: Device: The device IP Owner Controller: The controller that is considered the current owner of the device (for example, the configured OpenFlow master) Region: The region that the device belongs to Datapath ID: Datapath identifier for the device. A device can have multiple datapaths. Datapath ready: Default flows have been pushed To view a device s debug log for support purposes, select the desired device. In this example, the debug log for device is shown: Methods for configuring HA teaming There are a number of different ways for you to configure HA teaming. For the first method (using a script), a configuration file is required, see Defining inputs for teaming in a configuration file on page 109. For details on viewing teaming information in the controller UI, see Viewing your team configuration using the UI. Chose the method that best meets your needs. Using a Python script from a controller to configure a team 108 Aruba VAN SDN Controller 2.8 Administrator Guide

109 Use a Python script executed locally on a controller to configure a team of three controllers. Typically the scripts are used to configure a single team. The script uses a configuration file to define the inputs for configuring the team. The script can be used to do the following: Create a team Disband a team Create region(s) Delete region(s) Add device to a region Remove device from a region Using curl commands for REST APIs to configure a team, see Team configuration using curl commands on page 196. Defining inputs for teaming in a configuration file The first step in configuring teaming using a Python script is to define the inputs for teaming in a configuration file. The default configuration file is build_team.conf. The file is on the controller system at /opt/sdn/scripts/ teaming/build_team.conf. If you cannot access the /opt/sdn path, you need to enter sudo su first in order to see this directory path. You can edit this file to provide the input for team configuration or you can create a custom configuration file with a different name but the same format and in the same directory. You can create any number of configuration files. The following is an example of a build_team.conf file. # build_team.conf User=<sdn_controller_username> Password=<sdn_controller_password> Sleep_Time=20 Team_IP=##.##.##.### IP1=##.##.##.### IP2=##.##.##.### IP3=##.##.##.### Controller_IPs=$IP1,$IP2,$IP3 #Controller_IPs=##.##.##.###,##.##.##.###,##.##.##.### Region1_Name=RED #Region1_Prioritized_Controller_IPs=##.##.##.###,##.##.##.###,##.##.##.### Region1_Prioritized_Controller_IPs=$IP1,$IP2,$IP3 Region1_Device_Ranges=##.##.##.##-##,##.##.##.##-##,##.##.##.#-# Region1_Device_IPs=##.##.##.### Region2_Name=BLUE #Region2_Prioritized_Controller_IPs=##.##.##.###,##.##.##.###,##.##.##.### Region2_Prioritized_Controller_IPs=$IP2,$IP3,$IP1 Region2_Device_Ranges=##.##.##.##-##,##.##.##.##-##,##.##.##.#-# Region2_Device_IPs=##.##.##.##,##.##.##.###,##.##.##.# Region3_Name=GREEN #Region3_Prioritized_Controller_IPs=##.##.##.###,##.##.##.###,##.##.##.### Region3_Prioritized_Controller_IPs=$IP3,$IP1,$IP2 Region3_Device_Ranges=##.##.##.##-##,##.##.##.##-##,##.##.##.#-# Region3_Device_IPs=##.##.##.##,##.##.##.##,##.##.##.# Chapter 6 Configuring for High Availability 109

110 Edit the configuration file (for example vi build_team.conf) or create a custom configuration file to include the following inputs about the team configuration. You cannot change the parameter name, such as User or Team_IP. Some entries are optional as noted in the following table. You can create multiple configuration files, each for use with a different team configuration. Then you can use the same script (but with a different configuration file) to create another team. Parameter User Password Sleep_Time Team_IP IP1, IP2, IP3 Controller_IPs Region1_name Value (optional) The default login user name is sdn. Enter a different login user name if a user name other than sdn has been set for the controllers. Or leave the user name blank in the configuration file if you want the user to be prompted to enter it when they run the script. All controllers in the team you are configuring must have the same user name and password. (optional) The default login password is skyline. Enter a different login password if a different one has been set for the controllers. Or leave the password blank if you want the user to be prompted to enter it when they run the script. All controllers in the team you are configuring must have the same user name and password. (required) The default is 30 seconds. You can change this value if you want to allow more time for the controller to propagate the changes to other controllers. You may need to do this if, for example, your network is slow. If you leave this blank, the default 30 seconds is used. (required) The IP address to be assigned to the team. The team IP address is different from the individual controller IP addresses. It is used as a virtual address for connecting to the team leader. Physical IP addresses for the three controllers in the team. IP1, IP2, and IP3 are optional because alternately you can enter the IP addresses under Controller_IPs instead of as IP1, IP2, and IP3.If you enter an IP address for IP1, IP2, and IP3 you can use the notation $IP1, $IP2, and $IP3 in the Controller_IPs and Region parameters that follow. When running the team configuration script, it will check to verify that the IP addresses are valid.for requirements on the IP address you enter for team configuration, see High Availability best practices on page 101. (required) The order of the controllers in the team from master to primary slave and secondary slave. You can either enter the full IP addresses separated by commas or you can use the notation $IPx where x is the number from the values you defined above. For example $IP1, $IP2, $IP3. (required if specifying regions) User-defined name for the first region. You can specify 1, 2, or 3 regions in a team, or you can have a configuration with no regions. Table Continued 110 Aruba VAN SDN Controller 2.8 Administrator Guide

111 Region1_Prioritized_Controller_IPs Region1_Device_Ranges Region1_Device_IPs Region2, Region3 (required if specifying regions) The prioritized order of the controllers in this region from first to last. You can either enter the full IP addresses separated by commas or you can use the notation $IPx where x is the number from the values you defined above. For example $IP1, $IP2, $IP3. (optional but either range or device IPs are required if defining devices) The ranges for devices that you want to be configured in Region1. You can either define devices by range or by individual IP address.for requirements on the device IPs you enter for team configuration, see High Availability best practices on page 101. (optional but either range or device IP is required if defining devices) The device IPs configured in Region1. You can either define devices by range or by individual IP address.the devices must be configured with the IPs of all three controllers in a team. (optional) Enter inputs for other regions you want to define. You can specify 1, 2, or 3 regions in a team, or no regions. If you don t want to define Region2 or Region3 then leave all items for that region blank. Using a Python script from a controller to configure a team You can configure teaming using a Python script (conf_team.py) and a configuration file on any active controller you plan to include in the team. For details on the teaming requirements, see High Availability best practices on page 101. From a controller, the script allows the following teaming configuration actions: create a team, create regions within a team, add devices to a region, disband a team, delete regions within a team, remove devices from a region. To use the script you must ssh to one of the controllers you plan to include in the team. For example, ssh sdn@ (then enter the ssh password). You do not need to ssh to the other two controllers in the team. The Python script conf_team.py is run from the controller you ssh into. Edit a configuration file on that same controller to define the inputs you want to use when running the script. The default configuration file is build_team.conf. The file is on the controller system at /opt/sdn/scripts/teaming/ build_team.conf. (The /opt/sdn path may not be visible to the ssh user. The user must be part of the sdn group to see the directory structure). You can edit this file or you can create a custom configuration file with a different name, but the same format and in the same directory. For details on using the configuration file, see Defining inputs for teaming in a configuration file on page 109. Once you have all the inputs you want to use for team configuration in the configuration file, run the script. By running the script, /opt/sdn/scripts/teaming/conf_team.py you can access all the team configuration actions. To run the script: Procedure 1. Issue the command to run the script: python conf_team.py [config file name] [-v] Chapter 6 Configuring for High Availability 111

112 The [config file name] default is build_team.conf. If you don t specify a configuration file, the script uses the build_team.conf file in the same directory as the script. If you have created a custom configuration file, enter that configuration file name. Note that -v is optional. If you want to run the script in debug mode, add -v at the end of the command. 2. If prompted, enter user and password. The three controllers must have the same user name and password. 3. Check for messages or errors and respond to any prompts. 4. Select X/x to exit the script. 112 Aruba VAN SDN Controller 2.8 Administrator Guide

113 Chapter 7 Security The Aruba VAN SDN Controller communicates with different components, both internal and external to the controller, via secure channels. This section documents these channels, their defaults, and how to configure them in a deployment environment. SDN Controller authentication The SDN Controller identifies itself via Public-Key Infrastructure (PKI) for its communication with external subsystems and other controllers. It uses a Java keystore and truststore to keep its private key and public key respectively. These keys can be used for confidential and trusted communication with clients and keystone. For REST APIs, the controller uses bearer token authentication to authenticate the client. The client must present a valid token via the X-Auth-Header to authenticate itself with the controller. Since this means of token authentication are bearer tokens, use PKI to ensure trusted communication with keystone and clients, and to avoid unauthorized use of tokens. Make sure that the certificates that you use for both keystone and the controller are part of a valid trust chain. Token authentication is discussed further under SDN Controller keystore and truststore locations and passwords on page 115. The controller ships with a self-signed certificate. Therefore, it is recommended that the self-signed certificate be replaced by a certificate signed by a reputable Certificate Authority (CA). If you choose to replace the self-signed certificates with CA signed equivalents, see Changing the default controller keystore and truststore to use CA signed certificates. Also, the default password for the keystore and truststore should be changed as well. Enable (2-way SSL) mutually trusted PKI communication to require both the controller and keystone to present valid certificates before starting the communication. Hewlett Packard Enterprise strongly recommends that you change all default credentials to not expose any access to the controller. This includes changing the values from the defaults for the controller password, keystore and truststore passwords, the keystore admin token, and the controller service token (see Security procedure). Changing the default controller keystore and truststore to use CA signed certificates In a teamed environment, unique certificates are generated for each controller in the team. You must repeat the following procedure for each controller in the team. To create a CA-signed keystore and truststore, as the SDN user (for example, sudo - sdn): Procedure 1. From the Configurations screen of the controller UI: a. In the Advanced tab of the Configurations screen, select each of the following components and change the value of the selfsigned key to false: com.hp.sdn.api.impl.alertpostmanager com.hp.sdn.misc.adminrestcomponent com.hp.sdn.misc.servicerestcomponent b. Select the com.hp.sdn.adm.mgr.impl.hpws.hpwsinstallmanager configurable component and ensure that the following keys have the values indicated in the following table: Chapter 7 Security 113

114 Key keystore keystore.password selfsigned truststore truststore.password Value /opt/sdn/admin/keystore password is not displayed(enc()) false /opt/sdn/admin/truststore password is not displayed (ENC()) For more information about changing controller configurable components, see Modifying a component configuration. 2. Log in to the system running the SDN Controller as the sdn user and stop the controller. sudo service sdna stop 3. Back up your default /opt/sdn/admin/keystore and /opt/sdn/admin/truststore to a safe location. A Java keytool is used to create the new keystore and CSR. This tool can be found at /opt/sdn/ openjdk8-jre/bin/keytool. 4. As the sdn user, create a new keystore: cd /opt/sdn/admin rm keystore truststore /opt/sdn/openjdk8-jre/bin/keytool -genkey -alias serverkey -keyalg rsa -keysize keystore keystore 5. To support teaming, specify an IP address as the common name when configuring your server for the first and last name question. 6. Generate a CSR (Certificate Signing Request) for signing: /opt/sdn/openjdk8-jre/bin/keytool -keystore keystore -certreq -alias serverkey - keyalg rsa -file sdn-server.csr 7. Send the sdn-server.csr to a CA to be signed. The CA authenticates you and returns a signed certificate and its CA certificate chain. We assume the signed certificate from the CA is named signed.cer and the CA's certificate is root.cer. If root.cer is from your own internal CA, then you need to import root.cer into your browser as an authority. 8. Import the signed certificates into your keystore and truststore: a. Import the root.cer certificate into your keystore and truststore: /opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore keystore -file root.cer -alias CARoot /opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore truststore -file root.cer -alias CARoot b. Import the root-int.cer certificate into your keystore and truststore: /opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore keystore -file root-int.cer -alias CARoot /opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore truststore -file root-int.cer -alias CARoot 9. If you do not have an intermediate root-int.cer file, use a different alias such as CARootInt. For example: 114 Aruba VAN SDN Controller 2.8 Administrator Guide

115 /opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore keystore - file root-int.cer -alias CARootInt /opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore truststore -file root-int.cer -alias CARootInt 10. Replace your self-signed certificate in your serverkey entry with the signed certificate from your CA signed.cer: /opt/sdn/openjdk8-jre/bin/keytool -importcert -keystore keystore -file signed.cer -alias serverkey 11. Add the certificate from your CA to Linux trusted certs using root. For example: root@sdnctl1:/opt/sdn/admin# cp cacert.pem /usr/local/share/ca-certificates/ cacert.crt root@sdnctl1:/opt/sdn/admin# update-ca-certificates The following is an example of what you will see displayed during this process: Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done Running hooks in /etc/ca-certificates/update.d... Adding debian:cacert.pem done. done. root@sdnctl1:/opt/sdn/admin# 12. Start the controller: sudo service sdna start SDN Controller keystore and truststore locations and passwords The SDN Controller keystore and truststore are referenced by the following configurable components: com.hp.sdn.api.impl.alertpostmanager com.hp.sdn.misc.adminrestcomponent com.hp.sdn.misc.servicerestcomponent com.hp.sdn.adm.mgr.impl.hpws.hpwsinstallmanager The values for keystore and keystore.password contain the keystore location and encrypted keystore password respectively. The values for truststore and truststore.password contain the truststore location and encrypted truststore password respectively. The keystore and truststore location and password for each component must match the configured keystore and truststore location and password. Encryption Sensitive information such as tokens and passwords are stored encrypted on the SDN Controller. However, to encrypt and decrypt these properties, the controller requires a master key that is passed into the controller upstart script via an environment variable. To change the default master key (recommended): Procedure 1. Stop the following services: Chapter 7 Security 115

116 sudo service sdnc stop sudo service sdna stop 2. Change the default master key: sudo /opt/sdn/admin/sdnpass old_master_key new_master_key Built-in OpenFlow controller The Aruba VAN SDN Controller has a built-in OpenFlow controller for controller-to-switch communications. The OpenFlow controller component relies on PKI to establish mutual trust (2-way SSL) between itself and the OpenFlow switches that it manages. To establish TLS connections for controller-to-switch OpenFlow communications, Hewlett Packard Enterprise recommends the following: Use different store names for the built-in OpenFlow controller keystore and truststore than used for the Aruba VAN SDN Controller keystore and truststore. Use the same CA (certificate authority) to sign the controller and all device certificates. For information about configuring TLS, see the latest HPE OpenFlow Administrator Guide for your switch. Creating a keystore and truststore for OpenFlow switch communication The process for creating the OpenFlow keystore and truststore is similar to the steps outlined under Changing the default controller keystore and truststore to use CA signed certificates. Built-in OpenFlow controller keystore and truststore locations and passwords The Aruba VAN SDN Controller has a built-in OpenFlow controller for controller-to-switch communications. The configurations for the built-in OpenFlow controller keystore and truststore are located in the com.hp.sdn.ctl.of.impl.controllermanager component. The keystore and keystore.password keys store the location of the keystore and the password of the keystore respectively. Similarly, the truststore and truststore.password keys store the location of the truststore and the password of the truststore respectively. You can configure the com.hp.sdn.ctl.of.impl.controllermanager component in the Configurations screen Basic tab (screen example is shown below). A controller restart is required if these configurations are changed. The path to the keystore or truststore location must be specified as a relative path from the /opt/sdn/virgo directory. For example, to specify a location of /opt/sdn/config/of.jks enter the following: 116 Aruba VAN SDN Controller 2.8 Administrator Guide

117 ../config/of.jks Figure 64: Components that reference OpenFlow keystore and truststore REST authentication The Aruba VAN SDN Controller relies on token-based authentication to authenticate most of its REST APIs. All REST APIs except the /auth and /rsdoc APIs require an authentication token embedded in an X-Auth-Token header to be included with each REST request. The /auth API allows you to obtain a token, while the /rsdoc API provides REST API documentation information about the controller s REST API. The Aruba VAN SDN Controller REST API documentation is accessible from a web browser and in PDF format in the Hewlett Packard Enterprise Information Library for SDN. The RSdoc for your controller can also contain REST API documentation added by applications installed on the controller. Although the RSdoc API explorer interacts directly with the controller REST API, RSdoc is not intended as a management or configuration interface. Use caution when using the Try it out! button for POST or PUT methods because this action can result in changes to your current controller environment. Requests to the controller using the POST method of the cms/client/event resource can be authenticated using client certificate-based authentication instead of token-based authentication. For details on the Client Mapper Service that allows integration with an external policy manager such as Aruba ClearPass Policy Manager, see Using an external policy manager on page 234. When you use certificate-based authentication, the controller authenticates the REST API request by verifying the certificate presented by the client when the SSL connection is established against the client certificate in the controller truststore. Requirements for controller environment when using certificate-based authentication For the controller or for each controller in the team these requirements must be met: The certificate for the client, an external policy manager such as Aruba ClearPass, must be imported in to the truststore of the controller. The issuer CN (common name) of the certificate must be entered for the value of the clearpass.cert.cn key of the com.hp.sdn.cms.impl.clientmapperserviceprovider controller configurable component. Chapter 7 Security 117

118 Controller hybrid mode must be enabled (set to true). The value of the com.hp.sdn.cms.impl.clientmapperserviceprovider controller configurable component key clearpass.integration.enabled must be true. Requirements for the REST API request when using certificate-based authentication Certificate-based authentication can only be used for cms/client/event POST requests. The request must not include an X-Auth-Token in the request header. The URI for the request must use port For example: POST where CONTROLLER_IP is the IP address of the controller OpenStack Keystone used for user and token management The SDN Controller uses Openstack Keystone as an identity management for managing users, generating tokens, as well as token validation. The controller supports Keystone releases supporting the 2.0 REST API from Folsom up to the Juno release. It supports the following token authentication providers: UUID 32 character string (All Keystone releases) PKI CMS message containing service catalog, user roles, and metadata (Grizzly and later) PKIZ ZLIB compressed PKI token (Juno and later) The controller is configured by default to auto-detect the token provider. It can also be forced to use a specific provider. The auto detection logic determines that any token longer than 32 characters is PKI or PKIZ. Distinguishing between PKI and PKIZ is accomplished by detecting the PKIZ prefix which is prepended to PKIZ compressed tokens. UUID Authentication The UUID authentication follows this process: Procedure 1. The controller, upon receiving the username/password pair for a user, sends the pair along with a tenant/ project to the Keystone Identity Management service. 2. Keystone, upon receiving the username/password pair: Checks if the username/password is valid for the requested user and tenant/project If the username/password/tenant combination is valid: Generates a UUID token Stores the UUID token in its backend Sends a copy of the UUID token back to the controller 3. The controller caches the token and returns a copy to its client. 4. The controller s client uses this token on each API request to the controller. 5. Upon each user request, the controller sends this UUID back to Keystone for validation. 6. Keystone returns success or failure status to the controller. 7. On success Keystone grants access to its client to the API call, otherwise it would fail the call with an authorization failure message. This design requires every API request to call in to Keystone for validation. This approach does not scale well as the number of API calls increases. The PKI authentication mechanism addresses this issue by using a private/ 118 Aruba VAN SDN Controller 2.8 Administrator Guide

119 public key pair to produce a CMS message which can be verified by an endpoint without checking with Keystone for every API request. PKI Authentication The PKI authentication provider was introduced in the Grizzly release of Keystone. To use PKI tokens, keys and certificates need to be generated. In Grizzly, this is done by using the keystone-manage pki_setup command. Keystone becomes a (CA) by signing user tokens. The authentication process is as follows: Procedure 1. At startup, the controller: a. Starts a task that periodically downloads the CA and signing certificates from the Keystone server (configurable using the PKICertsDownloadHour item). b. Starts a task that periodically downloads the token revocation list from the Keystone server (configurable using the RevListPollPeriod item). item) 2. The controller, upon receiving the username/password pair for a user, sends the pair along with a tenant/ project to the Keystone Identity Management service. 3. Keystone, upon receiving the username/password pair: a. Checks whether the username/password is valid for the requested user and tenant/project. b. If the username/password/tenant combination is valid: Keystone builds a JSON message using: Service catalog details User role Metadata Produces a CMS message signing it using the private key. Strips the header and footer and then produces a URL safe base64 encoded token. Returns the token to the controller. c. The controller caches the token and returns a copy to its client. d. The controller s client uses this token on each API request to the controller. e. Upon each user request, the controller validates this token by: Checking whether the token is in its cache, if it generated the token. Checking whether the token is valid using signature verification with the signing certificate, if the token is not in its cache and not on the revocation. If the received token is compressed, the controller decompresses it before checking the signature. f. The revocation list is periodically retrieved from the Keystone server and is used to determine whether a token is revoked. g. The periodic certificate download results in the CA and signing certificates to be updated daily. Local vs Remote Keystone By default the Keystone server is assumed to be installed on the same machine (localhost) as the controller. A remote Keystone server can be specified using the ServerVIP configuration key in the AuthenticationManager component. Keystone controller configuration The following Keystone controller configuration is set in the controller UI Configurations screen in the System tab under the com.hp.sdn.adm.auth.impl.authenticationmanager component. The keys are described as follows: Chapter 7 Security 119

120 AdminToken Keystone admin token. ConnPoolEvictPeriod Keystone idle connection clean-up cycle in milliseconds. Minimum is 100. ConnPoolMaxActive Keystone maximum active connections. Minimum is 1. ConnPoolMaxIdle Keystone maximum idle connections. Minimum is 1. ConnPoolMinIdleTime Keystone minimum idle connection time in milliseconds. Minimum is ConnSSLClientAuth Keystone mutual authentication using TLS. ConnTimeout Keystone connection timeout in milliseconds. Minimum is 0. Keystore Keystone keystore location. KeystorePass Keystone keystore password. MaxCachedTokens Maximum number of cached tokens. Minimum is 0. PKICertsDownloadHour Hour in a 24 hour day (0-23) when PKI certificates download form the Keystore server occur. PKICertsPath Keystone PKI (signing and CA) certificates location. RevListPollPeriod Keystone PKI revocation list poll interval in seconds. ServerPort Keystone server port. ServerVIP Keystone server virtual IP. ServiceRole Role for shared secret. ServiceTenant Tenant (project) for shared secret. ServiceToken Shared secret for internal requests. ServiceTokenTimeout Timeout for shared secret, 0 for never. Minimum is 1. ServiceUser User for shared secret. Tenant Keystone tenant (only a single tenant is supported). TokenProvider Keystone token provider (Auto-Detect PKI PKIZ UUID) Truststore Keystone truststore location. TruststorePass Keystone truststore password. UserRole Keystone user role (only a single role is supported. Only a user having this role is allowed access to the controller. For information on Keystone, see the OpenStack Keystone documentation at developer/keystone/. Security Since tokens for either providers (UUID, PKI, or PKIZ) are bearer tokens, they should be protected by using mutually authenticated TLS. This can be accomplished by using valid PKI transport configuration as described in Changing the default controller keystore and truststore to use CA signed certificates: The controller must have: The valid trusted CA signed identity and CA certificates configured in the controller keystore The CA certificate must be configured in the controller truststore The authentication manager ConnSSLClientAuth should be set to true to enforce mutual authentication Keystone should be configured to: 120 Aruba VAN SDN Controller 2.8 Administrator Guide

121 Use valid trusted CA signed identity certificate Configure trusted CA to be used to validate client certificates Require SSL Require the client certificate to be valid Role-Based Access Control (RBAC) Aruba VAN SDN Controller supports limited RBAC (Role Based Access Control). The SDN Controller currently enforces a single role which has access to all controller features. By default, the single role is sdn-admin. The authenticated user must have this role in order to be granted access to the controller. You must ensure that Keystone is configured to grant this role. The applications installed on the SDN Controller can enforce RBAC to meet their security requirements. Assigning a user to a role To assign a user the sdn-admin role and give the user access to the desired SDN Controller: Procedure 1. Create a tenant (the example creates a test tenant): curl -H "X-Auth-Token:ADMIN" -H "Content-Type: application/json" -d '{"tenant": {"enabled": true, "name": "test-tenant", "description": "Test Tenant"}}' 2. List tenants: curl -H "X-Auth-Token:ADMIN" 3. Create a user: curl -H "X-Auth-Token:ADMIN" -H "Content-Type: application/json" -d '{"user": {" ": "tester@test.rose.hp.com", "password": "somepass", "enabled": true, "name": "test-user", "tenantid": "2c851897a09f483fa452e2de11511f71"}}' 4. List users: curl -H "X-Auth-Token:ADMIN" 5. Create a role: curl -H "X-Auth-Token:ADMIN" -H "Content-Type: application/json" -d '{"role": {"name": "test-role"}}' 6. List roles: curl -H "X-Auth-Token:ADMIN" 7. Assign a user to a role: curl -X PUT -H "X-Auth-Token:ADMIN"; <tenant-id>/users/<user-id>/roles/os-ksadm/<role-id> 8. List roles for a user for a given tenant: curl -X GET -H "X-Auth-Token:ADMIN" Example 1. List tenants Chapter 7 Security 121

122 curl -H "X-Auth-Token:ADMIN" tenants python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed :--:-- --:--:-- --:--: { "tenants": [ { "description": "", "enabled": true, "id": "575d62cc28bc403c ba6536d3", "name": "sdn" }, { "description": "Test Tenant", "enabled": true, "id": "fb2f0c68d410440baf67ba134733dbdb", "name": "test-tenant" } ], "tenants_links": [] } 2. Create a user as part of sdn tenant root@sdnctl1:/var# curl -H "X-Auth-Token:ADMIN" -H "Content-Type: application/ json" -d '{"user": {" ":"tester@test.rose.hp.com", "password": "somepass", "enabled": true, "name": "test-user", "tenantid":"575d62cc28bc403c ba6536d3"}}' :35357/v2.0/users {"user": {"username": "test-user", "name": "test-user", "id": "867e7e2e88644e73a4eee25e4b80c303", "enabled": true, " ": "tester@test.rose.hp.com", "tenantid": "575d62cc28bc403c ba6536d3"}} root@sdnctl1:/var# curl -H "X-Auth-Token:ADMIN" users python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed :--:-- --:--:-- --:--: { "users": [ { "enabled": true, "id": "4a4a30ce53b743798dd79d98f5ab7daf", "name": "sdn", "username": "sdn" }, { " ": "tester@test.rose.hp.com", "enabled": true, "id": "867e7e2e88644e73a4eee25e4b80c303", "name": "test-user", "tenantid": "575d62cc28bc403c ba6536d3", "username": "test-user" } ] } 3. List roles root@sdnctl1:/var# curl -H "X-Auth-Token:ADMIN" KSADM/roles python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current 122 Aruba VAN SDN Controller 2.8 Administrator Guide

123 Dload Upload Total Spent Left Speed :--:-- --:--:-- --:--: { "roles": [ { "description": "Default role for project membership", "enabled": "True", "id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_" }, { "id": "1719c0d3b647488da8ca7ff6a1d0288b", "name": "sdn-user" }, { "id": "c105e3dc4a484f e28f7483edc", "name": "sdn-admin" }, { "id": "67eb2907e94d43f7b3e e20bbc", "name": "test-role" } ] } 4. Assign user to sdn-admin role for the sdn tenant root@sdnctl1:/var# curl -X PUT -H "X-Auth-Token:ADMIN" 867e7e2e88644e73a4eee25e4b80c303/roles/OS-KSADM/c105e3dc4a484f e28f7483edc {"role": {"id": "c105e3dc4a484f e28f7483edc", "name": "sdn-admin"}} 5. Verify which roles this user has for the sdn tenant root@sdnctl1:/var# curl -X GET -H "X-Auth-Token:ADMIN" v2.0/tenants/575d62cc28bc403c ba6536d3/users/ 867e7e2e88644e73a4eee25e4b80c303/roles python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed :--:-- --:--:-- --:--: { "roles": [ { "description": "Default role for project membership", "enabled": "True", "id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_" }, { "id": "c105e3dc4a484f e28f7483edc", "name": "sdn-admin" } ] } API access requires authentication To authenticate, one needs to present username, domain, and password to the /auth API as below (using curl as an example): curl -sk -H 'Content-Type:application/json' -d '{"login": {"user":"sdn","password":"skyline","domain":"sdn"}}' Chapter 7 Security 123

124 Credential information (user name, password, domain, and authentication tokens) used in curl commands might be saved in the command history. For security reasons, Hewlett Packard Enterprise recommends that you disable command history prior to executing commands containing credential information. The above call returns this example JSON data structure that includes the authentication token, which, by default, expires in one hour: { } "record": { "domainid": "62e312edff47413fad7e1d7fa6ac7bc7", "domainname": "sdn", "expiration": , "expirationdate": " ", "token": "54a6f80a9ae243db89bfa05de4ced51d", "userid": "bca3dea8a28b457e99e899ae16b79634", "username": "sdn" "roles":["sdn-user","sdn-admin"], } Please guard this token information, as it can be used as an API key to gain access to your controller REST APIs. To gain access to the REST API, include the token in the X-Auth-Token header as in the following curl example: curl -sk -H "X-Auth-Token:54a6f80a9ae243db89bfa05de4ced51d" One can continue using the same token for different SDN Controller APIs within the default one hour period since token creation. If desired, one can change this default one hour timeout in the /etc/keystone/ keystone.conf file. (See the OpenStack Keystone Administration Guide for more information). Service and admin tokens The Service token is used for internal communication between controllers and is not exposed to the user. The Admin token is used for communication between controller and the Keystone server and is not exposed to the user. The values for these tokens can be seen in the controller UI in the Configurations screen in the System tab under the com.hp.sdn.adm.auth.impl.authenticationmanager component. All controllers in a team must have the same Service token to communicate successfully. For the Admin token, both the controller token value and the Openstack Keystone admin_token in the directory /etc/keystone/keystone.conf must match for successful authentication. Controller code verification All controller code is signed by Hewlett Packard Enterprise. Validating the certificate via jarsigner should return a Hewlett Packard Enterprise X.509 certificate similar to the following: X.509, CN=Hewlett-Packard, OU=HPGlobal, OU=Digital ID Class 3 - Java Object Signing, O=Hewlett-Packard, L=Andover, ST=Massachusetts, C=US [certificate is valid from 11/14/12 4:00 PM to 11/15/14 3:59 PM] 124 Aruba VAN SDN Controller 2.8 Administrator Guide

125 X.509, CN=VeriSign Class 3 Code Signing 2010 CA, OU=Terms of use at (c)10, OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US [certificate is valid from 2/7/10 4:00 PM to 2/7/20 3:59 PM] [CertPath not validated: null] If a controller jar or war file is tampered with, the jar verification fails, and the container does not start up. If an application is not signed by Hewlett Packard Enterprise, or has its certificate trusted by the controller (see section below), the application is not allowed to run on the controller. Adding certificates to the jar-signing truststore To deploy other signed applications onto the controller, use the Java keytool to import the public certificate that was used to sign the application jars and/or zips into the controller jar-signing truststore (/opt/sdn/admin/ sdnjar_trust.jks): /opt/sdn/openjdk8-jre/bin/keytool -importcert -keystore /opt/sdn/admin/ sdnjar_trust.jks -file signed_app.cer -alias mysignedcert The controller needs to be restarted for the new truststore to take effect. Running the SDN controller without Jar-Signing validation The SDN Controller enforces jar/zip-signing validation by default. For an experimental or development environment, where unsigned applications need to be deployed, jar/zip-signing validation can be turned off altogether: Procedure 1. Stop the SDN Controller: sudo service sdnc stop 2. Modify the /opt/sdn/virgo/bin/dmk.sh script to add the following option to the list of JMX_OPTS: -Dsdn.signedJar=none For example: cd $KERNEL_HOME; exec $JAVA_EXECUTABLE \ $JAVA_OPTS \ $DEBUG_OPTS \ $JMX_OPTS \ -XX:+HeapDumpOnOutOfMemoryError \ -XX:ErrorFile=$KERNEL_HOME/serviceability/error.log \ -XX:HeapDumpPath=$KERNEL_HOME/serviceability/heap_dump.hprof \ -Dsdn.signedJar=none \ -Djava.security.auth.login.config=$AUTH_LOGIN \ -Dorg.eclipse.virgo.kernel.authentication.file=$AUTH_FILE \ 3. Start the SDN Controller: sudo service sdnc start To enable jar/zip-signing validation, remove the line containing the -Dsdn.signedJar=none option from the /opt/sdn/virgo/bin/dmk.sh script and restart the controller. Revoking Trust Chapter 7 Security 125

126 Revoking trust via truststore The controller components rely on the public certificates in the respective truststore to establish trust with a given identity. Therefore, revoking trust from a client with a given public certificate amounts to removing its certificate from the respective truststore. To remove a given certificate from the truststore: List the certificates in your truststore:/opt/sdn/openjdk8-jre/bin/keytool list v -keystore truststore [-storepass password] Delete certificate from truststore:/opt/sdn/openjdk8-jre/bin/keytool delete alias certaliastruststore Revoking trust via CRL For the controller s REST API, a CRL (Certificate Revocation List) might also be specified to allow blacklisting of certain clients. This is done by modifying the /opt/sdn/virgo/configuration/tomcat-server.xml file to include the CRL file location in the SSL connector: <Connector port="8443" protocol="http/1.1" SSLEnabled="true" ciphers="tls_rsa_with_aes_128_cbc_sha, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA" maxthreads="150" scheme="https" secure="true" clientauth="false" sslenabledprotocols="tlsv1,tlsv1.1,tlsv1.2" keystorefile="../admin/keystore" keystorepass="skyline"/> For the change to take effect, restart the controller. SDN administrative REST API The main SDN Controller daemon (sdnc) is accompanied by an ancillary daemon process (sdna), which runs under user sdnadmin in order to grant it access to some elevated privileges. The administrative REST API can be used to securely perform various management functions in a privileged context. It would be undesirable for the main SDN Controller process to possess those privileges as it might be hosting execution of third-party code. The SDN Administrator daemon can be accessed via the REST API via HTTPS on port The access is secured through either token-based authentication or basic authentication, against the locally running Keystone server, which is the same as the main SDN Controller REST API. The following set of features are accessible through the administrative REST API: SDN Controller daemon (sdnc) stop/start/restart Adding/removing the team leader IP alias (required only when in team mode) Configure iptables rules to protect team communication If the iptables rule programming for Cassandra fails, the Cassandra server will not come up. In previous releases, the server would come up regardless of the iptables rule programming. Downloading the ZIP bundle of log files Uploading upgrade Debian bundles and installing/removing Debian packages Uploading upgrade ZIP bundles and executing upgrade commands System reboot The install process adds a number of sudoers entries for the sdnadmin user. These are as follows: /sbin/ifconfig /sbin/reboot /sbin/iptables 126 Aruba VAN SDN Controller 2.8 Administrator Guide

127 /usr/bin/service /usr/bin/at /usr/bin/dpkg /usr/sbin/arp /usr/bin/arping All, or any, of the above entries can be blocked or removed from the sudoers configuration. The /sbin/ ifconfig entry is only required when running in teamed mode. Otherwise the controller cannot migrate the team IP address from node to node as team leader changes. The /sbin/iptables is also required in teamed mode to secure team communication. The sdna process must be present and active for the SDN controller to function. The sdnc process will not start without sdna. Virgo admin UI access via localhost only You can access the Virgo admin UI by using a remote browser at address>:8443/admin. This should not be used under normal circumstances, but can be useful for debugging purposes. To change the credentials of this console, get root console access to the machine(s) running the Aruba VAN SDN Controller and edit the following file: /opt/sdn/virgo/configuration/org.eclipse.virgo.kernel.users.properties This file includes the following two entries: user.admin=sdn role.admin=admin where role.admin defines the user and user.admin defines the password. This file needs to be owned by user:sdn, group:sdn. Changes to this file require a restart of the controller to recognize the new credentials. To disable access to the Virgo Admin UI, either remove the following file or move it to a safe location outside the pickup directory. Virgo console access disabled by default The Virgo console is disabled by default as it is not security hardened. If you choose to enable it for debugging purposes, make sure you are in a trusted environment and disable it as soon as possible. To access the Virgo Admin WEB UI (GUI), copy the org.eclipse.virgo.management.console_3.6.2.release.jar file from the /opt/sdn/admin directory to the /opt/sdn/virgo/pickup directory. You must either be the sdn user on the SDN controller to copy the file or change the file ownership to sdn once it is copied. JMX console enabled for local access only The JMX console is only enabled for local access. This is used by the controller for metering and can also be used for debugging. The JMX console is not security hardened and should be enabled for remote access only in trusted environments. To enable JMX console remote access, edit /opt/sdn/virgo/bin/dmk.sh. The following line determines whether JMX allows remote access or not, in this case indicating local only access. -Dcom.sun.management.jmxremote.local.only=true \ Any changes to this file require a controller restart to recognize the change. Chapter 7 Security 127

128 Creating the Cassandra keystore and truststore To create the keystore and truststore: Procedure 1. Log in to the system running the SDN Controller and stop the controller. 2. As the sdn user (for example, su - sdn or sudo -i -u sdn), back up your default /opt/sdn/ cassandra/conf/.keystore and /opt/sdn/cassandra/conf/.truststore to a safe location. 3. Create a new keystore using the following commands (note the default password here is skyline): cd /opt/sdn/cassandra/conf rm.keystore.truststore /opt/sdn/openjdk8-jre/bin/keytool -genkey -alias serverkey -keyalg rsa -keysize keystore.keystore You must specify a fully-qualified domain for your server for the "first and last name" question as some CAs, such as VeriSign, expect it. 4. Generate a CSR (Certificate Signing Request) for signing: /opt/sdn/openjdk8-jre/bin/keytool -keystore.keystore -certreq -alias serverkey -keyalg rsa -file sdn-server.csr 5. Send the sdn-server.csr to a CA to be signed. The CA authenticates you and returns a signed certificate and its CA certificate chain. This procedure assumes that the signed certificate from the CA is named signed.cer and the CA's certificate is root.cer. 6. Import the signed root certificate into your keystores: In a team, you must add the certificate (and chain) from each other controller to the.truststore of all the other members on the team. /opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts -keystore.keystore -file root.cer -alias CARoot /opt/sdn/openjdk8-jre/bin/keytool -importcert -trustcacerts - keystore.truststore -file root.cer -alias CARoot 7. Replace your self-signed certificate in your serverkey entry with the signed certificate from your CA (signed.cer): /opt/sdn/openjdk8-jre/bin/keytool -importcert -keystore.keystore -file signed.cer -alias serverkey 8. Log in to the controller UI: 9. Select Configurations on the left navigation pane, select the System tab and then select the com.hp.sdn.teaming.impl.cassandraprocessmanager component. 10. Select Modify on the top. When the Modify System Configuration dialog box opens, update the location and password of the new keystore. Click Apply. 11. Restart the controller. 128 Aruba VAN SDN Controller 2.8 Administrator Guide

129 Cassandra keystore and truststore locations and passwords The Cassandra keystore and truststore are referenced by the com.hp.sdn.teaming.impl.cassandraprocessmanager component. To change the passwords keystore.password and truststore.password of this component: Procedure 1. From the controller UI, select Configurations, then select the System tab. 2. Select the com.hp.sdn.teaming.impl.cassandraprocessmanager component. 3. Select Modify. The stores are located in /opt/sdn/cassandra/config/.keystore and /opt/sdn/cassandra/ config/.truststore. Only a trusted authority should sign the certificates. You must install each of the certificates in the.truststore file of all of the nodes in the team. Security procedure Hewlett Packard Enterprise strongly recommends that you change all default credentials to prevent exposure of access to the controller. Change the values from the defaults for the following: Keystore password Truststore password Admin Token Service Token Jarsigning Procedure 1. Log into as the sdn user. 2. From the controller UI, select Configurations. Then select the System tab. 3. Select the component com.hp.sdn.adm.auth.impl.authenticationmanager. 4. Select Modify and change the default values for the following keys to the newly chosen credentials: Set the AdminToken key to the newly chosen Keystone (authentication) admin token. Set the ServiceToken to the newly chosen internal communication secret. Set the KeystorePass to the value that you will be using to secure the SSL Keystore. Set the TruststorePass to the value that you will be using to secure the SSL Truststore. 5. Specific to Keystone, set the ConnSSLClientAuth key to true. 6. Update the Keystone admin token in the file etc/keystone/keystone.conf. a. Change the admin token from the default admin_token=admin to admin_token=<newadmintoken> Where the <newadmintoken> is the newly chosen AdminToken value you entered in step 4. If the line is commented out, remove the # sign in front of the line. b. Restart the Keystone service (sudo service keystone restart). 7. Update the keystore password to match the newly chosen KeystorePass value you entered in step 4 using the following: /opt/sdn/openjdk8-jre/bin/keytool -storepasswd -storepass <OldKeystorePass> -new <newkeystorepass> -keystore /opt/sdn/admin/keystore Chapter 7 Security 129

130 8. Update the keystore s internal serverkey to match the newly chosen KeystorePass value you entered in step 4 using the following: /opt/sdn/openjdk8-jre/bin/keytool -keypasswd -alias serverkey -storepass <newkeystorepass> -keypass <oldkeystorepass> -new <newkeystorepass> - keystore /opt/sdn/admin/keystore 9. Update the truststore password to match the newly chosen TruststorePass value you entered in step 4 using the following: /opt/sdn/openjdk8-jre/bin/keytool -storepasswd -storepass <oldkeystorepass> -new <newkeystorepass> -keystore /opt/sdn/admin/truststore 10. Update the jar signing keystore password (named sdnjar_trust.jks): a. Use the keytool command to assign a new password. The default or old keystore password is skyline. /opt/sdn/openjdk8-jre/bin/keytool -storepasswd -storepass <oldkeystorepass> - new <newpass4sign> -keystore /opt/sdn/admin/sdnjar_trust.jks Where <newpass4sign> is a newly chosen password. This new password does not have to match the others. b. Update the dmk.sh to provide the new password as an environment variable for the running controller. Navigate to the /opt/sdn/virgo/bin directory as the sdn user. Open the dmk.sh file to edit. In the dmk.sh file, find the line containing XX:HeapDumpPath... After the XX:HeapDumpPath... line, add a new line-dsdn.trustpass=<newpass4sign> Save the dmk.sh file. c. Restart the sdnc service (sudo service sdnc restart) for the modified password to be read by the controller. When you have completed security configuration, restart Keystone service and restart the controller. Be sure to remove the visible passwords from the shell history. Security best practices Observing these rules can help to prevent unauthorized access to the controller: Do not enable shell history on your controller. Do not allow other users besides sdn, sdnadmin and the Linux user to have access to your controller system. Do not store your authentication token in plain text, such as a non-encrypted cookie. Do not use self-signed certificates in a production environment. Do not alter contents under /opt/sdn/cassandra and /opt/sdn/hazelcast. To prevent authentication tokens from being stolen: Always log out of the UI and close the web page, when you are done using it. Never leave a browser window open and unattended when you are accessing the UI. Never let someone who does not have access rights to the controller look over your shoulder while accessing the UI. Make sure Keystone is configured to expire tokens after a short period of time (a common industry practice is 20 minutes). Do not delete any iptables with the name hazelcast, cassandra-default, or cassadra-team, or any rules with the following ports: 5700, 7000, 7001, 7199, Do not manually override the iptables rules to allow or deny ports 5700, 7000, 7001, 7199, and Rules created for a team of 3 controllers on page 131 displays the rules created for a team of 3 controllers ( , , ) when running the sudo iptables -nl command: 130 Aruba VAN SDN Controller 2.8 Administrator Guide

131 Rules created for a team of 3 controllers # sudo iptables -nl Chain INPUT (policy ACCEPT) target prot opt source destination cassandra-team all / /0 cassandra-default all / /0 hazelcast all / /0 Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination cassandra-team all / /0 cassandra-default all / /0 hazelcast all / /0 Chain cassandra-default (2 references) target prot opt source destination ACCEPT tcp tcp dpt:7001 ACCEPT tcp tcp dpt:9160 ACCEPT tcp tcp dpt:7199 DROP tcp / /0 tcp dpt:7199 DROP tcp / /0 tcp dpt:9160 DROP tcp / /0 tcp dpt:7000 DROP tcp / /0 tcp dpt:7001 Chain cassandra-team (2 references) target prot opt source destination ACCEPT tcp tcp dpt:7001 ACCEPT tcp tcp dpt:7001 ACCEPT tcp tcp dpt:7001 ACCEPT tcp tcp dpt:7001 ACCEPT tcp tcp dpt:7001 Chain hazelcast (2 references) target prot opt source destination ACCEPT tcp tcp dpt:5700 ACCEPT tcp tcp dpt:5700 ACCEPT tcp tcp dpt:5700 ACCEPT tcp tcp dpt:5700 DROP tcp / /0 tcp dpt:5700 Chapter 7 Security 131

132 Chapter 8 Configuring OpenFlow instances Configuring OpenFlow Instances with Multiple VLANs Hewlett Packard Enterprise recommends that the OpenFlow instance VLAN membership be identical throughout the controlled network topology. If an OpenFlow instance contains a set of VLANs on one switch, then neighboring switches should also have an OpenFlow instance with the same set of VLANs. For network topologies that contain both ProVision and Comware OpenFlow instances, configure the ProVision aggregate mode and Comware Openflow instances to contain all VLANs. You should untag inter-switch links in the OpenFlow instance VLANs. Alternatively, you can tag these links in the OpenFlow instance VLANs. Default the untagged traffic to one of the OpenFlow instance VLANs. Hewlett Packard Enterprise recommends the following mixed ProVision/Comware topology switch configuration: ProVision ProVision Instance: Aggregate OpenFlow instance on all switches or virtualized only instances on all switchesinterface: Any valid configuration ProVision Comware Instance: ProVision Aggregate, Comware group = allinterface: Comware interswitch link port PVID in OpenFlow instance Comware Comware Instance: Consistent OpenFlow instance VLAN configuration throughout topologyinterface: Interswitch link port PVID in OpenFlow VLAN Configuring OpenFlow Instances with Single VLAN Identifier In a topology that contains both Comware and ProVision switches, if the ProVision switch OpenFlow instance is configured for a single VLAN, configure the Comware OpenFlow instance to a single VLAN as well. In some cases, when the ports of the inter-switch link between the ProVision and Comware devices are configured as tagged-only, the link from the Comware device to the ProVision device may not be discovered. In order for links to be correctly discovered in these configurations, configure the controller to insert a VLAN tag when injecting link discovery packets on the Comware device. Use the device REST API to configure the appropriate linkdiscoveryvlan that matches the VLAN identifier configured in the Comware device OpenFlow instance. Configuring linkdiscoveryvlan to discover all links from the Comware devices In this example, the topology consists of a ProVision 3800 on port 6 connected to a Comware 5500HI on port GigabitEthernet1/0/14: 132 Aruba VAN SDN Controller 2.8 Administrator Guide

133 The ProVision 3800 has the following configuration: openflow controller-id 1 ip controller-interface vlan 800 instance "1" member vlan 40 controller-id 1 version 1.3 enable exit enable exit vlan 1 name "DEFAULT_VLAN" no untagged 6,25,48 untagged 1-5,7-24,26-47,49-52 ip address dhcp-bootp exit vlan 40 name "VLAN40" tagged 6 no ip address exit The Comware 5500HI has the following configuration: interface GigabitEthernet1/0/14 port link-mode bridge port link-type trunk undo port trunk permit vlan 1 port trunk permit vlan 40 # openflow instance 1 controller 1 address ip classification vlan 40 loosen # Chapter 8 Configuring OpenFlow instances 133

134 Initially, the REST API reports the following devices: Initially, the REST API reports only one link; the link between the Comware 5500Hl and the ProVision 3800: The Rest API reports the link discovered when the controller injected link discovery packets to the ProVision 3800 OpenFlow instance. The ProVision switches insert a VLAN tag in packets injected by the controller when the egress ports are configured as tagged only. The 5500 port receives the properly tagged packet and forwards it to the controller, resulting in the discovered link. The corresponding link from the 3800 to the 5500 is missing. The controller-injected link discovery packets sent to port GigabitEthernet/1/0/14 on 5500 OpenFlow instance (the device with DPID 00:01:cc:3e:5f:6a:d3:80) were not tagged, therefore they were dropped by the receiving 3800 whose port 6 is configured for tagged traffic. Solution To ensure link discovery packets are tagged when sent out by tagged Comware device ports, configure the device linkdiscovervlan value for the Comware device with the tagged-only port using the device REST API. For 134 Aruba VAN SDN Controller 2.8 Administrator Guide

135 example, in the above topology, configure the 5500 device to set the linkdiscoveryvlan to 40 (the VLAN configured in the OpenFlow instances): Chapter 8 Configuring OpenFlow instances 135

136 The REST API query now reports that both links are discovered: When you configure a linkdiscoveryvlan for a device, the controller will always insert a 802.1Q header with the configured VLAN on each link discovery packet sent to all ports of the device, regardless of the actual port configurations, as the controller does not have any knowledge of the port configuration. Configuring OpenFlow instances to enable MAC group matching MAC group matching By supporting MAC group matching and MAC group tables, an OpenFlow instance can store flow entries that match MAC groups instead individual MAC IDs, freeing up space in the policy (TCAM) table of the physical switch for other flow entries. The list of the MAC addresses in the MAC group are defined in other tables (40 for source and 41 for destination). For example, to use MAC addresses to block access to the network for 10 devices owned by a particular group of users: Without support for MAC group matching, you would have to create a flow entry in the policy table of the switch for each MAC address, for a total of 10 flow entries. With support for MAC group matching and MAC group tables, you can create a MAC group, add the MAC addresses to the MAC group, and then create a single flow entry in the policy table to match that MAC group ID. The list of the MAC addresses in the MAC group are defined in other tables (40 for source and 41 for destination). For more information about programming flow entries for OpenFlow instances, see the following documents: 136 Aruba VAN SDN Controller 2.8 Administrator Guide

137 Aruba VAN SDN Controller Programming Guide Aruba VAN SDN Controller REST API Reference Switches that support MAC group tables and MAC group matching Beginning with version 2.6, the Aruba VAN SDN Controller supports the use of MAC groups and MAC tables for OpenFlow v1.3 instances on ProVision-based switches running release K/KA/KB/WB or later with the following restrictions: Support is provided for V2 and V3 modules only. MAC group tables are not supported when the switch is in V1 module compatible mode. The 2920 switch does not support destination MAC address matching or destination MAC group matching. Configuration rules for OpenFlow instances and MAC groups By default, MAC group tables are disabled. You must enable MAC group tables to use them. MAC group matching is supported in the standard match mode default pipeline only. Exactly one OpenFlow instance per physical switch can be enabled with MAC group match support. All OpenFlow instances on the physical switch must be disabled before you can enable the MAC group feature. MAC group tables do not support counters or idle-timeout metering. For example, if you want to count the number packets that match a MAC group, you must create a separate flow entry. The policy table cannot match on a source MAC and destination MAC address separately when MAC group tables are part of the pipeline. Enabling or disabling MAC group matching on an OpenFlow instance Prerequisites (SDN controller 2.8) The default domain and user settings are sdn. The default password setting is skyline. Credential information (user name, password, domain, and authentication tokens) used in curl commands might be saved in the command history. For security reasons, Hewlett Packard Enterprise recommends that you disable command history prior to executing commands containing credential information. Procedure 1. Install and start three standalone controllers in the network. (See the latest Aruba VAN SDN Controller Installation Guide.) 2. Optional: To improve security, you can change the username and password from the default settings on each of the standalone controllers in step Select any one of the controllers to use for configuring the team. 4. On the selected controller, acquire an Authentication Token. Use the following curl command, with the controller IP address, to acquire the token: curl --noproxy controller_ip> -X POST --fail -kssfl --url " -H "Content-Type: application/json" --data-binary '{"login": {"domain": "<domain>","user": "<user>","password": "<password>"}}' In this example, the default domain, user name, and password are used. Chapter 8 Configuring OpenFlow instances 137

138 curl --noproxy X POST --fail -kssfl --url " -H "Content-Type: application/json" --data-binary '{"login": {"domain": "sdn","user": "sdn","password": "skyline"}}' The command generates the authentication token 1759f214479e4ffd9504acb42123ef40. {"record":{"token":"1759f214479e4ffd9504acb42123ef40", "expiration": ,"expirationdate":" ", "userid":"b00cb0e94c9441d58011f980cf9635ae","username":"sdn","domainid": "a6701f6593d84fa5b8f23f9ab4ed69db","domainname":"sdn"}} 5. Determine the team configuration parameters: Parameter Team IP Address Controller IP Address Value The team IP address is different from the individual controller IP addresses. It is used as a virtual address for connecting to the team leader. IP address of a team member. When the virtual address is programmed on the team leader, gratuitous ARP is sent out. The virtual address moves from one controller to another in the team as the leader changes. If any security features are configured to prevent such a move, they should be reconfigured to allow the movement of team IP Address such that it remains reachable for the rest of the network. Enabling MAC groups Procedure 1. To enable source MAC groups, enter the following command, where instance-name is the name of the OpenFlow instance for which you are enabling MAC groups: openflow instance instance-name src-mac-grp-table. 2. To enable destination MAC groups, enter the following command, where instance-name is the name of the OpenFlow instance for which you are enabling MAC groups: openflow instance instance-name dst-mac-grp-table. Disabling MAC groups Procedure 1. To disable source MAC groups, enter the following command, where instance-name is the name of the OpenFlow instance for which you are disabling MAC groups: 2. openflow instance instance-name no src-mac-grp-table. 3. To disable destination MAC groups, enter the following command, where instance-name is the name of the OpenFlow instance for which you are disabling MAC groups: 4. openflow instance instance-name no dst-mac-grp-table. Disabling MAC groups Procedure 1. To disable source MAC groups, enter the following command, where instance-name is the name of the OpenFlow instance for which you are disabling MAC groups: 138 Aruba VAN SDN Controller 2.8 Administrator Guide

139 openflow instance instance-name no src-mac-grp-table. 2. To disable destination MAC groups, enter the following command, where instance-name is the name of the OpenFlow instance for which you are disabling MAC groups: openflow instance instance-name no dst-mac-grp-table. Chapter 8 Configuring OpenFlow instances 139

140 Chapter 9 Backing up and restoring This chapter describes controller backup and restore actions using curl commands. For information about the REST APIs related to backup and restore, see /restore and /backup in the RSdoc facility on the controller. Using a Google Chrome browser window on the controller, enter: Backing up and restoring Best Practices You cannot use RSdoc to download or upload files. Only one backup, restore, upload, or download operation can be active at any time on a given controller or controller team. Parallel operations are not supported. Use standard VM tools (such as Snapshot or Clone) to back up and restore the entire controller image. Use standard Linux server-based tools (such as rsync, LVM snapshot, and Amanda/Zmanda) to back up and restore the controller on bare metal. If you change the name of any of the following files, the controller backup will not be able to backup the renamed file. So before the backup you should manually back up the renamed files. Then after a restore, stop the controller service, copy the renamed files to the appropriate location and restart the controller. /opt/sdn/admin/keystore /opt/sdn/admin/truststore /opt/sdn/admin/sdnjar_trust.jks Backing up a controller A controller backup takes a snapshot of the controller state, and includes the following in a single file: Controller databases License compliance history and metrics log data In a teaming environment, the teaming configuration User repository folder (for user-installed applications) Controller configuration folder Application data for applications that have implemented backup/restore functionality. The controller does not back up the sdnjar_trust.jks file or existing keystore and truststore files in the /opt/sdn/admin directory. If you have CA signed certificates or you have added third party applications to the controller, you must back these files up separately and recover them after restoration. For backing up and restoring, Hewlett Packard Enterprise recommends that you use off-the-shelf solutions such as rsync or Amanda. If your applications use the controller's backup and restore service and you are using Cassandra, then consider the following: 140 Aruba VAN SDN Controller 2.8 Administrator Guide

141 To back up Cassandra, set the backuplockseconds configurable Cassandra lock timer parameter to the size of the data being backed up, the default is 10 minutes. Depending on the size of the backup, Cassandra could be locked for up to 10 minutes. If the applications that are using Cassandra encounter a lock timed out failure during the backup, increase the timer. The failure will generate an error message Cassandra lock timed out before backup was finished. in the /var/log/sdn/virgo/logs/log.log file. To configure the backuplockseconds parameter: 1. Login to the GUI. 2. Click Configurations. 3. From the list of configurations, click com.hp.sdn.teaming.impl.cassandraprocessmanager. 4. Click Modify. See Configuration components on page 37. Examples of curl commands in this guide use the --noproxy option, which is appropriate where execution of curl commands do not need a proxy to access controllers. If your network is set up such that a proxy is needed to access controllers, use the --proxy option. For details on curl proxy options, visit manpage.html. Backup operation A controller backup includes the controller configuration and databases in one *.zip file. Backups run in the background, and, except for locking the Cassandra database to prevent writes, do not interrupt system operation. Whether operating in a team or operating in standalone mode, each controller is backed-up as a single system. When the controller is deployed in a VM, standard VM backup/restore tools (such as Snapshot or Clone) can be used. When the controller is deployed on bare metal, standard Linux server-based backup/restore tools (such as rsync, LVM snapshot, and Amanda/Zmanda) can be used. To complete a teamed backup, no controller can be in a failed state. (A controller team must have three controllers.) On any controller or controller team, only one operation can run at any given time (backup, restore, upload, or download). Also, starting a new backup while another backup is being downloaded creates an error condition and halts the new backup. Only authenticated users are allowed to create and restore backups. In some cases the domain name is also required. The default domain name is sdn. The default username is also sdn. The default password is skyline. The controller does not save a non-default domain, user name, or pass-word across a backup. Changing these settings to non-default values and later backing up the controller, resets these settings to their defaults in the backup file. Later restoring the backup to the controller resets the domain, user name, and password to their default settings in the controller. For backup and restore of the Keystone configuration and database, see Backing up and restoring the Keystone configuration and database on page 148. If uploading a backup fails, then no backup version remains on the system. Starting a new backup replaces any earlier backup remaining in the controller. If a backup is being downloaded when a new backup is started, the new backup halts. Metering time-series data is not encompassed by the controller backup process. There can be a large amount of data, possibly tens of gigabytes in size, which is keyed to time. Not only is the time series data impractical to back up because of its size, but upon restoring it there is a likelihood that some of the restored data will not be Chapter 9 Backing up and restoring 141

142 usable because it will be older than the sliding window of time that metrics are retained for on the controller. However, there is one metering file that is backed up and restored. It contains a mapping of metric descriptor information (such as the ID of the application that created a metric and the metric's primary tag, secondary tag, and name) to the UID that was assigned to each metric. When a restore is performed, this file is restored, and any existing metering time-series data is deleted because it might not match the restored file. The mappings that are restored might, depending upon time elapsed since the backup was taken, be used to assign the same UID to a metric created following the restore (and subsequent controller restart) that was assigned to the metric before the backup was taken. This provides continuity for a metric across the time spanned between backup and restore because all data for the metric is keyed to the same UID. Thus, while time-series data from before the restore was not retained during the restore, UIDs used to key time-series data that was exported to external tools or storage before the restore will continue to be used for the same metrics. Backing up a controller Procedure Acquire the authentication token for the controller backup: curl --noproxy controller_ip -X POST --fail -kssfl --url " -H "Content-Type: application/json" --data-binary '{"login": {"domain": "domain","user": "user","password": "password"}}' Credential information (user name, password, domain, and authentication tokens) used in curl commands might be saved in the command history. For security reasons, Hewlett Packard Enterprise recommends that you disable command history prior to executing commands containing credential information. 2. If needed, increase the Cassandra backuplockseconds configuration using the GUI. See Configuration components on page Acquire the controller uid: curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kssfl --request GET --url " 4. Set the IP address of the controller using the following curl command: curl --noproxy controller_ip> --header "X-Auth-Token:auth_token" --fail -kssfl --request PUT " --data-binary '{"system":{"ip":"controller_ip"}}' 5. Perform the actual backup using the following curl command: curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kss --request POST --url " 142 Aruba VAN SDN Controller 2.8 Administrator Guide

143 6. Get the checksum to verify the backup file has not been corrupted. curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kss -- request GET --url " 7. Check on the status of a backup. curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kssfl --request GET --url " Downloading a backup from the controller to another location The backup file should be downloaded to a secure location. Choose the correct name now; you cannot rename the files later or you will get a file corruption error when you attempt to upload it for a restore. The file name must begin with sdn_controller_backup. Procedure 1. Download the Backup.zip File: curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kssfl --request GET --url " Recommended backup practices Do not run backup while making configuration changes. Instead, run the backup after completing configuration changes. Otherwise, an inconsistent system state could result with a subsequent restore. Always back up all of the controllers in a team after a configuration change. Just backing up a subset of the controllers is not sufficient. Back up all of controllers in a team at approximately the same time. (Team backups can be in sequence or in parallel). Do not allow days to pass in-between backups of different controllers in the same team. If any controller in a team fails to complete the backup, start the backup over for all members of the controller team. A completed backup should be downloaded from the controller to another location for safekeeping. Include the IP address in the backup filename, so you can easily determine which backup belongs to which controller in a team. Recommended file naming is: sdn_controller_backup_ip-address.zip Store the backup files you take off each controller in the team together, so they can easily be retrieved for a future restore. Restoring a controller from a backup Procedure 1. Uninstall the controllers to be restored. If this is a rollback to a previous state, uninstall all controllers. 2. Set CTL_RESTORE_INSTALL_MODE=True in the ~/.sdn_install_options file in the home directory. Chapter 9 Backing up and restoring 143

144 a. If this file is not present in the directory, create it with the CTL_RESTORE_INSTALL_MODE entry. b. If the file is already present, ensure that it includes the CTL_RESTORE_INSTALL_MODE entry. This entry directs the installer to perform the necessary changes to direct the controller to start in recovery/ restore mode, during which OpenFlow activity is suspended for the subject controller. 3. Re-install the failed controllers, making sure to use the same IP address configuration. During the reinstallation, log messages similar to the following appear in the Audit Log: sudo dpkg -i hp-sdn-ctl_1.11_amd64.deb Selecting previously unselected package hp-sdn-ctl. (Reading database files and directories currently installed.) Unpacking hp-sdn-ctl (from hp-sdn-ctl_1.11_amd64.deb)... Setup has detected a compatible jre-headless _25 Creating system group 'sdn'......done. Creating system user 'sdn'......done. Creating system user 'sdnadmin'......done. Configuring PostgreSQL database... * Restarting PostgreSQL 9.1 database server [ OK ]...done. Adding SDN-related items to Keystone... keystone stop/waiting keystone start/running, process done. Setting up hp-sdn-ctl (1.11)... Certificate was added to keystore CTL_RESTORE_INSTALL_MODE option is set SDN controller will be started in restore mode sdna start/running, process sdnc start/running, process Processing triggers for ureadahead... Do not re-install any applications before you complete the restore process. The restoration adds data from the backup file into the current database contents. If you re-install applications that are part of the controller backup, then those applications might end up with duplicate or conflicting entries in their database. If required, only re-install applications after you have completed all steps of the restore process. 4. Edit the /etc/sudoers file to add password-less access for the sdn user to run the required restore scripts: a. Open the /etc/sudoers file with this command: sudo /usr/sbin/visudo b. At the end of the file, add the following line: sdn ALL=(sdnadmin) NOPASSWD: /opt/sdn/cassandra/bin/caserver.sh c. Write out the file by entering CTRL-O, then enter CTRL-X to exit. If there are parse errors when you try to exit, the visudo program displays a warning and a syntax error. Enter e at the prompt to edit the file to correct the error. Do not exit the file until all parse errors are corrected. If the /etc/sudoers file contains parse errors, no users can use sudo to execute commands. 5. Edit the /opt/sdn/backup/restore.sh file: 144 Aruba VAN SDN Controller 2.8 Administrator Guide

145 a. On or near line 62 of the script, locate the command and change it to the following: sudo -u sdn rm -rf "$OPT_ROOT/cassandra/conf/" rm -rf "$OPT_ROOT/cassandra/conf/" b. On or near line 72 of the script, locate the command and change it to the following: sudo -u sdnadmin rm -rf "$OPT_ROOT/cassandra/commitlog/" rm -rf "$OPT_ROOT/cassandra/commitlog/" c. On or near line 79 of the script, locate the command and change it to the following: sudo -u sdn rm -rf "/var/lib/sdn/cassandra/" rm -rf "/var/lib/sdn/cassandra/" The following listing shows the corrected code snippets: 59 #pick the cassandra zip file and unpack 60 restore_log "Deleting cassandra configuration" 61 if [ -d "$OPT_ROOT/cassandra/conf" ]; then 62 rm -rf "$OPT_ROOT/cassandra/conf/" 63 check_and_exit $? 64 fi # 2. Clear all files in /var/lib/cassandra/commitlog 70 restore_log "Deleting the commitlog directory" 71 if [ -d "$OPT_ROOT/cassandra/commitlog/" ]; then 72 rm -rf "$OPT_ROOT/cassandra/commitlog/" 73 check_and_exit $? 74 fi restore_log "Deleting cassandra data directory" 78 if [ -d "/var/lib/sdn/cassandra/" ]; then 79 rm -rf "/var/lib/sdn/cassandra/" 80 check_and_exit $? 81 fi 6. Acquire the authentication token for the system restore: curl --noproxy controller_ip -X POST --fail -kssfl --url " -H "Content-Type: application/json" --data-binary '{"login": {"domain": "domain","user": "user","password": "password"}}' Credential information (user name, password, domain, and authentication tokens) used in curl commands might be saved in the command history. For security reasons, Hewlett Packard Enterprise recommends that you disable command history prior to executing commands containing credential information. Chapter 9 Backing up and restoring 145

146 7. Acquire the controller uid: curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kssfl --request GET --url " 8. Use the following curl command to set the IP address: curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kssfl --request PUT " --data-binary '{"system":{"ip":"controller_ip"}}' 9. Perform a single controller restore onto each controller needing restoration. a. Upload the backup files that will be restored: curl --noproxy controller_ip -X POST --fail -kssfl --url " -H "X-Auth-Token:auth_token" where path-and-file-name is the full path to the file and the filename. The filename MUST match the name you used during the backup. b. Initiate the restore: curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kss --request POST --url " For a controller team, wait for HA synchronization to complete to all the controllers and wait for the team to become connected. The team can take a few minutes to come back up. Be sure to verify that team status has all controllers as active and one of the team members is a leader. curl --noproxy controller_ip --header "X-Auth-Token:auth_token" --fail -kssfl --request GET --url " If less than a quorum of controllers are restored, then those controllers are updated to the latest state of the running team via HA synchronization. (A quorum is n/2+1 where n is the total number of controllers in a team. In a three-controller team, a quorum is two controllers.) If the entire team is restored, then each controller is reset to the previous backed-up state. 10. After the controller restore is complete, change the value of CTL_RESTORE_INSTALL_MODE to false in the ~/.sdn_install_options file on each controller so that it does not impact a future installation. This is because a future installation of the controller might not involve starting in recovery mode. It is possible to query the restore status by using the get command at v2.0/restore/status. Since the restore is not hitless, the REST query fails until the controller has successfully restarted. To restore a controller team, restore each controller as a standalone controller. See Distributed (team) backing up and restoring on page Aruba VAN SDN Controller 2.8 Administrator Guide

147 Attempting to restore a backup taken on any release prior to version 2.6 will not complete. 11. If you have files that were manually backed up prior to restoration, such as truststore or keystore files with CA signed certificates or certificates in the sdnjar_trust.jks file, do the following: a. Stop the controller. b. Copy the backed-up files to their original locations. c. Start the controller. Restore operation To restore a controller from a backup, it is necessary to re-install the controller. During a user-initiated restore operation, the controller GUI is not accessible and the controller is not fully functional until the restore is complete. In a controller team environment each active controller is restored as a single system. If a backed-up controller in a team fails, use single-system restore to restore the controller. The HA synchronization updates the controller to the latest version. The controller blocks traffic over OpenFlow ports during a restore operation. The controller ceases to operate during a restore operation. System restore requirements A system backup can be restored only to a system having the following: The same controller version that existed at the time the backup was taken. The same network settings (IP address) as were present at the backup. The same license ID as was in effect when the controller was installed. If you have modified any environment specific settings in files such as /opt/sdn/virgo/ options or /etc/init/sdnc.conf, ensure that the appropriate changes are made to these files after you re-install the controller and before you start the restore. For example, the network interface that the Virgo service uses (default: eth0) might be eth1 or another setting on some systems. Distributed (team) backing up and restoring In a team environment, all team members must successfully complete the backup. A team backup consists of using the single-system backup process. All controllers in the team must be active, and all of the backups in the team should be done either serially at approximately the same time, or in parallel. To complete a teamed backup, no controller can be in a failed state. (A controller team must have three controllers.) In a team environment, all team members must successfully complete the backup for the backup to be successful. A team restore consists of using the single-system restore process on each controller in the team. Like backups, a system restore in a team should be done either serially at approximately the same time, or in parallel. Two Chapter 9 Backing up and restoring 147

148 controllers must be up and running before either one can become active. If your team has one or more nodes, make sure that all controllers are up and Cassandra is running, as follows:./opt/sdn/cassandra/bin/nodetool ring grep -c 'Up' This command must return 3. You must login to each controller in sequence and run the following command, no matter how many controllers were actually restored:./opt/sdn/cassandra/bin/nodetool repair Do not attempt to run this command at the same time on different nodes. It must run to completion on one node before you run it on another node. This command may have a significant impact on disk and network I/O across all controllers in the team and it may take some time. When restoring a team, be sure to re-install all of the controllers, before initiating the actual restore on any of the controllers. Also, if backing up the team controllers was done serially, then the restore of the team controllers should be done in reverse order. A controller that fails a restore operation is not allowed to rejoin the team, and must be re-added as a new controller. Backing up and restoring the Keystone configuration and database Backup/Restore for the Keystone configuration and database are separate actions from the controller Backup/ Restore. The backup/restore does not backup any Keystone related configuration/credentials therefore any changes made to Keystone will be lost after the restore. These instructions apply to the default local Keystone instance (Keystone ) as specified in the Aruba VAN SDN Controller Installation Guide. If you are using a different Keystone installation, please follow the OpenStack instructions for backup/restore of the Keystone instance specific to your installation. For OpenStack documentation, visit Aruba VAN SDN Controller 2.8 Administrator Guide

149 Chapter 10 Metrics Viewing metric data Metric data created by the controller and applications can assist you when you are troubleshooting issues with the controller or network. The curl commands in this section interact with the metrics/apps REST API to display information about metric data collected by the controller. These commands must be used on individual controllers; they do not return information from controller teams. About metrics Metrics are created by controller components or applications and can be of several different types, such as timers, counters, gauges, and histograms. How metric values are processed When the controller component or application creates a metric, it specifies what to do with the resulting metric values. Metric values can be processed in one or more of the following ways: Persisted to disk as a time-series of values Exposed for inspection by the user via JMX Retained in memory for internal use by the component or application Metric identifiers When a metric is created, it is associated with the following text strings that, taken as a combination, uniquely identify the metric: Application ID (REQUIRED) Identifies the application creating the metric. For example, the application ID for the controller is com.hp.sdn. metric name primary tag secondary tag (REQUIRED) Describes the metric. This name is provided by the application that creates the metric. For example, an application that creates metrics to represent the characteristics of traffic on device ports might use names such as: framestransmitted TxFrames framesreceived RcvFrames Additional description of the metric, such as a category or a device. Primary and secondary tags are optional, but if a secondary tag is provided, a primary tag must be provided. These tags are provided by the application or component that creates the metric. For example, an application that creates metrics to represent the characteristics of traffic on device ports might use a primary tag to identify a device and a secondary tag to identify specific ports on the device. The resulting metrics are structured so that for each device (primary tag) there are multiple ports (secondary tags) that each have several metrics associated with them. Chapter 10 Metrics 149

150 Viewing the application IDs for applications that have persisted metrics to disk Use the application ID as input for metrics commands that require an application ID in the URL. Procedure 1. List the application IDs for all applications (including those that are part of the controller itself) that have persisted metric data to disk: curl --noproxy controller_ip -X GET \ --header "X-Auth-Token:token" --fail -kssfl \ --url " Command example curl --noproxy X GET \ --header "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" --fail -kssfl \ --url " Command output In the above example, only the base controller itself has persisted metric data to disk. The application id is: com.hp.sdn. The next example shows the JSON output as returned by the curl command: {"apps":[{"app_id":"com.hp.sdn","app_name":"hp VAN SDN Controller"}]} The next example shows the JSON output formatted for readability: { } "apps":[ { "app_id":"com.hp.sdn", "app_name":"hp VAN SDN Controller" } ] Viewing the metrics persisted by an application Procedure 1. To list all of the metrics available for an application, use the following curl command: curl --noproxy controller_ip -X GET \ --header "X-Auth-Token:token" --fail -kssfl \ --url " \?name=name&primary_tag=primary_tag&secondary_tag=secondary_tag" 150 Aruba VAN SDN Controller 2.8 Administrator Guide

151 Optional parameters Name and value pairs after the? character, separated by the & character, are optional parameters: name primary_tag The name of the metric. The controller lists only the metrics that have a name that matches the specified value. If you do not specify a name, the controller lists all metrics. The name of the primary tag. The controller lists only the metrics that have a primary tag that matches the specified value. secondary_tag The name of the primary tag. The controller lists only the metrics that have a secondary tag that matches the specified value. Usage If you specify more than one parameter, the controller lists only those metrics that match all the parameters you specify. If you do not specify a value for a parameter, the controller does not filter the results based on that parameter. Command example curl --noproxy X GET \ --header "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" --fail -kssfl \ --url " Command output This example shows a partial listing of the output from the example command. The uid for a metric is the unique identifier assigned to the metric on the controller. { "metrics":[ { "app_id":"com.hp.sdn", "type":"ratio_gauge", "name":"cpuloadsystem", "description":"the recent CPU usage of the system.", "primary_tag":"jvm", "secondary_tag":"operatingsystem", "jmx":false, "persistence":true, "summary_interval":"one", "uid":"42f65cd8-03c3-4cad d513e3c0f" }, { "app_id":"com.hp.sdn", "type":"gauge","name":"committedbytes", "description":"the amount of non-heap memory in bytes committed for the JVM to use.", "primary_tag":"jvm", "secondary_tag":"memorynonheap", "jmx":false, Chapter 10 Metrics 151

152 "persistence":true, "summary_interval":"one", "uid":"b82f5b a23-b5a8-bbda7eec44cb" }, { "app_id":"com.hp.sdn", "type":"gauge","name":"countterminated", "description":"number of JVM threads that had exited.", "primary_tag":"jvm", "secondary_tag":"threads", "jmx":false, "persistence":true, "summary_interval":"one", "uid":"0e9fe62e-01fd-42e9-88a6-f92021a5e786" },... { "app_id":"com.hp.sdn", "type":"rolling_counter", "name":"count", "description":"the number of JVM garbage collections actions during the sampling interval.", "primary_tag":"jvm", "secondary_tag":"garbagecollection", "jmx":false, "persistence":true, "summary_interval":"one", "uid":"d62c49d4-46b3-4c2c-be60-24f6fa6c6bf6" },... Viewing the primary tags for metrics persisted by an application Procedure 1. To list the primary tags associated with the metrics persisted by a specific application, use the following curl command: curl --noproxy controller_ip -X GET \ --header "X-Auth-Token:token" --fail -kssfl \ --url " Optional parameters Name and value pairs after the? character, separated by the & character, are optional parameters: name The name of the metric. 152 Aruba VAN SDN Controller 2.8 Administrator Guide

153 Usage If you do not specify a value for a parameter, the controller does not filter the results based on that parameter. Command example curl --noproxy X GET \ --header "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" --fail -kssfl \ --url " Command output The only primary tag associated with the controller itself in this example is jvm. { } "primaries":[ "jvm" ] Viewing the secondary tags for metrics persisted by an application Procedure 1. To list the the secondary tags associated with the metrics persisted by a specific application, use the following curl command: curl --noproxy controller_ip -X GET \ --header "X-Auth-Token:token" --fail -kssfl \ --url " Optional parameters Name and value pairs after the? character, separated by the & character, are optional parameters: name The name of the metric. The controller lists only the metrics that have a name that matches the specified value. primary_tag The name of the primary tag. The controller lists only the metrics that have a primary tag that matches the specified value. Usage If you specify more than one parameter, the controller lists only those metrics that match all the parameters you specify. If you do not specify a value for a parameter, the controller does not filter the results based on that parameter. Command example curl --noproxy X GET \ --header "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" --fail -kssfl \ Chapter 10 Metrics 153

154 --url " Command output { } "secondaries":[ "niodirectmemory", "operatingsystem", "threads", "garbagecollection", "memorynonheap", "memoryheap", "memorytotal", "niomappedmemory" ] Viewing the names of metrics persisted by an application Procedure 1. To list the names of the metrics persisted by a specific application, use the following curl command: curl --noproxy controller_ip -X GET \ --header "X-Auth-Token:token" --fail -kssfl \ --url " Optional parameters Name and value pairs after the? character, separated by the & character, are optional parameters: primary_tag The name of the primary tag. The controller lists only the metrics that have a primary tag that matches the specified value. secondary_tag The name of the primary tag. The controller lists only the metrics that have a secondary tag that matches the specified value. Usage If you specify more than one parameter, the controller lists only those metrics that match all the parameters you specify. If you do not specify a value for a parameter, the controller does not filter the results based on that parameter. Command example curl --noproxy X GET \ --header "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" --fail -kssfl \ --url " Command output 154 Aruba VAN SDN Controller 2.8 Administrator Guide

155 This example shows a partial listing of the output from the example command. {... } "names":[ "averagebufferusedbytes", "countdeadlocked", "buffercapacitybytes", "count", "countnew", "buffercount", "countwaiting", "filedescriptorsopen", "uptimems", "countterminated", "elapsedms", "counttimedwaiting", "countdaemon", ] "countblocked", "committedbytes" Viewing information about a persisted metric identified by its UID Procedure 1. To list information about a specific metric, which is identified by its UID (unique ID), use the following curl command: curl --noproxy controller_ip -X GET \ --header "X-Auth-Token:token" --fail -kssfl \ --url " Command example curl --noproxy X GET \ --header "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" --fail -kssfl \ --url " Command output { "metric":{ "app_id":"com.hp.sdn", "type":"gauge", "name":"usedbytes", "description":"the amount of heap memory currently being used by the JVM in bytes.", "primary_tag":"jvm", "secondary_tag":"memoryheap", "jmx":false, Chapter 10 Metrics 155

156 } } "persistence":true, "summary_interval":"one", "uid":"431b746e-e62e-4874-a801-b1438eaac635" Viewing the time-series values for a persisted metric identified by its UID Procedure 1. To list the time-series values for a specific metric, which is identified by its UID (unique ID), use the following curl command: curl --noproxy controller_ip -X GET \ --header "X-Auth-Token:token" --fail -kssfl \ --url " Optional parameters Name and value pairs after the? character, separated by the & character, are optional parameters: start end The earliest time to query for time-series data. The date and time in must be in the format yyyymm-dd+hh:mm. If you specify an end but do not specify a start, the value used for start is the time of the oldest instance of the metric that is within the configured age-out time. The most recent time to query for time-series data. The date and time in must be in the format yyyy-mm-dd+hh:mm. If you specify a start but do not specify an end, the value used for end is the time you enter the command. interval The interval each returned data point is to represent. This parameter is required if you specify the start or the end parameter. The interval you specify must be equal to or greater than the interval at which the metric s time-series data is persisted. Choose one of the following values: 1, 5, 15, 30, 60, day, and all, where the numeric values represent the number of minutes, day represents a 24- hour period, and all represents the summarized values over the lifetime of the metric data collection. Usage If you do not specify parameters, only the last persisted time-series value for the metric is returned. Specifying an interval summarizes the data, and enables you to quickly look for anomalous data. For example, instead of viewing 1440 discrete minute-by-minute values for a specific metric over the course of a day, you can request that each returned value represent 60 minutes so that the number of returned values is more manageable and can be more easily inspected. Summarizing the returned data for a period typically results in larger values for the metrics themselves and larger values for the milliseconds spanned. The computation of the values returned over longer intervals depends upon the type of metric. For example, gauge values are averaged over the data points encompassed in the summary, but counter values are summed over the summary interval. Command example curl --noproxy X GET \ 156 Aruba VAN SDN Controller 2.8 Administrator Guide

157 --header "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" --fail -kssfl \ --url " start= :00&interval=5" Command output { } "metric_values":{ "uid":"431b746e-e62e-4874-a801-b1438eaac635", "type":"gauge", "datapoint_count":6, "datapoints":[ { "update_time":"tue Sep 23 18:03:55 PDT 2014", "milliseconds_span":300000, "last": e8 }, { "update_time":"tue Sep 23 18:08:55 PDT 2014", "milliseconds_span":300000, "last": e8 }, { "update_time":"tue Sep 23 18:13:55 PDT 2014", "milliseconds_span":300000, "last": e8 }, { "update_time":"tue Sep 23 18:18:55 PDT 2014", "milliseconds_span":300000, "last": e8 }, { "update_time":"tue Sep 23 18:23:55 PDT 2014", "milliseconds_span":300000, "last": e8 }, { "update_time":"tue Sep 23 18:27:55 PDT 2014", "milliseconds_span":240000, "last": e8 } ] } Viewing all controller JVM metrics Many metrics are not persisted to disk as time-series data, often because they do not change over time. To view all metrics that are tracked by the controller with regard to the JVM, including those that are also persisted as time-series data: Chapter 10 Metrics 157

158 Procedure 1. Create a controller support report. 2. View the data in the support report under the jvm-metrics ID.jvm-metrics ID. Viewing current metric data using a JMX client Procedure 1. You can use JConsole or another JMX client to connect to the controller's JMX server to view metric data as it is updated in real time. Metrics that are viewable using a JMX client The jmx field of the MetricDescriptor used to create the TimeStampedMetric determines whether or not the metric can be viewed using a JMX client. The content exposed for each TimeStampedMetric is contingent on the type of TimeStampedMetric, but typically the most current values used by the TimeStampedMetric are visible as they are updated by the creator of the TimeStampedMetric. Prerequisites For JMX clients to connect to the JMX server of the controller, the following conditions must be true: Procedure 1. The JMX client must be installed on the same system as the controller. 2. No JMX clients are included with the controller or are among the prerequisites for installing it; they must be installed separately. For example, to use the JConsole JMX client, the openjdk-7-jdk package must be installed on the same system as the controller. 3. The controller must be configured to permit local JMX access. For more information about using JConsole and configuring local JMX access, see the JConsole documentation provided by Oracle at: Connecting to the JMX server using the JConsole JMX client Procedure 1. Start the JConsole JMX client. 2. From the New Connection screen, select Local Process. 158 Aruba VAN SDN Controller 2.8 Administrator Guide

159 For an example, see below. Figure 65: JConsole new connection 3. Choose a local connection to the JMX server instance and click Connect. After successfully connecting to that JMX server instance, a screen similar to the screen shown below is displayed. Figure 66: JConsole window Viewing metrics using JConsole JMX Procedure 1. To display the metrics for an application, expand the application folder in the left pane: a. To view metrics for the Aruba VAN SDN Controller and its embedded applications, expand the folder named Aruba VAN SDN Controller. b. To view metrics for an application installed on the controller, expand the folder for the application. Chapter 10 Metrics 159

160 The figure below shows an example of the expanded Aruba VAN SDN Controller application folder with the metrics displayed. Figure 67: JConsole window displaying Aruba VAN SDN Controller metrics 2. Expand the metric you want to view. The name displayed for each metric is a combination of the application ID, metric name, and if present the primary and secondary tags that were specified during the creation of its MetricDescriptor. This combination is unique among all TimeStampedMetric instances monitored for a specific application. 3. Select Attributes to display the metric attributes that are exposed via JMX. For persisted TimeStampedMetric instances, MsSpanned indicates the number of milliseconds that have accrued during the persistence interval that is underway but not yet complete. The figure shows an example of displaying the attributes for a metric. Figure 68: JConsole window displaying details about a metric Generating a controller support report The controller support report provides data, such as JVM metrics, that can be useful when troubleshooting. 160 Aruba VAN SDN Controller 2.8 Administrator Guide

161 Procedure 1. To create a controller support report, use the following curl command: curl --noproxy controller_ip -X GET \ --header "X-Auth-Token:token" --fail -kssfl \ --url " Optional parameters Name and value pairs after the? character, separated by the & character, are optional parameters: id The list of contributors to include in the returned support report data. For example, to include only items reported by the controller JVM metrics, specify the following string: id=jvm-metrics fields The set of fields, separated by commas, to be returned in the report. For example, to include only the title and content fields, specify the following string: fields=title,content If you specify more than one parameter, the support report includes only those items that match all the parameters you specify. If you do not specify a value for a parameter, the support report includes all items for that parameter. Command example curl --noproxy X GET \ --header "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" --fail -kssfl \ --url " Command output The following example contains a partial listing of the support report returned by the previous command: { "support_report":[ { "title":"alert Framework", "id":"alert", "content":[ "Alert-Topics: licensing", "Alert-Count: 7", "Data Retention Age Out: 14 days", "Data Trim Interval: 24 hours", "Data Trim Enabled: true", "Last trim conducted at: Mon Sep 22 19:15:20 PDT 2014" ]}, { "title":"alert Topic Listener", "id":"alert_listener", Chapter 10 Metrics 161

162 {... { "content":[ "No registered alert topic listeners" ]}, "title":"app Manager", "id":"app-management", "content":["installed Applications: 8", "Path Diagnostics, Version: SNAPSHOT, State: ACTIVE", "Link Manager, Version: SNAPSHOT, State: ACTIVE", "Node Manager, Version: SNAPSHOT, State: ACTIVE", "OpenFlow Link Discovery, Version: SNAPSHOT, State: ACTIVE", "OpenFlow Node Discovery, Version: SNAPSHOT, State: ACTIVE", "Path Daemon, Version: SNAPSHOT, State: ACTIVE", "Topology Manager, Version: SNAPSHOT, State: ACTIVE", "Topology Viewer, Version: SNAPSHOT, State: ACTIVE" ]}, "title":"jvm Metrics", "id":"jvm-metrics", "content":[ "Metric count: 44", "Last update time: Wed, 24 Sep :31:55 GMT", "Uptime: 10,037 minute(s)", "Memory", " Total", " Initial: 548,288 kb", " Committed: 740,032 kb", " Maximum: 4,301,824 kb", " Used: 477,308 kb", " Heap", " Initial: 524,288 kb", " Committed: 655,360 kb", " Maximum: 3,728,384 kb", " Used: 393,227 kb", " Usage: %", " Non-Heap", " Initial: 24,000 kb", " Committed: 84,672 kb", " Maximum: 573,440 kb", " Used: 84,081 kb", " Usage: %", "NIO Buffer Memory", " Direct", " Capacity: 0 bytes", " Used: 0 bytes", " Buffers: 0", " Mapped", " Capacity: 0 bytes", " Used: 0 bytes", " Buffers: 0", "Garbage Collection (last 1 minute(s))", " Executions: 0", " Elapsed time: 0 ms", "Threads", " Total count: 122", 162 Aruba VAN SDN Controller 2.8 Administrator Guide

163 ... " By Type", " Daemon: 65", " Non-daemon: 57", " By State", " Blocked: 0", " Deadlocked: 0", " New: 0", " Runnable: 7", " Terminated: 0", " Timed waiting: 24", " Waiting: 91", "Operating System", " CPU Usage", " System: %", " JVM: %", " File Descriptors", " Maximum: 8,192", " Open: 214", " Usage: %" ]}, Chapter 10 Metrics 163

164 Chapter 11 Troubleshooting REST API request returns HTTP code 401 Symptom Getting unauthorized HTTP code 401 for REST calls to the controller. Cause The role-based access control (RBAC) role is not authorized with the token. Aruba VAN SDN Controller 2.5 enforces a single role. By default the single role is sdn-admin. A user must have this role configured on the Keystone server for the domain (tenant) that the user belongs to. Action Procedure 1. Configure a user with the sdn-admin role on the Keystone server for the domain (tenant) that the user belongs to. The domain name and role configured for the user on the controller must match the domain name and role configured for that user in Keystone. a. Create a tenant: curl H "X-Auth-Token:ADMIN" H "Contant-Type: application/json" d '{"tenant": {"enabled": true, "name": "test-tenant", "description": "Test Tenant"}}' b. List tenants: curl H "X-Auth-Token:ADMIN" c. Create a user: curl H "X-Auth-Token:ADMIN" H "Contant-Type: application/json" d '{"user": {" ": "tester@test.rose.hp.com", "password": "somepass", "enabled": true, "name": "test-user", "tenantid": "2c851897a09f483fa452e2de11511f71"}}' <controller-ip>:35357/v2.0/users d. List users: curl H "X-Auth-Token:ADMIN" e. Create a role: curl H "X-Auth-Token:ADMIN" H "Contant-Type: application/json" d '{"role": {"name": "test-role"}}' <controller-ip>:35357/v2.0/os-ksadm/roles f. List users: curl H "X-Auth-Token:ADMIN" Aruba VAN SDN Controller 2.8 Administrator Guide

165 g. Assign role: curl X PUT H "X-Auth-Token:ADMIN" <tenant-id>/users/<user-id>/roles/os-ksadm/<role-id> h. List roles for a user: curl H "X-Auth-Token:ADMIN" <user-id> 2. Use the curl command to request authentication using the default username and password. You must include the keyword domain and the default domain name value, in this example sdn, as follows: curl -sk -H 'Content-Type:application/json' -d '{"login": {"user":"sdn","password":"skyline","domain":"sdn"}}' /sdn/v2.0/auth) Controller not listening on port TCP/8443 Symptom The SDN controller installed successfully but it is not listening on port TCP/8443. Cause You are running a version of Linux that is not supported. Action Procedure 1. Install the controller on Linux Ubuntu version LTS 64-bit server. Packets not received at the end point Symptom HTTP traffic is not received at the end point. Cause In some situations, a switch might not forward HTTP traffic. Action Procedure 1. Check the switch functionality and compatibility with your setup. In some cases, the firmware of the switch does not forward HTTP traffic if there is a copy for it. 2. Check the switch version and possible CPE fixes. In firmware 15.16, if there is an Openflow flow-mod with a copy action for HTTP traffic, it will not forward the HTTP traffic. This issue was fixed in a CPE branch for firmware Session expired message in the UI Symptom You see a message in the controller UI that your login session has expired. Chapter 11 Troubleshooting 165

166 Cause It has been longer than the defined Keystone session timeout since you logged in so your session has expired. The default Keystone login session timeout is 1 hour. Action Procedure 1. Log on to the controller as sudo user. 2. Open the /etc/keystone/keystone.conf file for editing. 3. Locate the line: #expiration= Modify that line by removing the comment (hash tag) and change the value to something larger. 5. Save the file and exit the editor. 6. Restart the Keystone server using this command: service keystone restart 7. Logout from the UI and log back in; the session timeout is increased to the new value. Error running the config_sdn.py script with date/time/ntp option Symptom You run the post install configuration script with the option to change the date and time or NTP server (python config_sdn.py d) and after entering the new date and time or NTP server information, the script will try to access the file /etc/net.conf but the script fails and you see a file permission denied error message. Cause Making changes to the date and time or NTP server information using the GUI will change the permission of the file /etc/net.conf. After that, using the post install script to change date and time or NTP server information (python config_sdn.py d) will fail because the script will try to access the /etc/net.conf file which has had the permission changed. Action Procedure 1. Do one of the following: Once you use the controller GUI to change date and time or NTP server, continue to make any subsequent changes to date and time or NTP server using the GUI rather than the script. Or you can reset the permissions on the /etc/ntp.conf file with the command: sudo chmod 644 ntp.conf. Licensing Redeem quantity error Symptom You see an error message that your license has a maximum redeem quantity. 166 Aruba VAN SDN Controller 2.8 Administrator Guide

167 Cause You specified a license quantity that exceeds what your license type supports. Action Procedure 1. Return to the My Network portal license selection screen. 2. Enter the correct quantity in the Redeem column for your license type: a. For an Aruba VAN SDN Ctrl Base SW w/ 50 node E-LTU license, the quantity must be 1. b. For Aruba VAN SDN Ctrl 50 node E-LTU or Aruba VAN SDN Ctrl HA E-LTU licenses, the quantity can be any quantity on your sales order. Install ID format errors Symptom You see an error message that your Install ID format is invalid. Cause You entered an invalid Install ID, or have not entered an Install ID. Action Procedure 1. Carefully check your Install ID. 2. Return to the license registration details screen and enter a valid value in the Install ID field. Install ID errors Symptom You see an error message that your Install ID is required. Cause The Install ID has not been entered in the portal during the registration process. Action Procedure 1. See the Aruba VAN SDN Controller Administrator Guide for instructions on how to enter your Install ID. Applications that use the Cassandra database are experiencing failures Symptom Applications that use the Cassandra database are experiencing failures, and there are log entries that indicate problems connecting to the Cassandra database. Chapter 11 Troubleshooting 167

168 Action Procedure 1. Log in as the sdnadmin user to Linux on the server on which the controller is installed. 2. Ensure that the Cassandra database is online: ~$ /opt/sdn/cassandra/bin/nodetool status If you see the following message, the Cassandra database is not online: Failed to connect to ' :7199': Connection refused 3. If the Cassandra database is not online, you must restart the controller to restart the Cassandra instance on that controller: a. Close any browser window in which the controller might be running. b. Restart the controller: ~$ sudo service sdnc restart 4. If the Cassandra database stops repeatedly, determine why by doing the following: a. Export the logs files. b. In the sdn-all-logs.zip file, check the /var/log/sdn/cassandra/system.log file for possible errors. Controller support log fills disk space, contains multiple Too many open files messages Symptom The controller support log exceeds its configured maximum size and potentially consumes all available disk space on the system on which the controller is installed. The log contains multiple instances of the following message: Unable to accept incoming connection: java.io.ioexception: Too many open files Cause The system on which the controller is installed has run out of file descriptors, either because it controls too many devices, links, and hosts, or some other process on the system has consumed a large number of file descriptors. Action Procedure 1. Ensure that the system on which the controller is installed conforms to the recommended hardware requirements for the number of devices, links, and hosts. For hardware recommendations, see the Aruba VAN SDN Controller and Applications Support Matrix. 2. Take one or more of following actions: a. Form a controller team and distributing ownership of the switches in the network across the team members such that each controller in the team controls one third of the switches in the network. b. Increase the system resources, such as the number of file descriptors, on the system on which the controller is installed. c. Install the controller on a larger system, such as one that conforms to the hardware recommendations appropriate to the size of the network deployment. Application management errors 168 Aruba VAN SDN Controller 2.8 Administrator Guide

169 Application not starting and in disabled state Symptom An application cannot start and is automatically moved to the DISABLED state. Cause The Application Management framework detected a failure to start an application in the OSGi runtime environment. Action Procedure 1. Correct the OSGi runtime conditions. 2. Enable the application via the GUI or a REST call. Application in transitive state Symptom An application is in a transitive state. Cause An unexpected error condition occurred when manipulating an application (such as file I/O exception or missing files). Action Procedure 1. Examine the log files for exceptions, determine the source of error and correct. 2. Uninstall the application. An application can only be uninstalled when it is trapped in a transitive state. 3. Upload the application. 4. Install the application. Application management exceptions Getting IllegalStateException: HTTP code 500 Symptom Getting IllegalStateException, HTTP code: 500. Cause Pushing a meter or group mod to a connected switch from a controller that is not an owner will not return the proper error message if the switch returns an error. Pushing flows, groups, or meters via a northbound REST API to any controller in the team is supported even if that controller is not the master of the given device. In that case, the controller will delegate the request to the controller who is the master of the switch and the master controller will handle the request. If the switch returns an error from the given request (for example, the meters table is full), the switch responds to the owning controller with a proper Openflow error. But when that error is sent back to the requesting controller, it is not parsed properly and results in an error. Chapter 11 Troubleshooting 169

170 Action Procedure 1. If you are pushing a meter or group mod to a connected switch from a controller, you must make that controller the master in order to handle the errors correctly. Getting UnsafeConfigurationException, HTTP code: 403 Symptom Getting UnsafeConfigurationException, HTTP code: 403. Applies to application, license, and configuration changes. Cause One member of a team is not active. All team members in a teamed environment must be active before you can make configuration, licensing, or application changes, or changes to regions. Otherwise, the configuration is not guaranteed to be synchronized with all the members of the team. Action Procedure 1. Indicates that the operation cannot be completed. Getting ApplicationDisableException, HTTP code: 500 Symptom Getting ApplicationDisableException, HTTP code: 500. Cause Occurs when an application status is STAGED or UPGRADE_STAGED, or that something has gone wrong as specified in error message. Action Procedure 1. Indicates that an application cannot be disabled. Getting ApplicationEnableException, HTTP code: 500 Symptom Getting ApplicationEnableException, HTTP code: 500. Cause Occurs when an application status is not DISABLED, or that something has gone wrong as specified in the error message. 170 Aruba VAN SDN Controller 2.8 Administrator Guide

171 Action Procedure 1. Indicates that an application cannot be enabled. Getting ApplicationInstallException, HTTP code: 500 Symptom Getting ApplicationInstallException, HTTP code: 500. Cause Occurs when an application status is not STAGED, or that something has gone wrong as specified in the error message. Action Procedure 1. Indicates that an application cannot be installed. Getting ApplicationUpgradeException, HTTP code: 500 Symptom Getting ApplicationUpgradeException, HTTP code: 500. Cause Occurs when an application status is not UPGRADE_STAGED, or that something has gone wrong as specified in the error message. Action Procedure 1. Indicates that an application cannot be upgraded Getting ApplicationUninstallException, HTTP code: 500 Symptom Getting ApplicationUninstallException, HTTP code: 500. Cause Occurs when something has gone wrong as specified in the error message. Action Procedure 1. Indicates that an application cannot be uninstalled. Getting ApplicationUploadException, HTTP code: 500 Symptom Getting ApplicationUploadException, HTTP code: 500. Chapter 11 Troubleshooting 171

172 Cause Occurs when an I/O error occurs while uploading the application to the controller. Action Procedure 1. Indicates that an application cannot be uploaded. Getting ApplicationValidationException, HTTP code: 400 Symptom Getting ApplicationValidationException, HTTP code: 400. Cause Occurs when the file format or contents is invalid, or when the signed jar verification failed (if enabled). Action Procedure 1. Indicates that an application zip file fails validation. OpenFlow errors Host location not learned by controller Symptom The host is not present in the node database maintained by the controller. The REST/Java API that gets the node information is missing on that host. Cause The ARP/DHCP/IP (if ip.learn=true in pureof mode) packets might not have reached the controller. Action Procedure 1. Make the host trigger the ARP/DHCP/IP packets. 2. Turn on the OpenFlow trace in the controller. 3. Check that those packets are reaching the controller using the PACKET_IN messages. Switches constantly being disconnected and reconnected Symptom The switches are constantly being disconnected and reconnected. Cause The pure OpenFlow loop topology requires packets to be flooded across the network. This can result in a lot of network traffic and interfere with the switches' ability to send echo packets. 172 Aruba VAN SDN Controller 2.8 Administrator Guide

173 Action Procedure 1. When running a looped topology in pure OpenFlow mode, be aware of how much traffic is being flooded. Some virtual switch implementations might get overwhelmed by the amount of traffic being generated. Unexpected network or service problems in hybrid mode Symptom Unexpected network or service problems. Cause You changed the hybrid.mode configuration of the controller without restarting and disabling the controller, then re-enabling each controlled OpenFlow instance in the OpenFlow switches. Action Procedure 1. Change the hybrid.mode configuration. See the Aruba VAN SDN Controller Administrator Guide for information about changing the hybrid.mode configuration. Troubleshooting teamed environments Controllers dropped from team or unable to form team Symptom A group of SDN controllers fail to form a team, or one or more controllers are dropped from the configured team. Cause The system clocks on one or more controllers are not synchronized with each other. Action Procedure 1. Synchronize all systems (even after a power cycle). 2. If the time is incorrect on boot up and restart the sdnc/sdna service. 3. If you are installing the controller on a Virtual Machine (VM), you must synchronize the host hypervisor with the NTP and NTP client running on the hypervisor host. Because there can be differences in the time reported by different NTP servers, Hewlett Packard Enterprise recommends that servers in a team be configured to use a centralized NTP daemon. If the servers for the controllers in the team are configured such that they connect to different NTP servers, change the configurations of the servers to use a centralized NTP daemon. Teaming framework does not run Symptom The teaming subsystem does not run. Chapter 11 Troubleshooting 173

174 Cause The Iptable rule programming for the teaming framework (Hazelcast) failed. Action Procedure 1. Enter: sudo iptables -D OUTPUT -p tcp --destination-port j DROP 2. Enter: sudo iptables -D INPUT -p tcp --destination-port j DROP 3. Restart the controller. Controller becomes suspended Symptom The controller is suspended and the RSdoc and UI are unavailable. Cause A controller transitions to the suspended state when quorum is lost or core services report critical health. When a team is partially created, a controller might never get quorum, and thus remain suspended, because team creation failed in other controllers. Action Procedure 1. Authenticate using the curl command: curl --noproxy [IP_ADDRESS] -X POST --fail -kssfl --url " 8443/sdn/v2.0/auth" --header "Content-Type: application/json" --data-binary '{"login":{"user":"sdn","password":"skyline","domain":"sdn"}}' For example: curl --noproxy X POST --fail -kssfl --url " :8443/sdn/v2.0/auth" --header "Content-Type: application/json" --databinary '{"login":{"user":"sdn","password":"skyline","domain":"sdn"}}' The curl command in this example generated the following response: {"record":{"token":"2ee24d3a87c345e98deaeaed4bf770ef","expiration": , "expirationdate":" ","userId":"06338e545b9a4f04b145a5ad7c541254", "username":"sdn","domainid":"1685f85a515e4983b4ea64fff9917ecc","domainname":"sdn" }} 2. Check systems status using the curl command: curl --noproxy [IP_ADDRESS] --header "X-Auth-Token:[AUTHENTICATION_TOKEN]" -- fail -kss -L -f --request GET --url " 174 Aruba VAN SDN Controller 2.8 Administrator Guide

175 For example: curl --noproxy header "X-Auth-Token: 2ee24d3a87c345e98deaeaed4bf770ef" --fail -kss -L -f --request GET --url " :8443/sdn/v2.0/systems" The curl command in this example generated the following response: {"systems":[{"uid":"0046a56e-a65e d9e-f1820a285e53","version":" ", "ip":" ","role":"member","core_data_version":16, "core_data_version_timestamp":" t19:30:37.587z","status":"suspended","se lf":true}, {"uid":"1ac1c1e7-1a6e ad3e d57db8","version":" ","ip":" ", "role":"member","core_data_version": 16,"core_data_version_timestamp":" T19:30:38.480Z", "status":"unreachable"},{"uid":"2533a1df-cced-44f9-b9bef6e3851da261","version":" ", "ip":" ","role":"member","core_data_version":16, "core_data_version_timestamp":" t19:30:38.933z","status":"unreachable"}] } 3. Check alerts. a. To get all alerts, use thecurl command: curl --noproxy [IP_ADDRESS] --header "X-Auth-Token:[AUTHENTICATION_TOKEN]" -- fail -kss -L -f --request GET --url " b. To get alerts from a time interval, use the following curl command: curl --noproxy [IP_ADDRESS] --header "X-Auth-Token:[AUTHENTICATION_TOKEN]" -- fail -kss -L -f --request GET --url " start=xxxx-xx-xxtxx:xx:xx.xxxz&end=xxxx-xx-xxtxx:xx:xx.xxxz" For example: curl --noproxy header "X-Auth-Token: 2ee24d3a87c345e98deaeaed4bf770ef" --fail -kss -L -f --request GET --url " start= t19:30:15.000z&end= t19:30:17.000z" The curl command in the preceding example generated the following response: {"alerts":[{"uid":"9ddf298e-a c-b d6bf3e85", "system_uid":"0046a56e-a65e d9e-f1820a285e53","topic":"healthmonitor", "org":"healthmonitor","ts":" t19:30:16.641z","sev":"critical","state": true, "desc":"health Monitor com.hp.sdn.adm.system.impl.quorumregistar changed state to CRITICAL > reason: No quorum"}, {"uid":"0e8deb7d-edcf-4000-a01d-db9bbabcc337","system_uid":"1ac1c1e7-1a6e ad3e d57db8", "topic":"healthmonitor","org":"healthmonitor","ts":" t19:30:16.905z", "sev":"critical","state":true,"desc":"health Monitor com.hp.sdn.adm.system.impl.quorumregistar changed state to CRITICAL > reason: No quorum"}]} Chapter 11 Troubleshooting 175

176 Unable to create team Symptom A group of SDN controllers fail to create a team. Cause The DNS server configured on each controller is unreachable. Action Procedure 1. Resolve the DNS server reachability issue. 2. If the DNS server cannot be reached, add each team member to the /etc/hosts file on each controller. Controller and application data differs among controllers in a team Symptom Controllers and applications in a team do not display the same data, or data appears to be out of synchronization between controllers in a team. Cause The system clocks for the controllers in the team are not synchronized. For controllers that are configured to use NTP (Network Time Protocol) but connect to different NTP servers, the server clocks can drift over time, resulting in a significant relative difference between clock times. Cassandra serializes write operations by timestamp, so differences between system clock times can result in inconsistent data across those systems. Action Procedure 1. Ensure that all the servers on which the controllers in the team are installed are configured to use NTP. 2. Ensure that the servers are configured to use a centralized NTP daemon. Application data is not synchronized after a controller rejoins the team Symptom Data displayed in the user interfaces and the output of programmatic interfaces of applications that use Cassandra differs among the controllers in a team after a controller rejoins the team after a long outage (such as more than 1 hour). Controller data and data for applications that do not use the Cassandra database are not affected. Cause After a Cassandra instance is offline for longer than is supported for automatic synchronization by the Cassandra database, the Cassandra databases in a controller team are not synchronized automatically. 176 Aruba VAN SDN Controller 2.8 Administrator Guide

177 Action Procedure 1. Stop the controller that was offline, remove that Cassandra instance from the cluster, delete all the Cassandra data on the controller, and then restart the controller: 2. Log in to Linux on the server on which the controller with the incorrect data is installed. Typically, this is the controller that came back online after a long outage. 3. Start a root user command shell: ~$ sudo su 4. Stop the controller and administrative services: ~# service sdnc stop ~# service sdna stop 5. Log in to Linux on a server on which a controller with the correct data installed. 6. Start a root user command shell: ~$ sudo su 7. Navigate to the /opt/sdn/cassandra/bin directory and enter the following command: /opt/sdn/cassandra/bin#./nodetool status 8. Copy the Host ID of the Cassandra instance that has a state of DN in the output of the command you entered in the previous step. For example, the last entry in the following output is the Cassandra instance for the controller that was stopped: Datacenter: datacenter1 ======================= Status=Up/Down / State=Normal/Leaving/Joining/Moving -- Addres Load Tokens Owns (effective) Host ID Rack UN GB % 4245b8ab-6c3c-4755-bb d3a4a24 rack1 UN GB % c172bbe2-799c-4adf-bd38-690dfa75ac79 rack1 DN MB % abec-4d80-a689-eb8b1f7f89d1 rack1 9. Remove the Cassandra instance from the cluster by entering the./nodetool removenode HostID command, where HostID is the identifier of the Cassandra instance to remove. For example: /opt/sdn/cassandra/bin#./nodetool removenode abec-4d80-a689- eb8b1f7f89d1 10. Log out of the server: /opt/sdn/cassandra/bin# exit 11. From the Linux prompt of the server for the controller you stopped, delete the Cassandra data: /opt/sdn/cassandra/data# rm -rf * /opt/sdn/cassandra/commitlog# rm * /opt/sdn/cassandra/saved_caches# rm * 12. Restart the controller and Cassandra database, then log out of the server: /opt/sdn/cassandra/bin# service sdnc start /opt/sdn/cassandra/bin# exit Chapter 11 Troubleshooting 177

178 Chapter 12 Websites SDN websites Hewlett Packard Enterprise Networking Information Library for SDN Hewlett Packard Enterprise Software-Defined Networking website Hewlett Packard Enterprise SDN community discussion forum Hewlett Packard Enterprise SDN App Store Hewlett Packard Enterprise Open Source download website community.arubanetworks.com/t5/software-defined- Networking-SDN/bd-p/SDN Networking Websites Hewlett Packard Enterprise Networking Information Library Hewlett Packard Enterprise Networking Software Hewlett Packard Enterprise Networking website Hewlett Packard Enterprise My Networking website Hewlett Packard Enterprise My Networking Portal Hewlett Packard Enterprise Networking Warranty General websites Hewlett Packard Enterprise Information Library For additional websites, see Support and other resources. 178 Aruba VAN SDN Controller 2.8 Administrator Guide

179 Chapter 13 Support and other resources Accessing Hewlett Packard Enterprise Support For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide website: To access documentation and support services, go to the Hewlett Packard Enterprise Support Center website: Information to collect Technical support registration number (if applicable) Product name, model or version, and serial number Operating system name and version Firmware version Error messages Product-specific reports and logs Add-on products or components Third-party products or components Accessing updates Some software products provide a mechanism for accessing software updates through the product interface. Review your product documentation to identify the recommended software update method. To download product updates: Hewlett Packard Enterprise Support Center Hewlett Packard Enterprise Support Center: Software downloads Software Depot To subscribe to enewsletters and alerts: To view and update your entitlements, and to link your contracts and warranties with your profile, go to the Hewlett Packard Enterprise Support Center More Information on Access to Support Materials page: Access to some updates might require product entitlement when accessed through the Hewlett Packard Enterprise Support Center. You must have an HPE Passport set up with relevant entitlements. Customer self repair Hewlett Packard Enterprise customer self repair (CSR) programs allow you to repair your product. If a CSR part needs to be replaced, it will be shipped directly to you so that you can install it at your convenience. Some parts do not qualify for CSR. Your Hewlett Packard Enterprise authorized service provider will determine whether a repair can be accomplished by CSR. Chapter 13 Support and other resources 179

180 For more information about CSR, contact your local service provider or go to the CSR website: Remote support Remote support is available with supported devices as part of your warranty or contractual support agreement. It provides intelligent event diagnosis, and automatic, secure submission of hardware event notifications to Hewlett Packard Enterprise, which will initiate a fast and accurate resolution based on your product's service level. Hewlett Packard Enterprise strongly recommends that you register your device for remote support. If your product includes additional remote support details, use search to locate that information. Remote support and Proactive Care information HPE Get Connected HPE Proactive Care services HPE Proactive Care service: Supported products list HPE Proactive Care advanced service: Supported products list proactivecareadvancedsupportedproducts Proactive Care customer information Proactive Care central Proactive Care service activation Warranty information To view the warranty for your product, see the Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products document, available at the Hewlett Packard Enterprise Support Center: Additional warranty information HPE ProLiant and x86 Servers and Options HPE Enterprise Servers HPE Storage Products HPE Networking Products Regulatory information To view the regulatory information for your product, view the Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products, available at the Hewlett Packard Enterprise Support Center: Additional regulatory information Hewlett Packard Enterprise is committed to providing our customers with information about the chemical substances in our products as needed to comply with legal requirements such as REACH (Regulation EC No 1907/2006 of the European Parliament and the Council). A chemical information report for this product can be found at: 180 Aruba VAN SDN Controller 2.8 Administrator Guide

181 For Hewlett Packard Enterprise product environmental and safety information and compliance data, including RoHS and REACH, see: For Hewlett Packard Enterprise environmental information, including company programs, product recycling, and energy efficiency, see: Documentation feedback Hewlett Packard Enterprise is committed to providing documentation that meets your needs. To help us improve the documentation, send any errors, suggestions, or comments to Documentation Feedback When submitting your feedback, include the document title, part number, edition, and publication date located on the front cover of the document. For online help content, include the product name, product version, help edition, and publication date located on the legal notices page. Chapter 13 Support and other resources 181

182 Appendix A curl commands curl commands The Aruba VAN SDN Controller provides a RESTful web service API. There are several tools available for accessing RESTful web service APIs, one of which is curl. This appendix shows some examples of accessing the controller's RESTul web service API with curl. For details on installing the curl application, see curl.haxx.se/download.html. The curl application has many options, which are described in detail in the curl manual (run "curl --manual") and at The examples in this appendix use minimal options and assume a non-scripted, command line mode of execution and no conflicts with a web proxy. Additional options can be used to customize your experience for your environment. The -k option should only be used when issuing the request against an Aruba VAN SDN Controller with a self-signed certificate, which is installed by default. If a CA signed certificate is installed, the - k option should not be used. See for further details. Credential information (user name, password, domain, and authentication tokens) used in curl commands might be saved in the command history. For security reasons, Hewlett Packard Enterprise recommends that you disable command history prior to executing commands containing credential information. Examples of curl commands in this guide use the --noproxy option, which is appropriate where execution of curl commands does not need a proxy to access controllers. If your network is set up such that a proxy is needed to access controllers, use the "--proxy" option. For details on curl proxy options, visit The following sections describe some typical curl commands: Export audit log data as a CSV file using curl commands on page 183 Licensing actions using curl commands on page 183 Application manager actions using curl commands on page 186 Viewing metric data using curl commands on page 193 Team configuration using curl commands on page 196 About the curl commands The backslash (\) character at the end of the line indicates that the command continues on the next line. In the Bash shell, which you use to enter curl commands, a backslash character that is followed by the newline character is removed from the input stream automatically such that the command is processed as if it were entered on a single line. When using a command in Linux, ensure that you replace any curly or smart quotation marks ( ) with straight quotation marks ("). Examples of curl commands in this document use the --noproxy option, which is appropriate where execution of curl commands does not need a proxy to access controllers. If your network is set up such that a proxy is needed to access controllers, use the --proxy option. Examples of curl commands in this document use the default user name and password for the controller. Your controller user names and passwords might have been changed. 182 Aruba VAN SDN Controller 2.8 Administrator Guide

183 For information about curl proxy options, see the man pages for curl. Getting an authorization token using a curl command Procedure 1. To get an authorization token using curl execute the following command: curl --noproxy controller_ip -X POST --fail -kssfl --url " 8443/sdn/v2.0/auth" -H "Content-Type: application/json" --data-binary '{"login": {"domain": "domain","user": "user","password": "password"}}' The output of the curl command contains the token. 2. Without including the quotation marks, copy the value displayed for token. Example Example input: $ curl --noproxy X POST --fail -kssfl \ --url " \ -H "Content-Type: application/json" \ --data-binary \ '{"login":{"user":"sdn","password":"skyline","domain":"sdn"}}' Example output: {"record":{"token":"237c78769e f ef26", expiration": ,"expirationdate":" ", "userid":"a4fc1cecad844bc280953f983bbdcc26","username":"sdn", "domainid":"ba4e20f1c232401e8f75e9f318c0ae8a","domainname":"sdn"}} Export audit log data as a CSV file using curl commands To export the audit log use the following command: curl [options] -H "X-Auth-Token: token"\ -H "Accept-Type: application/zip" \ \ -o zip-file-name To acquire the token for the command, see Getting an authorization token using a curl command. For example, to export the current content in the controller audit log in a file named auditlogexport.csv inside a zip file: curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ -H "Accept-Type: application/zip" \ \ -o auditlogexport.zip Licensing actions using curl commands Appendix A curl commands 183

184 Obtaining an install ID To acquire the token for the command, see Getting an authorization token using a curl command. Procedure 1. To obtain an Install ID, use the following command to obtain the SDN controller-assigned install_id value. : curl [options] -H "X-Auth-Token:token" \ Replace token with the token created in step 1. Replace controller_ip with your controller IP address. If you are installing a High Availability license, enter the IP address of the lead controller. A numerical install_id appears. For example: Record your install_id for use in the next part of the license registration process. Activating a license on the controller Using your license key, you must now activate a license on the controller, completing the license registration and activation process. Procedure 1. If your previous curl session has closed or timed out, re-enter the authentication command to obtain a new token. 2. Activate the license on the controller: curl [options] -H "X-Auth-Token:token" \ -d license_key \ a. Replace token with the token you obtained using the authentication command. b. Replace license_key with the key obtained in Registering your license and obtaining a license key. You can view the key by logging on to the My Network portal and selecting My Licenses, as shown in Viewing your license and other information. c. Replace controller_ip with your controller IP address. If you are installing a High Availability license, enter the IP address of the lead controller. The installed license information appears in JSON format, as shown below. See: example Installed license output { "license" : { "install_id" : , "serial_no" : 13, "license_metric" : "HA Controller", 184 Aruba VAN SDN Controller 2.8 Administrator Guide

185 } } "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 500, "license_type" : "PRODUCTION", "base_license" : false, "creation_date" : " T00:26: ", "activated_date" : " T00:26: ", "expiry_date" : " T00:26: ", "license_status" : "ACTIVE" Uninstalling licenses to prepare for transfer Prerequisites To uninstall licenses, see Installing, activating, uninstalling, or transferring licenses on page 49. Procedure 1. Obtain information about all installed licenses on your controller: curl [options] -H "X-Auth-Token:token" \ a. Replace token with the token created in step 1. b. Replace controller_ip with your controller IP address. If you are uninstalling a High Availability license, enter the IP address of the lead controller. The installed license information appears in JSON format, as shown below. All installed licenses output { "licenses" : [{ "install_id" : , "serial_no" : 12, "license_metric" : "Controller Node", "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 52, "license_type" : "PRODUCTION", "base_license" : true, "creation_date" : " T00:26: ", "activated_date" : " T00:26: ", "expiry_date" : " T00:26: ", "license_status" : "ACTIVE" },{ } "licenses" : { "install_id" : , "serial_no" : 13, "license_metric" : "HA Controller", "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 500, Appendix A curl commands 185

186 "license_type" : "PRODUCTION", "base_license" : false, "creation_date" : " T00:26: ", "activated_date" : " T00:26: ", "expiry_date" : " T00:26: ", "license_status" : "ACTIVE" }] } 2. Record each serial_no value. 3. Uninstall or deactivate each active license on your controller: curl [options] -H "X-Auth-Token:token" -d deactivate \ a. Replace token with the token you obtained using the authentication command. b. Replace controller_ip with your controller IP address. If you are installing a High Availability license, enter the IP address of the lead controller. c. Replace serial_number with the serial number of the license you want to deactivate. You can view the key by logging on to the My Network portal and selecting My Licenses, as shown in The license uninstall key appears in JSON format, as shown below. License uninstall key output { "license" : { "install_id" : , "serial_no" : 13, "license_metric" : "HA Controller", "product" : "HP VAN SDN Ctrl Base", "metric_qty" : 500, "license_type" : "PRODUCTION", "base_license" : false, "creation_date" : " T00:26: ", "activated_date" : " T00:26: ", "expiry_date" : " T00:26: ", "license_uninstall_key" : "MYOCD9JMCRRRM-IRTEQ2QUNBYCB-6Q6CJIEIJFKIQ- VAI2QUJBYC433" "license_status" : "INACTIVE" } } 4. Record your license_uninstall_key. Security token obtained from output, for example: The license_uninstall_key obtained from the example in the previous step is: MYOCD9JMCRRRM-IRTEQ2QUNBYCB-6Q6CJIEIJFKIQ-VAI2QUJBYC433 Application manager actions using curl commands curl commands can be used to perform actions on embedded or installed SDN applications. 186 Aruba VAN SDN Controller 2.8 Administrator Guide

187 Listing applications Procedure 1. Form curl [options] -H "X-Auth-Token:token" \ 2. Example of listing applications curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ 3. Example output: { } "apps": [ { "action": "NONE", "catalog_id": "", "deployed": " T19:22:49.536Z", "desc": "Path Diagnostic Utility", "download_url": "", "name": "Path Diagnostics", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.hp.sdn.ctl.diag", "vendor": "Hewlett-Packard", "version": " " }, { "action": "NONE", "catalog_id": "", "deployed": " T19:22:50.890Z", "desc": "Link Management", "download_url": "", "name": "Link Manager", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.hp.sdn.ctl.linkdisco", "vendor": "Hewlett-Packard", "version": " " } ] Appendix A curl commands 187

188 Listing information about an application Procedure 1. Form curl [options] -H "X-Auth-Token:token" \ 2. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ 3. Example output { } "app": { "action": "NONE", "catalog_id": "", "deployed": " T19:22:49.536Z", "desc": "Path Diagnostic Utility", "download_url": "", "name": "Path Diagnostics", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.hp.sdn.ctl.diag", "vendor": "Hewlett-Packard", "version": " " } Getting application health status Procedure 1. The HEAD command on health status returns only the response code rather than the entire message for management-type clients that want to poll for health status. Returns HTTP status as follows: 200 for healthy 290 for unhealthy 295 for critical 2. Form curl [options] -H "X-Auth-Token:" -w %{http_code} \ -X HEAD Aruba VAN SDN Controller 2.8 Administrator Guide

189 3. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" -w %{http_code} \ -X HEAD 4. Example output 200 Uploading an application (new or upgrade) Procedure 1. Form curl [options] -H "X-Auth-Token:token" \ -X POST \ 2. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ -X POST \ 3. Example output (new) { "app": { "action": "NONE", "catalog_id": "", "deployed": " T00:00:00.000Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "", "state": "STAGED", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "1.0.0" } } 4. Example output (upgrade) { "app": { "action": "NONE", "catalog_id": "", "deployed": " T23:04:25.955Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "", Appendix A curl commands 189

190 } } "state": "UPGRADE_STAGED", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "2.0.0" Installing a new application Procedure 1. Form curl [options] -H "X-Auth-Token:" \ -X POST \ -d install 2. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ -X POST \ -d install 3. Example output { "app": { "action": "NONE", "catalog_id": "", "deployed": " T21:46:39.845Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "1.0.0" } } Upgrading an application Procedure 1. Form curl [options] -H "X-Auth-Token:token" \ -X POST \ 190 Aruba VAN SDN Controller 2.8 Administrator Guide

191 -d upgrade 2. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ -X POST \ -d upgrade 3. Example output { } "app": { "action": "NONE", "catalog_id": "", "deployed": " T23:04:25.955Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "2.0.0" } Disabling an application Procedure 1. Form curl [options] -H "X-Auth-Token:token" \ -X POST \ -d disable 2. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ -X POST \ -d disable 3. Example output { "app": { "action": "NONE", "catalog_id": "", "deployed": " T23:04:25.955Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "", Appendix A curl commands 191

192 } } "state": "DISABLED", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "2.0.0" Enabling an application Procedure 1. Form curl [options] -H "X-Auth-Token:token" \ -X POST \ -d enable 2. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ -X POST \ -d enable 3. Example output { } "app": { "action": "NONE", "catalog_id": "", "deployed": " T23:04:25.955Z", "desc": "Gee Wiz event production", "download_url": "", "name": "GeeWiz", "product_id": "", "sku": "", "state": "ACTIVE", "uid": "com.geewiz", "vendor": "Gee Wiz, Inc.", "version": "2.0.0" } Removing a staged application Procedure 1. This curl request is used to remove a newly uploaded application before it is installed or upgraded. It has no output. 2. Form curl [options] -H "X-Auth-Token:token" \ -X POST \ 192 Aruba VAN SDN Controller 2.8 Administrator Guide

193 -d cancel 3. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ -X POST \ -d cancel Deleting an application Procedure 1. This curl request is used to shutdown and completely remove all application versions. It has no output. 2. Form curl [options] -H "X-Auth-Token:token" \ -X DELETE 3. Example curl -kss -H "X-Auth-Token:3d61f0d3e e6dbd82ec02c113" \ -X DELETE Viewing metric data using curl commands You can use curl commands view the metric data persisted by the controller components and installed applications. For more information, see Metrics on page 149. Managing SNMP keys Network management systems can use SNMP (Simple Network Management Protocol) to monitor networkattached devices for conditions that require administrative attention. As part of the switch identification process, the controller might use SNMP to obtain more specific information about the switch. SNMP-enabled switches can be configured with community names to provide more secure access. The Key Service component of the controller allows network administrators to configure SNMP security keys so that the controller can communicate with those switches using the secure key. Getting the SNMP keys Procedure 1. To get the SNMP keys known to the controller, enter the following curl command: curl --header "X-Auth-Token: token" -sx GET v2.0/net/keys/snmp Sample command curl --header "X-Auth-Token: 131eaa225ece4293bcebfd7f8e3cffd0" -sx GET localhost:8080/sdn/v2.0/net/keys/snmp Appendix A curl commands 193

194 Sample Response { } "SNMP": [ "Default SNMP key", "Test Key1", "Test Key2", "Test Key3" ] Adding SNMP keys Procedure 1. To add one or more SNMP keys to the controller: curl --header "Content-Type: application/json" --header "X-Auth-Token:token" -sx POST Where inputfile is a file in the local directory that contains the key information in JSON format. Sample command curl --header "Content-Type: application/json" --header "X-Auth-Token: aaea4e0782fa632ee9f04953" -sx POST /sdn/v2.0/net/keys Sample content of inputfile for an SNMP v1 key {"key": { "type": "SNMP", "description": "v1key", "snmp-config": { "snmpversion": "v1", "readcommunityname": "public", "writecommunityname": "public" } } } Sample content of inputfile for an SNMP v3 key {"key": { "type": "SNMP", "description": "authpriv", "snmp-config": { "snmpversion": "v3", "username": "authpriv", "authorization": { "type":"sha", "password":"myshapassword" }, "privacy": { "type":"aes", "password":"myaespassword" } } } } Deleting an SNMP key To delete an SNMP key from the controller, enter the following curl command: curl --header "X-Auth-Token: token" -sx DELETE v2.0/net/keys/snmp/description/description Where description matches the description for the key you want to delete. Sample command curl --header "X-Auth-Token: 131eaa225ece4293bcebfd7f8e3cffd0" -sx DELETE localhost:8080/sdn/v2.0/net/keys/snmp/description/"authpriv" 194 Aruba VAN SDN Controller 2.8 Administrator Guide

195 Managing NETCONF keys Some devices provide support NETCONF (Network Configuration Protocol). The Key Service component of the controller allows network administrators to configure NETCONF security keys so that the controller can communicate with those switches using the secure key. Getting the NETCONF keys Procedure 1. To get the NETCONF keys known to the controller: curl --header "X-Auth-Token: token" -sx GET v2.0/net/keys/netconf Sample command curl --header "X-Auth-Token: 131eaa225ece4293bcebfd7f8e3cffd0" -sx GET localhost:8080/sdn/v2.0/net/keys/netconf Sample Response { } "NETCONF": [ "Default Netconf key" ] Adding NETCONF keys Procedure 1. To add one or more NETCONF keys to the controller: curl --header "Content-Type: application/json" --header "X-Auth-Token:token" -sx POST Where inputfile is a file in the local directory that contains the key information in JSON format. Sample command curl --header "Content-Type: application/json" --header "X-Auth-Token: aaea4e0782fa632ee9f04953" -sx POST /sdn/v2.0/net/keys Sample content of inputfile for a NETCONF key {"key": { "type": "NETCONF", "description": "SampleNetconfKey" } } Deleting a NETCONF key To delete a NETCONF key from the controller, enter the following curl command: curl --header "X-Auth-Token: token" -sx DELETE v2.0/net/keys/netconf/description/description Appendix A curl commands 195

196 Where description matches the description for the key you want to delete. Sample command curl --header "X-Auth-Token: 131eaa225ece4293bcebfd7f8e3cffd0" -sx DELETE localhost:8080/sdn/v2.0/net/keys/snmp/description/"samplenetconfkey" Team configuration using curl commands The following curl commands can be used to configure teaming. Examples of curl commands in this section use the --noproxy option, which is appropriate where execution of curl commands does not need a proxy to access controllers. If your network is set up such that a proxy is needed to access controllers, use the --proxy option. For details on curl proxy options, visit Creating a team using curl Before creating a team ensure that your environment meets the requirements for teaming, see Requirements for controller teams on page 102. And for each controller that will become a member of the team, configure NTP to use a centralized NTP daemon, see Configuring controllers to use the same local NTP servers. Considerations when a controller team is formed using REST The default configuration of the SDN Controller is the system s eth0 interface. When a controller team is formed via REST with the team IP Address, an alias in the controller elected as leader will be configured automatically by the system and will attach to the eth0 interface by default. If the SDN Controller has multiple Ethernet interfaces, a different interface can be required for the team IP Address. In this case the configuration /etc/sdn/admin/ options might be changed using vim or emacs to reflect the desired configuration. sdncontroller:/opt/sdn/admin# cat options export ADMIN_OPTS="-Dcom.hp.sdn.admin.interface=eth0" Once the change has been made, the SDNA service must be restarted as shown with the following command: sdncontroller:/opt/sdn/admin#sudo service sdna restart sdna stop/waiting This change must be made for every active controller within the team and does not require that the team be deleted via REST. To view the team IP Address designation from the SDN Controller console or SSH session, use the ifconfig command: sdncontroller:$ ifconfig eth0 Link encap:ethernet HWaddr ac:16:2d:9a:62:60 inet addr: Bcast: Mask: inet6 addr: fe80::ae16:2dff:fe9a:6260/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets: errors:0 dropped:284 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (684.9 MB) TX bytes: (882.4 MB) Memory:f7f80000-f eth0:0 Link encap:ethernet HWaddr ac:16:2d:9a:62: Aruba VAN SDN Controller 2.8 Administrator Guide

197 lo inet addr: Bcast: Mask: UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Memory:f7f80000-f Link encap:local Loopback inet addr: Mask: inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets: errors:0 dropped:0 overruns:0 frame:0 TX packets: errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes: (32.8 MB) TX bytes: (32.8 MB) Configuring a controller team using curl This section describes configuring a controller team using curl commands. Team authentication is managed with iptables. The team communication channel is not encrypted. If you deploy the team in a highly secure environment, you can always use standard Linux IPSec functionality to encrypt traffic between the team members. You will need to setup bi-directional rules on each controller for communication with each of the other two controllers using the IP addresses specified during the team creation. For simplicity, you can configure host to host IPsec and secure all traffic between the controllers. However, all external communication channels other than teaming are already secured with SSL. If you only want to secure the team communication channel, the rules should be created to only encrypt port Do not use the team leader IP address in any of the IPsec rules. For more information on how to configure IPsec for Ubuntu Linux: Prerequisites (SDN controller 2.8) The default domain and user settings are sdn. The default password setting is skyline. Credential information (user name, password, domain, and authentication tokens) used in curl commands might be saved in the command history. For security reasons, Hewlett Packard Enterprise recommends that you disable command history prior to executing commands containing credential information. Procedure 1. Install and start three standalone controllers in the network. (See the latest Aruba VAN SDN Controller Installation Guide.) 2. Optional: To improve security, you can change the username and password from the default settings on each of the standalone controllers in step Select any one of the controllers to use for configuring the team. 4. On the selected controller, acquire an Authentication Token. Use the following curl command, with the controller IP address, to acquire the token: Appendix A curl commands 197

198 curl --noproxy controller_ip> -X POST --fail -kssfl --url " -H "Content-Type: application/json" --data-binary '{"login": {"domain": "<domain>","user": "<user>","password": "<password>"}}' In this example, the default domain, user name, and password are used. curl --noproxy X POST --fail -kssfl --url " -H "Content-Type: application/json" --data-binary '{"login": {"domain": "sdn","user": "sdn","password": "skyline"}}' The command generates the authentication token 1759f214479e4ffd9504acb42123ef40. {"record":{"token":"1759f214479e4ffd9504acb42123ef40", "expiration": ,"expirationdate":" ", "userid":"b00cb0e94c9441d58011f980cf9635ae","username":"sdn","domainid": "a6701f6593d84fa5b8f23f9ab4ed69db","domainname":"sdn"}} 5. Determine the team configuration parameters: Parameter Team IP Address Controller IP Address Value The team IP address is different from the individual controller IP addresses. It is used as a virtual address for connecting to the team leader. IP address of a team member. When the virtual address is programmed on the team leader, gratuitous ARP is sent out. The virtual address moves from one controller to another in the team as the leader changes. If any security features are configured to prevent such a move, they should be reconfigured to allow the movement of team IP Address such that it remains reachable for the rest of the network. Configuration procedure Procedure 1. Select any active controller to initially configure the team. 2. Enter the following curl command: curl -m noproxy member-1-ip --header X-Auth-Token:auth_token --fail -kss -- request POST --url --data-binary '{"team": {"ip":"team-ip","members":[{"ip":"member-1-ip"},{"ip":"member-2-ip"}, {"ip":"member-3-ip"}]}}' The member-1-ip should be the IP address of the controller chosen to configure the team. After executing the command in step 2, the team elects a team leader. The team leader then configures all team members and normal controller operation begins in the domain. The team creation command does not block until the team creation is complete. You will need to check the status of the system to verify on each controller that the team was successfully created. The team create POST command may now take up to 4 minutes to complete. You will need to make sure the read timeout for the client request is increased accordingly. Configuration example 198 Aruba VAN SDN Controller 2.8 Administrator Guide

199 This example shows a team of controllers configured with the following team member values: Team IP Address: Member IP Addresses: Domain: sdn (the default domain name) Username: myname Password: mypass It is not mandatory that the team IP address be in the same subnet as the member IP addresses. Other IP aliases can be used if the appropriate IP routes are present for the addresses to be reachable and usable. The following steps create and enable the team: 1. Acquire the authentication token: curl --noproxy X POST --fail -kssfl --url " -H "Content-Type: application/json" --data-binary '{"login": {"domain": "sdn","user": "myname","password": "mypass"}}' Command response {"record":{"token":"10f728e477cb4612b07069f339d0ca29","expiration": ,"expirationDate": ", "userid":"51802e12d16345fe9a c1a04e2","username":"sdn","domainid": "d45eca9bde1b4dc78bd7dff69ee9440d","domainname":"sdn"}} 2. Configure the controller team by using the team values and token from step 1: curl -m noproxy header X-Auth-Token: 10f728e477cb4612b07069f339d0ca29 --fail -kss --request POST --url --data-binary '{"team":{"ip":" ","members": [{"ip":" "}, {"ip":" "}, {"ip":" "}]}}' Since team creation is asynchronous, the response is always 202 unless the team configuration (JSON) is not valid or there is a problem configuring the local controller. Possible codes are: 202 Accepted 400 Bad request 401 Unauthorized 503 Service unavailable In case the team is not created in a quorum or if the team is partially created, an alert is posted. Example of an alert for a team partially created Team partially created: [Successes: , ], [Failures: ] The alert does not include the error description, however, the error description is added to the log files. Review the log files to get the cause of the partial team creation. Example of an alert for a team creation that failed in a quorum Appendix A curl commands 199

200 Team could not be created on a quorum If a team cannot be created in a quorum, delete the team and create it again. Error log for team configuration Table 5: Error log for team configuration Log message Build version not consistent on all the systems. Invalid configuration. Local member must be part of the team configuration. Team size must be greater than zero. A team has already been created. Team could not be created on a quorum. Team could not be deleted on a quorum. Team not configured on this system. Description Not all systems on the team have the same controller build version. Update the team as needed to have the same build version. The team configuration JSON is not valid. If the members list from the JSON configuration does not include the system where the team is being created (The local system). Aruba VAN SDN Controller 2.5 only supports a team of 3 controllers. Teaming is already running on the system. Team configuration has failed on a majority of systems. For example, a team of three systems has experienced failures on two systems. A team delete has failed on a quorum number of systems. An attempt has been made on a standalone controller to disband a team. Programming team alias ip-address failed. See Team alias node on page 202. Unprogramming team alias ip-address failed. See Team alias node on page 202. Table Continued 200 Aruba VAN SDN Controller 2.8 Administrator Guide

201 Log message Description Recovering from Partial Team Creation In case the team is not successfully created in all controllers, it is not possible to fix the failed controllers without disbanding the team. To recover from this failure it is recommended to disband the team, fix the problem in the controllers where the create operation failed, and try again. Team configuration can also fail if the iptables rules for Hazelcast or Cassandra are not successfully programmed on the Linux OS. Recovering from Partial Team Deletion If the team was not successfully disbanded in all controllers, the failed controllers might go to suspended mode because they might not have quorum they won t be able to connect to those controllers where the operation was a success. To recover from this failure it is recommended to disband the team on each failed controller so configuration files are removed and so the controllers transition to standalone mode. Or you may need to reinstall the controller. Table 6: Success log Message Team created. Team created with the following configuration: [Team IP: <team ip>, [Members<member list>]. Team disbanded. Programmed Team alias: <team ip>. Unprogrammed Team alias: <team ip>. Description The controller has completed all required steps to configure itself as part of the team. The controller has completed all required steps to configure itself as part of the team. The log entry includes the team configuration provided by the user. The controller has completed all required steps to configure itself as a standalone controller. The controller has configured the team IP address as an alias. This results in the election of a leader. The controller has removed the team IP address from the list of aliases. This results in the election of a leader. Appendix A curl commands 201

202 Table 7: Team IP error log Message Exception while checking alias: <team ip>, <exception> Team alias: <team ip> already programmed Exception while programming alias: <team ip>, <exception> Exception while unprogramming alias: <team ip>, <exception> Description The controller is unable to verify whether the team IP address has been configured as an alias. Represents an invalid state. The controller is trying to configure the team IP address when it has been already configured. The controller was unable to configure the team IP address as an alias. The controller was unable to remove the team IP address from the aliases. Team alias node An IP Address (North-Bound IP) alias is created on the node that is elected as team leader to allow a controller team to be accessible with a single IP Address no matter which controller is the leader. This IP Address is provided as part of the team configuration when creating a team. If the elected node stops being the team leader, the team IP Address must be removed from the interface because this address must be reassigned to the actual team leader. This must be done as the sdn user. If assigning or removing an alias fails, one of the following messages appears in the Alert log: Programming team alias ip-address failed Unprogramming team alias ip-address failed In either of these instances, the condition is logged and the team continues to operate. In this case you can manually program the team alias. You can configure the alias as follows: sudo ifconfig <alias interface> <alias IP> netmask <net mask address> up Example command sudo ifconfig eth0: netmask up You can disable the alias as follows: sudo ifconfig <alias interface> <alias IP> netmask <net mask address> down Example command sudo ifconfig eth0: netmask down Cassandra database maintenance in a team Some applications may choose to use the Cassandra database to store persistent data. For these applications, there is an extra maintenance step that you must run once every ten days to help maintain the performance of the database and the consistency across the team: 1. Make sure all controllers are up and Cassandra is running: /opt/sdn/cassandra/bin/nodetool ring grep -c 'Up' 202 Aruba VAN SDN Controller 2.8 Administrator Guide

203 This command must return Login to each controller in sequence and run the following command. The command must be run as the sdn user. /opt/sdn/cassandra/bin/nodetool repair Do not attempt to run this command at the same time on different controllers. It must run to completion on one controller before you run it on another. The command may have a significant effect on disk and network I/O across all controllers in the team and it may take some time. Run this command during a low usage window. Disbanding a team using curl Disbanding a team returns the teamed controllers to standalone operation. This action initiates the team delete. The REST call might return before the delete has completed. You must check the system to see the running state of the system. Procedure 1. Before disbanding a team, delete the region configuration for that team. (See Removing a region using curl on page 215.) 2. Acquire an authentication token for the team leader. (See step 4 of Prerequisites) 3. Using the token acquired in the step 2, disband the team: curl --noproxy team-ip --header "X-Auth-Token:auth_token" --fail -kssfl --request DELETE --url The deletion of the team can take up to 4 minutes to complete. Increase the read timeout for the client request accordingly. For the controller to be fully operational in standalone after a team is disbanded, you must completely re-install the controller on each node. Since team deletion is asynchronous, the response is always 202 unless there is a problem configuring the local controller as standalone. Possible codes are: 202 Accepted 400 Bad request 401 Unauthorized 503 Service unavailable If you get a returned error of 500, it might be that all regions have not been removed. In case the team is not deleted in a quorum or if the team is partially deleted, an alert is posted: Team partially deleted: [Successes: , ], [Failures: ] The alert does not include the error description, however, the error description is added to the log files. Review the log files to get the cause of the partial team deletion. To recover from this failure, Hewlett Packard Enterprise recommends that you delete the team on each failed controller, which will remove the configuration files and transition the controllers to standalone mode. Example of an alert for a team deletion that failed in a quorum Appendix A curl commands 203

204 Team could not be deleted on a quorum If a team cannot be deleted in a quorum, delete the failed controllers individually. Viewing the team configuration using curl To view your team configuration, do as follows: Procedure 1. Acquire an authentication token for the team leader. (See step 4 of Prerequisites) 2. Using the token acquired in Step 1, view the team configuration as follows: curl --noproxy team-ip --header "X-Auth-Token:auth_token" --fail -kssfl -- request GET --url curl --noproxy header "X-Auth-Token:<auth_token>" --fail -kssfl --request GET --url { "team": { "ip": " ", "revision":0 "members": [ { "ip": " " }, { "ip": " " }, { "ip": " " } ] } } Creating regions using curl To support High Availability (HA) for controllers to OpenFlow switches, create region configurations in the controllers using the REST APIs provided by the Device Owner Service. A region groups devices together with their controllers. Every region has a unique identifier (UID) assigned upon creation. Some REST commands will require that UID to manage the region. A region must have three controllers which must be specified in priority order for all devices within the region (master, primary slave, secondary slave). Devices in a region can be expressed as a list of individual IPv4 addresses, a list of IPv4 ranges, or a combination of both. The Device Owner Service provides high availability between devices and controllers and ensures the availability of a controller to the devices. The Device Owner Service also provides a measure of security; only devices explicitly included in a region can connect to the region s controllers; thus if no regions are defined for the teamed controllers, then no devices will be able to connect to the controllers. Putting the region configurations in place for a controller team ensures seamless failover and failback among the configured controllers for the specified network devices in a region. That is, when a controller experiences a fault, the Device Owner Service ensures that a slave controller immediately assumes the master role over the group of network devices for which the failed controller was master. Once the failed controller recovers and rejoins the team, the Device Owner Service ensures restoration of this controller s role; that is, the rejoining controller takes back the role for which it was configured with respect to the other network devices. If the controller was configured 204 Aruba VAN SDN Controller 2.8 Administrator Guide

205 to operate as the master in a region, then it would be restored to the master role. If it was configured to operate in the slave role, it would resume operation in the slave role. Once the region definition(s) are in place, the Device Owner Service ensures that a master controller is always available to the respective network element(s) even if the configured master fails or there is a disruption of the communication channel between the controller and the network device(s). IPv6 addresses occurring in any region field are not supported and will cause the region to be ignored. All region configuration operations (create, update, refresh, and delete) using the REST API require that every controller specified in the team, including the master controller and all slave controllers, be in an active state. If any controller in the region is in a suspended or unreacheable state, then the region configuration operations are disallowed. Regions and device ownership Regions in standalone environments In a standalone environment, regions are not enforced, and devices are owned by the standalone controller. Regions in teamed environments In a teamed environment, assigning devices and controllers to regions can ensure controller availability to devices when one of the following situations occur: A single controller fails. A single controller is disconnected from the other controllers in the team. A single device is disconnected from one or two controllers in the team. Each region is configured with a prioritized list of controllers, which are used by the Device Owner Service to assign specific roles to each controller: master The highest-priority controller is the configured master controller. The controller that has the role of master for a region is also considered the owner of the devices in the region. For a given device, only the controller that currently has the role of master controller can write to or modify the device. The controller that has the role of master at any given time might not be the configured master controller. For example, the configured master controller might be offline. slave Controllers with this role can read the configurations of the network devices that are managed by the region, but cannot write or modify those configurations. The slave controllers, in priority order, are the configured primary slave and the configured secondary slave. Controllers with a configured role of slave can be assigned the role of master controller temporarily, such as when the configured master controller is offline or a device has lost connection to both the configured master controller and the primary slave controller. Failover behavior within a region Device Owner Service triggers the failover operation in two cases: Controller failure : The Device Owner Service detects a controller failure in a team through notifications from the teaming subsystem. If Device Owner Service determines that the failed controller instance was a master for any devices within a region, it immediately elects an appropriate backup (slave) controller to assume the master role over the affected devices. Device disconnect : The Device Owner Service instance in a controller is notified of a communication failure with network device(s) through the Controller Service notifications. It instantly communicates with all Device Owner Service instances in the team to determine if the network device(s) in question are still connected to any of the backup Appendix A curl commands 205

206 (slave) controllers within the team. If that is the case, it elects one of the slaves to assume the master role over the affected network device(s).the first slave will be chosen as master if it still has connectivity with the device(s), and the second slave will be chosen as master if neither the configured master or first slave have connectivity with the device(s). In this example, the master controller fails. Although it is still in the region, it is unavailable to the devices for which it is the configured master and is no longer the master controller. The primary slave controller becomes the master controller. All 5 devices now belong to the failover master controller Example command curl --noproxy teamip --header "X-Auth-Token:token" -kss --request GET --url Example output { "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"region-red", "prioritizedcontrollerips":[ " ", " ", " " ], "devices":[ { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:01:44:31:92:5c:af:86", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:c8:cb:b8:dd:f0:c0", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:f0:92:1c:21:af:00", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ 206 Aruba VAN SDN Controller 2.8 Administrator Guide

207 } ] } ] }, { } ] { } "dpid":"00:01:cc:3e:5f:6b:19:00", "owningcontrollerip":" " "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:00:9c:02:e0:e4:00", "owningcontrollerip":" " } ] Failback behavior within a region When the configured master recovers from a failure and rejoins the team, or when the connection from the disconnected device(s) with the original master is resumed, Device Owner Service initiates a failback operation in which the master role is restored to the configured master as defined in the region definition. In this example, the configured master controller recovers from failure and is active again. The controller returns back to being a primary slave and all 5 devices belong to the configured master controller Example command curl --noproxy teamip --header "X-Auth-Token:token" -kss --request GET --url Example output { "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"region-red", "prioritizedcontrollerips":[ " ", " ", " " ], "devices":[ { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:01:44:31:92:5c:af:86", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", Appendix A curl commands 207

208 } ] } ] }, { }, { }, { } "datapaths":[ { "dpid":"00:1e:c8:cb:b8:dd:f0:c0", "owningcontrollerip":" " } ] "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:f0:92:1c:21:af:00", "owningcontrollerip":" " } ] "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:01:cc:3e:5f:6b:19:00", "owningcontrollerip":" " } ] "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:00:9c:02:e0:e4:00", "owningcontrollerip":" " } ] Adding a region using curl This POST command adds a region to those configured on the controller and propagates the modifications to each controller in the team. All controllers configured for the team must be available for such a configuration change to be permitted. In this example, we are adding a region with UID 713def9a-4f96-485f-990c-8924bc06c8d8 that has three controllers: (master), (primary slave), (secondary slave). Two devices are part of the region: and The devices IP range includes , , Example command curl --noproxy controllerip --header "X-Auth-Token:token" --header "Content- Type:application/json" -kss --request POST --url v2.0/owners --data-binary '{"region": {"name": "Region-Red", "prioritizedcontrollerips": [" ", " ", " "], "deviceips":[" ", " "],"deviceIpRanges": [" "]}}' 208 Aruba VAN SDN Controller 2.8 Administrator Guide

209 Example output { "region":{ } } "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"region-red", "prioritizedcontrollerips":[ " ", " ", " " ], "deviceips":[ " ", " " ], "deviceipranges":[ " " ] If deviceips or deviceipranges are not in numeric order, they are reordered in the response and in subsequent GET calls of the configuration. Adding a device to a region using curl This POST command adds a device with the specified IP address to the region with the specified UID and propagates the modifications to each controller in the team. A deviceip query parameter must be specified. You can only add one IP address at a time in order to know which device IP address is involved when an error condition is encountered. The new device IP address will be added to the deviceips list, unless it already appeared in the deviceips list or was encompassed by one of the ranges listed among the deviceipranges, in which case no change will be made. All controllers configured for the team must be available for such a configuration change to be permitted. In this example, for region UID 713def9a-4f96-485f-990c-8924bc06c8d8, device is added to the region. Example command curl --noproxy teamip --header "X-Auth-Token:token" -kss --request POST --url Example output {"result":"device IP address now exists in the region with UID '713def9a-4f96-485f-990c-8924bc06c8d8'"} To check if your device was added to the region, check the Team screen in the controller UI or see Getting the configuration of a specific region using curl on page 210. Getting the configuration of all regions using curl This GET command retrieves the configuration of all regions. The regions configuration may have been modified since controller startup to reflect the dynamic addition or removal of regions or devices within specific regions. In this example, there were no changes since controller startup and the configuration is the same is in Adding a region using curl on page 208. Example command Appendix A curl commands 209

210 curl --noproxy teamip --header "X-Auth-Token:token" -kss --request GET --url Example output { } "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"region-red", "prioritizedcontrollerips":[ " ", " ", " " ], "deviceips":[ " ", " " ], "deviceipranges":[ " " ] } ] Getting the configuration of a specific region using curl This GET command retrieves the configuration of the specified region. The regions configuration may have been modified since controller startup to reflect the dynamic addition or removal of regions or devices within specific regions. In this example, there were no changes to the region of interest since controller startup and the configuration is the same is in Adding a region using curl on page 208. Example command curl --noproxy teamip --header "X-Auth-Token:token" -kss --request GET --url Example output { } "region":{ "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"region-red", "prioritizedcontrollerips":[ " ", " ", " " ], "deviceips":[ " ", " " ], "deviceipranges":[ " " ] } 210 Aruba VAN SDN Controller 2.8 Administrator Guide

211 Determining whether a controller owns a specific device using curl The HEAD command returns a status (essentially a Boolean value) indicating whether the local controller owns the device corresponding to the specified IP address; this is decided based on the controller's locally cached data, which is guaranteed to be consistent with the receipt of DeviceOwnerEvents events. You must specify the deviceip query parameter. The command returns one of the following statuses: 204: The local controller owns the device. 404: The local controller does not own the device. Example command curl --noproxy controllerip -X HEAD --header "X-Auth-Token:token" -IL --insecure -- url Example output HTTP/ No Content Server: Apache-Coyote/1.1 X-FRAME-OPTIONS: deny Access-Control-Allow-Origin: * Access-Control-Allow-Methods: GET, POST, PUT, HEAD, PATCH Access-Control-Allow-Headers: Content-Type, Accept, X-Auth-Token Date: Wed, 11 Mar :34:07 GMT Getting the owning controller and devices for a region using curl This GET command retrieves the devices in the region with the specified UID that the specified controller currently owns. You can use an optional controllerip parameter to specify the controller IP address of interest; if no such address is specified, the local controller IP is used. In this example, for region UID 713def9a-4f96-485f-990c-8924bc06c8d8 and controller (master), the devices are , , , , and Regardless of how many IPs are configured for the devices in the region, this GET command indicates only those that are actually active and owned by the specified controller. Example command curl --noproxy controllerip --header "X-Auth-Token:token" -kss --request GET -- url Example output { "ownership":{ "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"region-red", "owningcontrollerip":" ", "devices":[ { "deviceip":" " }, { "deviceip":" " }, { "deviceip":" " }, Appendix A curl commands 211

212 } } ] { }, { } "deviceip":" " "deviceip":" " Getting the status of a specific region using curl This GET command retrieves the current status of the region with the specified UID, including its configured devices and the controller that currently owns each device. In this example, for region UID 713def9a-4f96-485f-990c-8924bc06c8d8, the command output shows the status of all the devices, including their datapaths. In this example all devices are owned by the master controller Example command curl --noproxy teamip --header "X-Auth-Token:token" -kss --request GET --url Example output { "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"region-red", "prioritizedcontrollerips":[ " ", " ", " " ], "devices":[ { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:01:44:31:92:5c:af:86", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:c8:cb:b8:dd:f0:c0", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { 212 Aruba VAN SDN Controller 2.8 Administrator Guide

213 } ] } ] }, { }, { } ] } "dpid":"00:1e:f0:92:1c:21:af:00", "owningcontrollerip":" " "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:01:cc:3e:5f:6b:19:00", "owningcontrollerip":" " } ] "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:00:9c:02:e0:e4:00", "owningcontrollerip":" " } ] Getting the status of all regions using curl This GET command retrieves the current status of all regions, including their configured devices and the controller that currently owns each device. You can filter the returned content of this command in order to get the current status for a specified device, allowing you to determine which region the device is configured in addition to which controller owns the device. You can also specify the optional datapathdetails parameter to enhance the output for devices to include DPID information consisting of each DPID associated with the device and the controller that currently owns each DPID. You can specify this parameter independently of the deviceip parameter, and the resulting output is similar to that when the optional datapathdetails parameter is specified for the GET /owners/ {region_uid}/state command. In this example, there is only one region therefore, the command output is the same as the Getting the status of a specific region using curl on page 212. Example command curl --noproxy teamip --header "X -Auth-Token:token" -kss --request GET --url Example output { "regions":[ { "uid":"713def9a-4f96-485f-990c-8924bc06c8d8", "name":"region-red", "prioritizedcontrollerips":[ " ", " ", " " Appendix A curl commands 213

214 } ] } ], "devices":[ { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:01:44:31:92:5c:af:86", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:c8:cb:b8:dd:f0:c0", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:f0:92:1c:21:af:00", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:01:cc:3e:5f:6b:19:00", "owningcontrollerip":" " } ] }, { "deviceip":" ", "owningcontrollerip":" ", "datapaths":[ { "dpid":"00:1e:00:9c:02:e0:e4:00", "owningcontrollerip":" " } ] } ] Removing a device from a region using curl This DELETE command removes a device with the specified IP address from the region with the specified UID and propagates the modifications to each controller in the team. A deviceip query parameter must be specified. You can only remove one IP address at a time in order to know which device IP address is involved when an error condition is encountered. If the deleted IP address appears among the deviceips addresses, it will be removed 214 Aruba VAN SDN Controller 2.8 Administrator Guide

215 from the list of IPs, and if the deleted IP address was encompassed by one of the ranges listed among the deviceipranges, the range will be transformed into one or more ranges and/or individual IP addresses to reflect the removal of the deleted device IP address and the original range will be removed. All controllers configured for the team must be available for such a configuration change to be permitted. In this example, for region UID 713def9a-4f96-485f-990c-8924bc06c8d8, device is removed. Example command curl --noproxy teamip --header "X-Auth-Token:token" -kss --request DELETE -- url Example output {"result":"device IP address no longer exists in the region with UID '713def9a-4f96-485f-990c-8924bc06c8d8'"} To check if your device was removed from the region, check the Team screen in the controller UI or see Getting the configuration of a specific region using curl on page 210. Removing a region using curl This DELETE command removes the region with the specified UID and propagates the modifications to each controller in the team. All controllers configured for the team must be available for such a configuration change to be permitted. In this example, region UID 713def9a-4f96-485f-990c-8924bc06c8d8 is deleted. Since only one region was configured, there are no regions configured for the team at this point. Therefore, no devices will be allowed to connect to the team until a new region is defined that encompasses the desired devices. Example command curl --noproxy teamip --header "X-Auth-Token:token" -kss --request DELETE -- url Example output {"result":"region with UID 713def9a-4f96-485f-990c-8924bc06c8d8 no longer exists"} Appendix A curl commands 215

216 Appendix B Scripts Scripts Restoring a controller The Restore.sh script restores a controller from a backup file. This script must have permissions set to 770 and be owned by the sdn user and sdn group: #!/bin/bash readonly OPT_ROOT="/opt/sdn" readonly VAR_LIB_SDN="/var/lib/sdn" readonly backupdir=${opt_root}"/backup" readonly targetdir=${backupdir}"/tmp/ com.hp.sdn.adm.backup.impl.backuprestorelegacymanager" readonly configdir=${opt_root}"/config/" readonly repodir=${opt_root}"/virgo/repository/usr" readonly backupfile=${backupdir}"/sdn_controller_backup*.zip" readonly LOG_FILE=${backupDir}"/restore.log" readonly INFO_FILE=${backupDir}"/info.bin" readonly metricsdir=${opt_root}"/virgo/metrics" WAIT_FOR_STOP=120 function restore_log { typeset script_name=${0##*/} typeset DATE_FORMAT=${DATE_FORMAT:-"+%b %e %H:%M:%S"} typeset LOG_PREFIX="$(whoami)@$(hostname)" } echo "$(date "$DATE_FORMAT") $LOG_PREFIX $script_name[$$]: $*" >> $LOG_FILE # For restore, clean virgo runtime environment function clean_virgo_runtime { export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 $OPT_ROOT/virgo/bin/startup.sh -clean -nostart } function get_sdnc_pid { echo $(ps -ef grep -w "/bin/bash" grep -w "/opt/sdn/admin/sdnc.sh" head - n 1 awk '{print $2}') } is_sdnc_running() { pid='get_sdnc_pid' [ "x" = "x$pid" ] && return 1 return Aruba VAN SDN Controller 2.8 Administrator Guide

217 } function restorecassandradata { # 1. shutdown the node - done by stopping sdnc /opt/sdn/cassandra/bin/caserver.sh status if [ $? -eq 0 ]; then restore_log "Cassandra is still running, attempting stop..." sudo -u sdnadmin /opt/sdn/cassandra/bin/caserver.sh stop check_stop_and_exit $? fi #pick the cassandra zip file and unpack restore_log "Deleting cassandra configuration" if [ -d "$OPT_ROOT/cassandra/conf" ]; then rm -rf "$OPT_ROOT/cassandra/conf/" check_and_exit $? fi restore_log "Unzipping the cassandra configuration" unzip -o $2 -d "$OPT_ROOT/cassandra/conf/" # 2. Clear all files in /var/lib/cassandra/commitlog restore_log "Deleting the commitlog directory" if [ -d "$OPT_ROOT/cassandra/commitlog/" ]; then rm -rf "$OPT_ROOT/cassandra/commitlog/" check_and_exit $? fi # 3. Delete system and other data directories restore_log "Deleting cassandra data directory" if [ -d "/var/lib/sdn/cassandra/" ]; then rm -rf "/var/lib/sdn/cassandra/" check_and_exit $? fi } # 4. pick the cassandra zip file and unpack restore_log "Unzipping the cassandra data directory" unzip -o $1 -d "/var/lib/sdn/cassandra/" chmod -R g=u /var/lib/sdn/cassandra function restoreteamconfig { # Restore the teaming config unzip -o $1 -d "$VAR_LIB_SDN" } function check_stop_and_exit { OUT=$1 if [[ $OUT -ne 0 && $OUT -ne 1 ]]; then restore_log "Stopping Cassandra failed and Restore failed : $OUT" rm $INFO_FILE exit 1 fi } function check_and_exit { OUT=$1 Appendix B Scripts 217

218 } if [ $OUT -ne 0 ]; then restore_log "Restore failed:$out" rm $INFO_FILE exit 1 fi function restorepostgre { restore_log "Restoring postgre database..." # unzip the zip of postgre unzip -o postgresqldata.zip -d $targetdir # delete the sdndb first and re-create it dropdb sdndb createdb -O sdn sdndb # remove the extra things that pg_dump back up sed -i '/REVOKE ALL ON SCHEMA public FROM PUBLIC/d' backuppg.sql sed -i '/REVOKE ALL ON SCHEMA public FROM postgres/d' backuppg.sql sed -i '/GRANT ALL ON SCHEMA public TO postgres/d' backuppg.sql sed -i '/GRANT ALL ON SCHEMA public TO PUBLIC/d' backuppg.sql } # this assumes that sdndb database already exists psql sdndb < backuppg.sql check_and_exit $? function restorelicenselogs { if [ -f licenselog.zip ] then restore_log "Restoring license history logs..." } fi # unzip the license logs unzip -o licenselog.zip -d /var/log/sdn/virgo/logs check_and_exit $? function restoremetricsdata { if [ -f metricsdata.zip ] then restore_log "Restoring metrics data..." # wipe out existing contents rm -rf $metricsdir/* } fi # unzip the metrics data unzip -o metricsdata.zip -d "$metricsdir" check_and_exit $? function command_exists { command -v $1 &> /dev/null; } 218 Aruba VAN SDN Controller 2.8 Administrator Guide

219 function wait_for_sdnc_stop { restore_log "Waiting for SDNC to stop..." pid='get_sdnc_pid' if [[ -z $pid ]]; then return 0; fi for tries in 'seq $WAIT_FOR_STOP'; do sleep 1 is_sdnc_running return 0 done } return 1 # check if unzip is present or not if command_exists unzip; then : else restore_log "Unzip is not installed" restore_log "Please install unzip utility and try again" check_and_exit 1 fi wait_for_sdnc_stop check_and_exit $? # extract the backup archive content cd $targetdir for file in 'ls -a *.*' do case $file in config.zip) restore_log "Restoring config files..." unzip -o $file -d $configdir check_and_exit $? ;; teamconfig.zip) restore_log "Restoring teaming Config files..." restoreteamconfig $file check_and_exit $? ;; userrepo.zip) restore_log "Restoring user repository..." rm -rf $repodir/* unzip -o $file -d $repodir check_and_exit $? ;; cassandradata.zip) restore_log "Restoring cassandra data files..." restorecassandradata $file cassandraconfig.zip check_and_exit $? ;; *) ;; esac done # restore postgre sql Appendix B Scripts 219

220 restorepostgre # restore licensing compliance history logs restorelicenselogs # restore metrics data restoremetricsdata # clean up virgo runtime environment clean_virgo_runtime # create links to /var/log/sdn/virgo [! -L /opt/sdn/virgo/serviceability ] && ln -s /var/log/sdn/virgo /opt/sdn/virgo/ serviceability # Change permissions in case the user IDs have changed since the backup. sudo chown -R sdn:sdn /opt/sdn /var/lib/sdn /var/log/sdn/virgo/logs/licensehistory.log restore_log "Turning off the restore mode..." # delete the restore.indicator file [ -f /opt/sdn/backup/restore.indicator ] && rm /opt/sdn/backup/restore.indicator restore_log "Restore done..." Backing up a controller team Because the scripts in this appendix cross page boundaries, be careful to avoid including the page number when copying a script. Copying a script one page at a time can prevent inclusion of page numbers. #!/bin/bash # # Copyright 2013 Hewlett Packard Co., All Rights Reserved. # # # Backup a Team # export BACKUP_DIR="/opt/sdn/backup" export BACKUP_TEAM_DIR="/opt/sdn/team_backup" export TEAM_BACKUP_STATUS_FILE="$BACKUP_TEAM_DIR/teamBackup_status" export TEAM_BACKUP_LOGFILE="$BACKUP_TEAM_DIR/teamBackup_log.log" export BACKUP_WAIT_COUNT=200 # this * 10 = seconds to wait for backup to finish export B_PID=$$ trap "exit 1" TERM #============================================================================== # F U N C T I O N S #============================================================================== # # Function validateteamlead ( ) # Validates configured node IP against the team leader IP. # function validateteamlead { leaderip='ifconfig grep -o $leaderip' if [ "$leaderip" == "" ]; then 220 Aruba VAN SDN Controller 2.8 Administrator Guide

221 teambackup_log "Run this script from the team lead node." exitbackup 1 fi teambackup_log "Leader node IP $leaderip is correctly configured." } # # Function validateteambackupstatus ( ) # Checks if a new backup can be started. # function validateteambackupstatus { TEAM_BACKUP_ON="backup_in_progress=true" # Check if any backup is going on now. if [ -e "$TEAM_BACKUP_STATUS_FILE" ]; then teambackup_log "Backup status file $TEAM_BACKUP_STATUS_FILE exists." backupstatus='cat $TEAM_BACKUP_STATUS_FILE' if [ "$backupstatus" == "$TEAM_BACKUP_ON" ]; then teambackup_log "Backup already in progress, aborting new backup..." exitbackup 1 fi fi rm -rf $BACKUP_TEAM_DIR mkdir $BACKUP_TEAM_DIR chmod 777 $BACKUP_TEAM_DIR echo $TEAM_BACKUP_ON>$TEAM_BACKUP_STATUS_FILE teambackup_log "No backup is currently in progress. A new backup can start." } # # Function backupnode ( <nodeindex> ) # Backs up a node. # function backupnode { local nodeindex=$1 local backuptoken=${nodeauth[$nodeindex]} local backupip=${iparr[$nodeindex]} local backupuuid=${nodeuuid[$nodeindex]} backupurl=" post $backupip $backuptoken "$backupurl" if [ $errorcode -ne 0 ]; then teambackup_log "Failed to start backup for $backupip." exitbackup 1 fi if [ "$sessionid" == "" ]; then teambackup_log "Failed to start backup on $backupip." exitbackup 1 fi echo $sessionid } # # Function downloadbackupset ( <nodeindex> ) # Downloads the backup file from each node to the team leader node, verifying the checkum. # function downloadbackupset { local nodeindex=$1 local backupauth=${nodeauth[$nodeindex]} local backupip=${iparr[$nodeindex]} local backupuuid=${nodeuuid[$nodeindex]} local filename="" if [ "$backupip" == "$leaderip" ]; then filename="$backup_team_dir/sdn_controller_backup_$backupip.leader.zip" else filename="$backup_team_dir/sdn_controller_backup_$backupip.zip" fi Appendix B Scripts 221

222 backupurl=" 'get $backupip $backupauth $backupurl > $filename' expected='get $nodeip "v2.0/backup/checksum"' actual=$(sha256sum "$filename" cut -d ' ' -f1) if [ "$expected"!= "$actual" ]; then echo "Checksum failure: expected $expected but got $actual." exitbackup 1 fi teambackup_log "Successfully copied backup MD5 file from $backupip." } } # # Function verifybackupstatus ( <nodeindex> ) # Verifies the success of the backup. # function verifybackupstatus { local nodeindex=$1 local backupip=${iparr[$nodeindex]} local backupurl=" backupstatus[$nodeindex]='get $backupip ${nodeauth[$nodeindex]} $backupurl' if [ "${backupstatus[$nodeindex]}" == "SUCCESS" ]; then teambackup_log "Backup completed successfully on $backupip." let "backup_complete = $backup_complete - 1" return fi } # # Function teambackupzip ( ) # Creates a single zip for all the team backup data. # function teambackupzip { teamzip='date tr ' ' '_' tr ':' '_'' teamzip="$backup_team_dir/sdn_team_backup_$teamzip.zip" rm -rf $BACKUP_TEAM_DIR/sdn_team_backup* $TEAM_BACKUP_STATUS_FILE zip -r $teamzip $BACKUP_TEAM_DIR/ rm -rf $BACKUP_TEAM_DIR/sdn_controller_backup* } # # Function remotebackupfilecopy ( ) # Copies the team backup zip to the specified remote location. # function remotebackupfilecopy { if [ "$remotepath" == "" ]; then teambackup_log "Team backup data was not copied to the remote location." return fi teambackup_log "Copying team backup to the remote location $remotepath..." scp $BACKUP_TEAM_DIR/sdn_team_backup* $remotepath } # # Function getsysinfo ( <authtoken> ) # Gets the SysInformation for the running node. # function getsysinfo { local leadauth=$1 local sysurl=" for i in {1..5}; do sysinfo='get localhost $leadauth "$sysurl"' if [ $errorcode -ne 0 ]; then teambackup_log "Failed to retrieve the system information." exitbackup Aruba VAN SDN Controller 2.8 Administrator Guide

223 fi [ "$sysinfo"!= "" ] && break sleep 5 done if [ "$sysinfo" == "" ]; then teambackup_log "Failed to retrieve the system information." exitbackup 1 fi } # # Function extractrole_nodeip ( <systeminfo>) # Extracts IP and role for all the nodes in a team. # function extractrole_nodeip { sysinfo=$1 iparr=($(echo $sysinfo tr -d '"' tr -d '[' tr -d ']' sed -e 's/\,/\n/g' grep -w "ip" cut -d ':' -f2-)) rolearr=($(echo $sysinfo tr -d '"' tr -d '[' tr -d ']' sed -e 's/\,/\n/g' grep - w "role" cut -d ':' -f2-)) numnodes=${#iparr[@]} teambackup_log "Number of nodes in the team is $numnodes." for (( i=0; i<=$numnodes; i++ )); do if [ "${rolearr[$i]}" == "leader" ]; then leaderip=${iparr[$i]} teambackup_log "The team leader is $leaderip." break fi done } # # Function teambackup_log ( <message> ) # Writes messages to the log for the team backup operation. # function teambackup_log { msg="$1" echo "$msg" tee -a $TEAM_BACKUP_LOGFILE } # # Function exitbackup ( <exitstatus> ) # Exits the backup. # function exitbackup { [ $1 -ne 0 ] && teambackup_log "Stopping backup/restore with errors." rm -rf $TEAM_BACKUP_STATUS_FILE kill -s TERM $B_PID exit $1 } # # Function get ( <ipaddr> <authtoken> <url> ) # Performs a GET. # function get { local getip=$1 local gettoken=$2 local geturl=$3 local attempts=0 while [ $attempts -lt 5 ]; do curl --noproxy $getip --header "X-Auth-Token:$getToken" \ --fail -kss -L -f --request GET --url "$geturl" errorcode=$? let "attempts = $attempts + 1" if [ 35 -eq $errorcode ]; then teambackup_log "SSL error on GET of $geturl, retrying..." continue; Appendix B Scripts 223

224 fi break; done } # # Function post ( <ipaddr> <authtoken> <url> <data>) # Performs a POST of the specified data. # function post { local postip=$1 local posttoken=$2 local posturl=$3 local postdata=$4 local attempts=0 while [ $attempts -lt 5 ]; do postres='curl --noproxy $postip --header "X-Auth-Token:$postToken" \ --fail -kss --request POST --url "$posturl" --data-binary "$postdata"' errorcode=$? let "attempts = $attempts + 1" if [ 35 -eq $errorcode ]; then teambackup_log "SSL error on POST to $posturl, retrying..." continue; fi break; done echo $postres } # # Function put ( <ipaddr> <authtoken> <url> <data> ) # Performs a PUT of the specified data. # function put { local putip=$1 local puttoken=$2 local puturl=$3 local putdata=$4 local attempts=0 while [ $attempts -lt 5 ]; do putres='curl --noproxy $putip --header "X-Auth-Token:$putToken" \ --fail -kss -L -f --request PUT "$puturl" --data-binary "$putdata"' errorcode=$? let "attempts = $attempts + 1" if [ 35 -eq $errorcode ]; then teambackup_log "SSL error on POST to $puturl, retrying" continue; fi break; done echo $putres } # # Function extractjsonstring ( <json> <fieldname> ) # Extracts the Json value corresponding to the field name. # function extractjsonstring { json=$1 field=$2 json='echo $json tr -d '"' sed -e 's/\,\ {/\n/g' grep -w "$field" \ cut -d ':' -f2-' echo $json } # # Function getauthtoken ( <ipaddr> ) 224 Aruba VAN SDN Controller 2.8 Administrator Guide

225 # Log-in and get the UID. # function getauthtoken { local nodeip=$1 url=" login="{ \"login\": { \"domain\": \"$domain\", \"user\": \"$user\", \"password\": \"$pass\" } }" # Attempt to authenticate and extract token if successful. auth=$(curl --noproxy $nodeip -X POST --fail -kssfl --url "$url" \ -H "Content-Type: application/json" --data-binary "$login" 2>&1) if [ $? -ne 0 ]; then teambackup_log "Unable to authenticate as user $user in $domain domain." exitbackup 1 fi authtoken='extractjsonstring "$auth" "token" sed '/^$/d'' if [ $restore_mode -ne 1 ] && [ "$authtoken" == "" ]; then teambackup_log "Failed to get the authentication token." exitbackup 1 fi echo $authtoken } #============================================================================== # M A I N #============================================================================== restore_mode=0 # Check for zip package. command -v zip &> /dev/null if [ $? -ne 0 ]; then echo "The zip package must be installed to use this script." exit 1 fi # Check the user specified script parameters. if [ $# -lt 2 ]; then echo "Usage : backupteam <user> <domain> [<user@ip:path>]" echo " <user> - user name to access the controller" echo " <domain> - domain of the controller" echo " [<user@ip:path>] - remote location to store backup file" echo " user - the login name for the system" echo " ip - the ip address of the system" echo " path - where to copy the file to on the remote system" exit 1 fi validateteambackupstatus user="$1" echo -n "Enter Controller Password: " read -s pass echo domain="$2" remotepath=$3 errorcode=0 # Get the authentication token for the local controller. leaderauth='getauthtoken localhost' # Get the system Information for the local controller. getsysinfo $leaderauth # Get the set of team IPs and their associated team roles. extractrole_nodeip $sysinfo (validateteamlead) # Initiate a backup on each node. for (( i=0; i<$numnodes; i++ )); do Appendix B Scripts 225

226 nodeauth[$i]='getauthtoken ${iparr[$i]}' uuidurl=" nodeuuid[$i]='get ${iparr[$i]} ${nodeauth[$i]} "$uuidurl?ip=${iparr[$i]}"' nodeuuid[$i]='extractjsonstring "${nodeuuid[$i]}" "uid" sed '/^$/d'' if [ "${iparr[$i]}" == "$leaderip" ]; then # Skip the leader backup backup, since it will be done last. leaderindex=$i continue fi backupnode $i teambackup_log "Started backup on ${iparr[$i]}." done # Verify the status of the backup on each node. backup_complete=$numnodes waittime=$(($backup_wait_count*10/60)) for (( k=0; k<$backup_wait_count; k++ )); do if [ $backup_complete -le 1 ]; then teambackup_log "Backup on all member nodes completed successfully." break fi sleep 10 for (( i=0; i<$numnodes; i++ )); do # Skip the leader node check, since it will be done last. [ "${iparr[$i]}" == "$leaderip" ] && continue # Backup already completed for this node, so continue. [ "${backupstatus[$i]}" == "SUCCESS" ] && continue verifybackupstatus $i done done if [ $backup_complete -gt 1 ]; then teambackup_log "Backup of all member nodes took longer than $waittime min. Aborting backup..." teambackup_log "To increase backup wait time, change BACKUP_WAIT_COUNT in the script." exitbackup 1 fi # Last, backup the leader node to avoid synchronization issues on a restore. backupnode $leaderindex teambackup_log "Started backup on leader ${iparr[$leaderindex]}." backup_complete=1 # Verify the backup on the leader node. for (( k=0; k<$backup_wait_count; k++ )); do sleep 10 verifybackupstatus $leaderindex if [ $backup_complete -le 0 ]; then teambackup_log "Backup on the leader node completed successfully." break fi done if [ $backup_complete -gt 0 ]; then teambackup_log "Backup of the leader node took longer than $waittime min. Aborting backup..." teambackup_log "To increase backup wait time, change BACKUP_WAIT_COUNT in the script." exitbackup 1 fi # Copy all the backup files from each node in the team onto the leader node. for (( i=0; i<$numnodes; i++ )); do downloadbackupset $i done # Create one zip for entire team and copy it to the specified remote location. teambackupzip 226 Aruba VAN SDN Controller 2.8 Administrator Guide

227 remotebackupfilecopy echo teambackup_log "The team was backed up successfully." exitbackup 0 Restoring a controller team Before running this script, re-install the controller. Otherwise an Error 404 condition results and the controller is not restored. See Restoring a controller from a backup. Because the scripts in this appendix cross page boundaries, be careful to avoid including the page number when copying a script. Copying a script one page at a time can prevent inclusion of page numbers. #!/bin/bash # # Copyright 2013 Hewlett Packard Co., All Rights Reserved. # # # Restore a Team # export BACKUP_DIR="/opt/sdn/backup" export BACKUP_TEAM_DIR="/opt/sdn/team_backup" export RESTORE_TEAM_DIR="/opt/sdn/team_restore" export TEAM_BACKUP_STATUS_FILE="$RESTORE_TEAM_DIR/teamRestore_status" export TEAM_BACKUP_LOGFILE="$RESTORE_TEAM_DIR/teamRestore_log.log" export RESTORE_BACKUP_FILESET="$RESTORE_TEAM_DIR/opt/sdn/team_backup" export B_PID=$$ trap "exit 1" TERM #============================================================================== # F U N C T I O N S #============================================================================== # # Function extract_zip_and_ip ( ) # Extracts the team backup zip and the backed up IP addresses. # function extract_zip_and_ip { unzip -o "$RESTORE_TEAM_DIR/sdn_team_backup*" -d $RESTORE_TEAM_DIR if [ $? -ne 0 ]; then teambackup_log "Failed to unzip the team backup file." exitbackup 1 fi teambackup_log "Extracted the team backup file successfully." rm -rf "$RESTORE_TEAM_DIR/sdn_team_backup*" backupip=($(ls $RESTORE_BACKUP_FILESET grep "zip$" sed "s/.zip//" \ sed "s/.leader//" sed "s/sdn_controller_backup_//")) numbackup=${#backupip[@]} teambackup_log "Found $numbackup backup file sets in the team backup file." } # # Function create_restoredir ( ) # Creates the team restore directory. # function create_restoredir { rm -rf $RESTORE_TEAM_DIR mkdir $RESTORE_TEAM_DIR chmod 777 $RESTORE_TEAM_DIR } # # Function validate_my_ip ( ) # Validates the configured node IP against the backed up IP addresses. Appendix B Scripts 227

228 # function validate_my_ip { for (( v=0; v<numbackup; v++ )); do myip='ifconfig grep -o "${backupip[$v]}"' if [ "$myip"!= "" ]; then teambackup_log "IP $myip is a valid member of the team." return fi done teambackup_log "IP $myip is not a valid member of the team, exiting." exitbackup 1 } # # Function upload_backup_file ( <systemip> <systemuuid> <authtoken> <zipfile> ) # Uploads backup file to the specific nodes of the team. # function upload_backup_file { local sysip=$1 local sysuuid=$2 local sysauth=$3 local uploadurl=" local zipfile=$4 if [! -f $zipfile ]; then teambackup_log "File $zipfile does not exist." exitbackup 1 fi curl --noproxy $sysip -X POST --fail -kssfl --url $uploadurl \ -H "X-Auth-Token:$sysAuth"\ if [ $? -ne 0 ]; then teambackup_log "Failed to upload backup $zipfile to $sysip." exitbackup 1 fi teambackup_log "Backup $zipfile uploaded successfully to $sysip." } # # Function restore_node ( <systemip> <systemuuid> <authtoken> ) # Restores a particular node. # function restore_node { local sysip=$1 local sysuuid=$2 local sysauth=$3 local restoreurl=" # Set the IP first. Ignore errors since this only works for standalone. put $sysip $sysauth " \ "{\"system\":{\"ip\":\"$sysip\"}}" > /dev/null 2>&1 restoresession='post $sysip $sysauth $restoreurl ' if [ $errorcode -ne 0 ]; then teambackup_log "Failed to start restore on node $sysip." exitbackup 1 fi teambackup_log "Started restore on node $sysip." } # # Function validate_node_status ( ) # Validates node status after the restore. # function validate_node_status { local sysip=$1 # Wait for the restore to complete. local sysurl=" for (( k=0; k<100; k++ )); do sleep Aruba VAN SDN Controller 2.8 Administrator Guide

229 authtoken='getauthtoken $sysip' [ "$authtoken" == "" ] && continue # Try to contact the system. data='get $sysip $authtoken "$sysur?ip=$sysip"' [ "$data" == "" ] && continue teambackup_log "Node:$sysIp came up successfully." && return done teambackup_log "Node:$sysIP failed to come up." exitbackup 1 } # # Function restore_nodes ( <ipaddrarray> ) # Restores only the specified node(s). # function restore_nodes { local leaderindex=-1 local restoreiparr=("$@") local numnodes=${#restoreiparr[@]} for (( i=0; i<$numnodes; i++ )); do # Get the auth token for a specific node. restoreauth[$i]='getauthtoken ${restoreiparr[$i]}' if [ "${restoreauth[$i]}" == "" ]; then teambackup_log "Failed to get the auth Token for ${restoreiparr[$i]}, can't start restore." exitbackup 1 fi uuidurl=" restoreuuid[$i]='get ${restoreiparr[$i]} ${restoreauth[$i]} "$uuidurl"' if [ "${restoreuuid[$i]}" == "" ]; then teambackup_log "Failed to get the UUID for ${restoreiparr[$i]}, can't start restore." exitbackup 1 fi restoreuuid[$i]='extractjsonstring "${restoreuuid[$i]}" "uid" sed '/^$/d'' teambackup_log "UUID for ${restoreiparr[$i]} is ${restoreuuid[$i]}" # Upload the backup files to a specific node. local ipfilename="sdn_controller_backup_${restoreiparr[$i]}*.zip" local zipfile='ls $RESTORE_BACKUP_FILESET/$ipFileName' upload_backup_file ${restoreiparr[$i]} ${restoreuuid[$i]} \ ${restoreauth[$i]} $zipfile # Check if this is the leader node from the backup set. local leaderzip='echo $zipfile grep "Leader"' [ "$leaderzip"!= "" ] && leaderindex=$i done # Start restore in the leader node first before all the other nodes. if [ $leaderindex -ne -1 ]; then restore_node ${restoreiparr[$leaderindex]} ${restoreuuid[$leaderindex]} \ ${restoreauth[$leaderindex]} fi # Verify the leader node is up after the restore. validate_node_status ${restoreiparr[$leaderindex]} # Continue restore on the remaining nodes. for (( i=0; i<$numnodes; i++ )); do # Skip the leader node; it's already done. [ $i -eq $leaderindex ] && continue # Restore the specified node. restore_node ${restoreiparr[$i]} ${restoreuuid[$i]} ${restoreauth[$i]} done sleep 200 # Validate that the restored nodes are up. for (( n=0; n<$numnodes; n++ )); do # Skip the leader node; it's already done. [ $n -eq $leaderindex ] && continue validate_node_status ${restoreiparr[$n]} Appendix B Scripts 229

230 done } # # Function teambackup_log ( <message> ) # Writes messages to the log for the team backup operation. # function teambackup_log { msg="$1" echo "$msg" tee -a $TEAM_BACKUP_LOGFILE } # # Function exitbackup ( <exitstatus> ) # Exits the backup. # function exitbackup { [ $1 -ne 0 ] && teambackup_log "Stopping backup/restore with errors." rm -rf $TEAM_BACKUP_STATUS_FILE kill -s TERM $B_PID exit $1 } # # Function get ( <ipaddr> <authtoken> <url> ) # Performs a GET. # function get { local getip=$1 local gettoken=$2 local geturl=$3 local attempts=0 while [ $attempts -lt 5 ]; do curl --noproxy $getip --header "X-Auth-Token:$getToken" \ --fail -kss -L -f --request GET --url "$geturl" errorcode=$? let "attempts = $attempts + 1" if [ 35 -eq $errorcode ]; then teambackup_log "SSL error on GET of $geturl, retrying..." continue; fi break; done } # # Function post ( <ipaddr> <authtoken> <url> <data>) # Performs a POST of the specified data. # function post { local postip=$1 local posttoken=$2 local posturl=$3 local postdata=$4 local attempts=0 while [ $attempts -lt 5 ]; do postres='curl --noproxy $postip --header "X-Auth-Token:$postToken" \ --fail -kss --request POST --url "$posturl" --data-binary "$postdata"' errorcode=$? let "attempts = $attempts + 1" if [ 35 -eq $errorcode ]; then teambackup_log "SSL error on POST to $posturl, retrying..." continue; fi break; done echo $postres 230 Aruba VAN SDN Controller 2.8 Administrator Guide

231 } # # Function put ( <ipaddr> <authtoken> <url> <data> ) # Performs a PUT of the specified data. # function put { local putip=$1 local puttoken=$2 local puturl=$3 local putdata=$4 local attempts=0 while [ $attempts -lt 5 ]; do putres='curl --noproxy $putip --header "X-Auth-Token:$putToken" \ --fail -kss -L -f --request PUT "$puturl" --data-binary "$putdata"' errorcode=$? let "attempts = $attempts + 1" if [ 35 -eq $errorcode ]; then teambackup_log "SSL error on POST to $puturl, retrying" continue; fi break; done echo $putres } # # Function extractjsonstring ( <json> <fieldname> ) # Extracts the Json value corresponding to the field name. # function extractjsonstring { json=$1 field=$2 json='echo $json tr -d '"' sed -e 's/\,\ {/\n/g' grep -w "$field" \ cut -d ':' -f2-' echo $json } # # Function getauthtoken ( <ipaddr> ) # Log-in and get the UID. # function getauthtoken { local nodeip=$1 url=" login="{ \"login\": { \"domain\": \"$domain\", \"user\": \"$user\", \"password\": \"$pass\" } } }" # Attempt to authenticate and extract token if successful. auth=$(curl --noproxy $nodeip -X POST --fail -kssfl --url "$url" \ -H "Content-Type: application/json" --data-binary "$login" 2>&1) if [ $? -ne 0 ]; then teambackup_log "Unable to authenticate as user $user in $domain domain." exitbackup 1 fi authtoken='extractjsonstring "$auth" "token" sed '/^$/d'' if [ $restore_mode -ne 1 ] && [ "$authtoken" == "" ]; then teambackup_log "Failed to get the authentication token." exitbackup 1 fi echo $authtoken } Appendix B Scripts 231

232 #============================================================================== # M A I N #============================================================================== restore_mode=1 selective_restore=0 # Check for unzip package. command -v unzip &> /dev/null if [ $? -ne 0 ]; then echo "The unzip package must be installed to use this script." exit 1 fi # Check the user specified script parameters. if [ $# -lt 3 ]; then echo "Usage : restoreteam <user> <domain> [<ip1> <ip2>...] <user@ip:path>" echo " <user> - user name to access the controller" echo " <domain> - domain of the controller" echo " [<ip1> <ip2>...] - ip(s) of node(s) to be restored; if none are specified all nodes are restored" echo " <user@ip:path> - remote location to retrieve backup file" echo " user - the login name for the system" echo " ip - the ip address of the system" echo " path - where to copy the file from on the remote system" exit 1 fi create_restoredir user="$1" echo -n "Enter Controller Password: " read -s pass echo domain="$2" file="" if [ $# -eq 3 ]; then teambackup_log "Starting the team restore. This will restore all the nodes in a team." file=$3 else teambackup_log "Starting selective restore on specified IPs. This restore will happen only on the specified nodes." count=0 selective_restore=1 for ip in "$@"; do restoreip[$count]=$ip let "count = $count + 1" done fileindex=$(($# - 1)) file=${restoreip[$fileindex]} && unset restoreip[$fileindex] fi # Upload the team backup file from the user specified location. scp $file $RESTORE_TEAM_DIR if [ $? -ne 0 ]; then teambackup_log "Failed to upload team backup file to the node." exitbackup 1 fi # Unzip the team backup file. extract_zip_and_ip # Validate the IP address of the node. validate_my_ip # Restore the node(s). if [ $selective_restore -eq 1 ]; then restore_nodes ${restoreip[@]} else restore_nodes ${backupip[@]} fi echo 232 Aruba VAN SDN Controller 2.8 Administrator Guide

233 teambackup_log "The team was restored successfully." exitbackup 0 Appendix B Scripts 233

234 Appendix C Using an external policy manager Using an external policy manager By integrating the controller with an external policy manager such as Aruba ClearPass Policy Manager, you can get information about a client device based on its activity in the network. Aruba ClearPass Policy manager can push information about a client device to any other server using its REST API. A client device could be a laptop, desktop, any wireless device, or any server that is connected to the network. Events are generated based on a user login, logout or rejected event. The Client Mapper Service is an internal controller application that combines information known about a network client by the controller, such as host IP address, host MAC addresses, and the connected datapath and port, with information about the network client known by an outside policy manager, such as the Aruba ClearPass Policy Manager, to provide information about network clients, including user information, device information, and location information. External SDN applications can use the information about a client and perform appropriate actions. Currently the information is available on the controller via the REST API only. For REST API details, see the Aruba VAN SDN Controller REST API Reference. To integrate the controller with Aruba ClearPass Policy Manager, you must configure ClearPass Policy Manager to use the controller as the external server. You must also configure ClearPass Policy Manager to use the controller s Client Mapper Service POST REST API to post user events. For every event posted to the controller by the Aruba ClearPass Policy Manager, the Client Mapper Service posts a corresponding ClientEvent to the controller. For details on configuring ClearPass Policy Manager to meet these requirements, see the Aruba ClearPass Policy Manager documentation. Authentication of Client Mapper Service related REST API requests can be either token-based authentication or client certificate-based authentication. For details, see REST authentication on page Aruba VAN SDN Controller 2.8 Administrator Guide

235 Appendix D Performance testing Performance testing Measuring flows (packets) per second For measuring flows-per-second for performance testing, disable the additional processing required by learn.ip key of the com.hp.sdn.disco.of.node.ofipdiscoverycomponent component by setting the value of the key to false. Procedure 1. From the navigation menu, select Configurations. 2. In the Basic tab, select the com.hp.sdn.disco.of.node.ofipdiscoverycomponent component. 3. Click Modify. Figure 69: Display the learn.ip option 4. For the learn.ip key, enter false in the Value box. 5. Click Apply to set the new learn.ip configuration and close the window. When flow measurement tasks are complete, set the learn.ip key to true (its default value). Flow measurement results can vary based on the type of server used for the controller and on the server configuration. Appendix D Performance testing 235

236 Appendix E Examples of Metrics Examples of Metrics The SDN controller has a subsystem for tracking metric values over time. Metric values are held as a time series which becomes available to the user via JMX or may be persisted to disk. For metric values that are persisted to disk the time-series values for each individual metric may be persisted at intervals of 1, 5, or 15 minutes; the component or application creating the metric may choose to use the default persistence. Each persisted value represents the value of the metric over the elapsed interval represented by the metric. Persisted metric values are retained on the controller disk for the period of time configured via the metric manager configuration. Persisted metric values may be extracted from the controller using a series of REST calls. Persisted value Consider the value of a counter that incremented from 100 to 145 during a 5 minute interval. By taking the value at the end of the interval (145), less the value at the start of the interval (100) the value persisted for the interval would be 45. The amount of time during the interval over which the metric value was accrued is also persisted. If the counter value was accrued over only the last 3 minutes of the 5 minute interval, then the normalized rate of accrual over the interval would be 15 counts per minute or 75 counts for the entire 5 minute interval. The user can inspect the most recent value of the exposed metric using any JMX client (e.g. jconsole). The values exposed will show the absolute value of each such metric. Metric values that are retained only in memory and not exposed to the user via persistence and/or JMX may be used internally by a component or application. The metric REST API consists of several distinct commands that may be used in combination to determine which metrics have persisted time-series values resident on the controller, what each metric represents, and to retrieve time series values for specific metrics of interest. Note this API operates only on a single controller at a time; it is 236 Aruba VAN SDN Controller 2.8 Administrator Guide

237 not team-aware and does not return values that span a controller team. The metrics applications commands will display the application IDs including the controller itself that have persisted metrics to disk. Figure 70: Metrics options Displaying the application IDs The metrics/apps command will display the application IDs for applications including the controller itself that have persisted metrics to disk. In this example, only the base controller itself has persisted metrics to disk. Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - kssfl --url " Curl output {"apps":[{"app_id":"com.hp.sdn", "app_name":"hp VAN SDN Controller"}]} Appendix E Examples of Metrics 237

238 Result The application ID for the controller is com.hp.sdn. It can be plugged into other metric REST API calls that require an{app_id} value in their URL. All metrics available The following command lists all of the metrics available for a specific application (the controller itself in this example); because of the number of metrics available the output is cut off. This output describes the metrics; it does not represent the time-series values for them. Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - kssfl --url " Curl output for app_id=com.hp.sdn 238 Aruba VAN SDN Controller 2.8 Administrator Guide

239 {"metrics": [{"app_id":"com.hp.sdn","type":"ratio_gauge","name":"cpuloadsystem","description": "The recent CPU usage of the system.","primary_tag":"jvm","secondary_tag": "operatingsystem","jmx":false,"persistence":true,"summary_interval": "ONE","uid":"42f65cd8-03c3-4cad d513e3c0f"},{"app_id": "com.hp.sdn","type":"gauge","name":"committedbytes","description": "The amount of non-heap memory in bytes that is committed (guaranteed) for the JVM to use.","primary_tag": "jvm","secondary_tag":"memorynonheap","jmx":false,"persistence": true,"summary_interval":"one","uid":"b82f5b a23-b5a8-bbda7eec44cb"}, {"app_id": "com.hp.sdn","type":"gauge","name":"countterminated","description": "Number of JVM threads that had exited.","primary_tag":"jvm","secondary_tag": "threads","jmx":false,"persistence":true,"summary_interval": "ONE","uid":"0e9fe62e-01fd-42e9-88a6-f92021a5e786"},{"app_id": "com.hp.sdn","type":"gauge","name":"uptimems","description": "The uptime of the JVM in milliseconds.","primary_tag":"jvm","jmx": false,"persistence":true,"summary_interval":"one","uid": " c0e4-4b4c-987e-79a690a541af"},{"app_id":"com.hp.sdn","type": "GAUGE","name":"committedBytes","description": "The amount of heap memory in bytes that is committed (guaranteed) for the JVM to use.","primary_tag": "jvm","secondary_tag":"memoryheap","jmx":false,"persistence": true,"summary_interval":"one","uid":"4cb1a4be-3a8d-4f69-a626-2c5ae134c7e3"}, {"app_id":"com.hp.sdn","type": "GAUGE","name":"usedBytes","description": "The total amount of memory currently being used by the JVM in bytes.","primary_tag": "jvm","secondary_tag":"memorytotal","jmx":false,"persistence":true,"summary_interva l": "ONE","uid":"fe56c3fd ec4-af d58"},{"app_id":"com.hp.sdn","type": "GAUGE","name":"usedBytes","description":"The amount of heap memory currently being used by the JVM in bytes.", "primary_tag":"jvm","secondary_tag":"memoryheap","jmx":false,"persistence": true,"summary_interval":"one","uid":"431b746e-e62e-4874-a801-b1438eaac635"}, {"app_id":"com.hp.sdn","type": "GAUGE","name":"usedBytes","description":"The amount of non-heap memory currently being used by the JVM in bytes.", "primary_tag":"jvm","secondary_tag":"memorynonheap","jmx":false,"persistence":true, "summary_interval":"one","uid": "afa9a4b2-856a-4f69-8abf-a4775fd0f2e7"}, {"app_id":"com.hp.sdn","type":"rolling_counter","name":"count","description": "The number of garbage collections undertaken by the JVM during the sampling interval.","primary_tag": "jvm","secondary_tag":"garbagecollection","jmx":false,"persistence": true,"summary_interval":"one","uid":"d62c49d4-46b3-4c2c-be60-24f6fa6c6bf6"}, {"app_id": "com.hp.sdn","type":"ratio_gauge","name":"usage","description": "The ratio of non-heap memory currently being used by the JVM to maximum non-heap memory requested by the JVM.","primary_tag":"jvm","secondary_tag":"memoryNonHeap","jmx": false,"persistence":true,"summary_interval":"one","uid":"fafefbb9-e e-9ddfdb76961f8958"}, {"app_id":"com.hp.sdn","type":"ratio_gauge","name":"averagebufferusedbytes","descri ption": "The average bytes used in each mapped memory buffer associated with the JVM.","primary_tag": "jvm","secondary_tag":"niomappedmemory","jmx":false,"persistence":true,"summary_int erval": "ONE","uid":"25a38f14-1ea3-4bc fe4c53a587dd"}, {"app_id":"com.hp.sdn","type":"gauge","name": "bufferusedbytes","description":"the total amount of mapped buffer memory that the JVM is using in bytes.", Appendix E Examples of Metrics 239

240 "primary_tag":"jvm","secondary_tag":"niomappedmemory","jmx":false,"persistence":tru e,"summary_interval": "ONE","uid":"c715e388-87dc-4f05-a430-c3c39e73615e"}, {"app_id":"com.hp.sdn","type":"gauge","name": "bufferusedbytes","description":"the total amount of direct buffer memory that the JVM is using in bytes.", "primary_tag":"jvm","secondary_tag":"niodirectmemory","jmx":false,"persistence":tru e,"summary_interval": "ONE","uid":"f7301f57-e c-af8f-cf3dae376232"}, {"app_id":"com.hp.sdn","type":"gauge","name": "buffercapacitybytes","description":"the total capacity in bytes of direct buffer memory associated with the JVM.", "primary_tag":"jvm","secondary_tag":"niodirectmemory","jmx":false,"persistence":tru e,"summary_interval": "ONE","uid":"fbcebed0-21a2-4b8a-8e cddefee8"}, {"app_id":"com.hp.sdn","type":"ratio_gauge","name": "averagebuffercapacitybytes","description":"the average capacity in bytes of each mapped memory buffer associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioMappedMemory","jmx": false,"persistence":true,"summary_interval":"one","uid":"42aee5ea-934d-4816-a1fdf2f8f99e5160"}, {"app_id":"com.hp.sdn","type":"gauge","name":"countrunnable","description": "Number of JVM threads that were executing.","primary_tag":"jvm","secondary_tag": "threads","jmx":false,"persistence":true,"summary_interval":"one","uid": "b3acc121-e c b0fcfb7"},{"app_id":"com.hp.sdn","type": "GAUGE","name":"countTimedWaiting","description":"Number of JVM threads waiting up to a specified period for another thread to perform a particular action.", "primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence": true,"summary_interval":"one","uid":"9c6928b5-9ca3-4e9b-956f ea2b8a"}, {"app_id":"com.hp.sdn","type":"ratio_gauge","name":"filedescriptorsusage", "description":"the ratio of file descriptors open on the operating system to the maximum supported file descriptors.","primary_tag":"jvm","secondary_tag": "operatingsystem","jmx":false,"persistence":true,"summary_interval":"one","uid": "466e03fa-a9b7-4ba5-b7cd-d697dba9b1be"},{"app_id":"com.hp.sdn","type": "RATIO_GAUGE","name":"averageBufferCapacityBytes","description": "The average capacity in bytes of each direct memory buffer associated with the JVM.", "primary_tag":"jvm","secondary_tag":"niodirectmemory","jmx":false,"persistence": true,"summary_interval":"one","uid":"393bc882-0fc4-4eea-9aaa-97acd716a0c5"}, {"app_id": "com.hp.sdn","type":"gauge","name":"countblocked","description": "Number of JVM threads that were blocked.","primary_tag":"jvm","secondary_tag": "threads","jmx":false,"persistence":true,"summary_interval":"one","uid": "f0be8e6d-a456-4b8c-83cb-d156629b9aec"},{"app_id":"com.hp.sdn","type": "ROLLING_COUNTER","name":"elapsedMs","description":"The number of milliseconds for which garbage collection was undertaken by the JVM during the sampling interval.", "primary_tag":"jvm","secondary_tag":"garbagecollection","jmx":false,"persistence": true,"summary_interval":"one","uid":"b550925d-c8d a3ab-ec37d498fd8f"}, {"app_id": "com.hp.sdn","type":"gauge","name":"countdeadlocked","description": "Number of JVM threads that were in deadlock.","primary_tag":"jvm","secondary_tag": "threads","jmx":false,"persistence":true,"summary_interval":"one","uid": "505b577c d5-90b26ac2be85"},{"app_id":"com.hp.sdn","type": "GAUGE","name":"countWaiting","description":"Number of JVM threads waiting indefinitely for another thread to perform a particular action.","primary_tag": "jvm","secondary_tag":"threads","jmx":false,"persistence":true,"summary_interval": "ONE","uid":"cb00d09b-1d9c-4f96-a5c9-8a6cd06ad5bd"},{"app_id":"com.hp.sdn","type": "GAUGE","name":"bufferCount","description":"The number of direct memory buffers associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioDirectMemory","jmx": false,"persistence":true,"summary_interval":"one","uid":"cdcbb352-d001-4be0-a7fd- 240 Aruba VAN SDN Controller 2.8 Administrator Guide

241 c6d614c75f1a"}, {"app_id":"com.hp.sdn","type":"gauge","name":"countnew","description": "Number of JVM threads that had not yet started.","primary_tag":"jvm","secondary_tag": "threads","jmx":false,"persistence":true,"summary_interval":"one","uid": "0e67f839-48b d83-ac7828c742f6"},{"app_id":"com.hp.sdn","type": "GAUGE","name":"countDaemon","description":"Number of JVM threads that were live daemon threads.","primary_tag":"jvm","secondary_tag":"threads","jmx": false,"persistence":true,"summary_interval":"one","uid":"7328dd03-57fd-4baab741-25dab77446fc"}, {"app_id":"com.hp.sdn","type":"ratio_gauge","name":"cpuloadjvm","description": "The recent CPU usage of the JVM process.","primary_tag":"jvm","secondary_tag": "operatingsystem","jmx":false,"persistence":true,"summary_interval":"one","uid": "cc82ee87-80eb-417b-8d fbf24b63"}, {"app_id":"com.hp.sdn","type":"gauge","name": "buffercapacitybytes","description":"the total capacity in bytes of mapped buffer memory associated with the JVM.","primary_tag":"jvm","secondary_tag":"nioMappedMemory","jmx":false,"persistence":true,"summary_interval":"ONE","uid":"6c08d248-e6dd-4d96- b3de-9ee6c32825d3"}, {"app_id":"com.hp.sdn","type":"ratio_gauge","name":"usage","description": "The ratio of heap memory currently being used by the JVM to maximum heap memory requested by the JVM.","primary_tag":"jvm","secondary_tag":"memoryHeap","jmx": false,"persistence":true,"summary_interval":"one","uid":"bc729aec-4bc4-453e-8e9b-6c 2fff5eeaef"}, {"app_id":"com.hp.sdn","type":"ratio_gauge","name":"averagebufferusedbytes","descri ption": "The average bytes used in each direct memory buffer associated with the JVM.","primary_tag": "jvm","secondary_tag":"niodirectmemory","jmx":false,"persistence":true,"summary_int erval": "ONE","uid":"0cac91dd-4f d2a104362bd3"}, {"app_id":"com.hp.sdn","type":"gauge","name": "filedescriptorsopen","description":"the number of file descriptors open on the operating system.", "primary_tag":"jvm","secondary_tag":"operatingsystem","jmx":false,"persistence":tru e, "summary_interval":"one","uid":"60bcbb b-bbc9-b403a1ce9b56"},{"app_id": "com.hp.sdn","type":"gauge","name":"counttotal","description":"total (daemon and non-daemon) number of live JVM threads.","primary_tag":"jvm","secondary_tag":"threads","jmx": false,"persistence":true,"summary_interval":"one","uid":"fc9d9166- f525-4b7a-93a f31"}, {"app_id":"com.hp.sdn","type":"gauge","name":"buffercount","description": "The number of mapped memory buffers associated with the JVM.","primary_tag": "jvm","secondary_tag":"niomappedmemory","jmx":false,"persistence":true,"summary_int erval": "ONE","uid":"dbe9e2fc-f5a5-42d7-a4e9-45bb6c5d0d8d"},{"app_id":"com.hp.sdn","type": "GAUGE","name":"countNonDaemon","description":"Number of JVM threads that were live non-daemon threads.","primary_tag":"jvm","secondary_tag":"threads","jmx":false,"persistence": true,"summary_interval":"one","uid":"e13bce02-45f0-47e3-b38e-284d30bc84af"}, {"app_id": "com.hp.sdn","type":"gauge","name":"committedbytes","description": "The total amount of memory in bytes that is committed (guaranteed) for the JVM to use.", "primary_tag":"jvm","secondary_tag":"memorytotal","jmx":false,"persistence": true,"summary_interval":"one","uid":"8c6e a8-4cbb-a2e d2e36a"}]} Appendix E Examples of Metrics 241

242 Result For each metric listed, one can see the its type, its associated application ID, its name, its primary and secondary tags, whether it is persisted, whether it is exposed via JMX, and its summary interval. Also displayed for each metric is the unique ID (uid) assigned to the metric on the controller. Other metric REST API calls can be used to view specific subsets of this data. 242 Aruba VAN SDN Controller 2.8 Administrator Guide

243 Lists primary tags associated with a specific application Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - kssfl --url " Curl output for app_id=com.hp.sdn {"primaries":["jvm"]} Result The only primary tag associated with the controller is jvm. Secondary tags associated with a specific application The secondary tags associated with a specific application are listed using the following REST call. Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - Appendix E Examples of Metrics 243

244 kssfl --url " secondaries" Curl output for app_id=com.hp.sdn {"secondaries":["niodirectmemory","operatingsystem","threads","garbagecollection", "memorynonheap","memoryheap","memorytotal","niomappedmemory"]} Result Several secondary tags are associated with the primary tag jvm along with several subcategories of jvm metric: memoryheap metrics and threads metrics, among others. Metric names associated with a specific application Metric names associated with a specific application are displayed using the following call. 244 Aruba VAN SDN Controller 2.8 Administrator Guide

245 Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - kssfl --url " Curl output for app_id=com.hp.sdn {"names":["averagebufferusedbytes","countdeadlocked","buffercapacitybytes","count", "countnew","buffercount","countwaiting","filedescriptorsopen","uptimems","cou ntterminated", "elapsedms","counttimedwaiting","countdaemon","countblocked","filedescriptorsusage", "averagebuff ercapacitybytes","cpuloadsystem","counttotal","bufferusedbytes","usedbytes", "usage","countnondaemon","countru nnable","cpuloadjvm","committedbytes"]} Appendix E Examples of Metrics 245

246 Result Optional query parameters are provided in each of the calls. To see the metric names for specific primary tags and optionally secondary tags, you may be specific in the call so that only the applicable metric names are displayed. Primary and secondary tags Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail Aruba VAN SDN Controller 2.8 Administrator Guide

247 kssfl --url " metrics/apps/<app_id>/names? primary_tag=<primary_tag>, secondary_tag=<secondary_tag>" Curl output for app_id=com.hp.sdn primary_tag=jvm&secondary_tag=memoryheap): {"names":["usedbytes","usage","committedbytes"]} Result Metric names are specific to JVM heap memory. The UID can be obtained once the specific metric of interest is identified via the earlier call. Optional query parameters to filter the output and list the metrics associated with an application ID may be employed. Filter primary, secondary and metric name Filter for a primary tag of jvm, a secondary tag of memoryheap, and a metric name of usedbytes. Appendix E Examples of Metrics 247

248 Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - kssfl --url " <app_id>? primary_tag=jvm&secondary_tag=memoryheap&name=usedbytes" Curl output for app_id=com.hp.sdn, primary_tag=jvm, secondary_tag=memoryheap, name=usedbytes {"metrics":[{"app_id":"com.hp.sdn","type":"gauge","name":"usedbytes","description": "The amount of heap memory currently being used by the JVM in bytes.","primary_tag": "jvm","secondary_tag":"memoryheap","jmx":false,"persistence":true,"summary_interval ": "ONE","uid":"431b746e-e62e-4874-a801-b1438eaac635"}]} 248 Aruba VAN SDN Controller 2.8 Administrator Guide

249 Result Detailed information about the metric can be retrieved using a specific metric UID. This same information is encompassed by the previous metric listing. Time-series data for a metric The following example shows time-series data for a metric. The time for which the actual metric value was accrued is shown in each 18 interval in number of milliseconds spanned. The ast value is the metric value for the indicated time, spanning the indicated number of milliseconds ending at that time. Appendix E Examples of Metrics 249

250 Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - kssfl --url " values? start=<start_time>&interval=<interval_value>" Curl output for app_id=com.hp.sdn, metric_uid=431b746e-e62e 4874 a801 b1438eaac635,start= :00,interval=1 {"metric_values":{"uid":"431b746e-e62e-4874-a801- b1438eaac635","type":"gauge","datapoint_count": 25,"datapoints":[{"update_time":"Tue Sep 23 17:59:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:00:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:01:55 PDT 2014","milliseconds_span": 60001,"last": E8},{"update_time":"Tue Sep 23 18:02:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:03:55 PDT 2014","milliseconds_span": 59999,"last": E8},{"update_time":"Tue Sep 23 18:04:55 PDT 2014","milliseconds_span": 60001,"last": E8},{"update_time":"Tue Sep 23 18:05:55 PDT 2014","milliseconds_span": 59999,"last": E8},{"update_time":"Tue Sep 23 18:06:55 PDT 2014","milliseconds_span": 60001,"last": E8},{"update_time":"Tue Sep 23 18:07:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:08:55 PDT 2014","milliseconds_span": 59999,"last": E8},{"update_time":"Tue Sep 23 18:09:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:10:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:11:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:12:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:13:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:14:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:15:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:16:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:17:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:18:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:19:55 PDT 2014","milliseconds_span": 60000,"last": E8},{"update_time":"Tue Sep 23 18:20:55 PDT 2014","milliseconds_span": 60001,"last": E8},{"update_time":"Tue Sep 23 18:21:55 PDT 2014","milliseconds_span": 59999,"last": E8},{"update_time":"Tue Sep 23 18:22:55 PDT 2014","milliseconds_span": 60001,"last": 250 Aruba VAN SDN Controller 2.8 Administrator Guide

251 E8},{"update_time":"Tue Sep 23 18:23:55 PDT 2014","milliseconds_span": 59999,"last": E8}]}} Result The computation of values returned over longer intervals depends upon the type of metric. Gauge values as shown in this example are averaged over the data points encompassed in the summary. Counter values are summed over the summary interval in which histogram values are combined. Appendix E Examples of Metrics 251

252 Returned data for a period results Summarizing the returned data for a period results in (generally) larger values for the metrics themselves and larger values for the milliseconds spanned. Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - kssfl --url " values? start=<start_time>&interval=<interval_value>" Curl output for app_id=com.hp.sdn, metric_uid=431b746e-e62e 4874 a801 b1438eaac635,start= :00,interval=5 {"metric_values":{"uid":"431b746e-e62e-4874-a801- b1438eaac635","type":"gauge","datapoint_count":6,"datapoints": [{"update_time":"tue Sep 23 18:03:55 PDT 2014","milliseconds_span":300000,"last": E8},{"update_time": "Tue Sep 23 18:08:55 PDT 2014","milliseconds_span":300000,"last": E8}, {"update_time": "Tue Sep 23 18:13:55 PDT 2014","milliseconds_span":300000,"last": E8}, {"update_time": "Tue Sep 23 18:18:55 PDT 2014","milliseconds_span":300000,"last": E8}, {"update_time": "Tue Sep 23 18:23:55 PDT 2014","milliseconds_span":300000,"last": E8}, {"update_time": "Tue Sep 23 18:27:55 PDT 2014","milliseconds_span":240000,"last": E8}]}} 252 Aruba VAN SDN Controller 2.8 Administrator Guide

253 Result Support report This report may be generated via the following REST API. Appendix E Examples of Metrics 253

254 Curl equivalent command curl --noproxy <controller_ip> -X GET --header "X-Auth-Token:<token>" --fail - kssfl --url " Curl output controller_ip:8443/sdn/v2.0/support {"support_report": [{"title":"alert Framework","id":"alert","content":["Alert-Topics: licensing","alert-count: 7","Data Retention Age Out: 14 days","data Trim Interval: 24 hours","data Trim Enabled: true","last trim conducted at: Mon Sep 22 19:15:20 PDT 2014"]},{"title":"Alert Topic Listener", "id":"alert_listener","content":["no registered alert topic listeners"]}, {"title":"app Manager", "id":"app-management","content":["installed Applications: 8","Path Diagnostics, Version: SNAPSHOT, State: ACTIVE","Link Manager, Version: SNAPSHOT, State: ACTIVE", "Node Manager, Version: SNAPSHOT, State: ACTIVE","OpenFlow Link Discovery, Version: SNAPSHOT, State: ACTIVE","OpenFlow Node Discovery, Version: SNAPSHOT, State: ACTIVE","Path Daemon, Version: SNAPSHOT, State: ACTIVE","Topology Manager, Version: SNAPSHOT, State: ACTIVE","Topology Viewer, Version: SNAPSHOT, State: ACTIVE"]}, {"title":"audit Log Framework","id":"audit_log","content":["Audit Log Count: 0","Data Retention Age Out: 365 days","data Trim Interval: 24 hours","data Trim Enabled: true", "Last trim conducted at: Mon Sep 22 19:15:20 PDT 2014"]},{"title":"Server Environment", "id":"env","content":["os architecture: amd64","os Name: Linux","OS Version: generic", "Java Vendor: Oracle Corporation","Java Version: b04","Java Name: OpenJDK 64- Bit Server VM","Available processors (cores): 4","Max Heap: [3641Mb]","Heap: [640Mb]","Heap used: [386Mb]","Start Date: Tue Sep 16 19:14:57 PDT 2014","UpTime: 6 Days, 23 Hours","HP VAN SDN Controller Version: "]}, {"title": "JVM Metrics","id":"jvm-metrics","content":["Metric count: 44","Last update time: Wed, 24 Sep :31:55 GMT","Uptime: 10,037 minute(s)","memory"," Total"," Initial: 548,288 kb"," Committed: 740,032 kb"," Maximum: 4,301,824 kb"," Used: 477,308 kb", " Heap"," Initial: 524,288 kb"," Committed: 655,360 kb"," Maximum: 3,728,384 kb", " Used: 393,227 kb"," Usage: %"," Non-Heap"," Initial: 24,000 kb", " Committed: 84,672 kb"," Maximum: 573,440 kb"," Used: 84,081 kb"," Usage: %","NIO Buffer Memory"," Direct"," Capacity: 0 bytes"," Used: 0 bytes", " Buffers: 0"," Mapped"," Capacity: 0 bytes"," Used: 0 bytes"," Buffers: 0","Garbage Collection (last 1 minute(s))"," Executions: 0"," Elapsed time: 0 ms", 254 Aruba VAN SDN Controller 2.8 Administrator Guide

255 "Threads"," Total count: 122"," By Type"," Daemon: 65"," Non-daemon: 57", " By State"," Blocked: 0"," Deadlocked: 0"," New: 0"," Runnable: 7", " Terminated: 0"," Timed waiting: 24"," Waiting: 91","Operating System", " CPU Usage"," System: %"," JVM: %"," File Descriptors"," Maximum: 8,192"," Open: 214"," Usage: %"]}, {"title":"licensing","id": "licensing","content":["number of licenses Found: None"]}]} Appendix E Examples of Metrics 255