Cisco ACI with OpenStack OpFlex Architectural Overview

Transcription

1 First Published: February 11, 2016 Last Modified: March 30, 2016 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA USA Tel: NETS (6387) Fax:

2 THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS. THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY. The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of the UNIX operating system. All rights reserved. Copyright 1981, Regents of the University of California. NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental. Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R) 2016 Cisco Systems, Inc. All rights reserved.

3 CONTENTS Preface Preface v Audience v Document Conventions v Related Documentation vii Documentation Feedback ix Obtaining Documentation and Submitting a Service Request ix CHAPTER 1 Overview 1 About OpenStack and Cisco ACI 1 Extending OpFlex to the Compute Node 1 CHAPTER 2 Solution Architecture 3 ACI with OpenStack Physical Architecture 3 OpFlex ML2 Software Architecture 4 Logical OpenStack Topology 6 Distributed Neutron Services 7 Mapping OpenStack and ACI Constructs 8 OpFlex NAT Operations 9 IP Subnets Required for NAT 9 OVS NAT and External Routing 10 Optimized DHCP and Metadata Proxy Operations 10 Optimized DHCP Services 11 Optimized Metadata Services 11 APIC OpenStack VMM Integration 12 iii

4 Contents iv

5 Preface This preface includes the following sections: Audience, page v Document Conventions, page v Related Documentation, page vii Documentation Feedback, page ix Obtaining Documentation and Submitting a Service Request, page ix Audience This guide is intended primarily for data center administrators with responsibilities and expertise in one or more of the following: Virtual machine installation and administration Server administration Switch and network administration Document Conventions Command descriptions use the following conventions: Convention Description bold Bold text indicates the commands and keywords that you enter literally as shown. Italic Italic text indicates arguments for which the user supplies the values. [x] Square brackets enclose an optional element (keyword or argument). v

6 Document Conventions Preface Convention [x y] {x y} [x {y z}] variable string Description Square brackets enclosing keywords or arguments separated by a vertical bar indicate an optional choice. Braces enclosing keywords or arguments separated by a vertical bar indicate a required choice. Nested set of square brackets or braces indicate optional or required choices within optional or required elements. Braces and a vertical bar within square brackets indicate a required choice within an optional element. Indicates a variable for which you supply values, in context where italics cannot be used. A nonquoted set of characters. Do not use quotation marks around the string or the string will include the quotation marks. Examples use the following conventions: Convention screen font boldface screen font italic screen font < > [ ]!, # Description Terminal sessions and information the switch displays are in screen font. Information you must enter is in boldface screen font. Arguments for which you supply values are in italic screen font. Nonprinting characters, such as passwords, are in angle brackets. Default responses to system prompts are in square brackets. An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line. This document uses the following conventions: Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual. Caution Means reader be careful. In this situation, you might do something that could result in equipment damage or loss of data. vi

7 Preface Related Documentation Warning IMPORTANT SAFETY INSTRUCTIONS This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided at the end of each warning to locate its translation in the translated safety warnings that accompanied this device. SAVE THESE INSTRUCTIONS Related Documentation The Application Centric Infrastructure documentation set includes the following documents that are available on Cisco.com at the following URL: application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html. Web-Based Documentation Cisco APIC Management Information Model Reference Cisco APIC Online Help Reference Cisco APIC Python SDK Reference Cisco ACI Compatibility Tool Cisco ACI MIB Support List Downloadable Documentation Knowledge Base Articles (KB Articles) are available at the following URL: us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/ tsd-products-support-series-home.html Cisco Application Centric Infrastructure Controller Release Notes Cisco Application Centric Infrastructure Fundamentals Guide Cisco APIC Getting Started Guide Cisco ACI Basic Configuration Guide Cisco ACI Virtualization Guide Cisco APIC REST API User Guide Cisco APIC Object Model Command Line Interface User Guide Cisco APIC NX-OS Style Command-Line Interface Configuration Guide Cisco APIC Faults, Events, and System Messages Management Guide Cisco ACI System Messages Reference Guide Cisco APIC Layer 4 to Layer 7 Services Deployment Guide Cisco APIC Layer 4 to Layer 7 Device Package Development Guide vii

8 Related Documentation Preface Cisco APIC Layer 4 to Layer 7 Device Package Test Guide Cisco ACI Firmware Management Guide Cisco ACI Troubleshooting Guide Cisco APIC NX-OS Style CLI Command Reference Cisco ACI Switch Command Reference, NX-OS Release 11.0 Verified Scalability Guide for Cisco ACI Cisco ACI MIB Quick Reference Cisco Nexus CLI to Cisco APIC Mapping Guide Application Centric Infrastructure Fabric Hardware Installation Guide Cisco NX-OS Release Notes for Cisco Nexus 9000 Series ACI-Mode Switches Nexus 9000 Series ACI Mode Licensing Guide Cisco Nexus 9332PQ ACI-Mode Switch Hardware Installation Guide Cisco Nexus 9336PQ ACI-Mode Switch Hardware Installation Guide Cisco Nexus 9372PX and 9372PX-E ACI-Mode Switch Hardware Installation Guide Cisco Nexus 9372TX ACI-Mode Switch Hardware Installation Guide Cisco Nexus 9396PX ACI-Mode Switch Hardware Installation Guide Cisco Nexus 9396TX ACI-Mode Switch Hardware Installation Guide Cisco Nexus 93128TX ACI-Mode Switch Hardware Installation Guide Cisco Nexus 9504 NX-OS Mode Switch Hardware Installation Guide Cisco Nexus 9508 ACI-Mode Switch Hardware Installation Guide Cisco Nexus 9516 ACI-Mode Switch Hardware Installation Guide Cisco Application Centric Infrastructure (ACI) Simulator Documentation The following Cisco ACI Simulator documentation is available at cloud-systems-management/application-centric-infrastructure-simulator/tsd-products-support-series-home.html. Cisco ACI Simulator Release Notes Cisco ACI Simulator Installation Guide Cisco ACI Simulator Getting Started Guide Cisco Nexus 9000 Series Switches Documentation The Cisco Nexus 9000 Series Switches documentation is available at switches/nexus-9000-series-switches/tsd-products-support-series-home.html. Cisco Application Virtual Switch Documentation The Cisco Application Virtual Switch (AVS) documentation is available at support/switches/application-virtual-switch/tsd-products-support-series-home.html. viii

9 Preface Documentation Feedback Cisco Application Centric Infrastructure (ACI) Integration with OpenStack Documentation Cisco ACI integration with OpenStack documentation is available at cloud-systems-management/application-policy-infrastructure-controller-apic/ tsd-products-support-series-home.html. Documentation Feedback To provide technical feedback on this document, or to report an error or omission, please send your comments to We appreciate your feedback. Obtaining Documentation and Submitting a Service Request For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What's New in Cisco Product Documentation at: Subscribe to What s New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation as an RSS feed and delivers content directly to your desktop using a reader application. The RSS feeds are a free service. ix

10 Obtaining Documentation and Submitting a Service Request Preface x

11 CHAPTER 1 Overview This chapter contains the following sections: About OpenStack and Cisco ACI, page 1 Extending OpFlex to the Compute Node, page 1 About OpenStack and Cisco ACI Cisco Application Centric Infrastructure (ACI) is a comprehensive policy-based architecture that provides an intelligent, controller-based network switching fabric. This fabric is designed to be programmatically managed through an API interface that can be directly integrated into multiple orchestration, automation, and management tools, including OpenStack. Integrating ACI with OpenStack allows dynamic creation of networking constructs to be driven directly from OpenStack requirements, while providing additional visibility within the ACI Application Policy Infrastructure Controller (APIC) down to the level of the individual VM instance. OpenStack defines a flexible software architecture for creating cloud-computing environments. The reference software-based implementation of OpenStack allows for multiple Layer 2 transports including VLAN, GRE, and VXLAN. The Neutron project within OpenStack can also provide software based Layer 3 forwarding. When utilized with ACI, the ACI fabric provides an integrated Layer 2 and Layer 3 VXLAN-based overlay networking capability that can offload network encapsulation processing from the compute nodes onto the top-of-rack or ACI leaf switches. This architecture provides the flexibility of software overlay networking in conjunction with the performance and operational benefits of hardware-based networking. Extending OpFlex to the Compute Node OpFlex is an open and extensible policy protocol designed to transfer declarative networking policies such as those used in Cisco ACI to other devices. Utilizing OpFlex, the policy model native to ACI can be extended all the way down into the virtual switches running on OpenStack Nova compute hosts. This OpFlex extension to the compute host allows ACI to use Open vswitch (OVS) to support common OpenStack features such as Source NAT (SNAT) and Floating IP in a distributed manner. The OpFlex-based OpenStack drivers support two distinct modes of deployment. The first approach is based on the Neutron API and Modular Layer 2 (ML2), which are designed to provide common constructs such as network, router, and security groups that are familiar to Neutron users. The second approach is native to the Group-Based Policy abstractions for OpenStack, which are closely aligned with the declarative policy model 1

12 Extending OpFlex to the Compute Node Overview used in Cisco ACI. The content of this document is focused specifically on details of the OpFlex ML2-based driver. See cisco.com for further information and specific deployment guidance for the OpFlex GBP-based driver. 2

13 CHAPTER 2 Solution Architecture This chapter contains the following sections: ACI with OpenStack Physical Architecture, page 3 OpFlex ML2 Software Architecture, page 4 Logical OpenStack Topology, page 6 Mapping OpenStack and ACI Constructs, page 8 OpFlex NAT Operations, page 9 Optimized DHCP and Metadata Proxy Operations, page 10 APIC OpenStack VMM Integration, page 12 ACI with OpenStack Physical Architecture A typical architecture for an ACI fabric with an OpenStack deployment consists of a Nexus 9000 Spine/Leaf topology, an APIC cluster, and a group of servers to run the various control and compute components of OpenStack. OpenStack can be deployed in many ways, but a very basic test architecture would consist of at least one OpenStack Controller server which is also acting as the Neutron network node, and two or more OpenStack compute nodes to host Virtual Machine (VM) instances. An ACI External Routed Network connection as a Layer 3 connection outside of the fabric can be used to provide connectivity outside the OpenStack cloud. 3

14 OpFlex ML2 Software Architecture Solution Architecture Note The configurations validated for this deployment guide utilized Cisco UCS C-Series rack mount servers in standalone mode. Third-party standalone rack servers running OpenStack can also be supported. Support for systems running UCS Manager with Fabric Interconnects attached to the ACI fabric is planned for a future release. Figure 1: Example ACI with OpenStack Physical Topology OpFlex ML2 Software Architecture The Modular Layer 2 framework in OpenStack allows integration of networking services based on TypeDrivers, and MechanismDrivers. Common networking type drivers include local, flat, vlan, vxlan, etc. OpFlex is added as a new network type through ML2, with an actual packet encapsulation of either VXLAN or VLAN on the host defined in the OpFlex configuration. A mechanism driver is enabled to communicate networking requirements from the Neutron server(s) to the Cisco APIC cluster. The APIC mechanism driver translates Neutron networking elements such as a network (segment), subnet, router, or external network into APIC constructs within the ACI Policy Model. The OpFlex ML2 software stack also currently utilizes a modified Open vswitch Package, and local software agents on each OpenStack compute host that communicate with the Neutron server(s) and OVS. An OpFlex proxy from the ACI leaf switch exchanges policy information with the Agent-OVS instance in each compute host, effectively extending the ACI switch fabric and policy model into the virtual switch. This results in a cohesive system that can apply networking policy anywhere in the combined virtual and physical switching fabric starting from the virtual port where a VM instance attaches to the network.the Figure below illustrates the interaction of the OpFlex ML2 APIC Driver with the ACI Fabric, and the extension of the OpFlex proxy down into the Agent-OVS service on the compute host. 4

15 Solution Architecture OpFlex ML2 Software Architecture Note The OpFlex ML2 APIC Driver for integration into Neutron runs on the server running the neutron-server service. This server may be a controller node running other OpenStack software elements, or be dedicated to the Neutron function. High Availability configurations with multiple Neutron servers are also supported. Figure 2: OpenStack with ACI architecture with OpFlex ML2 On the compute node, the neutron-opflex-agent service receives information about OpenStack endpoints from the ML2 Driver software on the Neutron server. This information is stored locally in the endpoint files located in /var/lib/opflex-agent-ovs/endpoints. The agent-ovs service uses the endpoint information to resolve policy for the endpoints through the OpFlex Proxy on the connected ACI leaf switch. The agent-ovs then programs policy on OVS using Open Flow for policies that can be enforced locally. 5

16 Logical OpenStack Topology Solution Architecture Non-local policies are enforced on the upstream leaf switch. The Figure below illustrates the interaction between the Opflex modules running on the compute node, and OVS. Figure 3: OpFlex Agent Architecture on the Compute Host Logical OpenStack Topology OpenStack defines multiple network connection requirements for the server nodes providing cloud services. Communication paths need to be defined and provided for Management traffic, Tenant Data, and External networking requirements, as well as API communication between the various OpenStack services. Deployments may also dedicate a network segment for storage traffic or other specific needs. An ACI switching fabric is able to provide networking services to meet all of these requirements. Server connectivity can consist of either separate physical interfaces, virtualized network adapters such as the Cisco VIC, or a managed blade server system such as Cisco UCS B-Series. Management and API Network(s) This network segment is for administrative secure shell access to OpenStack servers, as well as API communication directly to the servers and between OpenStack functions. The Management and API functions may also be broken out into different network segments, the example configurations in this guide use a single network segment for both. External Network With an ACI fabric integrated with OpFlex, the External Network path is provided by an External Routed Network configuration in the APIC. An External network in a system running the Neutron L3-Agent is a network on the outside of a software-based routing function. An External Network utilizes NAT services to allow hidden and overlapping IPv4 address space to be used by Tenants. Tenant Data Network A Tenant network in OpenStack can be dynamically created by a Tenant to provide connectivity between VM instances in the cloud, and also to connect to cloud-based routing services to other Tenant or External networks. The segment ID's assigned to Tenant networks with OpFlex are tracked by the ACI fabric, and consist of VXLAN between leaf switches, with either VXLAN or VLAN between leaf switch and server. 6

17 Solution Architecture Distributed Neutron Services Distributed Neutron Services OpenStack Neutron defines common networking constructs and services required by VM instances operating in the cloud environment. Both availability and scale of Neutron services can become a concern when implementing all of these functions on a single server, or a small cluster of servers. The OpFlex ML2 Driver software stack provides the ability to distribute these network services across the compute nodes in the cluster, using a scale-out approach to reduce the load on any single instance of a service while increasing overall service availability. The following OpenStack Neutron services may be distributed to compute nodes when using the OpFlex ML2 Driver software: NAT for External Networks The Opflex ML2 Driver approach for supporting External networks distributes Source NAT and Floating IP functions for OpenStack to the Open vswitch of the compute hosts. Packets destined for IP addresses not defined in the private OpenStack Tenant space are automatically NATted prior to egressing a compute host. The translated packets are then routed to the external routed network defined in the APIC. Distributed NAT services are inherent in the solution. Layer 3 Forwarding The Neutron reference software implementation of Layer 3 Agent is replaced by a combination of Layer 3 forwarding in the ACI fabric, as well as local forwarding within a compute node. If two VMs connected to the same OpenStack Tenant router reside on the same compute node, Layer 3 traffic between them will be forwarded by OVS and remain local to that physical server. Distributed Layer 3 for traffic local to a compute node is inherent in the solution. DHCP Reference Neutron software implementations have a DHCP Agent service centralized on the Neutron server(s). The OpFlex ML2 driver software allows a distributed DHCP approach using the agent-ovs service. Distributing the DHCP function across the compute nodes keeps DHCP Discovery, Offer, Request and Acknowledge (DORA) traffic local to the host and ensures reliable allocation of IP addressing to VM Instances. Central Neutron address management functions communicate DHCP addressing and options to the local agent-ovs over the management network. This optimized DHCP approach is enabled by default in the solution, but can be reverted to the traditional centralized mode if desired. Metadata Proxy OpenStack VM's can receive instance-specific information such as instance-id, hostnames, and SSH keys from the Nova Metadata Service. This service is normally reached through a Neutron service acting as a proxy on behalf of OpenStack VM instances. The OpFlex ML2 software allows this proxy function to be distributed to each of the compute hosts. This optimized metadata proxy is disabled by default, and either a traditional centralized or the distributed approach may be configured. 7

18 Mapping OpenStack and ACI Constructs Solution Architecture The logical topology diagram in the Figure below illustrates the connections to OpenStack network segments from Neutron server(s) and compute hosts, including the distributed Neutron services. Figure 4: Logical OpenStack Network Connectivity with Distributed Neutron Services Note The Management/API network for OpenStack may be connected to servers using an additional Virtual NIC on a common uplink with tenant networking to the ACI fabric, or via a separate physical interface. Mapping OpenStack and ACI Constructs Cisco ACI uses a policy model to enable network connectivity between endpoints attached to the fabric. OpenStack Neutron uses more traditional Layer 2 and Layer 3 networking concepts to define networking configuration. The OpFlex ML2 driver translates the Neutron networking requirements into the necessary ACI policy model constructs to achieve the desired connectivity. The Table below illustrates OpenStack Neutron constructs, and the corresponding APIC policy objects that will be configured when they are created. OpenStack Object Neutron Instance Tenant/Project Tenant Network APIC Object ACI Tenant(s), VMM Domain APP Profile or Separate ACI Tenant Endpoint Group + Bridge Domain 8

19 Solution Architecture OpFlex NAT Operations OpenStack Object Subnet Security Groups/Rules Router External Network APIC Object Subnet N/A (Linux iptables rules are maintained per-host) Contract+EPG+Bridge Domain Layer 3 Out / Outside EPG By default, the OpFlex ML2 Driver associates an entire instance of OpenStack Neutron to a single ACI tenant, and names this tenant according to the apic_system_id setting in the /etc/neutron/plugins/ml2/ml2_conf_cisco_apic.ini file. This allows the ACI administrator to manage each cloud instance connected to the fabric as a single entity, and not generate a large number of ACI tenants in the APIC for a fabric that is being used for multiple systems. In this mode, separate OpenStack tenants are defined in the APIC as different Application Profiles. There is an alternative option that can be configured at installation time in the ml2_conf_cisco_apic.ini file using the setting single_tenant_mode = False which tells the system to create a new ACI Tenant for each OpenStack tenant. This results in a 1:1 correlation between OpenStack tenant and ACI tenant, and generates ACI tenants named according to the convention_<apic_system_id>_<openstack tenant name> for each OpenStack tenant. If multi-tenant mode is used the value apic_name_mapping = use_uuid should also be set in the ml2_conf_cisco_apic.ini file for proper system function. OpFlex NAT Operations The OpFlex ML2 Driver software brings the capability to support Network Address Translation (NAT) functions in a distributed manner using the local OVS instance on each OpenStack compute node. This distributed approach increases the availability of the overall solution, and offloads the central processing of NAT from the Neutron server L3-agent used in the reference implementation. IP Subnets Required for NAT Three distinct IP subnets are required to take full advantage of external network functionality with the OpFlex ML2 driver. This is a different approach from the default Neutron external network behavior that typically uses a single external subnet for these functions. Link Subnet This subnet represents the actual physical connection to the external next-hop router outside of the fabric. This will be assigned to a routed interface, sub-interface, or SVI depending on the configuration. Source-NAT Subnet The term Source-NAT or SNAT in OpenStack is used to describe allowing VM instances to initiate connections with destinations outside of the cloud by sharing an address on the external network. This subnet is used for Port Address Translation (PAT) allowing multiple VM s to share an outside-routable IP address. A single IP address is assigned to each compute host and Layer 4 port number manipulation is used to maintain unique session traffic. 9

20 OVS NAT and External Routing Solution Architecture Floating IP Subnet The term Floating IP in OpenStack is used when a VM instance is allowed to claim a distinct static NAT address to support inbound connections to the VM from outside the cloud. The Floating IP subnet will be the subnet assigned within OpenStack to the Neutron external network entity. Traffic egressing the cloud will carry a source IP address of either the SNAT subnet or Floating subnet. The routing hops external to ACI needs to have routes back to these subnets, either through a dynamic routing protocol or static configuration to allow return traffic to find its way back to OpenStack. OVS NAT and External Routing With the NAT function itself occurring in the local OVS of the compute node, the physical switches in the ACI fabric need only to route external traffic to and from the external next-hop router. This external routing is taken care of through a Virtual Routing and Forwarding (VRF) instance associated with the Layer-3 Out. This L3-Out VRF has an interface associated with the physical link to the external next-hop router. The same VRF also has an interface with an IP address of the assigned Source NAT subnet, and the Floating IP subnet. A loopback interface is also present in this VRF for routing protocol interaction. A graphical representation of the subnet architecture to support this NAT approach is shown in Figure below. Figure 5: OpFlex Local OVS NAT Subnet Architecture The L3-Out VRF associated with the OpenStack Neutron external network processes NAT traffic that egresses OVS on the compute host. Non-NAT traffic is processed by a Tenant VRF based on the OpenStack Tenant/Project association of the VM Instance. Optimized DHCP and Metadata Proxy Operations The OpFlex ML2 Driver software stack provides optimized traffic flow and distributed processing to provide DHCP and Metadata Proxy services for VM instances. These services are designed to keep as much processing 10

21 Solution Architecture Optimized DHCP Services and packet traffic local to the compute host. The distributed elements communicate with centralized functions to ensure system consistency. Optimized DHCP Services The reference OpenStack Neutron architecture utilizes the neutron-dhcp-agent service running on the Neutron server(s) to provide all DHCP communication to VM instances over the OpenStack tenant networks. The neutron-dhcp-agent provides central IP address management, and also communicates with each VM instance for DHCP Discovery, Offer, Request, and Acknowledgement (DORA) functions. The OpFlex optimized DHCP approach instead provides all DORA services locally on the compute host via the agent-ovs service. The distributed services communicate over the Management network to the Neutron server(s) for allocation of IP addressing and DHCP options. This architecture keeps the bulk of the packet traffic required to issue a DHCP lease local to the compute host itself, while also offloading the processing of this interaction from the Neutron server(s). An illustration of this DHCP architecture is provided in the Figure below. Figure 6: OpFlex based DHCP Architecture Optimized Metadata Services The reference OpenStack Neutron architecture for Metadata delivery to VM instances provides a centralized proxy service running on the Neutron server(s). This proxy service looks up instance information in Nova API, then adds HTTP headers and redirects the Metadata request to the Nova Metadata service. The Metadata requests from VM instances are transmitted on the OpenStack Tenant networks. The OpFlex optimized Metadata Proxy approach instead provides Metadata delivery using distributed Metadata Proxy instances running on each compute host. The agent-ovs service reads the OpFlex service file and programs a flow in OVS to direct Metadata service requests to the local neutron-metadata-agent. This local agent runs in a separate Linux namespace on the compute host. The Metadata Proxy function then accesses Nova-API and Nova Metadata Service running on the OpenStack Controller over the Management 11

22 APIC OpenStack VMM Integration Solution Architecture network to deliver VM-specific metadata to each VM instance. An illustration of this Metadata Proxy architecture is shown in the Figure below. Figure 7: OpFlex based Metadata Proxy Architecture APIC OpenStack VMM Integration Cisco ACI supports integration with multiple Virtual Machine Manager (VMM) systems, including OpenStack. This integration provides direct APIC visibility to the OpenStack compute nodes, including a detailed listing of all of the VM instances on each node along with the virtual interface information for each learned port. An example VM Networking view of OpenStack hypervisors is shown in the Figure below. Figure 8: APIC VM Networking Hypervisors View The VM Networking section of the APIC web interface also provides a view by Distributed Virtual Switch (DVS). Each DVS corresponds to an OpenStack network, which may be distributed across multiple compute nodes. The listing includes details of where each compute node and ACI leaf each VM instance is connected. 12

23 Solution Architecture APIC OpenStack VMM Integration This listing is instrumented with sort and filtering capabilities to locate any VM by IP or MAC address. An example VM Networking view of OpenStack DVS instances is shown in the Figure below. Figure 9: APIC VM Networking DVS View 13

24 APIC OpenStack VMM Integration Solution Architecture 14