Integration of Multi-Hypervisors with Application Centric Infrastructure

Transcription

1

2 Integration of Multi-Hypervisors with Application Centric Infrastructure BRKAPP-9005 Bradley Wong Principal Engineer

3 The Application Centric Infrastructure (ACI) is adopting an innovative approach to addressing these challenges, through normalisation of different hypervisor encapsulations together with tight integration of the Virtual Machine Manager (VMM) of choice, providing a single point of management for both physical and virtual infrastructure as well as the applications that run on top of them. This session will address how the ACI fabric handles single and multihypervisor environments, and how the ACI controller provides integration into different VMMs for a single point of management... BRKAPP-9005 ABSTRACT

4 Introduction to ACI

5 Cisco ACI Logical Network Provisioning of Stateless Hardware Web App DB Outside (Tenant VRF) QoS Filter QoS Service QoS Filter APIC ACI Fabric Non-Blocking Penalty Free Overlay Application Policy Infrastructure Controller 5

6 ACI Network Profile Policy-Based Fabric Management Application Extend the principle of Cisco UCS Manager service profiles to the entire fabric Network profile: stateless definition of application requirements - Application tiers - Connectivity policies - Layer 4 7 services - XML/JSON schema Fully abstracted from the infrastructure implementation - Removes dependencies of the infrastructure - Portable across different data centre fabrics Web Tier Storage App Tier ## Network Profile: Defines Application Level Metadata (Pseudo Code Example) <Network-Profile = Production_Web> <App-Tier = Web> <Connected-To = Application_Client> <Connection-Policy = Secure_Firewall_External> <Connected-To = Application_Tier> <Connection-Policy = Secure_Firewall_Internal & High_Priority>... <App-Tier = DataBase> <Connected-To = Storage> <Connection-Policy = NFS_TCP & High_BW_Low_Latency>... Storage DB Tier The network profile fully describes the application connectivity requirements 6

7 Application Policy Model and Instantiation Application policy model: Defines the application requirements (application network profile) Web Tier Application Client App Tier Storage DB Tier Storage Policy instantiation: Each device dynamically instantiates the required changes based on the policies APIC VM VM VM VM VM VM VM All forwarding in the fabric is managed through the application network profile IP addresses are fully portable anywhere within the fabric Security and forwarding are fully decoupled from any physical or virtual network attributes Devices autonomously update the state of the network based on configured policy requirements 7

8 Application Awareness Application-Level Visibility ACI Fabric provides the next generation of analytic capabilities PetStore Event Triggered Events or Queries Actions: No new hosts or VMs Evacuate hypervisors Re-balance clusters Per application, tenants, and infrastructure: Health scores Latency Atomic counters Resource consumption PetStore Dev Leaf 1 and 2 Spine 1 3 Atomic counters PetStore Prod Leaf 2 and 3 Spine 1 2 Atomic counters PetStore QA Leaf 3 and 4 Spine 2 3 Atomic counters Integrate with workload placement or migration APIC VXLAN Per-Hop Visibility Physical and Virtual as One 8

9 Providers Service Profile Service Graph ACI Layer 4-7 Service Integration Centralised, Automated, And Supports Existing Model Elastic service insertion architecture for physical and virtual services Helps enable administrative separation between application tier policy and service definition APIC as central point of network control with policy coordination Automation of service bring-up/tear-down through programmable interface Supports existing operational model when integrated with existing services Service enforcement guaranteed, regardless of endpoint location Application Admin Service Admin App Tier A Web Server Server begin Policy Redirection Chain Security 5 Security 5 Chain Defined Stage 1 inst inst Firewall.... Stage N inst inst Load Balancer end App Tier B Web App Server Server 9

10 Multi-Hypervisor-Ready Fabric Virtual Integration APIC Network Admin APIC ACI Fabric Integrated gateway for VLAN, VxLAN, and NVGRE networks from virtual to physical Normalisation for NVGRE, VLAN VXLAN VLAN NVGRE VLAN VXLAN VLAN VXLAN, and VLAN networks ESX Hyper-V KVM Customer not restricted by a choice of hypervisor Fabric is ready for multihypervisor Application Admin VMware Microsoft Red Hat XenServer Hypervisor Management VMware Microsoft Red Hat PHYSICAL SERVER 10

11 Open Ecosystem Framework Full-Featured, Programmable API And Data Model Northbound API Rapid integration with existing management frameworks System Management NetQoS HP CA Technologies SolarWinds Arbor Networks Tivoli Software NetBrain InfoVista Automation Tools Puppet Labs Opscode Python CFEngine OpenStack Tenant- and application-aware Hypervisor Management VMware XenServer Microsoft Red Hat KVM Orchestration Frameworks CloudStack OpenStack VMware Eucalyptus Nebula Object-Oriented Centralised Automation RESTful XML/JSON Open Ecosystem Framework Comprehensive Programmability and System Access Southbound API Publish data model Open source Enables application portability Microsoft XenServer Red Hat KVM *Only straight chains supported at FCS 11

12 ACI Fabric Policy Constructs

13 application rules of how application communicates to the external private or public networks a set of network requirements specifying how application components communicate with each other Contract Access Control QoS Network Services web app db Network Profile VM VM VM The Outside application-centric VM network VM policy VM network Virtual Patch Panel a collection of endpoints connecting to the network VMs, physical compute, Component Tier End Point Group 13

14 End-points Things that connect to the fabric and use it to interface with other things A compute, storage or service instance attaching to a fabric ACI Fabric NIC vnic... end-points [ ] 14

15 End-points Things that connect to the fabric and use it to interface with other things A compute, storage or service instance attaching to a fabric... A collection of end-points with identical network behaviour form a End Point Group (G) 15

16 End-point Groups (Gs) G APP SERVER policies G WEB.. Allows to specify rules and policies on groups of physical or virtual end-points without understanding of specific identifiers and regardless of physical location. Can flexibly map into application tier of multi-tier app segmentation construct (ala VLAN) a security construct ESX port group end-point group [ G ] 16

17 Tenant L3, L2 Isolation G G APP SERVER G WEB... network profile subnet Tenant self-contained tenant definition outside representable as a recursive structured text BD document subnet subnet BD With or without flooding semantics L3 context (isolated tenant VRF) 17

18 EXAMPLE: Three-tier APP Outside NW Public NW Private subnet subnet consume G WEB G APP G DB provide infra shared services consume consume consume web bundle consume java bundle provide provide provide provide mgmt bundle consume sql bundle provide L3 context bd bd bd 18

19 Overview of Hypervisor Integration

20 ACI Fabric Architecture ACI VXLAN (evxlan) Header Ethernet Header IP Header Payload FCS Outer Ethernet Outer IP Outer UDP evxlan Inner Ethernet Inner IP Header Payload New FCS 8 Bytes Flags Flags/DR E Source Group VXLAN Instance ID (VNID) M/LB/SP 1 Byte N L Rsvd I Rsvd N: The N bit is the nonce-present bit L: The L bit is the Locator-Status-Bits field enabled bit I: The I bit is the Instance ID bit, Indicates the presence of the VXLAN Network ID (VNID) field. When set, it indicates that the VNID field is valid 20

21 ACI Fabric Integrated Overlay ACI VXLAN (evxlan) Header All Tenant traffic within the Fabric is tagged with an ACI VXLAN (evxlan) header which identifies the policy attributes of the application end point within the fabric At the ingress port the Fabric translates an external identifier which can be used to distinguish different application end points via the ACI evxlan tagging format VT avt ACI VXLAN (evxlan) header identifies the attributes of the application end point within the fabric Policy attributes are carried by every packet Flags External Identifiers are localised to specific Leaf or Leaf ports (unless external requirements for consistency, e.g. downstream networks) SRC Group Outer IP Outer IP VNID 802.1Q NVGRE VXLAN Eth MAC Eth IP IP IP Eth IP MAC IP Payload Payload Payload Payload Payload Payload 21

22 ACI Fabric Integrated Overlay Multi-Hypervisor Encapsulation Normalisation IP Fabric Using evxlan Tagging Normalised Encapsulation Any to Any VT evxlan IP Payload Localised Encapsulation VXLAN VNID = Q VLAN 50 VXLAN VNID = NVGRE VSID = 7456 All traffic within the ACI Fabric is encapsulated with an extended VXLAN (evxlan) header External VLAN, VXLAN, NVGRE tags are mapped at ingress to an internal evxlan tag Forwarding is not limited to, nor constrained within, the encapsulation type or encapsulation overlay network External identifies are localised to the Leaf or Leaf port, allowing re-use and/or translation if required Outer IP Outer IP 802.1Q NVGRE VXLAN Eth MAC Eth IP IP IP Eth IP Normalisation of Ingress Encapsulation Payload Payload Payload Payload Payload 22

23 Hypervisor Integration with ACI VMM Domains Multiple Virtual Machine Managers (VMMs) likely on a single Fabric vcenter SCVMM Each VMM and associated Virtual hosts are grouped within APIC Called VMM Domain VMM Domain 1 VMM Domain 2 23

24 Hypervisor Integration with ACI VMM Domains & VLANs 16M Virtual Networks VLAN ID only gives 4K Gs (12 bits) Scale by creating pockets of 4K Gs Map to scope of live migration VMM Domain 1 4K Gs VMM Domain 2 4K Gs Place VM anywhere Live migrate within VMM domain 24

25 Hypervisor Integration with ACI VMM Domains 16M Virtual Networks VLAN ID only gives 4k Gs (12 bits) Scale by creating pockets of 4k Gs VMM Domain 1 4K Gs VMM Domain 2 4K Gs Map to scope of live migration Place VM anywhere Live migrate within VMM domain 25

26 Hypervisor Integration with ACI VMM Domains & VLANs 16M Virtual Networks VLAN ID only gives 4K Gs (12 bits) VNID 6032 Scale by creating pockets of 4K Gs Map to scope of live migration VLAN 5 VMM Domain 1 4K Gs VLAN 16 VMM Domain 2 4K Gs Place VM anywhere Live migrate within VMM domain 26

27 G Spanning Across VMM Domains Gs can take different network identities across VMM Domain Applications can be deployed across VMM Domains VM Mobility is not allowed between VMM Domain due to vcenter/scvmm limitation VMM Domain 1 vcenter vshield Hosts VMM Domain 1 Web G App G VMM Domain 2 Hosts VMM Domain 1 4k Gs DB G vcenter vshield App G VM VM VM VM VM VM VM VM VM 27

28 Recommended Practice for VLAN Networks Well separated VMM Domains Separate VLAN name space when VMM domains share TOR Best Practice for VMM Domain definition vcenter 1 vcenter 2 VMM Definition to avoid vcenter 1 vcenter 1 VLAN range Overlapping name space on the same TOR VLAN range

29 OpFlex A Flexible, Extensible Policy Protocol OPFLEX is a new extensible policy resolution protocol designed for declarative management of any data centre infrastructure. Unlike legacy protocols such as OVSDB, OPFLEX was designed to offer: APIC Policies Who can talk to whom What about Topology control Ops stuff Declarative resolution Push + Pull API support Abstract policies rather than device-specific configuration Opflex Agent Opflex Agent Opflex Agent Opflex Agent Flexible, extensible definition of using XML / JSON Support for any device vswitch, physical switch, network services, servers, etc. Opflex Proxy Legacy API Opflex Agent Firewall Opflex Agent Hypervisor Switch Opflex Agent ADC 29

30 Hypervisor Integration with ACI Endpoint Discovery Virtual Endpoints are discovered for reachability & policy purposes via 2 methods: APIC Control Plane Learning: - Out-of-Band Handshake: vcenter APIs - Inband Handshake: OpFlexenabled Host (N1KV, Windows Server 2012, etc.) Data Path Learning: Distributed switch learning Control (OpFlex) Data Path Data Path VMM Control (vcenter API) LLDP used to resolve Virtual host ID to attached port on leaf node (non-opflex Hosts) OpFlex Host DVS Host 30

31 Design Considerations VLAN-Based Hypervisor Networks APIC Hosts are assigned VLAN ID to G binding through VMM & APIC Integration Intermediate L2 nodes not managed need to manage VLANs on these for each VMM Domain Provision all VLANs for VMM Domain Endpoint location discovered through stitching LLDP TLVs (non OpFlex-enabled Hosts) 31

32 Design Considerations VXLAN & NVGRE-based Hypervisor Networks APIC Hosts are assigned VNID and VSID to G binding through VMM & APIC Integration Infra-VLAN is extended out to front-panel tenant ports - Infra-VLAN needs to be provisioned on intermediate L2 Nodes Endpoint location discovered though stitching LLDP TLVs (non OpFlex-enabled Hosts) Provision Infra VLAN for Hypervisors 32

33 Integration with VMware DVS

34 ACI Fabric and VMware DVS Integration F/W G WEB APIC Application Network Profile L/B G APP WEB PORT GROUP APP PORT GROUP DB PORT GROUP G DB How does ACI Fabric implement policy? - Assigning s to Gs What are s in virtual environment? - VM vnics How does VMware apply network configuration? - Port Groups How are Gs exposed to VMware? - Map Gs to Port Groups VM VM VM 34

35 Cisco ACI Hypervisor Integration VMware DVS APIC 5 Create Application Policy F/W Application Network Profile G WEB L/B GAP P G DB APIC Admin 9 Push Policy (Lazy) ACI Fabric 1 Cisco APIC and VMware vcenter Initial Handshake 6 Automatically Map G To Port Groups 4 Learn location of ESX Host through LLDP 2 Create VDS VIRTUAL DISTRIBUTED SWITCH VI/Server Admin vcenter Server 8 Instantiate VMs, Assign to Port Groups 7 3 Create Port Groups Attach Hypervisor to VDS WEB PORT GROUP APP PORT GROUP DB PORT GROUP Web App HYPERVISOR DB Web Web HYPERVISOR DB 35

36 Integration with Microsoft

37 Microsoft Azure Pack Integration Integration with Microsoft requires: - Windows Server Systems Centre 2012 R2 with SPF - Windows Azure Pack Azure Pack provides single pane of glass for Definition, creation, management of their cloud service Divided into Provider (Admin) portal and Consumer Self-Service (Tenant) portal Cisco ACI Service Plugin enables management of Network Infrastructure through APIC REST API Service Plans Users Web Sites Service Provider Provider Portal VMs SQL Web Sites Apps Database VMs ACI Service Bus Customer Consumer Self-Service Portal R2 w/ Service Provider Foundation 37

38 Microsoft Azure Pack Integration Admin Experience Add & Configure service providers for this deployment (APIC IP Address, Login Credentials, etc.) Usage & Billing statistics per user and other admin functions 38

39 Microsoft Azure Pack Integration Tenant Experience Services this account has access to Resources of ACI service currently created and consumed by this tenant Application Network Profiles are created through Azure Pack, and pushed to APIC using REST APIs 39

40 ACI Azure Pack Integration APIC 1 APIC Admin (Basic Infrastructure) 7 ACI Fabric 3 2 Pull Policy on leaf where attaches Get VLANs allocated for each G Push Network Profiles to APIC Create Application Policy 1 Create VM Networks Instantiate VMs 6 Indicate Attach to attached leaf when VM starts APIC Plugin SCVMM Plugin OpFlex Agent OpFlex Agent OpFlex Agent HYPERVISOR HYPERVISOR HYPERVISOR Azure Pack Tenant Azure Pack \ SPF Web App Web App DB Web Web DB 40

41 Nexus 1000V ACI Edition

42 Cisco ACI - Application Virtual Switch (AVS) NETWORK VIRTUALISATION SUPPORT AV S Application Virtual Switch AV S AV S HYPERVISOR HYPERVISOR HYPERVISOR PURPOSE BUILT VIRTUAL MEMBER OF ACI OPTIMAL TRAFFIC STEERING INTEGRATED VISIBLITY THROUGH APIC (PHYSICAL AND VIRTUAL) COMMON MANAGEMENT MODEL THROUGH APIC MULTI-HYPERVISOR SUPPORT OPEN APIS 42

43 Nexus 1000V Integration Overview OpFlex Control protocol - Control channel - VM attach/detach, link state notifications VEM extension to the fabric vsphere 5.0 and above BPDU Filter/BPDU Guard SPAN/ERSPAN Port level stats collection Southbound OpFlex API VM VM VM VM AVS Hypervisor Manager vsphere 43

44 Cisco ACI Hypervisor Integration VMware N1KV VEM APIC 5 Create Application Policy F/W Application Network Profile G WEB L/B GAP P G DB APIC Admin 9 Push Policy (Lazy) ACI Fabric 1 Cisco APIC and VMware vcenter Initial Handshake 6 Automatically Map G To Port Groups 4 Learn location of ESX Host through OpFlex OpFlex Agent OpFlex Agent VI/Server Admin vcenter Server 8 Instantiate VMs, Assign to Port Groups Create N1KV VDS Create Port Groups Attach Hypervisor to VDS NEXUS 1000V VDS WEB PORT GROUP APP PORT GROUP DB PORT GROUP Web App HYPERVISOR DB Web Web HYPERVISOR DB 44

45 Nexus 1000V Switching Modes NS Non switching mode, FEX mode LS Local switching within Gs on the same host, similar behaviour as ESX VDS Non switching mode Local switching mode Punt to Leaf for all traffic Hypervisor Punt to Leaf for intra-g traffic Hypervisor VM G Web VM VM G App VM VM G Web VM VM G App VM 45

46 Cisco AVS Differentiation with ACI Hypervisor Networking VDS/OVS AVS No Switching Yes Yes Local Switching Yes Yes Full Switching(routing etc) No Yes Optimal Traffic Steering No Yes Local (on-host) Policy Enforcement No Yes Single Point of Management with APIC No Yes, Robust Atomic Counters No Yes End-to-End Visibility Yes Enhanced Consistency Across Hypervisors No Yes Enhanced NX-OS No Yes Ease of Install/Upgrade Separate Integrated 46

47 Integration with OpenStack

48 OpenStack Components Initial Focus on Networking (Neutron) 48

49 OpenStack Neutron Networking Model Tenant Router Network: external Network Security Group Subnet Port Security Group Rule L3 + External Net Extension Core API Sec Grp Extension 49

50 OpenStack Deployment Management Network Provider Networks dhcp-agent *-plugin-agent neutron-l3-agent L2B/OVS L2B/OVS Network Node Data Network nova-compute nova-compute nova-compute *-plugin-agent nova-compute *-plugin-agent *-plugin-agent *-plugin-agent Compute Node Compute L2B/OVS Node Compute Node Compute Node API Network nova-api nova-scheduler neutron-server keystone mysql, rabbit... Neutron PIugin Cloud Controller Node External Network Internet API Network is typically routable to enable public access 50

51 Cisco ACI Model Tenant Outside Network App Profile Bridge Domain Context (VRF) Contract Subnet Subject Endpoint Group 51

52 Cisco ACI Model Neutron API Mapping OpenStack ACI Tenant Tenant No equivalent Application Profile Network G Subnet Subnet Security Group Handled by Host Security Group Rule Handled by Host Router Context Network:external Outside 52

53 OpenStack Managed Networks Controller Node ACI Fabric offers hardware VXLAN encapsulation Flexible placement of endpoints APIC APIC Plugin converts networks to End Point Groups APIC Plugin Neutron Networking OVS Plugin Distributed L3 gateway L2 Extension to physical servers, etc. Leverage standard OVS Network mapped to segment ID VLAN / VXLAN IPTables available for security group functions OVS Network A V(X)LAN /24 Network B V(X)LAN /24 IPTables OVS Network A V(X)LAN /24 Network C V(X)LAN /24 OVS Network A V(X)LAN /24 Network B V(X)LAN /24 Host 1 Host 2 Host 3 OVS Network A V(X)LAN /24 Network C V(X)LAN /24 IPTables IPTables IPTables Host 4 53

54 ACI OpenStack Integration - FCS APIC 3 Create Application Policy APIC Admin (Performs Steps 3) 5 Push Policy ACI Fabric 2 Automatically Push Network Profiles to APIC Create Network, Subnet, Security Groups, Policy 1 NETWORK ROUTING SECURITY OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH NEUTRON NOVA 4 Web App Web App DB Web Web DB OpenStack Tenant (Performs Steps 1,4) Instantiate VMs HYPERVISOR HYPERVISOR HYPERVISOR 54

55 ACI Neutron Plugin OVS driver Neutron API ML2 Plugin APIC driver APIC REST API APIC APIC Plugin for Fabric using ML2 framework Translates Neutron primitives to ACI policy model ML2 allows plugin to select network technology Existing Neutron functions only KVM KVM KVM KVM KVM KVM KVM OVS OVS OVS OVS OVS OVS OVS 55

56 Group-based Policy in OpenStack Messy mapping ACI to current OpenStack components Endpoint Groups (Ports + Security Groups) Contracts (Security Groups + Security Group Rules) Goal : Introduce ACI model into OpenStack Starting with Groups and Group based Policies 56

57 Group-based Policy in OpenStack Group-Based Policy Model Extensions (ACIcompatible) Dashboard Automation Compute Networking Storage Team meets weekly at 8am PT ACI Fabric Merchant Silicon OpenFlow Software Overlay Etc. 57

58 ACI OpenStack Integration Post-FCS Create Application Network Profile 1 F/W L/B Application Network Profile G WEB L/B G APP G DB NEUTRON NOVA 4 Web App Web App DB Web Web DB OpenStack Tenant (Performs step 1,4) Instantiate VMs HYPERVISOR HYPERVISOR HYPERVISOR 2 Automatically Push Network Profiles to APIC APIC 3 Create Application Policy F/W L/B Application Network Profile G WEB L/B G APP G DB ACI Admin (manages physical network, monitors tenant state) 5 Push Policy ACI Fabric 58

59 APIC Managed Networking ACI Fabric also supports OpenStack through the addition of an Operator Managed API Layer Operator Managed API Layer for Compute and Network VLAN / VXLAN coordination agent Highlights: No OpenStack changes required! - OpenStack running OVS Plugin or even nova network Network Policy defined by APIC in terms of Gs, Contracts, etc. Requires Operator Managed API Layer OVS APIC OVS OVS OVS Plugin OVS Plugin OVS Plugin OVS Host 1 Host 1 Host 1 Host 1 Map VLAN / VXLAN to G 59

60 Operator Managed API Layer APIC Managed Networking Create Network/Subnet for each G NEUTRON NETWORK NEUTRON SUBNET 2 NEUTRON NOVA 4 Web App Web App DB Web Web DB OpenStack Tenant (Performs step 1,4) Instantiate VMs, Join Network HYPERVISOR HYPERVISOR HYPERVISOR APIC 1 Create Application Policy F/W L/B Application Network Profile G WEB L/B G APP G DB ACI Admin (manages physical network, monitors tenant state) 5 ACI Fabric Push Policy 60

61 Q & A

62 Complete Your Online Session Evaluation Give us your feedback and receive a Cisco Live 2014 Polo Shirt! Complete your Overall Event Survey and 5 Session Evaluations. Directly from your mobile device on the Cisco Live Mobile App By visiting the Cisco Live Mobile Site Visit any Cisco Live Internet Station located throughout the venue Polo Shirts can be collected in the World of Solutions on Friday 21 March 12:00pm - 2:00pm Learn online with Cisco Live! Visit us online after the conference for full access to session videos and presentations. 62

63