Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

Transcription

1 NET1350BUR Deploying NSX on a Cisco Infrastructure Jacob Rapp jrapp@vmware.com Paul A. Mancuso pmancuso@vmware.com #VMworld #NET1350BUR

2 Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. #NET1350BUR CONFIDENTIAL 2

3 Agenda 1 NSX Anywhere 2 NSX Design 3 Nexus Switching Fabric 4 UCS Connectivity 5 NSX on ACI 6 Summary & Question #NET1350BUR CONFIDENTIAL 3

4 Agenda 1 NSX Anywhere 2 NSX Design 3 Nexus Switching Fabric 4 UCS Connectivity 5 NSX on ACI 6 Summary & Question #NET1350BUR CONFIDENTIAL 4

5 Apps Driving Infrastructure Requirements Infrastructure independent, with standards-based interoperability where necessary Security wrapped around the VM, container & microservice Works across hypervisors, application frameworks, clouds People, Process and Tooling Model #NET1350BUR CONFIDENTIAL 5

6 Infrastructure Independent Any L2/L3 Fabric Controlled Communication Controlled Communication Micro- Segment Controlled Communication Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful Stateful DFW DFW 3 rd Party Services 3 rd Party Services 3 rd Party Services 3 rd Party Services Any Transport Consistent policy Any IP network Up to 150ms Universal Distributed Logical Router Controlled Communication Controlled Communication Micro- Segment Controlled Communication Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful Stateful DFW DFW 3 rd Party Services 3 rd Party Services 3 rd Party Services 3 rd Party Services Public clouds ACI Fabric Single/Multi Pod Or Any L2/L3 Fabric #NET1350BUR CONFIDENTIAL 6

7 Security Everywhere Control Points Data Center Cloud Campus/Branch Mobile Endpoint Context Telemetry: Application Network Intelligence Policy Complete framework spanning use cases and service stack #NET1350BUR CONFIDENTIAL 7

8 Multi-Hypervisor, Cloud and Frameworks Branch offices On-Premise Data Center vcloud Air Network Public clouds VMworld 2017 Content: Not for Virtual Desktop (VDI) New app frameworks publication Mobile Devices (Airwatch) Security Inherently Secure Infrastructure Automation IT at the Speed of Business Application Continuity Data Center Anywhere #NET1350BUR CONFIDENTIAL 8

9 People, Process and Tooling Model Ending State Blended, cross-functional People Speed, agility, standardization Processes VMworld 2017 Content: Not for Converged, correlated publication Tooling Beginning State Siloed, specialized Slow, error prone, inconsistent Domain-specific, hardware focused Reseach paper detailing operational changes: #NET1350BUR CONFIDENTIAL 9

10 Agenda 1 NSX Anywhere 2 NSX Design 3 Nexus Switching Fabric 4 UCS Connectivity 5 NSX on ACI 6 Summary & Question #NET1350BUR CONFIDENTIAL 10

11 Software Defined Data Center Delivers Freedom Network hardware centric application and service deployment Difficult operational model Complex protocol mix Hardware ASIC dependent Finite service offering Micro- Stateful DFW Micro- Stateful Micro- Micro- Segment Stateful DFW Micro- 3 rd Party Segment Stateful DFW DFW Micro- Micro- 3 rd Party Segment Stateful DFW Segment Stateful 3DFW 3 rd Party Services Segment rd Party Segment Stateful DFW 3 rd Party 3 rd Party Services Segment Stateful DFW 3 rd Party Services Services 3 rd Party Services Services Services Services Micro- Segment Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful DFW Stateful DFW Decouple applications from hardware infrastructure: Simple, Scalable Service model #NET1350BUR CONFIDENTIAL 11

12 NSX vsphere Cluster Design Tiered Logical Switches Web Application Database Transit VXLAN Management (VLAN) VM1 VM5 Transport Zone (VLAN) VM2 VM3 VM5 VM6 VMworld 2017 Content: Not for Compute Clusters Edge Cluster(s) Management Cluster Physical Network NSX Edge Control VM NSX Controller Cluster NSX Manager publication Transport Subnet A /24 Transport Subnet B /24 #NET1350BUR CONFIDENTIAL 12

13 VMkernel Networking Span of VLANs IP Stacks Default vmotion VXLAN SVI 66: /26 SVI 77: /26 SVI 88: /26 SVI 99: /26 VLAN 66 Mgmt /26 DGW: VLAN 77 vmotion /26 DGW: VLAN 88 VLAN Trunk (802.1Q) VXLAN /26 DGW: Layer 2 or Layer 3 Uplinks VLAN 99 or distribution vsphere Host (ESXi) Storage /26 DGW: Span of VLANs #NET1350BUR CONFIDENTIAL 13

14 NSX VXLAN Data and Control Plane NSX VXLAN Control Plane Unicast mode or Hybrid Mode Unicast preferred with ACI infrastructure NSX Data Plane VMKernel VTEP encaps/decaps Transport Zone VXLAN VTEP 3 VXLAN VTEP L2 Frame L2 IP UDP VXLAN L2 Payload IP Frame UDP VXLAN L2 Payload L2 Frame VM Sends a standard L2 Frame Source Hypervisor (VTEP) encapsulates VXLAN, UDP & IP Headers Physical Network forwards frame as standard IP frame Destination Hypervisor (VTEP) decapsulate headers Original L2 Frame delivered to VM #NET1350BUR CONFIDENTIAL 14

15 NSX VXLAN Infrastructure Transport Zone VXLAN prepared ESXi clusters VTEP(s) 1 prepared VDS per cluster enabled for VXLAN Logical Switch as dvportgroup Support for VMware vds managed by vsphere VTEP VM MAC1 vsphere Host VM MAC2 Compute VDS VTEP VMworld 2017 Content: Not for VXLAN 5002 VXLAN prepared Hosts VM MAC3 Edge VDS VM MAC4 VTEP3 VTEP4 vsphere Host publication VXLAN Transport Network #NET1350BUR CONFIDENTIAL 15

16 VDS Uplink Design NSX host preparation Creates VXLAN dvuplink Consistent for all hosts using that VDS Consistent teaming policy Recommended teaming mode Route Based on Originating Port LACP is not possible from UCS blade VMworld 2017 Teaming Mode Route based on Originating Port Route based on Source MAC hash NSX Support Multi- VTEP LACP Route based on IP Hash (Static EtherChannnel) Explicit Failover Order Uplink 2 x 10G/40G Nexus/ACI Port Configuration All Active Standard All Active Standard Flow based All active Flow based All active Single link active vpc Port-Channel - LACP Content: Not for publication vpc Port-Channel LACP mode OFF Standard Route based on Physical NIC Load (LBT) Standard #NET1350BUR CONFIDENTIAL 16

17 Connect Your Workloads to a Physical Network Route and switch where you can. VLAN subnets Transit subnet VXLAN subnets VM1 VM5 VM2 VM5 VM3 VMworld 2017 Web Application Database Compute/Edge Clusters Physical Network VXLAN VLAN Bridge if you must. VM1 VM5 Content: Not for publication VM2 VM5 VM3 Each bridged segment is a separate subnet #NET1350BUR CONFIDENTIAL 17

18 Agenda 1 NSX Anywhere 2 NSX Design 3 Nexus Switching Fabric 4 UCS Connectivity 5 NSX on ACI 6 Summary & Question #NET1350BUR CONFIDENTIAL 18

19 NSX Infrastructure Requirements Only Two Requirements 1) MTU of ) IP Connectivity NSX is AGNOSTIC to underlay network topology L2 or L3 switched infrastructure #NET1350BUR CONFIDENTIAL 19

20 Jumbo MTU Nexus 5000 and 6000 VDS Max MTU is 9000 Byte Nexus 5xxx, 56xx and 6xxx requires MTU to be changed with Policy-Map L3 requires per interface MTU change All links belonging to fabric must be enabled with Jumbo MTU VMworld 2017 All L2 interfaces Only global configurations Create policy-map: policy-map type network-qos jumbo class type network-qos class-default mtu 9216 Apply policy-map: system qos service-policy type network-qos jumbo Layer 3 Interface interface Vlan151 SVI Interface no ip redirects ip address /27 hsrp 1 ip description VXLAN Transport Zone no shutdown mtu 9216 Content: Not for publication interface Ethernet2/12 Layer 3 Interface description L3 Link to Spine no switchport speed duplex full mtu 9216 ip address /31 no shutdown #NET1350BUR CONFIDENTIAL 20

21 Jumbo MTU Nexus 7000 and 9000 VDS Max MTU is 9000 Byte Nexus 7xxx, 9xxx Series L2 only requires global configuration L3 requires per interface MTU change All links belonging to fabric must be enabled with Jumbo MTU VMworld 2017 Layer 2 Interface system jumbomtu 9216 Global configurations interface Ethernet1/9 description to esx-vmnic3-vmk switchport mode trunk switchport trunk allowed vlan spanning-tree port type edge trunk mtu 9216 Layer 2 MTU channel-group 9 mode active Layer 3 Interface interface Vlan151 SVI Interface no ip redirects ip address /27 hsrp 1 ip description VXLAN Transport Zone no shutdown mtu 9216 Content: Not for publication interface Ethernet2/12 Layer 3 Interface description L3 Link to Spine no switchport speed duplex full mtu 9216 ip address /31 no shutdown #NET1350BUR CONFIDENTIAL 21

22 Cisco DC Topology L2 Pod NSX is Agnostic Pod components can be any mix of 9k / 7k/ 6k / 5k / 2k Compute Cluster A Compute Cluster B L3 L2 POD A 95xx L3 Core L3 95xx 95xx 95xx L2 95xx 95xx 93xx 93xx 93xx 93xx 93xx 93xx 93xx 93xx VMworld 2017 Content: Not for POD B VLANs & IP Subnet Defined at 95xx for POD A SVI Interface VLAN ID IP Subnet Management A.x/24 publication vmotion A.x/24 Storage A.x/24 VXLAN A.x/24 VLANs & IP Subnet Defined at 95xx for POD B SVI Interface VLAN ID IP Subnet Management B.x/24 vmotion B.x/24 UCS B-Series VLAN ID 100, 101 & 102 Scope UCS B-Series VLAN ID 200, 201 and 203 Scope Storage B.x/24 VXLAN B.x/24 VXLAN VLAN ID Transport Zone Scope (extends across ALL PODs/clusters) #NET1350BUR CONFIDENTIAL 22

23 Cisco DC Topologies L3 Design NSX is Agnostic Compute Cluster A Compute Cluster B L3 L2 POD A 95xx L3 Core 95xx 95xx 95xx 95xx 95xx L3 93xx 93xx 93xx 93xx 93xx 93xx 93xx 93xx L2 POD B VLANs & IP Subnet Defined at each ToR SVI Interface VLAN ID IP Subnet Management R_ID.x/24 vmotion R_ID.x/24 Storage R_ID.x/24 VXLAN R_ID.x/24 UCS B-Series VLAN ID 100, 101 & 102 Scope UCS B-Series VLAN ID 100, 101 & 102 Scope VXLAN VLAN ID Transport Zone Scope (extends across ALL PODs/clusters) #NET1350BUR CONFIDENTIAL 23

24 Compute Cluster A Compute Cluster B Cisco DC Topologies VXLAN Leaf Spine - Leaf can be: 9xxx, 7xxx, 6xxx, 56xx UCS B-Series Spine UCS B-Series L3 Spine 95xx 95xx 95xx 95xx 56xx 56xx 93xx 93xx 93xx 93xx or UCS B-Series VLAN ID 100, 101 and 102 Scope VXLAN VLAN ID Transport Zone Scope (extends across ALL PODs/clusters) VLANs & IP Subnet Defined at each ToR SVI Interface VLAN ID IP Subnet Management x/24 vmotion x/24 Storage x/24 VXLAN x/24 60xx UCS B-Series 60xx DC Core Internet/DMZ distribution Border Leaf Mgt / Edge Cluster Cisco s Prime or NFM may also provide underlay and VXLAN management. (NFM supports only 9ks) #NET1350BUR CONFIDENTIAL 24

25 Agenda 1 NSX Anywhere 2 NSX Design 3 Nexus Switching Fabric 4 UCS Connectivity 5 NSX on ACI 6 Summary & Question #NET1350BUR CONFIDENTIAL 25

26 NSX Connectivity Ideals vsphere Host and UCS Interconnectivity UCS Fabric Interconnects End-Host mode vpcs to Nexus switching vsphere Compute connectivity UCS vnics shared or dedicated vsphere dvuplinks equals number of vmnics VTEPs with Src ID teaming vsphere Edge connectivity Preferably UCS C-Series Separate connection, bypass FIs 95xx L3 L2 DC Fabric 93xx UCS 95xx 93xx Edge Leaf or Edge Cluster UCS C-Series DC Core Internet/DMZ distribution #NET1350BUR CONFIDENTIAL 26 VPN VPN VPN VPN

27 NSX Edge Routing for UCS C Series & Intel NICs Recommended design for Edge routing: Two Uplinks per ESG Per ESG, establish adjacency to each ToR (redundancy) VLAN backed networks routing neighbors on the ToRs A/B Map each VLAN of the dvportgoup to each dvuplink VLANs used for dynamic routing are local to each router Use Source ID ebgp is used between NSX ESG and routers A/B Equally applicable to OSPF Default route must follow the uplink status VMworld 2017 Loss of Uplinks will withdraw routes on that fabric and upstream link Default route advertised downstream ebgp Peering VLAN 10 SVI vnic1 Uplink A VLAN 10 95xx 9K-A vds-edge 9K-B 95xx VLAN 20 SVI Content: Not for publication vnic2 Uplink B VLAN 20 Pair of NSX Edges per ESXi host #NET1350BUR CONFIDENTIAL 27

28 NSX Edge Routing for UCS B Series Recommended design for Edge routing Dedicated UCS vnics pinned for routing Port-Channel connection between FI and ToRs Redundancy and scaling bandwidth Per ESG, establish an adjacency per ToR (Redundancy) Each ESG, bound to one active dvuplink ebgp is used between NSX ESG and N9Ks Equally applicable to OSPF Default route must follow the uplink status Loss of Uplinks will withdraw routes on that fabric and upstream link VMworld 2017 Content: Not for Default route advertised downstream VLAN 10 SVI Dedicated non-vpc pinned UCS FI-A 1 vnic maps to Fabric A VLAN 10 95xx 9K-A vds-edge 9K-B publication 95xx VLAN 20 SVI UCS FI-B 1 vnic maps to Fabric B VLAN 20 ebgp Peering Pair of NSX Edges per ESXi host #NET1350BUR CONFIDENTIAL 28

29 VDS Design, Uplink & Traffic Mapping Recommended UCS B-Series setup UCS B-Series Blade vnic 1 93xx 6248 (A) 6248 (B) 2204 FEX vnic FEX vnic 3 Nexus 93XX NX-OS Mode vnic 4 VMNIC 0 VMNIC 1 VMNIC 2 VMNIC 3 VDS - 1 MGMT, vmotion, NFS, VXLAN & Bridging 93xx VDS 2 Routing Traffic Type VMkernel IP Storage VMkernel vmotion VMkernel VXLAN VTEP 1 VMkernel VXLAN VTEP - 2 VMkernel Mgmt Bridging PG Routing VLAN PG 10 Routing VLAN PG 20 Teaming Mode LBT LBT SRC_ID SRC_ID Explicit Failover SRC_ID SRC_ID SRC_ID #NET1350BUR CONFIDENTIAL 29

30 Agenda 1 NSX Anywhere 2 NSX Design 3 Nexus Switching Fabric 4 UCS Connectivity 5 NSX on ACI 6 Summary & Question #NET1350BUR CONFIDENTIAL 30

31 NSX Network Virtualization with ACI underlay Cloud Management Platform vrealize, OpenStack, Custom NSX vcenter NSX Network Virtualization provides Cloud automation Integrated NSX service deployment Embedded security deployment Provides P 2 V integrated services and security Inherent Services: LB, DHCP, NAT, VPN, and 3 rd party service insertion Customer Benefits No dependency on infrastructure for service mobility Choice of underlay network Opportunistic leveraging of fabric E.g -> ACI and network infrastructure isolation #NET1350BUR CONFIDENTIAL 31

32 NSX Over ACI Recommendations ACI Fabric Ideals Layer 2 Fabric: Single Tenant Fewer Contract Needs Map Static vsphere EPs Map NSX Edge to ACI Border Fabric Infrastructure Minimum Requirements: 1 Physical Domain 1 External Routing Domain 2 VLAN Pools (Int & Ext) 1 AEP (Leaf & Switch Policies, Int & Int Sel Policies, etc..) Tenant: NSX over ACI Tenant Separate tenant (not common) 1 Application (Network) Profile 4 EPGs (base epgs) 4 Bridge Domains, 1 VRF 2 L3Outs; North and South #NET1350BUR CONFIDENTIAL 32

33 NSX Over ACI: ACI Infrastructure NSX Overlay Compute VMKernel ACI Contracts ACI EPGs Layer 2 ACI Fabric VM1 VM3 VM5 VM2 VM4 Web LS APP LS DB LS Compute VDS Edge VDS VMworld 2017 Content: Not for VM6 MGMT vmotion Storage Transport Peer VLANs ACI Infrastructure Supports attachment of hosts Define phy domain of host attachment VLANs, switch interfaces, and policies in use publication Domains, Physical and External Create Application Profile Defines EPGs Networks Private Networks Bridge Domains External L2 and L3 connectivity #NET1350BUR CONFIDENTIAL 33

34 ACI Infrastructure Screen Shot 4 EPGs Specify Physical domain and define Static Ports along with a VLAN encapsulation Contracts unnecessary for Intra EPG communication #NET1350BUR CONFIDENTIAL 34

35 NSX Edge Mapping to ACI Border Leaf Compute Clusters Compute workloads Services (Tenant LB, NAT, etc) Distributed routing, switching, DFW Edge Clusters ECMP routing with ACI border leafs ESGs single active uplink per dvuplink Vmkernel VXLAN Overlay Compute ACI Spine EPG/VLAN 10 L3Out DC Core Edge North bound L3Outs L3 L2 Border Leafs EPG/VLAN 20 L3Out ECMP Edges Routing Adjacency VM1 VM3 VM5 Control VM #NET1350BUR CONFIDENTIAL 35

36 NSX Edge BGP AS Configuration (Prod) BGP Connectivity DC Core to ACI Border Leafs ebgp connection DC Core: AS 1 ACI Leafs: AS ACI Border Leafs to ESGs ebgp connection ACI Leafs: AS ESGs: ESGs to DLR CVM ebgp connection ESGs: DLR Control VM: ACI Spines MPBGP AS This AS must NOT be used by ACI for L3Out Neighboring ACI Spine EPG/VLAN 1209 Vmkernel VXLAN Overlay BGP L3Out E1 E2 Control VM BGP L3 L2 E3 L3Outs North BGP 1 Border Leafs EPG/VLAN 1210 L3Out E4 DC Core Edge Cluster BGP BGP #NET1350BUR CONFIDENTIAL 36

37 ACI Tenant Networking Networking Bridge domains VRFs or Private Networks vzany contract for EPG collection for VRF Allow all L3Out and IP Mgmt traffic to communicate externally External Routed Networks NSX to ACI border leafs ACI border leafs to Nexus DC cloud (not pictured) Default Route Leak Policy Allow Default route #NET1350BUR CONFIDENTIAL 37

38 ACI L3Outs NSX Edges L3 External Routing Per ACI Leaf config BG Peer defined Utilize defined SVI per leaf, per VLAN SVI s IP address per leaf node Define switch node, ports and transit vlan #NET1350BUR CONFIDENTIAL 38

39 ACI BGP Peer Connectivity BGP Peer Connectivity Profile Define BGP peer, BGP Prefix policy for prefix distribution, and relative BGP neighbor attributes #NET1350BUR CONFIDENTIAL 39

40 NSX Edge Mapping to ACI Border Leaf L3Out Edge Connectivity Transit VLANs 1209: /29 Edges: E1:210, E3:211 ACI SVI: : /29 Edges: E2:.218, E3:.220 ACI SVI:.217 Vmkernel VXLAN Overlay Compute /29 VLAN 1209 ACI Spine E1 Edge Cluster Transit VXLAN E2 Control VM L3 L2 E3 DC Core L3Outs North Border Leafs /29 VLAN 1210 E4 ECMP Transit VXLAN /29 DLR Forwarding IP:.185 DLR Protocol IP:.186 E1 E4: ( ) #NET1350BUR CONFIDENTIAL 40

41 Multi Site Heterogenous DR Universal Logical Switches VMworld 2017 L2/L3 DCI Any L2/L3 DCI Policy Everywhere - Stateful Firewall - Network Introspection - Identity Firewall - Decoupled - Less replacement cycles Content: Not for publication #NET1350BUR CONFIDENTIAL 41

42 Agenda 1 NSX Anywhere 2 NSX Design 3 Nexus Switching Fabric 4 UCS Connectivity 5 NSX on ACI 6 Summary & Question #NET1350BUR CONFIDENTIAL 42

43 NSX Design Guides Reference Design: Deploying NSX with Cisco UCS and Nexus 9000 Infrastructure Design Guide for Vmware NSX running with a Cisco ACI Underlay Fabric NSX-V Multi-Site Options and Cross-VC NSX Design Guide VMworld 2017 VMware NSX for vsphere Network Virtualization Design Guide version Content: Not for publication #NET1350BUR CONFIDENTIAL 43

44

45