NET1949BU Seamless Network Connectivity for Virtual and Bare-metal s with NSX Suresh Thiru Sridhar Subramanian VMworld 2017 Content: Not for publication VMworld 2017 - NET1949BU
Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. 2
Agenda 1 NSX Everywhere 2 Bare-metal Use Cases 3 NSX Solutions 4 Design Considerations and Best Practices 3
Application Drives Infrastructure What does this mean for Networking and Security? Infrastructure independent Security wrapped around the VM, container, microservice Works across Hypervisors, Clouds, Application frameworks
Evolution of Server Computing Introduces new Networking and Security requirements VM Explosion of VM and mobility led to network virtualization Dynamic Container Native container network with multi-tenancy, micro-segmentation, and common tools for day 2 operations Public Cloud Networking And Security Services Full visibility and control with consistent operation across private and public cloud Static Bare-metal Seamless connectivity and security for physical workloads (Legacy App, DB, Storage, Security Appliance)
New Silos Leads to Operational Inefficiencies Challenges: Different technology stacks, processes, teams, and expertise VM Container VMworld 2017 Content: Not for Public Cloud Bare-metal publication
NSX Everywhere: A Platform For All s VM Container VMworld 2017 Public Cloud Bare-metal Content: Not for publication Uniform Networking & Security services across private & public clouds Single pane of glass management Supports any physical network infrastructure PV FW
NSX Architecture Extended to Support All s Central Management to manage networking and security policies Cloud Consumption Management Plane Control Plane Data Plane SW RT FW Virtual Switch Hypervisor SW RT FW Virtual Switch Container on Hypervisor or Bare-metal* SW RT FW Virtual Switch Guest VM in Public Cloud NSX Manager NSX Controller RT FW LB VPN Edge Router NSX Edge Services PV Virtual Switch ESXi OVSDB TOR Bare-metal Server * NSX support for containers on bare-metal is planned for future release 8
NSX Platform Journey vsphere Delivered entire networking and security services in software for vsphere Multi Hypervisor Extended NSX to KVM and Openstack Containers Integrated NSX with PaaS and Container orchestrator for cloud-native apps Public Cloud Extended NSX to native cloud workload and cloud services Baremetal NSX benefits extended to bare-metal
Focus For Rest of The Session vsphere Refer to NET1535BU - NSX Design Reference Design for SDDC with NSX and vsphere Multi Hypervisor Refer to NET1510BU - Introduction to NSX-T Architecture Containers Refer to CNA1091BU - One-Stop Container Networking: Cloud Foundry, Kubernetes, Docker, and More Public Cloud Refer to MMC2046BU - Using VMware NSX for Enhanced Networking and Security for AWS Native s Baremetal This Session
Bare-metal Use Cases
Use Case 1: Integration of non-virtualized s Typically necessary for integrating a non-virtualized appliance L2 as well as L3 A gateway takes care of the on ramp/off ramp s VMworld 2017 Virtual To GATEWAY Overlay-backed s Content: Not for publication VLAN Overlay 12
Use Case 2: Migration Of To Virtual workloads migrated in phases to virtual form factor Temporary, bandwidth not critical 3 s BEFORE DURING MIGRATION AFTER Virtual To GATEWAY 0 Virtual s 2 s Virtual To GATEWAY 1 Virtual s 0 s Virtual To GATEWAY 3 Virtual s VLAN Overlay VLAN Overlay VLAN Overlay
Use Case 3: Migration Of VLAN-Backed Virtual s VLAN-backed Virtual workloads Migrated in phases to Overly-backed Virtual workloads Temporary, bandwidth not critical 3 VLANbacked s BEFORE DURING MIGRATION AFTER Virtual To GATEWAY 0 Overlaybacked s 2 VLANbacked s Virtual To GATEWAY 1 Overlaybacked 0 VLANbacked s Virtual To GATEWAY 3 Overlaybacked VLAN Overlay VLAN Overlay VLAN Overlay
NSX Solutions
Guiding Principle: Routing Vs Bridging Routed Connectivity to workloads Standard Routing protocols (OSPF and BGP) ECMP Scale-out, failure isolation with routing L2 L3 L2 s Bridged Connectivity to workloads Flat Broadcast domain limiting size and scale Single Active bridge for a VXLAN-VLAN pair Virtual To GATEWAY L2 s VLAN-backed Virtual workload Overlay VLAN Overlay VLAN Route when you Can, Bridge when you Must! 16
Practical Example with Exadata Server in separate L3 Subnet Edge Services Gateway External Network VPN Distributed Logical Router L3 Oracle Exadata Server Web and App Tier in Overlay App Tier and Exadata Are in different Subnets Edge Gateway provides routed North- South to physical network Performance & Scale with ECMP Most commonly deployed by Oracle & Enterprise Web1 App1 NET1416BU NSX-T Logical Routing
Practical Example when Exadata and APP Server in same Subnet Edge Services Gateway External Network VPN Distributed Logical Router L3 VMworld 2017 Content: Not for Web and App Tier in NSX Overlay App to Exadata Bridging via the Virtual to Gateway realizable in Two ways 1. NSX Software Bridge Design in a separate VM publication 2. HW Gateway Design by enabling Top-Of-Rack Network switch to provide the function Web1 App1 Virtual To GATEWAY Oracle Exadata Bare-metal Server App tier and Exadata share the same subnet
Summary of Bridging Options For Virtual To Connectivity SW Bridge Pros Independent Of Switch HW or SW Scale-out with little investment High performance VXLAN to VLAN gateway in hypervisor kernel Cons Density of workloads mapping to different VxLAN VLAN pairs HW Gateway Pros Offers Higher Bandwidth and portdensity for workloads Useful in racks where no Hypervisor can be deployed Fast Failover and Redundancy features from HW Vendors Cons Reduces Virtualization benefits by introducing Hardware dependency SW Agent* Pros Common NSX Stack for workload connectivity across Bare-metal servers, Hypervisors, Containers and Public Cloud Paves way for security Of workload at OS layer Cons Legacy OS versions not supported *This is NOT a shipping option today and is in exploration stage 19
Software Bridge - Recorded Demo 192.168.1.10 VLAN 16 VLAN NSX SW Bridge hosted in a Hypervisor Instance Overlay-backed Overlay 192.168.1.20
Software Bridge DEMO With NSX-T
Hardware Bridge Recorded Demo 172.16.10.11 Overlay-backed HV1 10.114.221.196 Database Logical Switch VNI 5000 NSX Controllers 10.114.221.235-237 VTEP Ethernet18 Arista Switch as Hardware Gateway 10.114.211.105 VLAN 160 172.16.10.10 22
Up Configuration Next: Configuration of the Replication of the Arista Cluster Hardware Gateway 23
Up Configuration Next: Registration of the Arista of the Hardware Arista Hardware Gateway Gateway into NSX 24
Up Registration Next: Binding of the a Arista Logical Hardware Switch to Gateway a into Port/VLAN NSX 25
Binding a Logical Switch to a Port/VLAN 26
Customer Case Study Large Electronics Manufacturing Company Deployment Region: Global Deployment Scale : 1 st Phase: 26 Hosts 2 nd Phase: 30 Hosts in 2 nd DC Management: Log Insight NSX version: 6.2.3 3 rd Party Integration: Arista Hardware VTEP NSX Features Used: HW Gateway DFW 27
Network Topology for Case Study Compute Racks VXLAN ID 500X Database Racks VXLAN VLAN Storage Disk VLAN X Arista HW Gateway Key Takeaways Use Case:- Shared storage service with controlled access for compute Rack VMs. Problem: Compute rack VMs need shared storage access from a nonvirtualized disk. Conditions: VMs cannot talk to each other Solution: HW Gateway Solution used to bridge VM traffic to VLAN X on which storage disk attached to get shared service DFW used to prevent VM to VM communication 28
Design Considerations and Best Practices VMworld 2017 Content: Not for publication
Software Bridge vs. Hardware Gateway A single bridging instance per Logical Switch Bandwidth limited by single bridging instance L2 network must be extended to reach all the physical devices VLAN extended between racks VMworld 2017 Several Hardware Gateways can be deployed at several locations simultaneously With Hardware Gateways, VLANs can be kept local to a rack and don t need to be extended Content: Not for publication VXLAN VLAN Non-virtualized devices (part of the same L2 segment) Compute Racks VLAN 10 VLAN 10 Database Racks Compute Racks VLAN 10 VLAN 20 Database Racks 30
Redundancy considerations With Software Bridge VLAN Y VLAN Y Control VM-0 ACTIVE SW Bridge Hypervisor Control VM-0 DOWN SW Bridge Hypervisor Control VM-1 STANDBY SW Bridge Hypervisor Control VM-1 ACTIVE SW Bridge Hypervisor Virtual (Logical Switch X) Virtual (Logical Switch X) SW Bridge functionality for a given VLAN/VXLAN par can only be active on a single Hypervisor Recommendation: Introduce redundancy by selecting Standby Hypervisor that will host SW Bridge and take over upon failure Standby is determined by location of Control VM that the User configures 31
Hardware Based Solution re-introduces Hardware Dependency VXLAN VLAN Compute Racks VLAN 10 VLAN 20 Database Racks Hardware Gateway does not natively support Distributed Routing or Distributed Firewall switch HW and SW versions need to be certified with NSX The Hardware Based model invalidates the benefits of virtualization 32
Redundancy Consideration With Hardware Gateway Hardware Gateway1 workload 1 VLAN 10 LS VNI:5000 Loop Network Switch in backdoor VLAN 10 workload 2 Hardware Gateway2 The OVSDB based mechanism is currently not aware of any form of redundancy Several Hardware Gateways can be active for the same Logical Switch A backdoor connection could result in a loop Recommendation: Only connect hosts to the Hardware Gateway 33
Best Practices For Redundancy With Hardware Gateway Host Based Redundancy Hardware Gateway1 VLAN 10 LS VNI:5000 workload Hardware Gateway2 VLAN 10 Active/Standby uplink No L2 connection must be made between switches 34
Best Practices For Redundancy With Hardware Gateway Port Channel Based Redundancy HV HV Several physical Hardware Gateways presented as a single logical one to NSX Data Plane: View Data Plane: Logical View Most Hardware vendors offer a distributed port channel based solution
Security Considerations For Bare-metal s Virtual To communication STOP STOP Edge Firewall VPN VPN STOP Edge Firewall STOP STOP Security Groups Network Network STOP Distributed or Edge FW can regulate V-P traffic NSX integration with Partner FW manager can regulate V-P traffic closest to the workload NSX Manager Eg.Partner Firewall Mangement Console 36
Security Considerations For Bare-metal s To Flows Analysis & ACL Recommendations* Search, Analytics and Micro-segmentation Modeling Across Virtual, & Cloud VMworld 2017 vrealize Network Insight (vrni) leverageable analyzing flows from virtual, physical (Netflow) and cloud. V-to-V, V-to-P and P-to-P Micro-segmentation models, application tier definition and firewall/acl rules recommendation for physical end points / IPs Content: Not for publication Scale out architecture for large scale flow collection. No agents. NetFlow (from physical) IPFIX (from vsphere) AWS Flow Logs PAR4377BU NSX Advanced Security Virtual Public Cloud *This is NOT a shipping option today and is currently under development 37
Key Takeaways VM Container Route when you can and Bridge only when you must Public Cloud Recommended Order Of Bridging Solutions For Bare-metal workloads SW Bridge Hardware Gateway Bare-metal Secure Bare-metal servers with native NSX solution or with NSX integrated partner solution PV FW 38
Relevant Sessions and References Sessions NET1535BU NET1536BU NET1863BU NET1416BU CNA1091BU MMC2046BU References Reference Design for SDDC with NSX and vsphere: Part 1 & 2 NSX-T Advanced Architecture Concepts NSX-T Logical Routing One-Stop Container Networking: Cloud Foundry, Kubernetes, Docker, and More Using VMware NSX for Enhanced Networking and Security for AWS Native s NSX for vsphere Network Virtualization Design Guide (Ver 3.0) https://communities.vmware.com/docs/doc-27683 39