Architecting a VMware NSX Solution for VMware Cloud Providers

Transcription

1 VMware vclud Architecture Tlkit fr Service Prviders Architecting a VMware NSX Slutin fr VMware Clud Prviders Versin 2.9 January 2018 Michael Haines and Jeffrey Mre

2 2018 VMware, Inc. All rights reserved. This prduct is prtected by U.S. and internatinal cpyright and intellectual prperty laws. This prduct is cvered by ne r mre patents listed at VMware is a registered trademark r trademark f VMware, Inc. in the United States and/r ther jurisdictins. All ther marks and names mentined herein may be trademarks f their respective cmpanies. VMware, Inc Hillview Ave Pal Alt, CA VMware vclud Architecture Tlkit fr Service Prviders

3 3 VMware vclud Architecture Tlkit fr Service Prviders

4 Cntents Intrductin Dcument Purpse... 8 Technlgy Mapping Glssary f Terms NSX fr vsphere Overview Deplyment Mdel Cnsideratins Deplyment Sizing Cnsideratins Clud Service Offerings Design Cnsideratins Architecture Overview Netwrk Requirements and Tplgies vcenter Server Design vsphere Cluster Design NSX Manager NSX Cntrller Cluster VXLAN Design Cnsideratins Transprt Zne Design VMware NSX Distributed Firewall Service Cmpser NSX Edge Services Gateways VMware NSX Distributed Lgical Ruter NSX Lgical Switches Key Use Cases Custmer On-Premises-t-Hsted Clud Cnnectivity Securing Applicatins and Netwrks in the VMware Clud Prvider Prgram Micr-Segmentatin On-Demand Creatin f Lgical Netwrks VMware NSX Dynamic Ruting Scenari (Prvider/Tenant) with MPLS Independent Netwrking and Security Functins NSX Prvider Edge Independent f vclud Directr Availability NSX Manager NSX Cntrller Cluster VMware vclud Architecture Tlkit fr Service Prviders

5 6.3 NSX Edge Services Gateway NSX Distributed Lgical Ruter Cntrl VMs Manageability Clud Cnsumptin Mdels NSX fr vsphere Lgging Cnsideratins Management Interfaces Perfrmance and Scalability Perfrmance f Netwrking and Security in a Virtualized Envirnment Scalability f Virtualized Clud Envirnments Recverability NSX Manager Recverability NSX Cntrller Recverability NSX Edge Services Gateway Recverability Distributed Firewall Recverability VMware vsphere Distributed Switch Recverability VMware vcenter Server Recverability Security NSX fr vsphere Cmpnent Security Integratin with vcenter Single Sign-On Rle-Based Access Cntrl NSX fr vsphere Hardening Operatinal Cnsideratins NSX Manager Operatinal Cnsideratins NSX Cntrller Operatinal Cnsideratins NSX Distributed Firewall Operatinal Cnsideratins Appendix A: NSX fr vsphere Prt and Prtcl Requirements Appendix B: Reference Dcuments VMware vclud Architecture Tlkit fr Service Prviders

6 List f Figures Figure 1. VMware NSX Netwrk and Security Functins Figure 2. Netwrking and Security Virtualizatin Functinal Cmpnents Figure 3. Single vsphere r vcenter Managing All NSX Cmpnents Figure 4. Dedicated vsphere r vcenter Server Managing All NSX Cmpnents Figure 5. Architecture Overview Figure 6. 3-Tier Tplgy Figure 7. Leaf-Spine Fabric Design Figure 8. VMware NSX L2VPN Using a VLAN/VXLAN-Based Slutin Figure 9. VMware NSX L2VPN Using VXLAN-t-VXLAN-Based Slutin Figure 10. Securing Applicatins with NSX Edge Firewall and NAT Figure 11. Securing Applicatins with Micr-Segmentatin Figure 12. VMware NSX Dynamic Ruting Scenari with ebgp (External BGP) and OSPF Figure 13. Edge Gateway Deplyment by vclud Directr fr Service Prviders (VLAN-t-VXLAN) Figure 14. Prvider Edge Cnfiguratin Independent f vclud Directr Platfrm Figure 15. Number f Ndes Required t Maintain HA in a VMware NSX Cluster Figure 16. Stateful Active/Standby HA Deplyment fr NSX Edge Services Gateway Figure 17. Standalne Deplyment fr NSX Edge Services Gateway Figure 18. ECMP High Availability Deplyment fr NSX Edge Services Gateway Figure 19. NSX fr vsphere Lgging Envirnment Figure 20. QS Layer 3 and DSCP Layer Figure 21. Scalability Deplyment Example with NSX Edge Services Gateway Figure 22. VMware NSX Management Plane VMware vclud Architecture Tlkit fr Service Prviders

7 List f Tables Table 1. Glssary f Terms... 9 Table 2. NSX Manager Specificatin Table 3. NSX Cntrller Cluster VM Specificatin Table 4. NSX Edge Services Prperties Limits Based n Deplyment Size Table 5. NSX fr vsphere Rles and Permissins Table 6. NSX fr vsphere Permissins Scpes Table 7. NSX Manager Minimum Requirements Table 8. NSX Manager Required Open Prts Table 9. NSX Cntrller Status Table 10. NSX Cntrller Nde Rle Status Table 11. NSX fr vsphere Prts and Prtcls VMware vclud Architecture Tlkit fr Service Prviders

8 Intrductin As new and existing VMware Clud Prviders cntinue t ffer new cmpute and strage ptins t their tenants, they can als cnsider expanding their prtfli t ffer sftware-defined netwrking, security, and cmpliance functins within the clud infrastructures. Using the VMware NSX platfrm t create a sftware-defined netwrking and security slutin addresses many f the challenges f deplying a virtualized infrastructure in the clud, and can help service prviders realize the visin f the sftware-defined data center (SDDC) architecture frm VMware. Sftware-defined netwrking and security prvides a way fr clud cnsumers t build a myriad f lgical netwrking services, such as firewalls and advanced netwrking, that are independent f the underlying physical netwrk infrastructure. 1.1 Dcument Purpse The purpse f this dcument is t help the VMware Clud Prviders understand the key design cnsideratins when implementing a VMware NSX based sftware-defined netwrking and security slutin. 8 VMware vclud Architecture Tlkit fr Service Prviders

9 Technlgy Mapping 2.1 Glssary f Terms Table 1. Glssary f Terms Term NAT (netwrk address translatin) Distributed virtual switch VMCI (Virtual machine cmmunicatin interface) VLAN (virtual lcal area netwrk) VMkernel vnic VTEP (VXLAN Tunnel Endpint) VXLAN (Virtual Extensible LAN) Security tag Definitin In hsted netwrking, a type f netwrk cnnectin that enables yu t cnnect yur virtual machines t an external netwrk when yu have nly ne IP netwrk address and the hst cmputer uses that address. NAT passes netwrk data between ne r mre virtual machines and the external netwrk. It identifies incming data packets intended fr each virtual machine and sends them t the crrect destinatin An abstract representatin f multiple hsts defining the same virtual switch (same name, same netwrk plicy) and prt grup. An infrastructure that prvides cmmunicatin between a virtual machine and the hst perating system and between tw r mre virtual machines n the same hst. The VMCI Sckets API facilitates develpment f applicatins that use the VMCI infrastructure. A sftware-managed lgical segmentatin f a physical LAN. Netwrk traffic within each segment is islated frm traffic in all ther segments. In VMware ESXi, a high-perfrmance perating system that ccupies the virtualizatin layer and manages mst f the physical resurces n the hardware, including memry, physical prcessrs, strage, and netwrking cntrllers. A virtual netwrk interface card that is cnfigured n tp f a system's physical netwrk adapter. Endpints f the VXLAN cmmunicatin which encapsulate/decapsulate the virtual machine traffic int/frm a VXLAN header. An verlay technlgy which encapsulates/tunnels MAC frames at Layer 2 int a UDP header. Prvides actinable security psture f the VM wrklad. 9 VMware vclud Architecture Tlkit fr Service Prviders

10 2.2 NSX fr vsphere Overview The verall gal f netwrk and security virtualizatin is t at a minimum achieve the same functinality f physical netwrk and security cmpnents and prvide that functinality in a virtualized lgical cmpnent. This ability is enabled by prviding a centralized management platfrm t rganize the multiple virtualized netwrking and security functins within a single interface. The netwrking and security functins supprted by the VMware NSX platfrm are shwn in the fllwing figure. Figure 1. VMware NSX Netwrk and Security Functins VMware NSX supprts the fllwing functins: Switching Extensin f L2 segment IP subnets anywhere in the fabric, regardless f the physical netwrk design. Ruting Layer 3 lgical ruting between IP subnets withut traffic ging ut t the physical ruter. This ruting, perfrmed in the hypervisr kernel with minimal CPU r memry verhead, prvides an ptimal data path fr ruting traffic within the virtual infrastructure (East-West cmmunicatin). Similarly, VMware NSX Edge services gateway prvides an ideal centralized pint fr seamless integratin with the physical netwrk infrastructure, handling cmmunicatin with the external netwrk (Nrth-Suth cmmunicatin). Distributed firewall (DFW) Security enfrcement implemented as a kernel mdule and prviding a virtual NIC level firewall. This enables firewall rule enfrcement in a highly scalable manner, withut creating bttlenecks n physical appliances. The firewall is distributed in the kernel, and therefre, has minimal CPU verhead s it can perfrm at line-rate speed. Lgical lad balancing Supprt fr L4 L7 lad balancing with the ability t prvide SSL terminatin. VPN SSL VPN services t enable L2 and L3 VPN services. IPsec VPN Service prvider cnfigures tw NSX Edge ndes and creates a site-t-site tunnel between the tw edges. The netwrks behind the tw edges are reachable with the site-t-site slutin, prviding the ability t intercnnect tw different netwrks. L2 VPN Service prvider can extend a netwrk acrss bundaries such that the VMs being extended are unaware f r require any change in their ruting r MAC addresses. 10 VMware vclud Architecture Tlkit fr Service Prviders

11 SSL VPN-Plus Service prvider ffers this user-based slutin, where an NSX Edge is prvisined with SSL VPN and the private netwrk behind the NSX Edge is reachable thrugh the end user s machine after cnnected thrugh the SSL VPN client. Cnnectivity t physical netwrks L2 and L3 gateway functins are supprted within VMware NSX fr vsphere t prvide cmmunicatin between wrklads deplyed in lgical and physical spaces. Individual VMware NSX cmpnents prvide the fllwing functinality: NSX Edge services gateway Multi-functinal netwrking and security virtualized cmpnent that prvides supprt f bth cntrl plane and data plane functins, such as netwrk address translatin (NAT), dynamic ruting prtcls (OSPF, ibgp, ebgp), static ruting, firewall, Identity-Based Firewall, lad balancing, DHCP/DNS supprt, and VPN functinality with a primary fcus n Nrth-Suth traffic. Distributed lgical ruter Netwrking virtualized platfrm that prvides supprt f bth cntrl plane and data plane functins f ruting prtcls (OSPF, BGP) with a primary fcus n East-West traffic. Distributed firewall Distributed firewall services integrated with the vsphere kernel fr ptimized perfrmance and functinality. VMware NSX Cntrller cluster Virtual appliance that prvides the cntrl plane functin fr the L3 ruting and L2 switching cmpnents. VMware NSX Manager Virtual appliance that centralizes the prvisining f lgical netwrking cmpnents and manages the cnnectin f virtual machines and strage bjects t the netwrking functins. VMware NSX API Restful API fr interfacing with external prgrams, such as clud management prtals r rchestratin engines. Varius VMware NSX cmpnents, as shwn in the fllwing figure, supprt the netwrking and security virtualizatin functins t prvide an verall end-t-end netwrk and security virtualizatin slutin. Figure 2. Netwrking and Security Virtualizatin Functinal Cmpnents 11 VMware vclud Architecture Tlkit fr Service Prviders

12 Deplyment Mdel Cnsideratins VMware NSX can be deplyed in a number f different cnfiguratins depending n yur requirements fr scalability and manageability. This sectin highlights sme f the mst cmmn deplyment mdels fr VMware NSX. 3.1 Deplyment Sizing Cnsideratins The cnfiguratin f the VMware NSX fr yur clud and hsting ffering can be categrized int tw main scenaris deplyment in small/medium data centers and deplyment in large-scale data centers. The fllwing cnsideratins are valid when deplying VMware NSX in bth scenaris: VMware vcenter Server and NSX Manager 6.1.x have a ne-t-ne mapping relatinship. That is, there is ne NSX Manager (NSX dmain) per vsphere r vcenter Server instance. (This implies the scale limit f vcenter Server gverns the scale f the verall VMware NSX deplyment.) As part f the installatin prcess, NSX Cntrller instances must be deplyed n the same vcenter Server that NSX Manager is cnnected t. The main difference in small/medium data center cmpared with large-scale data center designs is usually the number f vcenter Server instances deplyed and hw they map t NSX Manager. Nte VMware NSX versin 6.2 can supprt multiple vcenter Server instances. This is ut f scpe fr this versin f the dcument Small/Medium Data Center Deplyment When deplying a VMware NSX slutin within yur clud and hsting service ffering, cnsider the fllwing: A single vcenter server is typically deplyed t manage all the VMware NSX cmpnents. VMware recmmends deplying separate edge and management clusters t accmmdate future grwth. The fllwing figure shws a typical deplyment cnfiguratin f the vcenter Server and the VMware NSX cmpnents. 12 VMware vclud Architecture Tlkit fr Service Prviders

13 Figure 3. Single vsphere r vcenter Managing All NSX Cmpnents Large Data Center Deplyment Large-scale clud envirnments primarily use a dedicated vsphere r vcenter Server fr the management cluster. vsphere r vcenter Server is typically already deplyed befre VMware NSX is intrduced in the architecture. When that happens, ne r mre dedicated vcenter Server instances are added t the management cluster t manage the resurces f the VMware NSX dmain (edge and cmpute clusters) as shwn in the fllwing figure. Figure 4. Dedicated vsphere r vcenter Server Managing All NSX Cmpnents This design apprach has the fllwing advantages: Avids circular dependencies, because the management cluster is utside f the dmain it manages Prvides mbility f management cluster fr remte data center peratin Supprts integratin with existing vsphere and VMware vcenter fferings Prvides ability t deply mre than ne VMware NSX dmain 13 VMware vclud Architecture Tlkit fr Service Prviders

14 Upgrade f the main vcenter Server des nt affect the VMware NSX dmains Supprts use f site recvery and ther explicit-state management systems 3.2 Clud Service Offerings This dcument fcuses n three main service mdels t enable VMware Clud Prviders t deliver a unified hybrid clud experience t their custmers: Hsting (Managed r Unmanaged) VMware Clud Prvider Prgram Pwered Hsting Services ffer all the benefits f a dedicated sftwaredefined data center and are engineered n vsphere, s they are 100 percent cmpatible with end custmers n-premises vsphere envirnments. This ffers a unified hybrid clud experience with the same advantages f imprved availability, recverability, perfrmance, and scalability t run yur business-critical applicatins with cnfidence. The hsting slutin can either be managed by the prvider r self-managed VMware vsphere Client Cnsumptin The NSX Manager cmpnent integrates with the VMware vsphere Web Client and prvides a Netwrking and Security plug-in that allws cnsumptin directly frm the NSX Manager fr sufficiently privileged users. This mdel is typically used when the cnsumer f the hsting services has full access t the platfrm and apprpriate knwledge t perate the sftware-defined netwrking slutin effectively Private Clud (Managed r Unmanaged) VMware Clud Prvider Prgram Pwered Private Clud Services are engineered n the VMware vrealize Suite, and is 100 percent cmpatible with end custmers n-premises vsphere envirnments. This prvides a unified hybrid clud experience with dedicated sftware-defined data centers, which can ffer the required self-service cnsumptin, availability, perfrmance, and scalability t run yur businesscritical applicatins in the clud. The private clud slutin can either be managed by the prvider r selfmanaged VMware vrealize Autmatin Cnsumptin Users f VMware vrealize Autmatin can take advantage f the VMWare NSX sftware-defined netwrking and security capabilities by cnfiguring multi-machine blueprints that create netwrks ndemand, r cnsume existing netwrks that are cnnected upstream within the data centers. Users can als islate their deplyments using firewall plicies, lad balancers, and NAT services within their blueprints. Advanced users f vrealize Autmatin can als create additinal value-add services t cnsume advanced features f VMware NSX thrugh the VMware vrealize Orchestratr plug-in r REST API Public Clud VMware Clud Prvider Prgram Public Clud Services are engineered n the VMware vclud Suite with vsphere and vclud Directr at its cre. This unique cmbinatin prvides cmplete multi-level security and a multi-tenant architecture that reduces cmplexity and makes plicy implementatin cnsistent with yur internal data center and the VMware Clud Prvider Prgram, ffering a unified hybrid clud experience t the cnsumers VMware vclud Directr fr Service Prviders Cnsumptin Within VMware vclud Directr fr Service Prviders, the sftware-defined netwrking and security services are presented t the end users thrugh the vclud Directr UI and API. End users have the 14 VMware vclud Architecture Tlkit fr Service Prviders

15 ability t cnfigure edge netwrking services, such as NAT, firewall, DHCP, VPN, and lad balancer services. They als have the ability t create ruted r islated netwrks directly thrugh the UI. This is all cntained within the tenancy bundaries f each vclud Directr Organizatin. Nte Sftware-defined netwrking and security capabilities can be added t all three f the clud service mdels t enhance the clud service and functinality VMware vclud Directr and VMware NSX VMware NSX is a direct replacement fr the VMware vclud Netwrking and Security prduct, and prvides an in-place upgrade path frm vclud Netwrking and Security which retains all existing cnfiguratins and prvides backwards cmpatibility with vclud Netwrking and Security APIs, and where the NSX Cntrller based VXLAN can be cnsumed immediately. NSX 6.1.x is fully supprted with vclud Directr. 15 VMware vclud Architecture Tlkit fr Service Prviders

16 Design Cnsideratins 4.1 Architecture Overview The fllwing figure highlights an example architecture verview where a custmer has deplyed a VMware NSX based slutin acrss their data center. The example uses a leaf-and-spine netwrk tplgy and dedicated edge and management racks. Figure 5. Architecture Overview 16 VMware vclud Architecture Tlkit fr Service Prviders

17 4.2 Netwrk Requirements and Tplgies VMware NSX can be implemented n tp f any existing r new netwrk tplgy. VMware Clud Prviders typically have a number f different netwrk tplgies, depending n what services they typically ffer t their custmers. This sectin highlights sme f the cmmn netwrk tplgies leveraged when deplying VMware NSX and is based n the VMware NSX (NSX-V) fr vsphere Netwrk Virtualizatin Design Guide Classic Cre/Aggregatin/Access Layer Tplgies The classic cre, aggregatin, and access (3-tier) tplgy has been cmmnplace in mst enterprises and service prviders fr many years and prvides a scalable mdular architecture fr netwrking services. Figure 6. 3-Tier Tplgy Applicatins which required a Layer 2 adjacency had t be cnnected within the same Pd, because each Pd was separated by a Layer 3 wide-area netwrk (WAN) cnnectin. This was ne f the primary reasns fr the intrductin fr a leaf-spine fabric design. 17 VMware vclud Architecture Tlkit fr Service Prviders

18 4.2.2 Leaf-and-Spine Fabric Design The evlutin f the leaf-and-spine design evlved based n the fllwing requirements: Addressing the increasing demand f traffic fr East-West cmmunicatin Ability t deply applicatins independent f the Layer 2 fabric within each Pd Figure 7. Leaf-Spine Fabric Design The evlved leaf-and-spine design als cllapsed the number f netwrking layers frm three lgical layers (cre/aggregatin/access) t tw lgical layers (leaf-spine). Nte A brder leaf is a special leaf nde that supprts the external cnnectin t the WAN r Internet. As mentined in the VMware NSX fr vsphere (NSX-V) Netwrk Virtualizatin Design Guide., the fllwing tpics must be cnsidered when defining the end-t-end netwrk requirements: Simplicity Scalability High-bandwidth Fault-tlerance Quality f Service (QS) This dcument prvides further detail t these areas f fcus. 18 VMware vclud Architecture Tlkit fr Service Prviders

19 4.3 vcenter Server Design vcenter Server is the management and cntrl cmpnent fr the vsphere platfrm. Service prviders can deply any number f vcenter Server ndes t supprt the required scale and management. Typically, there are tw types f vcenter Server instances that VMware Clud Prviders deply: Management vcenter Server is leveraged t hst all management clusters and is nt under the cntrl f VMware NSX. Resurce/paylad vcenter Server is used t hst all the resurce/paylad clusters fr the end user wrklads and is under the cntrl the VMware NSX Design Cnsideratins NSX Manager instances pair with the vcenter Server instances n a 1:1 basis, s the VMware NSX slutin scales in a mdular fashin with the vcenter Server and can be implemented n a Pd-based apprach t scalability that includes netwrk and security services. Avid pairing the management vcenter Server with an NSX Manager because this culd lead t a circular dependency impacting the management cmpnents with the distributed firewall. 4.4 vsphere Cluster Design vsphere clusters are physical grups f ESXi hsts that are gruped tgether t create a pl f resurces. Service prviders can design their cluster tplgy t meet their needs based n factrs such as csts, security, and manageability. Typically, there are three types f clusters that service prviders deply: Management Leveraged t hst all management cmpnents. Edge services Used t hst all netwrking services appliances. Resurce/paylad Where the end custmer virtual wrklads are lcated Design Cnsideratins Fr increased availability, distribute cluster hsts acrss racks within the data center s that a rack failure has limited impact n yur cluster peratin. Fr large deplyments, cnsider deplying an NSX Edge cluster fr all netwrking services appliances fr Nrth/Suth traffic. This enables the prvider t be deterministic as t where netwrk traffic exits the data center and limits the need t present service VLANs t all paylad hsts. If leaf-and-spine netwrk tplgy is leveraged and management hsts are distributed acrss racks, verify that the management L2 netwrking is extended acrss the racks s that the management netwrk is available in bth racks. 19 VMware vclud Architecture Tlkit fr Service Prviders

20 4.5 NSX Manager NSX Manager is the management cmpnent fr NSX fr vsphere and is typically lcated in the management cluster. The key functins f the NSX Manager are: Deplyment and management f the cntrller cluster Preparatin f the vsphere ESXi hsts (installatin f the VIBs) Deplyment f the edge services gateways and assciated services (firewall, NAT, ruting, and s n) Acts as the target fr NSX REST API calls Design Cnsideratins Deply an NSX Manager per vcenter server as f versin 6.1.x. NSX Manager has a direct 1:1 mapping with the vcenter Server. Deply the NSX Manager appliance t the management cluster and prtect with VMware vsphere High Availability t imprve availability. The NSX Manager appliance must be cnfigured fr apprpriate cnfiguratin backup thrugh the NSX Manager user interface ptins. The cnfiguratin can be backed up t a remte lcatin ndemand r scheduled in line with existing business RPOs. NTP and ther supprting infrastructure services, such as DNS, must be cnfigured accrding t the best practices highlighted in the design guides. The NSX Manager is a virtual appliance, which is deplyed with the fllwing specificatins. Table 2. NSX Manager Specificatin Attribute Memry Specificatin 12 GB vcpu 4 Strage 60 GB 20 VMware vclud Architecture Tlkit fr Service Prviders

21 4.6 NSX Cntrller Cluster VMware NSX Cntrller instances are typically deplyed t the NSX Edge cluster and are respnsible fr the fllwing functins: Respnsible fr the switching and ruting mdules in the hypervisrs Remve the VXLAN dependency n multicast ruting/pim in the physical netwrk Prvide suppressin f ARP bradcast traffic in VXLAN netwrks Prvide the cntrl plane t distribute netwrk infrmatin t ESXi hsts Are clustered fr scale-ut and high availability Design Cnsideratins The NSX Cntrller cluster must be deplyed in an dd number f ndes. The current limitatin with versin 6.1.x is three ndes. This is t maintain majrity in the event f a nde failure. Distribute the NSX Cntrller VMs acrss ESXi hsts within the NSX Edge cluster. Leverage VM:VM antiaffinity rules t achieve this. The NSX Cntrller VMs are deplyed as virtual appliances with the fllwing resurce specificatins. Table 3. NSX Cntrller Cluster VM Specificatin Attribute Memry CPU Reservatin Specificatin 4 GB 2048 Mhz vcpu 4 Strage 20 GB Nte Mdifying settings is unsupprted and memry reservatin is nt required. 21 VMware vclud Architecture Tlkit fr Service Prviders

22 4.7 VXLAN Design Cnsideratins VXLAN (Virtual Extensible LAN) is an verlay technlgy that is used by VMware NSX t decuple the netwrking services frm the physical netwrk. The physical netwrk then becmes a backbne netwrk used t transprt verlay traffic as quickly as pssible. The main functins that VXLAN enables are: Rapid deplyment f virtual netwrks int the data center Mbility f wrklads acrss Layer 3 bundaries Large-scale multi-tenancy, allwing a service prvider t extend their netwrk beynd the VLAN limit f 4,096 netwrks within their data center Design Cnsideratins The ESXi hsts VTEP netwrk interfaces must be cnfigured fr at least 1,600 MTU thrugh the netwrk switches. This is t allw fr the extra header bits that are applied t the packet size by VXLAN. The cncept f VXLAN replicatin t address scalability using Unicast Tunnel EndPints (UTEP) and/r Multicast Tunnel EndPints (MTEP) is dcumented in the design guide. Hwever, cnsider the fllwing aspects fr a design implementatin: Layer 2 deplyments (L2 tplgy has n bundary fr selecting UTEP and MTEP, because all VTEPs are n the same subnet) Small deplyment Unicast mde is a recmmended cnfiguratin. Large deplyment Hybrid mde is mre suited because the UTEP functin cannt identify a VTEP bundary (VTEP are n same subnet) t prvide efficient BUM replicatin per lgical switch and thus scales very well. Layer 3 deplyments In mst cases, unicast deplyment wrks because Layer 3 tplgy prvides a VTEP IP addressing bundary, and therefre, UTEP efficiently replicates frame pre LS. N need fr PIM because traffic is all unicast. Fr very large deplyments, hybrid mde is recmmended prviding MTEP-based BUM replicatin as well eliminating cnfiguratin f L3 multicast (PIM). Hybrid mde deplyments: VMware recmmends cnfiguring the external physical switch with IGMP querier alng with IGMP snping (this is an industry standard best practice fr mst switches, including Cisc, Arista, Dell, and Brcade). If yu accidentally frget t cnfigure IGMP querier in the physical switch, as lng as IGMP snping is defined, the hypervisr will send a jin t the cnfigured multicast address. Hybrid mde is preferred when large L2 multicast traffic frm VMs requires replicatin. Be aware f multicast reserved address space and avid using multicast addresses that will result in bradcast: 22 VMware vclud Architecture Tlkit fr Service Prviders

23 4.8 Transprt Zne Design The VMware NSX transprt zne defines the bundaries f which ESXi hsts the VMware NSX lgical switches (VXLANs) can be extended acrss. The ESXi hsts participating within a transprt zne need t cmmunicate with each ther ver a VXLAN Tunnel Endpint (VTEP) cnnectin. VMware Clud Prviders can cnfigure transprt znes in different ways depending n their clud service mdel. Fr example, a public clud service might leverage a single transprt zne acrss their data center fr simplicity, whereas a hsting r private clud mdel might leverage a transprt zne per tenant t avid replicating all netwrks acrss all hsts within the data center Design Cnsideratins Span the transprt zne acrss all the ESXi hsts r clusters that the end custmers VMs must reside n s that all specified hsts can service the required netwrk traffic. Verify that the transprt zne is extended t the NSX Edge clusters s that East/West traffic can get t the NSX Edge cluster befre traversing the Nrth/Suth edge service gateways. Islated transprt znes can be used t imprve security where required. These transprt znes are nly applied t the required clusters. VMware NSX prvides flexibility fr the VXLAN transprt which des nt require cmplex multicast cnfiguratins n the physical netwrk t be in place. This flexibility is prvided thrugh different VXLAN replicatin mdes yu can chse depending n yur netwrk fabric. They are: Unicast All replicatin ccurs using unicast. This is applicable t small deplyments. Multicast The entire replicatin is ffladed t the physical netwrk and requires IGMP querier as well as multicast ruting fr L3 (PIM). It is the hst that prvides the necessary querier functin. Hwever, an external querier is recmmended fr manageability. Hybrid Lcal replicatin is ffladed t the physical netwrk, while remte replicatin ccurs thrugh unicast. This is the mst practical replicatin mde withut the cmplexity f multicast mde and nly requires IGMP snping/querier and des nt require L3 PIM. All VXLAN replicatin mdes require an MTU f 1,600 bytes. 23 VMware vclud Architecture Tlkit fr Service Prviders

24 4.9 VMware NSX Distributed Firewall The VMware NSX distributed firewall prvides L2-L4 stateful firewall services t any wrklad in the VMware NSX envirnment. The distributed firewall is embedded in the ESXi kernel, scales hrizntally with the ESXi hsts, and perfrms at line rate. It is designed t prvide prtectin, islatin, and segmentatin f East/West traffic within the data center envirnment. VMware Clud Prviders can leverage this functinality t ffer zer-trust micr-segmentatin t their custmers hsted wrklads, and cntrlled islated access t shared management resurces where required Design Cnsideratins Distributed firewall enfrcement is applied at the vnic level f the VMs. If the management cmpnents are under cntrl f VMware NSX, the cmpnents must be excluded frm participatin within the distributed firewall t avid circular dependencies. Fr example, yu culd edit a rule that blcks access t the vcenter Server. Cllapsing applicatin tiers t cmmn services with each applicatin tier having its wn lgical switch: Better fr managing dmain (web and database) specific security requirements. Easier t develp segmented islatin between applicatin tiers (web-t-database cmpared with web-t-applicatin granularity). Requires explicit security between applicatin tiers. Cllapsing all applicatin tiers int single lgical switch: DMZ mdel Better fr managing grup/applicatin-wner specific expertise. Applicatins cntainer mdel. Suits the applicatin as tenant mdel. Simpler security grup cnstruct per applicatin tier. Security plicy between different applicatins cntainer is required. Zer-trust security. Multiple DMZ lgical netwrks. Default deny_all within DMZ segments. External t internal prtectin by multiple grups. A DFW plicy can be applied t varius bjects in the Virtual Inventry such as: Security Tags, IP Sets, MAC Sets, VMs, Prt Grups and Lgical Switches, Flders, Clusters, as well as user grup identity infrmatin frm Active Directry. 24 VMware vclud Architecture Tlkit fr Service Prviders

25 4.10 Service Cmpser Service Cmpser prvides plicy bject and plicy enfrcement pints (PEPs) t help yu prvisin and assign netwrk and security services t applicatins in a virtual infrastructure. Yu map these services t a security grup, and the services are applied t the virtual machines in the security grup Security Grups Security grups are lgical grupings created t define what needs t be prtected by the VMware NSX distributed firewall r similar devices. A typical strategy is t add vcenter Server inventry bjects as security grup members. The underlying firewall rules cnfigured within the kernel are IP-based, despite being abstracted as bjects at the cnfiguratin layer. This requires VMware Tls t be run in all virtual machines s that their addresses are reprted in the vcenter Server. Membership f a security grup can be achieved in a number f ways ranging frm vcenter Server bjects, security tags, IPsets, MACsets r ther security grups, directry grups, r regular expressins Security Plicy A VMware NSX security plicy is a cllectin f netwrking and security services. The services that can be added include: Endpint services Data security, anti-virus, vulnerability management Distributed firewall rules Netwrk intrspectin services Design Cnsideratins Where security grups are leveraged, VMware Tls must be installed n the virtual machines t btain full management functinality. VMware NSX distributed firewall requires up-t-date IP infrmatin frm the VM t be reprted in vcenter. When yu have many security grups t which yu need t attach the same security plicy, create an umbrella security grup that includes all these child security grups, and apply the cmmn security plicy t the umbrella security grup s that the VMware NSX distributed firewall uses ESXi hst memry efficiently. 25 VMware vclud Architecture Tlkit fr Service Prviders

26 4.11 NSX Edge Services Gateways NSX Edge services gateway is a multi-functinal virtualized netwrking and security cmpnent that prvides supprt f bth cntrl plane and data plane functins, such as netwrk address translatin (NAT), ruting prtcls (OSPF, ibgp, ebgp), firewall, lad balancing, DHCP/DNS supprt, and VPN functinality with a primary fcus n the Nrth-Suth traffic. The NSX Edge services gateway must be deplyed as an HA pair t address high availability requirements. This creates a VM:VM anti-affinity rule t supprt the HA functin. Fr imprved thrughput fr the ruting capabilities, the prvider can implement equal-cst multi-path (ECMP) high-availability. With this mdel we can deply up t eight ECMP edge devices t imprve thrughput and availability. The NSX Edge services gateway must be deplyed in the crrect size prfile as driven by netwrk functinal and perfrmance requirements. NSX Edge services gateway appliance deplyments are typically cnfigured with the fllwing resurces: X-Large = 6 x vcpu, 8,192 MB vram (high-perfrmance firewall + lad balancer + ruting) Quad-Large = 4 x vcpu, 1,024 MB vram (high-perfrmance firewall) Large = 2 x vcpu, MB vram Cmpact = 1 x vcpu, 512 MB The fllwing table lists ther cnfiguratin prperty limits fr different size deplyments. Table 4. NSX Edge Services Prperties Limits Based n Deplyment Size Netwrk Functin Value (Cmpact / Large / X-Large / Quad-Large) NSX Edge services gateways 2,000 Nte HA des nt change the scaling requirements fr NSX Edge Interfaces 10 (internal, uplink, r trunk) Nte With trunk, 200 sub-interfaces per NSX Edge Ruter NAT rules per NSX Edge services gateway Static rutes per NSX Edge services gateway BGP rutes per NSX Edge services gateway BGP neighburs per NSX Edge services gateway BGP rutes redistributed 2,000 (all sizes) 2,048 (all sizes) 20K / 50K / 250K / 250K 10 / 20 / 50 / 50 N limit 26 VMware vclud Architecture Tlkit fr Service Prviders

27 Netwrk Functin OSPF rutes per NSX Edge services gateway OSPF adjacencies per NSX Edge services gateway OSPF rutes redistributed Ttal number f rutes Value (Cmpact / Large / X-Large / Quad-Large) 20K / 50K / 100K / 100K 10 / 20 / 40 / 40 2K / 5K / 20K / 20K 20K / 50K / 250K / 250K Firewall Firewall rules per NSX Edge services gateway Cncurrent cnnectins per hst (cmpact/all ther) 2, K / 1 M Lad balancing Lad balancer VIPs per ESXi 64 Lad balancer pls per ESXi 64 Lad balancer servers per pl 32 DHCP DHCP pls per NSX Edge services gateway 20K IPsec / VPN IPsec sites per NSX Edge services gateway (nly fr pre-6.1, n limit fr 6.1 r later) IPsec tunnels per NSX Edge services gateway / 1,600 / 4,096 / 6, VMware vclud Architecture Tlkit fr Service Prviders

28 4.12 VMware NSX Distributed Lgical Ruter The VMware NSX distributed lgical ruter prvides supprt f bth cntrl plane and data plane functins f ruting prtcls (OSPF, BGP) with a primary fcus n the East-West traffic. VMware Clud Prviders can leverage distributed lgical ruting functinality t address the scale requirements fr ruted interfaces and ptimize the East/West netwrking traffic within the data center. The prvider can als run dynamic ruting prtcls between the distributed lgical ruter and the NSX Edge ruter f external physical ruting devices Design Cnsideratins The VMware NSX distributed lgical ruter can scale up t 1,000 lgical interfaces, which gives the prvider the ability t allw the end users within the clud envirnment the ability t deply up t 1,000 netwrks within this distributed lgical ruter. Fr increased availability the distributed lgical ruter cntrl VM must be deplyed in an HA pair. HA is prvided in an active/standby cnfiguratin. The distributed lgical ruter is heavily dependent n the NSX Cntrller cluster. Verify that the cntrller cluster is up and running befre making any changes t the distributed lgical ruter NSX Lgical Switches A lgical switch is mapped t a unique VXLAN, which encapsulates the virtual machine traffic and carries it ver the physical IP netwrk. Lgical switches are islated by nature within the clud platfrm. Each lgical switch is its wn L2 bradcast dmain. A clud cnsumer r prvider can create lgical switches that span their area f the infrastructure within the transprt zne. As the lgical switch is expanded acrss the transprt zne, and inherently, the distributed virtual switch and clusters, this enables the virtual machine t be mved acrss the data center with VMware vmtin. The NSX Cntrller cluster cntrls lgical switches and maintains infrmatin abut virtual machines, ESXi hsts, lgical switches, and VXLANs. The cntrl plane mde decuples NSX fr vsphere frm the physical netwrk and handles the bradcast, unknwn unicast, and multicast (BUM) traffic within the lgical switches. All lgical switches created within the transprt zne inherit VMware NSX transprt zne settings. (This behavir can be verwritten by the custmer.) Other ptins t cnsider when designing the lgical switch cntrl plane are described in fllwing sectins Multicast Mde In multicast mde, the cntrl plane uses multicast IP addresses n the physical netwrk. VMware recmmends this cnfiguratin when upgrading frm existing VXLAN deplyments. This design requires the cnfiguratin f PIM/IGMP n the physical netwrk Design Cnsideratins Requires IGMP and IGMP snping cnfiguratins thrughut the physical netwrk which adds cmplexity t the cnfiguratin and is nt always available n the netwrk. Multicast IP addresses must be reserved n the physical netwrk. The use f multicast mde reduces the verhead incurred n the surce hst VTEPs. 28 VMware vclud Architecture Tlkit fr Service Prviders

29 Unicast Mde In unicast mde, the NSX Cntrller ndes handle the cntrl plane. All replicatin is cnfigured lcally n the hst. N multicast IP addresses r physical netwrk cnfiguratins are required fr this mde t perate Design Cnsideratins N requirement fr multicast cnfiguratins n the physical netwrks. The use f unicast mde increases the verhead n the surce hsts VTEPs Hybrid Mde Hybrid mde is an ptimized versin f unicast mde where lcal traffic replicatin fr the subnet is ffladed t the physical netwrk. Operatin in this mde requires IGMP snping n the first-hp switch and IGMP querier must be available, but the requirement fr PIM is remved Design Cnsideratins IGMP snping cnfiguratin is required n the physical netwrk. Multicast IP addresses must be reserved n the physical netwrk. 29 VMware vclud Architecture Tlkit fr Service Prviders

30 Key Use Cases This sectin highlights sme f the key VMware Clud Prvider use cases fr VMware NSX. Emplyment f the use cases might vary depending n the cnsumptin mdel and service mdel that the prvider ffers. The prvider might als chse t ffer sme f the use cases as managed services that their peratins teams can execute n behalf f the end custmers. 5.1 Custmer On-Premises-t-Hsted Clud Cnnectivity One key VMware Clud Prvider use case is t prvide services that enable the end custmers t cnnect their n-premises vsphere implementatins t the hsted clud service. With VMware NSX, there are several ptins available t create a cmmn netwrk between the custmer and prvider. IPsec VPN The cnsumer can cnfigure an IPsec VPN service frm their hsted clud NSX Edge gateway device that is cnfigured t pair with a third-party VPN endpint r standalne NSX Edge in the custmer s data center. The VPN cnnectivity is achieved ver L3 cnnectivity. L2VPN The cnsumer can create a L2 VPN service frm their hsted clud NSX Edge services gateway device that is cnfigured t pair with a standalne NSX Edge device in the custmer s data center. The L2 VPN stretches the same Layer 2 netwrk between sites. VMware NSX supprts L2VPN cnnectivity fr bth VLAN-backed and VXLAN-backed netwrks as described in the VMware NSX fr vsphere Administratin Guide NSX 6.1 fr vsphere, and this capability can be leveraged between private and public clud envirnments (NSX versin 6.1) as shwn in the fllwing figures. 30 VMware vclud Architecture Tlkit fr Service Prviders

31 Figure 8. VMware NSX L2VPN Using a VLAN/VXLAN-Based Slutin As depicted in the figure, a VMware NSX Edge services gateway must be deplyed in the private and public clud envirnments. In the case f a VLAN-backed netwrk in the private clud, a standalne NSX Edge gateway must be used fr the end-t-end deplyment (edge services gateway is deplyed withut the entire site being VMware NSX enabled). This allws fr the seamless migratin f VLAN-based r VXLAN-based wrklads between lcatins. 31 VMware vclud Architecture Tlkit fr Service Prviders

32 Figure 9. VMware NSX L2VPN Using VXLAN-t-VXLAN-Based Slutin Nte The data center cnnectivity ptins can either be self-serviced by the end users r prvider managed, depending n the service mdel ffered. 32 VMware vclud Architecture Tlkit fr Service Prviders

33 5.2 Securing Applicatins and Netwrks in the VMware Clud Prvider Prgram VMware Clud Prviders can enable their custmers t cnfigure security rules n the NSX Edge service gateways. This allws the end user t create assciated applicatin firewall rules and NAT rules s that their applicatins are apprpriately secured within the clud envirnment. This use case is particularly useful fr service prviders wh ffer direct Internet cnnectivity and public IP addressing frm within the custmer s NSX Edge services gateway. Figure 10. Securing Applicatins with NSX Edge Firewall and NAT 33 VMware vclud Architecture Tlkit fr Service Prviders

34 5.2.1 Cnsumptin Mdels vclud Directr fr Service Prviders Public clud services built with vclud Directr fr Service Prviders can ffer self-service cnsumptin f NSX Edge services, which include NSX Edge firewall and NAT. The prvider can als ffer this as a managed service. vrealize Autmatin Private clud services built with vrealize Autmatin can cnsume NSX Edge services thrugh definitin f apprpriate service blueprints. The service prvider can als create fferings fr API-driven cnfiguratin f the NSX Edge gateway services if required. vsphere Web Client Hsting services prviders can give their end custmers full access t VMware NSX functinality, which includes the cnfiguratin f the edge gateway, firewall, and NAT services. The prvider culd als ffer this as a managed service. 5.3 Micr-Segmentatin Micr-segmentatin with VMware NSX can enable VMware Clud Prviders t implement zer-trust security and prtectin f sensitive virtual machine wrklads in the clud envirnment. By using VMware NSX distributed firewalls, VMware NSX micr-segmentatin can prvide clud wrklads that reside n the same Layer 2 segment a similar level f islatin and segmentatin t wrklads n separate Layer 2 segments. This allws fr mre granular and efficient security fr clud wrklads. VMware Clud Prviders can prvide micr-segmentatin in the vsphere Web Client fr the Hsted Clud Service mdel r thrugh the cnsumptin f multi-machine blueprints fr the Private Hsted Clud Service mdel. An example f using micr-segmentatin with the distributed firewall platfrm might be in the case where the service prvider wants t prtect the back end infrastructure, which ffers billing, patch, and mnitring services. This wuld allw fr the prtectin f East/West traffic while the edge services gateway firewall prvides the Nrth/Suth prtectin. 34 VMware vclud Architecture Tlkit fr Service Prviders

35 Figure 11. Securing Applicatins with Micr-Segmentatin 5.4 On-Demand Creatin f Lgical Netwrks VMware Clud Prviders can enable end users the ability t create lgical netwrks n-demand. The lgical switches are islated by default, but can be cnfigured t rute t upstream VMware NSX distributed lgical ruters r edge services gateways fr cnnectivity t ther areas f the data center r egress pints Cnsumptin Mdels vclud Directr fr Service Prviders Public clud services built with vclud Directr fr Service Prviders can ffer self-service cnsumptin f lgical switches thrugh the vclud Directr user interface r API. They can either be islated r ruted netwrks. vrealize Autmatin Private clud services built with vrealize Autmatin can ffer autmated creatin f new lgical switches within a multi-machine blueprint. The netwrks can be islated r cnnected t an upstream distributed lgical ruter. vsphere Web Client Hsting services prviders can give their end custmers full access t VMware NSX functinality, enabling them t create islated r ruted lgical switches. 35 VMware vclud Architecture Tlkit fr Service Prviders

36 5.5 VMware NSX Dynamic Ruting Scenari (Prvider/Tenant) with MPLS As prvider and tenant functinal requirements begin t expand in the public clud, there might be a need t enable VMware NSX dynamic ruting prtcls, such as OSPF, t multiple netwrk and security elements fr bth prvider and tenant envirnments. When cnnecting t a third-party MPLS backbne, yu can use BGP (external BGP/eBGP) as a dynamic ruting prtcl t exchange netwrk infrmatin with the lcal backbne prvider. The fllwing figure prvides an example f a Dedicated Private Clud scenari where the service prvider ffers an envirnment fr a single tenant. Figure 12. VMware NSX Dynamic Ruting Scenari with ebgp (External BGP) and OSPF The fllwing are the design cnsideratins f this use case: OSPF can be cnfigured between the tenant distributed lgical ruter (t redistribute cnnected rutes) and the management NSX Edge services gateway (t redistribute cnnected and static rutes). Prvider Edge NSX Edge services gateway A defines a shared OSPF Area 0. This supprts end-t-end cnnectivity frm the tenant lgical netwrks t the prvider management netwrks. 36 VMware vclud Architecture Tlkit fr Service Prviders

37 Dynamic ruting is disabled between the prvider management NSX Edge services gateway and the physical management ruter. Static rutes fr the management netwrks are created n the prvider management NSX Edge services gateway. BGP filters are created n the tenant prvider NSX Edge services gateway t deny cllectin f rutes frm the WAN edge ruter. There is a large amunt f netwrk infrmatin frm the WAN backbne prvider that des nt need t be cllected in the lcal prvider envirnment. Overlapping IP addresses are unsupprted fr the Internal Tenant Netwrks (lgical switches). 5.6 Independent Netwrking and Security Functins Certain netwrking and security virtualizatin features can currently be deplyed independent f the VMware NSX functinality by using the Edge Gateway cmpnent, which is cntrlled and deplyed by vclud Directr in legacy cmpatibility mde. As shwn in the fllwing figure, the Edge Gateway can supprt the intercnnectivity f the virtual machines that are cnnected thrugh the VXLAN-backed infrastructure. This infrastructure is the Org VDC Netwrk VXLAN500x cnnected t the physical nrthbund L3/L2 netwrk thrugh a VLAN-backed infrastructure (vclud Directr External Netwrk VLAN 101). Figure 13. Edge Gateway Deplyment by vclud Directr fr Service Prviders (VLAN-t-VXLAN) 37 VMware vclud Architecture Tlkit fr Service Prviders

38 5.7 NSX Prvider Edge Independent f vclud Directr VMware NSX cmpnents can be intrduced int service prvider clud fferings independent f the clud management platfrm t include prvider-facing functinality as highlighted in the fllwing figure. Figure 14. Prvider Edge Cnfiguratin Independent f vclud Directr Platfrm An NSX Edge services gateway called the Prvider Edge is intrduced t leverage all f the VMware NSX functinality, such as L2VPN and L2 bridging. The Prvider Edge prvides the VLAN-t-VXLAN ruting functin (VLAN101 t vclud Directr external netwrk VXLAN5001), as the Edge Gateway did in the previus use case. This allws fr an additinal level f separatin f the netwrking and security functins between the prvider (transit transprt zne) and the tenant (prvider VDC transprt zne) envirnments. 38 VMware vclud Architecture Tlkit fr Service Prviders

39 Availability 6.1 NSX Manager Because the NSX Manager is a virtual machine, the recmmendatin is t apprach the tpic f resiliency and verall high availability in the same way as ther vsphere cmpnents, by utilizing the vsphere HA functinality. That way, NSX Manager can be mved dynamically t ther parts f the infrastructure in case f a failure. In such a situatin, the NSX management plane is impacted, while the already deplyed lgical netwrks (data plane) cntinue t perate. 6.2 NSX Cntrller Cluster When NSX Cntrller clusters are deplyed, a master cntrller nde is chsen thrugh an electin prcess where its rle is t allcate resurces t individual cntrller ndes and determine when a nde has failed. The electin prcess f a master requires a majrity vte f all active and inactive cntrller ndes and is the primary reasn fr the dd number f ndes within a deplyed cntrller cluster as described in the NSX fr vsphere design guide. Figure 15. Number f Ndes Required t Maintain HA in a VMware NSX Cluster Nte The VMware NSX 6.1.x slutin supprts three-nde clusters. The additinal ptins are prvided t illustrate the majrity vte mechanism prcess. 39 VMware vclud Architecture Tlkit fr Service Prviders

40 6.3 NSX Edge Services Gateway The NSX Edge services gateway high availability (HA) feature prvides that an NSX Edge services gateway appliance is always available by installing an active pair f NSX Edge services gateways in the virtualized infrastructure. There is the ptin t enable high availability either when installing the NSX Edge services gateway device r n an already deplyed NSX Edge services gateway instance. The primary NSX Edge services gateway appliance is in the active state and the secndary appliance is in the standby state. The NSX Edge services gateway replicates the cnfiguratin f the primary appliance fr the standby appliance. VMware recmmends that the primary and secndary appliance be created n separate resurce pls and datastres. If yu create the primary and secndary appliances n the same datastre, the datastre must be shared acrss all hsts in the cluster. In this way, the high availability appliance pair can be deplyed n different ESXi hsts. If the datastre is a lcal strage, bth virtual machines are deplyed n the same hst. In the peratin f the primary appliance, it maintains a heartbeat with the standby appliance and sends service updates thrugh an internal interface. If a heartbeat is nt received frm the primary appliance in the specified time, which is cnfigurable, the primary appliance is declared dead. The standby appliance mves t the active state and takes ver the interface cnfiguratin f the primary appliance. The standby appliance als starts the NSX Edge gateway services that were running n the primary appliance Stateful Active/Standby HA Deplyment fr NSX Edge Services Gateway This design fllws the redundancy mdel where a pair f NSX Edge services gateways is deplyed fr each tenant. One NSX Edge gateway functins in active mde (that is, actively frwards traffic and prvides the ther lgical netwrk services), whereas the ther unit is in standby state, waiting t take ver shuld the active NSX Edge gateway fail. 40 VMware vclud Architecture Tlkit fr Service Prviders

41 Figure 16. Stateful Active/Standby HA Deplyment fr NSX Edge Services Gateway 41 VMware vclud Architecture Tlkit fr Service Prviders

42 6.3.2 Standalne Deplyment fr NSX Edge Services Gateway A standalne HA mdel fr NSX Edge services gateway HA inserts tw independent NSX Edge appliances between the distributed lgical ruter and the physical netwrk as shwn in the fllwing figure. This cnfiguratin is supprted when running NSX 6.x. Nte Starting with NSX 6.1, yu can als chse t implement the ECMP mdel fr high availability as described in the next sectin. Figure 17. Standalne Deplyment fr NSX Edge Services Gateway 42 VMware vclud Architecture Tlkit fr Service Prviders

43 6.3.3 Equal-Cst Multi-Path High Availability fr NSX Edge Services Gateway In the ECMP mdel, distributed lgical ruting and NSX Edge capabilities have been imprved t supprt up t eight equal-cst paths in their frwarding table. This means that up t eight active NSX Edge instances can be deplyed at the same time and all the available cntrl and data planes are fully utilized, as shwn in the fllwing figure. This HA mdel prvides tw main advantages: An increase in available bandwidth fr Nrth/Suth cmmunicatin (up t 80 Gbps per tenant). Reduced traffic utages (in terms f percentage f affected flws) fr NSX Edge failure scenaris. Nte As f the NSX fr vsphere release, there is an ptin t disable the edge services gateway firewall functin when enabling the ECMP feature if required. Figure 18. ECMP High Availability Deplyment fr NSX Edge Services Gateway 43 VMware vclud Architecture Tlkit fr Service Prviders

44 6.4 NSX Distributed Lgical Ruter Cntrl VMs Deplying a distributed lgical ruter als deplys a cntrller VM that lives within the edge cluster. Yu can specify that the cntrl VMs use high availability, which deplys an active/standby specificatin t imprve availability. The primary functin f the distributed lgical ruting feature in the VMware NSX platfrm is t prvide an ptimized and scalable way f handling East/West traffic in a data center. When ruting between virtual netwrks, these Layer 3 netwrks are distributed in the ESXi hypervisr. Here the distributed lgical ruter ptimizes the ruting and data path, and supprts bth single-tenant r multi-tenant deplyments. Fr example, a netwrk cntains tw VNIs that have the same IP addressing. With this scenari, tw different distributed lgical ruters must be deplyed with ne distributed lgical ruter cnnecting t tenant A and ne t tenant B. It is the jb f the NSX Manager t cnfigure and manage the ruting service. During the cnfiguratin prcess, the NSX Manager deplys the lgical ruter cntrl virtual machine and then pushes the lgical interface (LIF) cnfiguratins t each hst thrugh the cntrl cluster. The lgical ruter cntrl virtual machine is the cntrl plane cmpnent f the ruting prcess and the lgical ruter cntrl virtual machine supprts bth the OSPF and BGP prtcls. 44 VMware vclud Architecture Tlkit fr Service Prviders

45 Manageability 7.1 Clud Cnsumptin Mdels NSX fr vsphere is designed t be cnsumed thrugh a self-service prtal r REST API, depending n the service mdel that the clud prvider wants t achieve, and dictates hw the prvider and the end users cnsume VMware NSX resurces Hsting Slutin Thrugh the vsphere Web Client, hsting services prviders can give their end custmers full access t the VMware NSX functinality t create and manage netwrking resurces Private Clud Slutin Private clud services built with vrealize Autmatin ffer autmated creatin f VMware NSX netwrking and security services within a multi-machine blueprint r thrugh the API (using vrealize Orchestratr) Public Clud Slutin Public clud services built with vclud Directr fr Service Prviders ffers self-service cnsumptin f NSX Edge netwrking and security services thrugh the vclud Directr user interface r API. 7.2 NSX fr vsphere Lgging Cnsideratins All VMware NSX cmpnents, such as NSX Cntrller, VMware NSX Virtual Switch, and NSX Edge, prvide detailed netwrk visibility and data. The VMware NSX platfrm ffers centralized reprting and mnitring, distributed perfrmance and scale, and is designed fr autmatin. VMware NSX is built n a REST API prvided by NSX Manager, and all peratins can be perfrmed prgrammatically thrugh scripting r higher-level languages. 45 VMware vclud Architecture Tlkit fr Service Prviders

46 Figure 19. NSX fr vsphere Lgging Envirnment ESXi hsts run a syslg service (vmsyslgd) that prvides a standard mechanism fr lgging messages frm VMkernel and ther system cmpnents. ESXi can als be cnfigured t send the lgs acrss the netwrk t a VMware vrealize Lg Insight server. There are multiple levels f lgging t cnsider. Nte Cnfiguratin f the vrealize Lg Insight service n ESXi can be perfrmed using hst prfiles, the vsphere cmmand-line interface, r the advanced cnfiguratin ptins in the VMware vsphere Client. The fllwing lg files are related t NSX and must be sent t an apprpriate lg cllectin service such as vrealize Lg Insight: Distributed firewall packet lgs can be fund at /var/lg/dfwpktlgs.lg. Distributed firewall userwrld agent lgs are lcated at /var/lg/vsfwd.lg. Netcpa (userwrld agent) lgs can be fund at /var/lg/netcpa.lg. This lg file cntains messages regarding cntrller-t-hst cmmunicatin details. Lgical switch (VXLAN), distributed lgical ruter and VMware Internetwrking Service Insertin Platfrm (VSIP) kernel mdule lgs are available at /var/lg/vmkernel.lg. The lgical switch related lgs will be tagged with vxlan, the distributed lgical ruter related lgs will be tagged with vdrb, and the VSIP-related lgs will be tagged with vsip. DVS lgs are als available at /var/lg/vmkernel.lg 46 VMware vclud Architecture Tlkit fr Service Prviders

47 7.3 Management Interfaces The fllwing sectin describes management interfaces fr NSX fr vsphere cmpnents that must be specifically enabled Distributed Lgical Ruter Cntrl Virtual Machine When a distributed lgical ruter is deplyed, the lgical ruter cntrl virtual machine is als and it handles all cntrl plane cmmunicatins fr the distributed lgical ruter. The distributed lgical ruter prvides a management interface cnfiguratin thrugh the user interface, which supprts management services, such as SSH, fr remte cnnectivity. The lgical ruter cntrl VM cmmunicates with the NSX Cntrller thrugh a VMCI interface. Nte The lgical ruter cntrl VM des nt have an actual IP address assigned althugh the management interface is cnnected t the same management virtual distributed switch prt grup as the NSX Cntrller VMware NSX Distributed Firewall Mnitring The VMware NSX distributed firewall must have enugh memry t avid drpping traffic. The firewall administratr is ntified f the lack f available memry by the fllwing methds: An alert sent when a new rule cannt be cnfigured due t the shrtage. A syslg message that states the distributed firewall cannt create new cnnectins due t the shrtage. If the rule relating t the flw creatin has lgging turned n, a secnd message is generated t indicate that the packet was als drpped. Freeing memry n a hst, by mving a guest t anther hst, fr example, reslves the issue. If the distributed firewall virtual CPUs reach a maximum limit, packets might als be drpped. If lgging is enabled fr that flw, a lg message is als generated fr the drpped packets. In an All Failure scenari, packets are discarded and the distributed firewall perates in a fail-clsed mde until the failure is remedied. 47 VMware vclud Architecture Tlkit fr Service Prviders

48 Perfrmance and Scalability 8.1 Perfrmance f Netwrking and Security in a Virtualized Envirnment Quality f Service (QS Layer 3) and Differentiated Services (DSCP Layer 2) NSX fr vsphere allws trust f the Differentiated Service Cde Pint (DSCP) marking riginally applied by a virtual machine, r explicitly mdifying and setting the DSCP value at the lgical switch level. In bth cases, the DSCP value is prpagated t the uter IP header f VXLAN encapsulated frames. This enables the external physical netwrk t priritize the traffic based n the DSCP setting n the external header. Bth quality f service (QS) and DSCP are gd examples f hw physical and virtual netwrking can wrk tgether under a cmmn set f rules. Using bth QS and DSCP which are netwrking standards, allws netwrk switches t priritize certain netwrk traffic ver thers, which in turn helps yur critical wrklads get the netwrk pririty required t meet business demands. Yu can verify that the applicatin traffic flwing thrugh the physical netwrk infrastructure is priritized by using the fllwing: Class f Service (CS): Layer 2 tag Differentiated Services Cde Pint (DSCP) marking: Layer 3 tag Traffic can be classified in different ways. In a Layer 2 frame, the 802.1q header cntains the infrmatin fr the class f service (CS). The first 16 bits are always 0x8100, which means that the header cntains a VLAN tag. The class f service is in the next 3 bits fllwed by a flag that indicates whether t fragment. Layer 3 has a different field called DSCP that has 6 bits. The first three values typically match the first three CS bits. At the bundary between Layers 2 and 3, the switch can take the CS and ther factrs like the surce r destinatin address and match that t a Layer 3 DSCP value. Because DSCP has mre ptential values, it can be mre specific abut the service that it is ging t prvide. Figure 20. QS Layer 3 and DSCP Layer 2 48 VMware vclud Architecture Tlkit fr Service Prviders

49 8.2 Scalability f Virtualized Clud Envirnments The service prvider tplgy described in earlier sectins can be scaled ut as shwn in the fllwing figure. The figure shws nine tenants served by the NSX Edge services gateway n the left and the remaining nine by the NSX Edge services gateway n the right. Service prviders can easily prvisin additinal NSX Edge services gateways t serve additinal tenants. Figure 21. Scalability Deplyment Example with NSX Edge Services Gateway External Netwrks NSX ESG X-Large (Rute Aggregatin Layer) Transit VXLAN 5100 Tenant NSX ESG Tenant NSX ESG Tenant DLR Tenant DLR VXLAN VXLAN VXLAN VXLAN VXLAN VXLAN Scalability f NSX fr vsphere Cmpnents There is a ne-t-ne mapping between NSX Manager and vcenter Server in versin 6.1.x. Shuld the inventry f a prtin f the hybrid clud exceed the limits supprted by a single vcenter Server, a new NSX Manager must be deplyed alng with any new vcenter Server added. Transprt znes can be extended and scaled larger by adding mre vsphere cmpute and NSX Edge clusters until vcenter Server limits are reached. There is a limit f 1,000 distributed lgical ruters per ESXi hst when using NSX fr vsphere and abve. As a design cnsideratin, if yu want t exceed the 1,000 distributed lgical ruter limit in a VMware NSX dmain, yu must create multiple transprt znes with different clusters in each transprt zne. There are many factrs that determine the scalability limits in NSX fr vsphere 6.1.x and 6.2, as well as ther vcenter Server limits which will likely be exceeded befre reaching the limit n the NSX fr vsphere cmpnents. Nte Fr scaling the thrughput f the Nrthbund cnnectin t the Internet, refer t Sectin 6.3.3, Equal-Cst Multi-Path High Availability fr NSX Edge Services Gateway. 49 VMware vclud Architecture Tlkit fr Service Prviders

50 Recverability The tpic f recverability fr netwrk and security virtualizatin within in a VMware Clud Prvider Prgram envirnment relates t the ability t back up and restre the fllwing assciated VMware NSX cmpnents: NSX Manager NSX Edge NSX firewall rules NSX Service Cmpser Virtual distributed switch vcenter Server At a minimum, service prviders must make regular backups f NSX Manager and vcenter Server t restre the system state in the event f a catastrphic failure. The verall backup frequency and schedule might vary based n business need and peratinal prcedures set up by peratinal teams. Hwever, VMware recmmends having the same number f NSX backups as there are cnfiguratin changes. NSX Manager backups can be made n demand r n an autmated hurly, daily, r weekly basis. T restre the system state after a failure, the recmmended timeframe t make backups is the fllwing: Befre an NSX r vcenter Server upgrade. After an NSX r vcenter Server upgrade. During r after Day Zer deplyment and cnfiguratin f VMware NSX cmpnents (creatin f cntrllers, lgical switches, distributed lgical ruter, NSX Edge cmpnents, security, and firewall plicies). Fllwing infrastructure changes. After any majr Day2 changes. Synchrnize VMware NSX cmpnent backups (NSX Manager and NSX Cntrller) with yur backup schedule fr ther dependent cmpnents (vcenter Server, clud management systems, peratinal tls, and s n). This will capture the entire system state at a given time, and give yu a stable state in time t which yu can rll back. 50 VMware vclud Architecture Tlkit fr Service Prviders

51 9.1 NSX Manager Recverability Yu can back up NSX Manager data by perfrming an n-demand backup r a scheduled backup. Backups can be scheduled n an hurly, daily, r weekly basis. Yu can back up and restre yur NSX Manager data, including system cnfiguratin, events, and audit lg tables. Cnfiguratin tables are included in every backup. Backup and restre can be cnfigured frm the NSX Manager virtual appliance web interface r thrugh the REST API. Restre is nly supprted n the same NSX Manager versin as the backup. The backup file is saved t a remte lcatin that the NSX Manager can access via FTP r SFTP. Nte Save yur FTP server IP r hst name, credentials, directry details, and pass phrase. These are used when yu want t restre the backup Restring NSX Manager Backups VMware recmmends restring a backup n a newly deplyed NSX Manager appliance: Restring t an existing NSX Manager installatin might als wrk, but is nt fficially supprted r tested in-huse. Internal testing is dne with the assumptin that the existing NSX Manager has failed, requiring that a new NSX Manager appliance be deplyed. The newly deplyed NSX Manager appliance VM n which the restre is perfrmed must be the same versin as the NSX Manager appliance frm which the backup was taken. T restre an available backup, the Hst IP Address, Username, Passwrd, Backup Directry, Filename Prefix, and Passphrase fields in the Backup Lcatin screen must have values that identify the lcatin f the backup t be restred. Nte Take screenshts f the ld NSX Manager appliance settings screen r nte them s that they can be used t specify IP infrmatin and backup lcatin infrmatin n a freshly deplyed NSX Manager appliance. 9.2 NSX Cntrller Recverability There is an NSX Cntrller snapsht buttn in the user interface t take NSX Cntrller cluster snapshts. A snapsht is the database snapsht f the cntrller cluster. Take NSX Cntrller snapshts at the same time as the NSX Manager backup. Befre taking the snapsht backup, verify the fllwing: All f the cntrllers are in the nrmal state. The cluster has frmed a majrity (qurum). 9.3 NSX Edge Services Gateway Recverability All NSX Edge cnfiguratins (distributed lgical ruter cntrl VMs and edge gateways) are backed up as part f NSX Manager backup. If NSX Manager cnfiguratin is intact, VMs n an inaccessible r failed NSX Edge appliance can be redeplyed anytime frm the vsphere Web Client by selecting Netwrking and Security > NSX Edges > Actins > Redeply. If yu want t get the cnfiguratin f a standalne NSX Edge gateway, yu can use REST API calls. These calls are useful if yu want t preserve the cnfiguratin f a standalne NSX Edge gateway fr future use r reference. This cnfiguratin might be useful in the event that yu want t recreate a single NSX Edge gateway with an existing NSX Manager. Details n hw t use the REST API t manage VMware NSX can be fund in the VMware NSX fr vsphere API Guide and VMware NSX API Guide. 51 VMware vclud Architecture Tlkit fr Service Prviders

52 9.4 Distributed Firewall Recverability A user can exprt the firewall rules cnfiguratin and save them t a central lcatin. All firewall rules including Service Cmpser rules are exprted. The saved cnfiguratin can be used as a backup r imprted fr use in an NSX Manager envirnment. Nte When yu lad an imprted firewall cnfiguratin, if yur current cnfiguratin cntains rules managed by Service Cmpser, these are verridden after the imprt. If Service Cmpser rules in yur cnfiguratin were verridden by the laded cnfiguratin, click Actins > Synchrnize Firewall Cnfig in the Security Plicies tab within Service Cmpser. 9.5 VMware vsphere Distributed Switch Recverability Yu can exprt VMware vsphere Distributed Switch and distributed prt grup cnfiguratins t a file. The file preserves valid netwrk cnfiguratins, enabling distributin f these cnfiguratins t ther deplyments. This functinality is available nly with vsphere Web Client 5.1 r later. Nte A best practice is t exprt the vsphere Distributed Switch cnfiguratin befre preparing the cluster fr VXLAN. 9.6 VMware vcenter Server Recverability See the VMware vcenter Server dcumentatin fr vcenter Server backup and restre prcedures and best practices. Fr example, see the VMware vcenter Server 5.5 Availability Guide at 52 VMware vclud Architecture Tlkit fr Service Prviders

53 Security 10.1 NSX fr vsphere Cmpnent Security The NSX Manager generates self-signed certificates fr each f the hsts and cntrllers, which are used t secure cntrl plane cmmunicatins. This cntrl plane cmmunicatin is secured with TLS encryptin by using the certificates that are managed by the NSX Manager. Install a CA-signed certificate fr the NSX Manager t secure bth the management interface and API endpint n prt 443. The Pivtal RabbitMQ brker certificates n the NSX Manager used fr cmmunicatin with the ESXi hsts are uniquely generated n first bt Integratin with vcenter Single Sign-On NSX Manager instances n each site are cnfigured t integrate with the VMware vcenter Single Sign- On service assciated with the vcenter Server resurce t which they are bund. This facilitates the secure authenticatin f vcenter Server users within NSX fr vsphere and als any f the identity stres cnfigured under vcenter Server, including LDAP, Active Directry, and NIS directries. The integratin is set thrugh the NSX Manager user interface by supplying the address and prt f the vcenter Single Sign-On server. The NTP settings fr the NSX Manager are cnfigured s that it is in sync with the time f the vcenter Single Sign-On service. Authenticatin using this methd is highly time-sensitive, s verify that the cmpnents invlved are nt subject t drift Rle-Based Access Cntrl NSX fr vsphere utilizes a rle-based access cntrl (RBAC) apprach t granting permissins t users r grups. Pre-existing rles are present in the NSX fr vsphere envirnment and users are then assigned t rles t inherit the assciated permissins. The default rles are described in the fllwing table. Table 5. NSX fr vsphere Rles and Permissins Rle Enterprise Administratr NSX fr vsphere Administratr Security Administratr Auditr Permissins NSX fr vsphere peratins and security. NSX fr vsphere peratins nly (fr peratins such as install virtual appliances, and cnfigure prt grups). NSX fr vsphere security nly (fr peratins such as defining data security plicies, creating prt grups, and creating reprts fr NSX fr vsphere mdules). Read-nly rights. 53 VMware vclud Architecture Tlkit fr Service Prviders

54 In additin t granting permissins using rles, it is als necessary t specify the scpe f access that the user r grup will have t the system. The scpe levels are shwn in the fllwing table. Table 6. NSX fr vsphere Permissins Scpes Scpe N restrictin Limit access scpe Descriptin Full access t the NSX fr vsphere system. Access nly t a specified NSX Edge device. Bth the Enterprise Administratr and VMware NSX Administratr rles can be assigned nly t vcenter Server resurces. Their scpe is glbal, s it is nt pssible t apply restrictins NSX fr vsphere Hardening This sectin prvides high-level recmmendatins fr the mst effective methds f evaluating and securing the NSX fr vsphere platfrm, data center, and clud infrastructure built using NSX fr vsphere, specifically v6.1. The recmmendatins are gruped in t the fllwing categries: Cmmn Management plane Cntrl plane Data plane Infrmatin fr each f these categries is prvided in the VMware NSX fr vsphere Hardening Guide available at This guide is intended fr users in varius rles, including netwrk and security architects, security fficers, virtual infrastructure administratrs, clud infrastructure architects, clud administratrs, clud custmers, clud prviders, and auditrs. Additinally, individuals and rganizatins that are seeking a starting pint fr the netwrk and security cntrls t cnsider when adpting r building a netwrk and security infrastructure will find the recmmendatins helpful. VMware engages with varius partners t perfrm security assessments f the NSX fr vsphere platfrm and specific design and architecture deplyments. These assessments als fcus n newer features such as the integratin f sftware-defined netwrking (SDN) and sftware-defined data center (SDDC). The assessment f the NSX fr vsphere platfrm is primarily fcused n netwrking and security attacks, cnfiguratin issues, secure defaults, and prtcls in use. Using a cmbinatin f targeted surce cde review, active and fuzz testing, as well as ther methds, these assessments lcate and determine whether any significant vulnerabilities exist. Left unchecked, many f these issues (separately, r in cncert) culd result in a cmplete data center cmprmise. S, keep in mind as yu design data center and clud architecture and system slutins that yu must take the required steps and make the apprpriate architectural design decisins t avid r mitigate issues that might arise in yur wn envirnment. Despite the inherent risks, sftware-defined netwrking paired with netwrk and security virtualizatin ffers a myriad f benefits and allws fr entirely sftware-defined data centers, a key part f the VMware visin fr current and future prducts. Yu must als address the ptential and inherent risks f this new platfrm as yu wrk with the VMware NSX platfrm technlgy. One f the true values f sftware-defined netwrking and security is it allws agile mvement f virtual machines and netwrks and security services between physical hsts and the data center as cmpared t physical netwrking. The dynamic nature f this technlgy requires that underlying hsts be fully cnnected at the physical and IP layer. With these new ptins fr cnnectivity, hwever, als cme sme risks. All sftware has flaws, and the re-implementatin f cre netwrking prtcls, parsers, and 54 VMware vclud Architecture Tlkit fr Service Prviders

55 switching methds will repeat and likely inherit histric vulnerabilities frm lder methds f physical netwrking and security. As an example, denial-f-service (DS) attacks have becme a much greater issue nw. In the physical netwrking wrld, dedicated hardware handles much f the parsing and ruting f packets. In a sftware netwrking and security wrld, it is the sftware cmpnent that must parse, reparse, perfrm table lkups, and generally be aware f encapsulatin, fragmentatin and s n, spending much mre CPU time deciding hw t handle each packet. A ptential sftware bug in any stage f this packet handling can lead t resurce exhaustin, sftware crashes, and ther scenaris that result in DS and pssibly a lss f netwrking and security services fr hundreds f hsts, and als might affect the entire data center. Sftware-defined netwrking and security als extends traditinal netwrk and security attacks t multiple data centers. Traditinally lcal attacks, such as ARP spfing, can nw be cnducted acrss Layer 3 netwrks in gegraphically diverse lcatins. Additinally, if any vulnerability in the sftware netwrk and security stack allws these attacks t leak nt the physical netwrk, physical hsts in multiple data centers affecting multiple custmers can als be cmprmised. In a very real sense, sftware-defined netwrking and security as it is currently designed relies n virtual machine cntainment. If a virtual machine escape is ever perfrmed r if an attacker discvers a technique fr sending un-encapsulated packets n physical netwrks, expected security will be lst. As described previusly, every physical hst must be cmpletely cnnected at the IP and physical layer, expsing an extremely brad attack surface. Once an attacker has a methd f sending and receiving data n this physical netwrk, the attacker can mve laterally between hsts unabated by firewalls r ruters, as these are n lnger security relevant devices. Sftware-defined netwrking and security is a pwerful technlgy that is necessary fr rganizatins and cmpanies t take advantage f, nw and in the future. Hwever, like all sftware, sftware-defined netwrking and data centers can be fragile and netwrking and security vulnerabilities have brad ramificatins nt traditinally realized in physical netwrking platfrms. As yu lk at recurring weaknesses, these are gd candidates fr systematic fixes as well as areas that require additinal testing. These can als be cnsidered in secure guidelines and threat mdeling. Cnsider the fllwing: insufficient cntrl, management and data plane security requirements Much f the NSX fr vsphere platfrm can be prtected with TLSv1/SSL (if prperly cnfigured), but cnsistent usage and strng defaults are still elusive. When prtecting the NSX Manager, as well as the management REST APIs, use TLS v1.2, because the cntrl plane uses TLS in all ther cmmunicatins. 55 VMware vclud Architecture Tlkit fr Service Prviders

56 Operatinal Cnsideratins 11.1 NSX Manager Operatinal Cnsideratins The NSX Manager is the management plane virtual appliance that helps cnfigure lgical switches and cnnect virtual machines t these lgical switches. It als prvides the management user interface an entry pint fr the NSX API, which helps autmate deplyment and management f the lgical netwrks thrugh a clud management platfrm. In the NSX fr vsphere architecture, NSX Manager is tightly cnnected t the vcenter Server managing the cmpute infrastructure. In fact, there is a 1:1 relatinship between the NSX Manager and vcenter Server and, upn installatin, the NSX Manager registers with vcenter Server and injects a plug-in int the vsphere Web Client fr cnsumptin within the web management platfrm. Figure 22. VMware NSX Management Plane 56 VMware vclud Architecture Tlkit fr Service Prviders