Data Center Design for the Midsize Enterprise

Size: px

Start display at page:

Download "Data Center Design for the Midsize Enterprise"

Monica Hall
5 years ago
Views:

2 Data Center Design for the Midsize Enterprise Jerry Hency Technical Marketing Engineer, Data Center Group

3 Data Center Design for the Midsize Enterprise Terminology and Goals for this session Midsize Enterprise/Organization Designs: Minimum requirement a dedicated pair of Data Center switches. The transition point upwards from true SMB collapsed-core designs. Distinct Layer 2/3 boundary, with Data Center oriented feature set. Support current and future needs: Right-sizing the Data Center, not just large scale. Build using components that will also transition easily into larger designs. Make Data Center design choices: Topology options: from single layer designs to spine/leaf data center fabrics. Tradeoffs of components to fill topology roles. Client Access/Enterpr ise Data Center WAN/Internet Edge L L2 3

4 Agenda Introduction Data Center Requirements for Midsize Organizations Building an Access Pod Single-Layer Design Examples Moving to a Spine/Leaf Fabric Conclusion 4

5 Midsize Data Center Design Challenges This session will provide example designs which are: Flexible: Entry-level design models for smaller organizations to take advantage of Nexus Unified Fabric features and easily scale up when they are ready. Practical: Balancing cost with port count, software features, and hardware capabilities. Easy to use: providing deployment and operational simplicity for organizations managing a growing network with a small staff. Choose which platform requirements to prioritize: Port types, speeds, cable plant: 1/10GigE, FEX, vpc, Fibre Channel, FCoE High Availability features: Dual Supervisors, ISSU, Spine/Leaf Design Data Center Interconnect (DCI): Overlay Transport Virtualization (OTV), MPLS, VPLS Fabric Integration and Orchestration: Programmability, API s, Controller-based options 5

6 Server and Storage Needs Drive Design Choices Form Factor Unified Computing Fabric 3 rd Party Blade Servers Rack Servers (Non-UCS Managed) Storage Protocols Fibre Channel FCoE IP (iscsi, NAS) Virtualization Requirements vswitch/dvs/ovs Nexus 1000v APIs/Programmability/Orchestration Connectivity Model 10 or 1-GigE Server ports NIC/HBA Interfaces per-server NIC Teaming models FC FCoE iscsi NFS/ CIFS VM VM VM VM VM VM 6

7 NORTH - SOUTH TRAFFIC Data Center Fabric Requirements Varied North-South communication needs with endusers and external entities. Increasing East-West communication: clustered applications and workload mobility. High throughput and low latency requirements. Increasing high availability requirements. Public Cloud Mobile API Orchestration/ Monitoring Internet DATA CENTER FABRIC Enterprise Network Offsite DC Site B FC FCoE iscsi / NAS Storage Automated provisioning and control with orchestration, monitoring, and management tools. Server/Compute EAST WEST TRAFFIC Services 7

Cisco Intercloud Fabric Workload Portability for the

Capacity Augmentation Disaster Recovery Provider

VM Cloud APIs PRIVATE CLOUD Secure Extender Secure

8 Cisco Intercloud Fabric Workload Portability for the Hybrid Cloud Director Dev/Test Control of Shadow IT Capacity Augmentation Disaster Recovery Provider Platform Administration portal Workload management VM VM Cloud APIs PRIVATE CLOUD Secure Extender Secure network extension Workload mobility PUBLIC CLOUD Cisco Powered 8

9 Agenda Introduction Data Center Requirements for Midsize Organizations Building an Access Pod Single-Layer Design Examples Moving to a Spine/Leaf Fabric Conclusion 9

10 Access Pod basics: Compute, Storage, and Network Different Drawing, Same Components To Data Center Aggregation or Network Core Access/Leaf Switch Pair UCS Fabric Interconnect System Storage Array 10

11 Access Pod Features: Virtual Port Channel (vpc) vpc provides port-channel link aggregation across a pair of separate physical switches. Allows the creation of resilient Layer-2 topologies based on Link Aggregation. Spanning Tree Protocol (STP) is no longer the primary means of loop prevention. Provides more efficient bandwidth utilization since all links are actively forwarding. vpc maintains independent control and management planes. Two peer vpc switches are joined together to form a vpc domain. Physical Topology Si Virtual Port Channel L2 Si Non-vPC Logical Topology vpc 11

Access Pod Features: Nexus 2000 Fabric Extension Using FEX provides Top-of-Rack presence in more racks with fewer points of management, less cabling, and lower cost.

12 Access Pod Features: Nexus 2000 Fabric Extension Using FEX provides Top-of-Rack presence in more racks with fewer points of management, less cabling, and lower cost. In a straight-through or single-homed FEX configuration, each Nexus 2000 FEX is only connected to one parent switch. FEX parent switch may be Nexus 5000, 6000 or Series. Nexus 2000 includes 1/10GigE TOR models with 10 or 40GigE uplinks, plus the B22 models for use in blade server chassis from HP, Dell, Fujitsu, and IBM. End/Middle of Row Switching with FEX Nexus Parent Switch Nexus 2000 FEX Design note: Verify platform-specific FEX compatibility and scale numbers on cisco.com. Dual NIC Active/Standby Server Dual NIC 802.3ad Server 12

13 Nexus Fabric Features: Enhanced vpc (EvPC) Dual-homed FEX with addition of dual-homed servers In an Enhanced vpc configuration, server NIC teaming configurations or single-homed server connections are supported on any port. No vpc orphan ports on FEX in the design. Nexus 6000/5600/5500 All components in the network path are fully redundant. Supported FEX parent switches are Nexus 6000, 5600 and FEX Provides flexibility to mix all three server NIC configurations (single NIC, Active/Standby and NIC Port Channel). Design Notes: Port Channel to active/active server is not configured as a vpc. N7000 planned to support dual-homed FEX (without dual-homed servers) targeted in NX-OS Single NIC Dual NIC Active/Standby Dual NIC 802.3ad

14 Nexus Fabric Features: Unified Ports and FCoE Seamless transport of both storage and data traffic at the server edge Unified Ports: May be configured to support either native Fibre Channel or Ethernet Available on Nexus 5500/5600UP switches, or as an expansion module on Nexus Any Unified Port can be configured as: Ethernet Fibre Channel Traffic or Fibre Channel Fibre Channel Traffic Fibre Channel over Ethernet (FCoE): FCoE allows encapsulation and transport of Fibre Channel traffic over a shared Ethernet network Traffic may be extended over Multi-Hop FCoE, or directed to an FC SAN SAN A / B isolation is maintained across the network SAN-A FCoE Links Nexus Ethernet/FC Switches Servers with CNA SAN-B FC Disk Array

15 Planning Physical Data Center Pod Requirements Map physical Data Center needs to a flexible fabric topology. Term Server, Management Switch (2) N5672UP (2)N2232 FEX Plan for growth in a modular, podbased repeatable fashion. Your own pod definition may be based on compute, network, or storage requirements. Access Pod TOR switching becomes the leaf switches of a spine/leaf topology. Today s Server Racks Storage Arrays PATCH Network/Storage Rack (32) 1RU Rack Servers Compute Rack How many current servers/racks, and what is the expected growth? Tomorrow s Data Center Floor 15

16 Data Center Service Integration Approaches Data Center Service Insertion Needs Firewall, Intrusion Prevention Application Delivery, Server Load Balancing Network Analysis, WAN Optimization Physical Appliances/Switch Modules Typically introduced at Layer 2/3 Boundary; spine/aggregation or services leaf switches. Traffic direction with VLAN provisioning, Policy-Based Routing, WCCP. Physical DC Service Appliances (Firewall, ADC/SLB, etc.) Network Core L L2 Virtualized Services Deployed in a distributed manner along with virtual machines. Traffic direction with vpath and Nexus 1000v. Application Centric Infrastructure (ACI) automated framework for service insertion. Virtual DC Services in Software VM VM VM VM VM VM Virtualized Servers with Nexus 1000v and vpath 16

17 Agenda Introduction Data Center Requirements for Midsize Organizations Building an Access Pod Single-Layer Design Examples Moving to a Spine/Leaf Fabric Conclusion 17

18 Single Layer DC, Fixed/Semi-Modular Switching Nexus 5600 Data Center Switches: 5672UP: 1RU, 48 1/10GE + 6 QSFP (16 Unified Ports) 56128P: 2RU, 48 1/10GE + 4 QSFP, 2 expansion slots (24 1/10GE-Unified Port + 2 QSFP module available) Non-blocking, line-rate Layer-2/3 switching with low latency ~1 µs. FCoE plus 2/4/8G Fibre Channel options. Hardware-based Layer-2/3 VXLAN, NVGRE. Dynamic Fabric Automation (DFA) capable. Design Notes: DCI (OTV, MPLS, VPLS, LISP) may be provisioned through separate Nexus 7000 or ASR WAN Routers ISSU not supported with Layer-3 on Nexus 5000/6000 L L2 1Gig/100M Servers Client Access 10-GigE UCS C-Series WAN / DCI 10 or 1-Gig attached UCS C-Series Nexus 5600 FC FCoE iscsi / NAS Nexus 2000 FEX

19 Single Layer switching plus FEX design Reference Nexus 5672UP: 48 x 10GE ports plus 6 40GE QSFP. Using 2232PP FEX: 32 1/10GE host, 8 x 10 GE uplinks. Client Access WAN / DCI Example is connecting all 8 uplinks per-fex to maintain 4:1 oversubscription, 4 ports per-parent. N5672UP: 48 x 10 GE / 4 uplinks = 12 FEX Using 4 x 10 breakout on each QSFP provides 24 additional 10GigE port count to account for: Uplinks to Client Access, WAN/DCI vpc Peer link Storage arrays, Service Devices Direct-attached 10GigE Servers L L2 Nexus x 2232 FEX = 384 1/10 GE ports (platform limit 24 FEX) FC FCoE iscsi / NAS

20 Single Layer Data Center plus UCS Fabric Alternate Server Edge 1: UCS Fabric Interconnects Client Access Typically 4 8 UCS Chassis per Fabric Interconnect (FI) pair. (Maximum is 20). WAN / DCI UCS Manager can also manage UCS C- Series rack servers on the same system. Add UCS Director to provide management and orchestration of the unified infrastructure. Example DC Switching Components: 2 x Nexus 5672UP Layer 3 and Storage Licensing 2 x Nexus 2232PP/TM-E Nexus 5672UP UCS Fabric Interconnect s L L2 FCoE iscsi / NAS FC Optional design: direct-attach storage to UCS Fabric Interconnect UCS Blade Chassis UCSM managed C-Series Servers

21 Single Layer Data Center with Nexus B22 FEX Alternate Server Edge 2: Third-Party Blade Server Client Access B22 FEX allows Fabric Extension directly into compatible 3rd-party chassis. WAN / DCI Provides consistent network topology for multiple 3rd-party blade systems. FCoE on FEX uplinks, or MDS/Nexus SAN connected to server HBA s. Example Components: 2 x Nexus 5672UP s L3 and Storage Licensing 4 x Nexus B22 HP, Dell, Fujitsu, and IBM blade chassis supported, see data sheet for model specifics. Nexus 5672UP UCS C-Series Cisco B22 FEX for Blade Chassis Access L L2 FCoE iscsi / NAS FC

22 Single Layer Data Center, Modular Chassis High-Availability, Modular 1/10/40/100 GigE Client Access Nexus 7700 example topology, common asics and software shared with Nexus 7000 platform. Concurrent support for: DCI capability with OTV, LISP, MPLS, VPLS FabricPath, FCoE and FEX Dual-Supervisor High Availability. Layer-2/3 In Service Software Upgrade (ISSU) Virtual Device Contexts (VDC) Layer-2/3 VXLAN in hardware on F3 card. Dynamic Fabric Automation support; NX-OS 7.1 Spine+ Leaf VDCs L L2 WAN OTV VDCs Nexus 7706 iscsi / NAS Design Notes: For native Fibre Channel add Nexus/MDS SAN. N7k FCoE directly to FEX, support planned for NX-OS or 1-Gig attached UCS C-Series

23 Single Layer Data Center, ACI-Ready Platform Nexus 9000 switching platforms enable migration to Application Centric Infrastructure (ACI). May also be deployed in standalone NX-OS mode (without APIC controller). 9396PX: 48 1/10GigE SFP+ ports, 12 QSFP 9504: Small-footprint HA modular platform Basic vpc and straight-through FEX supported as of NX-OS 6.1(2)I3(1) VXLAN Layer-2/3 in hardware IP-based storage support Low latency, non-blocking Layer-2/3 switching ~1µs Design Notes: OTV, LISP DCI may be provisioned through separate Nexus 7000 or ASR 1000 WAN Routers. Fibre Channel or FCoE support requires separate MDS or Nexus 5500/5600 SAN switching. (Future FCoE capable) ISSU support on 9300 targeted for 2HCY14. L L2 1Gig/100M Servers Client Access 10-GigE UCS C-Series WAN / DCI 10 or 1-Gig attached UCS C-Series Nexus 9396PX FEX iscsi / NAS

Working with 40 Gigabit Ethernet Reference Nexus family switches support QSFPbased 40 Gigabit Ethernet interfaces.

* 40 Gigabit Ethernet cable types: Direct-attach copper [QSFP <-> QSFP] and [QSFP <-> 4 x SFP+].

SR4 uses bit-spray over 4 fiber pairs within a 12 fiber MPO/MTP connector to reach up to 100/150m on multimode OM3/OM4 CSR4 is a higher

24 Working with 40 Gigabit Ethernet Reference Nexus family switches support QSFPbased 40 Gigabit Ethernet interfaces.* On most platforms, splitter cables can be used to provision 4x10GigE ports out of 1 QSFP.* 40 Gigabit Ethernet cable types: Direct-attach copper [QSFP <-> QSFP] and [QSFP <-> 4 x SFP+]. Passive cables at 1/3/5m, active cables at 7 and 10m. SR4 uses bit-spray over 4 fiber pairs within a 12 fiber MPO/MTP connector to reach up to 100/150m on multimode OM3/OM4 CSR4 is a higher powered SR4 optic with reach up to 300/400m on multimode OM3/OM4 LR4 uses CWDM to reach up to 10km on a single-mode fiber pair. * Verify platform-specific support of capabilities and roadmap QSFP-40G-CR4 direct-attach cables QSFP+ to 4-SFP+ direct-attach cables (splitter) QSFP-40G-SR4 with direct MPO and 4x10 MPO-to-LC duplex splitter fiber cables 24

Multimode Fiber QSFP SR 12-Fiber infrastructure TX RX 4 x 10G 4 x 10G QSFP SR 12-Fiber infrastructure Duplex (two strand) multimode fiber with

25 QSFP-BIDI vs. QSFP-40G-SR4 Run 40 GigE over existing duplex multimode cable plant QSFP-BIDI Duplex Multimode Fiber TX/RX 2 x 20G 2 x 20G TX/RX QSFP-BIDI Duplex Multimode Fiber QSFP SR 12-Fiber infrastructure TX RX 4 x 10G 4 x 10G QSFP SR 12-Fiber infrastructure Duplex (two strand) multimode fiber with Duplex LC connectors at both ends Use of duplex multimode fiber lowers cost of upgrading from 10G to 40G by leveraging existing 10G multimode infrastructure 12-Fiber ribbon cable with MPO connectors at both ends Higher cost to upgrade from 10G to 40G due to 12-Fiber infrastructure 25

26 Configuration Best Practices: vpc with Layer-2, Layer-3

vpc Options: Auto-Recovery By default, both parents must be present for a newly connected vpc to be brought active Auto-Recovery Allows vpc s to be established with only a single parent present.

27 vpc Options: Auto-Recovery By default, both parents must be present for a newly connected vpc to be brought active Auto-Recovery Allows vpc s to be established with only a single parent present. Addresses multiple scenarios: After a power failure with a partial restore where only one parent switch is present. New vpc-attached devices to be configured or powered on during a hardware issue with one of the parent switches. Ongoing operations based off of either the configured vpc Primary or Secondary parent when one is down for any reason. vpc Domain N6004-a(config)# vpc domain 10 Missing vpc Peer N6004-a(config-vpc-domain)# auto-recovery 27

vpc Options: Orphan Ports Suspend Keepalive An orphan port is a device attached to only one member of a vpc pair. Intended for devices that do not support portchannel.

28 vpc Options: Orphan Ports Suspend Keepalive An orphan port is a device attached to only one member of a vpc pair. Intended for devices that do not support portchannel. Other devices should be dually connected by vpcs. If the vpc peer-link were to go down, the vpc secondary peer device shuts all its vpc member ports as well as designated orphan ports. Configure switch ports for single attached devices (like Firewall or Load Balancer) as orphan-port. Configuration allows consistent behavior of orphan ports with vpc member ports. Active/Standby Server NIC teaming also uses Orphan Ports. Active or Standby S1 -Primary vpc peer-link Server with Active/Standby NIC Teaming S1(config)# int eth 1/1 S1(config-if)# vpc orphan-ports suspend S2(config)# int eth 1/1 S2(config-if)# vpc orphan-ports suspend S2-Secondary Active or Standby Orphan port 28

29 vpc Options: vpc Peer Switch Unifies Spanning Tree processing across vpc peers For use on vpc pairs acting as root bridge of an STP domain (Not needed on FabricPath edge) Allows ongoing STP processing without a root bridge transition in the event of a switch failure. STP configuration and priority settings must be identical on both peer switches vpc Peer-link operates in forwarding state for all vpc VLANs Physical representation vpc Primary vpc Secondary Logical representation vpc Primary vpc Secondary S1 Root vpc Peer-link Peer-switch Root S2 S1 Root Root Peer-switch S2 vpc1 vpc2 S,0,S 29

vpc Options: vpc Peer Gateway Non-RFC compliant end hosts: The vpc peer-gateway functionality allows a vpc switch to act as the active gateway for packets that are addressed to the router

vpc Peer Gateway Feature Allows a vpc peer to respond to both the HSRP virtual and the real MAC address of both itself and it s peer Physical IP A Physical MAC A Virtual IP Virtual MAC

30 vpc Options: vpc Peer Gateway Non-RFC compliant end hosts: The vpc peer-gateway functionality allows a vpc switch to act as the active gateway for packets that are addressed to the router physical MAC address of the vpc peer. Some non-compliant devices use the MAC address of the sender device (Switch physical MAC instead of virtual MAC) Certain NAS devices (i.e. NETAPP Fast-Path or EMC IP-Reflect) have been found to do this. vpc Peer Gateway Feature Allows a vpc peer to respond to both the HSRP virtual and the real MAC address of both itself and it s peer Physical IP A Physical MAC A Virtual IP Virtual MAC Physical MAC B Switch A Physical IP B Physical MAC B Virtual IP Virtual MAC Physical MAC A Switch B Layer 2/3 Layer-2 Access VLAN 100 VLAN

31 vpc Options: ip arp synchronize When the peer-link connection is first established, perform an ARP bulk-sync using CFS over Ethernet to the peer switch Improves convergence times for layer 3 flows after recovery of a peer relationship ARP TABLE IP1 MAC1 VLAN 100 IP2 MAC2 VLAN 200 P SVIs S ARP TABLE IP1 MAC1 VLAN 100 IP2 MAC2 VLAN 200 S1(config-vpc-domain)# ip arp synchronize S2(config-vpc-domain)# ip arp synchronize IP1 MAC1 IP2 MAC2 P S Primary vpc Secondary vpc 31

32 Routing Protocol Peering between vpc Peers Deployment specifics for Nexus Switches Nexus 5000/6000 series only support using a VLAN over the vpc Peer-Link Do not provision a separate physical link for router peering on Nexus 5000/6000 Nexus 7000 series allow use of a separate physical port channel for Layer-3 Peering This is optional and can provide greater control of behavior for service integrations Layer-3 Links Layer-3 Core Layer-3 Links Layer-3 Core L3 SVI s with VLAN over Shared Peer Link Nexus 5/6000 Layer 2/3 Separate L3 Physical Port-Channel Nexus 7000 Layer 2/3 Nexus Access Layer-2 Nexus Access Layer-2 32

33 Agenda Introduction Data Center Requirements for Midsize Organizations Building an Access Pod Single-Layer Design Examples Moving to a Spine/Leaf Fabric Conclusion 33

Designing Switching with Oversubscription: Balancing Cost and Performance Oversubscription: Most servers will not be consistently filling a 10 GigE interface.

34 Designing Switching with Oversubscription: Balancing Cost and Performance Oversubscription: Most servers will not be consistently filling a 10 GigE interface. A switch may be a line-rate non-blocking device, but still introduce oversubscription into an overall topology by design. Consider Ethernet-based storage traffic when planning ratios, keep plans on the conservative side. Example device numbers assuming all ports connected: Nexus 5672UP: 48x10Gig + 6x40Gig uplink = 48:24 or 2:1 oversubscription Nexus 2232PP FEX: 32x10Gig + 8x10Gig uplink = 32:8 or 4:1 oversubscription Actual oversubscription can be controlled by how many ports and uplinks are physically connected. 3:1 4:1 Spine Leaf FEX Servers 34

35 Value of FabricPath/vPC+ in Spine/Leaf Designs Adding FabricPath to a traditional physical DC Topology vpc becomes vpc+ when used at the edge of a FabricPath network, the Peer Link also runs FabricPath. FabricPath Benefits: FabricPath Spine Topology flexibility beyond the vpc limitation of using switches in pairs. Leaf Ease of configuration. Completely eliminates STP from running between Leaf and Spine. vpc+ FEX No Orphan Port isolation on Leaf switch vpc Peer-link loss. Improved Multicast and routing support with vpc+. VM VM VM UCS Rack Servers 35

VXLAN Overlay Encapsulation Dynamic network segmentation across traditional Layer-3 boundaries Overlay encapsulations allow fabric segmentation beyond VLAN limits

Nexus 9000, 7000-F3, 6000X, and 5600 platforms support VXLAN in hardware.

36 VXLAN Overlay Encapsulation Dynamic network segmentation across traditional Layer-3 boundaries Overlay encapsulations allow fabric segmentation beyond VLAN limits for greater flexibility and scale. Software-only VXLAN implementations can provide Layer-2 workload mobility, but with limited visibility into the physical network. Nexus 9000, 7000-F3, 6000X, and 5600 platforms support VXLAN in hardware. An optimal control plane will utilize the benefits of VXLAN encapsulation, while integrating directly with the underlying physical network. VTEP VTEP VTEP VTEP Outer MAC Header Outer IP Header Outer UDP Header VXLAN Header Original Ethernet Frame FCS 36

37 Dynamic Fabric Automation (DFA) Modular building blocks for migration to an automated fabric Integration with cloud orchestration platforms and supports dynamic workload mobility. Provides a distributed default gateway in the leaf layer to handle traffic to and from any subnet or VLAN. Implements segment-id in frame header to eliminate hard VLAN scale limits and support multi-tenancy. Provides central point of fabric management (CPOM) for network, virtual-fabric and host visibility. Auto-configuration of new switches to expand the fabric using POAP, also provides cable plan consistency checking. Spine Nexus 7k,6k Leaf Nexus 7k, 6k, 5k DFA Central Point of Management DCNM DCNM DFA Fabric Client Access Border-Leaf Nexus 7k, 6k WAN / DCI 37

38 Application Centric Infrastructure (ACI) APIC controller-managed fabric based on Nexus 9000 hardware innovations Centralized provisioning and abstraction layer for control of the switching fabric. Application Policy Infrastructure Controller Client Access WAN / DCI Simplified automation with an applicationdriven policy model. APIC APIC APIC Controller provides policy to switches in the fabric but is not in the forwarding path. Normalizes traffic to a VXLAN encapsulation with Layer-3 Gateway and optimized forwarding. ACI Spine ACI Fabric Decouples endpoint identity, location, and policy from the underlying topology. Provides for service insertion and redirection. ACI Leaf Border-Leaf Nexus

39 Spine/Leaf Data Center and Dual-tier Switching Design Examples

40 Migration from single-layer to spine/leaf fabric Larger switches more likely to becoming spine (or aggregation) layer. Smaller switches more likely to becoming leaf/access layer. Nexus 7000, 6004 or 9500 Single Layer Spine Layer-3 gateway can migrate to spine switches or to border-leaf switch pair. Leaf Spine switches can support leaf switch connections, plus some FEX and directattached servers during migration. Nexus 5000 or 9300 Single Layer Spine/Leaf Data Center Fabric 40

41 Expanded Spine/Leaf Nexus Data Center Fabric Introduction of Spine layer, and FabricPath forwarding Data Center switching control plane distributed over Dual Layers. Spine: FabricPath switch-id based forwarding, but also providing Layer-3 and service integration. Leaf: Physical TOR switching or FEX aggregation for multiple racks. Multi-hop FCoE with dedicated links. Example Components: 2 x Nexus 6004, 2 x Nexus 5672UP Layer-3 and Storage Licensing 12 x Nexus 2232PP/TM-E FabricPath enabled between tiers for configuration simplicity and future expansion. L L2 FabricPath Forwarding WAN /DCI 10 or 1-Gig attached UCS C-Series Nexus 6004 Spine Nexus 5600 Leaf FCoE iscsi / NAS FC 41

42 Adding Access Pods to Grow the Fabric Modular expansion with added leaf-switch access pods Data Center switching control plane distributed over Dual Layers. Spine: FabricPath switch-id based forwarding, but also providing Layer-3 and service integration. Leaf: Physical TOR switching or FEX aggregation for multiple racks. Multi-hop FCoE with dedicated links. Example Components: 2 x Nexus 6004, 4 x Nexus 5672UP Layer-3 and Storage Licensing 24 x Nexus 2232PP/TM-E FabricPath enabled between tiers for configuration simplicity and future expansion. Nexus 5600 Leaf L L2 Rack Server Access with FEX WAN /DCI Nexus 6004 Spine Rack Server Access with FEX FCoE iscsi / NAS FC 42

43 Modular, High Availability Data Center Fabric Virtual Device Contexts partitioning the physical switch WAN Nexus 7700 FabricPath Spine, 5672UP Leaf High Availability spine-switching design with dualsupervisor. VDCs allow OTV and Storage functions to be partitioned on common hardware. Add leaf pairs for greater end node connectivity. Add spine nodes for greater fabric scale and HA. FCoE support over dedicated links and VDC. L L2 Core VDC Spine VDC OTV VDC Storage VDC FCoE iscsi / NAS Specific Nexus features utilized: Integrated DCI support with OTV, LISP, MPLS, and VPLS. Feature-rich switching fabric with FEX, vpc, FabricPath, FCoE. Investment protection of a chassis-based switch. Rack Server Access with FEX Rack Server Access with FEX 43

44 FabricPath with vpc+ Best Practices Summary Manually assign FabricPath physical switch ID s to easily identify switches for operational support. Configure all leaf switches with STP root priority, or use pseudo-priority to control STP. Ensure all access VLANs are mode fabricpath to allow forwarding over the vpc+ peer-link which is a FabricPath link. Use vpc+ at the Layer-3 gateway pair to provide active/active HSRP, or use Anycast HSRP. Set FabricPath root-priority on the Spine switches for multi-destination trees. Enable overload-bit under FabricPath domain to delay switch forwarding state on insertion into fabric set-overload-bit on-startup <seconds> L L2 FabricPath SW-ID: 101 VPC+ Domain 10 FabricPath SW-ID: 102 VPC+ Domain

Dual-Layer, ACI-Ready Data Center NX-OS Standalone mode with vpc NX-OS standalone model allows deployment of vpc multi-tier switching architecture without using the APIC controller.

45 Dual-Layer, ACI-Ready Data Center NX-OS Standalone mode with vpc NX-OS standalone model allows deployment of vpc multi-tier switching architecture without using the APIC controller. Allows expansion beyond a single pair of leaf switches with a 40GigE fabric using hardware that is ACI-ready. vpc is used for connectivity between switching layers in a traditional Aggregation/Access topology. Design Notes: Layer-3 connectivity and services would move to designated leaf switches in an ACI fabric. IP-based storage now, future support of FCoE. L L2 Client Access Rack Servers/FEX WAN / DCI UCS Blade System Nexus 9500 iscsi / NAS Nexus 9396PX Leaf 45

46 Data Center Fabric Provisioning and Operations Evolving toolsets and platforms to utilize based on YOUR requirements. Open Programmability NX-API/JSON OnePK: Python Java, C Automated and Optimized Networking DDNM DCNM OpenDaylight Controller Application Centric Infrastructure Deployment APIC OpFlex Policy Protocol Nexus Switching Fabric Designs 46

47 Summary: Scalable Midsize Data Center Designs Midsize Data Centers can benefit from the same technology advances as larger ones. Smaller Nexus platforms allow building feature-rich fabrics at an entry-level scale. Example designs allow future migration to controller-based fabric solutions such as DFA, and ACI. Both direct programmability and controllerbased options are available for all Nexus switching platforms. Plan ahead for re-use of components in new roles as needs change. 47

48 Related Sessions Session-ID Session Name BRKDCT-3445 Building Scalable Networks with NX-OS and Nexus 7000 BRKDCT-2081 BRKDCT-2121 BRKDCT-2334 BRKDCT-2385 BRKDCT-2404 Cisco FabricPath Technology and Design Virtual Device Context (VDC) Design and Implementation Real World Data Center Deployments and Best Practice Session Cisco Dynamic Fabric Automation Architecture VXLAN Deployment Models BRKDCT-2000 BRKDCT-2006 Introduction to Application Centric Infrastructure Integration of Hypervisors and L4-7 Services into an ACI Fabric 48

49 Complete Your Online Session Evaluation Give us your feedback and you could win fabulous prizes. Winners announced daily. Complete your session evaluation through the Cisco Live mobile app or visit one of the interactive kiosks located throughout the convention center. Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online 49

50 Continue Your Education Demos in the Cisco Campus Walk-in Self-Paced Labs Table Topics Meet the Engineer 1:1 meetings 50

Scalable Midsize Data Centre Designs

Scalable Midsize Data Centre Designs James Gilarte Virtual Systems Engineer 3 4 House Keeping Rules This is an interactive session. Feel free to ask questions I will repeat them for the audience You can