UCS Networking Deep Dive

Size: px

Start display at page:

Download "UCS Networking Deep Dive"

Elfrieda Cobb
5 years ago
Views:

1 UCS Networking Deep Dive BRKCOM

2 Agenda Overview / System Architecture Physical Architecture Logical Architecture Switching Modes of the Fabric Interconnect Fabric Failover Ethernet Switching Modes Recommendations SAN / LAN Northbound Connections FC/FCoE Storage direct attach, NAS appliance port Port Channel and vpc uplinks, traffic flows and failures Adapter Offerings UCS Generation-2 Hardware 2

3 Overview

4 Unified Computing System (UCS) Unified Fabric Single Point of Management Stateless Servers with Virtualised Adapters 4

card UCS Blade Server Chassis Flexible bay configurations UCS Blade or Rack

5 UCS Building Blocks UCS Manager Embedded manages entire system UCS Fabric Interconnect 20 Port 0Gb FCoE 40 Port 0Gb FCoE UCS Fabric Extender Remote line card UCS Blade Server Chassis Flexible bay configurations UCS Blade or Rack Server Industry-standard architecture UCS Virtual Adapters Choice of multiple adapters 5

6 Cisco UCS Networking: Physical Architecture SAN A ETH ETH 2 SAN B MGMT MGMT Uplink Ports OOB Mgmt Fabric Switch Server Ports 600 Fabric A Cluster 600 Fabric B Fabric Extenders I O M I O M I O M Chassis Chassis 20 I O M Virtualised Adapters Compute Blades Half / Full width A CNA B200 B A B250 6 CNA CNA B

7 Cisco UCS Networking: Physical Architecture SAN A ETH ETH 2 SAN B MGMT MGMT Uplink Ports OOB Mgmt Fabric Switch Server Ports 600 Fabric A Cluster 600 Fabric B Fabric Extenders Virtualised Adapters Compute Blades Half / Full width I O M A Chassis CNA B200 I O M B VIC FEX FEX Rack Mount Management Plane 7

8 Network Interface Virtualisation (NIV) vnic (LIF) Host presented PCI device managed by UCSM. VIF Policy application point where a vnic connects to UCS fabric VNTag An id appended to the packet which contains the source and destination ID used for switching within the UCS fabric. Fabric Interconnect vhba IOM veth Adapter vnic 8 vfc Service Profile (Server) Blade Cable Virtual Cable (VNTag)

600-A Abstracting the Logical Architecture What you see

Adapter What you get 600-A vfc veth Dynamic, Rapid

9 600-A Abstracting the Logical Architecture What you see Switch Eth / IOM A Cable 0GE A 600-A vfc IOM A 0GE A veth Adapter What you get 600-A vfc veth Dynamic, Rapid Provisioning State abstraction Location Independence Blade or Rack Adapter Blade vhba vnic Service Profile (Server) Blade vhba (Server) vnic Physical Cable Virtual Cable (VN-Tag) 9

10 Hardware Components

11 UCS 600 Hardware Architecture

204-IOM Architecture Components Woodside ASIC Aggregates traffic to/from 32 hostfacing 0G Ethernet ports from/to 8 network-facing 0G Ethernet ports

Woodside Interfaces HIF (Backplane ports) NIF (FabricPorts) BIF CIF Control IO - 4 Fabric Ports to Interconnect Redwood ASIC L2 Switch Aggregates

12 204-IOM Architecture Components Woodside ASIC Aggregates traffic to/from 32 hostfacing 0G Ethernet ports from/to 8 network-facing 0G Ethernet ports CPU (also referred to as CMC) Controls Redwood and perform other chassis management functionality FLASH DRAM EEPROM Chassis Management Controller Woodside Interfaces HIF (Backplane ports) NIF (FabricPorts) BIF CIF Control IO - 4 Fabric Ports to Interconnect Redwood ASIC L2 Switch Aggregates traffic from BMCs on the server blades No local switching All traffic from HIFs goes upstream for Switching Chassis Signals Switch 8 Backplane Ports to Blades 2

13 UCS IOM Slot to Uplink Pinning Number of Active Fabric Links -Link 2-Link 4-Link Blades pinned to fabric link All the HIF ports pinned to the active link,3,5,7 to link- 2,4,6,8 to link-2,5 to link- 2,6 to link-2 3,7 to link-3 4,8 to link-4 HIFs are statically pinned by the system to individual fabric ports. Only,2,4 links are supported for pinning. 3 is not a valid pinning configuration. On a link failure, only blades pinned to the NIF are brought down 3

14 Blade Blade 2 Blade 3 Blade 4 Blade 5 Blade 6 Blade 7 Blade 8 Server Ports Fabric Ports Individual Links Static Pinning (IOM-FI) Static Pinning done by the system dependent on number of fabric ports,2 4 (2^x) are valid links for initial pinning IOM Fabric Interconnect 4

15 Blade Blade 2 Blade 3 Blade 4 Blade 5 Blade 6 Blade 7 Blade 8 Server Ports Fabric Ports Individual Links Fabric Port Failure Pinned HIFs are brought down Fabric Interconnect Other blades unaffected IOM 5

16 Blade Blade 2 Blade 3 Blade 4 Blade 5 Blade 6 Server Ports Fabric Ports Blade 7 Blade 8 Unused Link Individual Links Re-ack of chassis Blades re-pinned to valid number of links,2 or 4 HIF s brought down/up for repinning May result in unused links Addition of links requires reack of chassis. IOM Fabric Interconnect 6

17 Ethernet Switching Modes

Switching Modes: End Host Spanning Tree 600 A Fabric A VNIC 0 Server 2 VLAN 0 L2 Switching LAN veth 3 veth VNIC 0 Server MAC Learning MAC Learning Server vnic pinned to an Uplink port No Spanning

18 Switching Modes: End Host Spanning Tree 600 A Fabric A VNIC 0 Server 2 VLAN 0 L2 Switching LAN veth 3 veth VNIC 0 Server MAC Learning MAC Learning Server vnic pinned to an Uplink port No Spanning Tree Protocol Reduces CPU load on upstream switches Reduces Control Plane load on 600 Simplified upstream connectivity UCS connects to the LAN like a Server, not like a Switch Maintains MAC table for Servers only Eases MAC Table sizing in the Access Layer Allows Multiple Active Uplinks per VLAN Doubles effective bandwidth vs STP Prevents Loops by preventing Uplink-to- Uplink switching Completely transparent to upstream LAN Traffic on same VLAN switched locally 8

traffic is forwarded to server only if it arrives on pinned uplink port.

19 End Host Mode Unicast Forwarding Server to server traffic on the same VLAN is locally switched Uplink port to Uplink port traffic not switched Each server link is pinned to an uplink port / port-channel Uplink Ports 600 LAN Deja-Vu Server 2 RPF Network to server unicast traffic is forwarded to server only if it arrives on pinned uplink port. This is termed as the Reverse Path Forwarding (RPF) check VLAN 0 veth veth 3 Packet with source MAC belonging to a server received on an uplink port is dropped (Deja-Vu Check) VNIC 0 Server 2 VNIC 0 Server 9

20 End Host Mode Multicast Forwarding Broadcast traffic is pinned on exactly one uplink port (or port-channel) i.e., it is dropped when received on other uplinks All multicast groups are pinned to same uplink port (port-channel) Server to server multicast traffic is locally switched RPF and deja-vu check also applies for multicast traffic Uplink Ports 600 B veth veth 3 B VNIC 0 LAN Broadcast Listener All VLANs B VNIC 0 Server 2 Server 20

21 Switching Modes: Switch 600 A MAC Fabric A VNIC 0 Server 2 VLAN 0 L2 Switching Root LAN veth 3 veth VNIC 0 Server Learning Fabric Interconnect behaves like a normal Layer 2 switch Server vnic traffic follows VLAN forwarding Spanning tree protocol is run on the uplink ports per VLAN Rapid PVST+ Configuration of STP parameters (bridge priority, Hello Timers etc) not supported VTP is not supported currently MAC learning/aging happens on both the server and uplink ports like in a typical Layer 2 switch Upstream links are blocked per VLAN via Spanning Tree logic 2

22 Fabric Failover

23 Fabric Failover Fabric provides NIC failover capabilities chosen when defining a service profile veth L 600-A L2 L2 600-B L veth Traditionally done using NIC bonding driver in the OS Provides failover for both unicast and multicast traffic Works for any OS. Physical Cable Virtual Cable IOM 0GE PHY Adapter Cisco VIC M8KR Menlo M7KR vnic 0GE IOM VIRT Adapter OS / Hypervisor / VM 23

24 Fabric Failover with Bare Metal OS Slam Dunk: Cisco UCS simplifies the redundancy No OS NIC Teaming configuration required Simple single NIC design Fabric failures hidden from OS NIC stays UP L L 600-A L2 L2 600-B MAC A Server w/ Cisco VIC or Menlo vnic MAC A MAC A 600 sends gratuitous ARP Everything to gain WINDOWS / LINUX Nothing to lose Implicit MAC Learned MAC 24

Fabric Failover with HYPER-V Slam Dunk: Cisco UCS provides the missing redundancy L L 600-A L2 L2 600-B MAC A MAC C MAC B Server w/ Cisco VIC or Menlo vnic HYPER V soft switch VM MAC

25 Fabric Failover with HYPER-V Slam Dunk: Cisco UCS provides the missing redundancy L L 600-A L2 L2 600-B MAC A MAC C MAC B Server w/ Cisco VIC or Menlo vnic HYPER V soft switch VM MAC A MAC C VM MAC B MAC C MAC A MAC B Hyper-V soft switch only uses () NIC Fabric Failover provides the missing redundancy Everything to gain Nothing to lose Implicit MAC Learned MAC 25

26 Fabric Failover with Hypervisor Pass-Through Default setting Cannot be disabled L L 600-A L2 L2 600-B MAC B MAC B MAC A MAC A Single adapter per VM with Redundancy Load Sharing Alternating fabric Round Robin Server w/ Cisco VIC Dynamic vnic VM-FEX Dynamic vnic 2 VM MAC A VM 2 MAC B Implicit MAC Learned MAC 26

27 Ethernet Switching Modes Recommendations

Scalability Spanning Tree protocol is not run in EHM hence control plane is unoccupied EHM is least disruptive to upstream network BPDU Filter/Guard,

28 Scalability Spanning Tree protocol is not run in EHM hence control plane is unoccupied EHM is least disruptive to upstream network BPDU Filter/Guard, Portfast enabled upstream MAC learning does not happen in EHM on uplink ports. Current MAC address limitation on the 600 ~4.5K. Recommendation: End Host Mode 28

29 Pinning for Deterministic Traffic Flows Dynamic pinning Server ports pinned to an uplink port/port-channel automatically Static pinning Specific pingroups created and associated with adapters Static pinning allows traffic management if required for certain applications / servers A veth 3 veth Fabric A VNIC 0 VNIC 0 Server X Oracle DEFINED: PinGroup Oracle APPLIED: PinGroup Oracle Pinning Switching 29

30 Fabric Failover Fabric Failover is only applicable in EHM. NIC teaming software required to provide failover in Switch mode. 600-A L2 L2 600-B Physical Cable Virtual Cable veth IOM 0G E L L 0G E IOM veth PHY Adapter Cisco VIC M8KR Menlo M7KR vnic VIRT Adapter OS / Hypervisor / VM 30

31 Active/Active use of Uplinks End Host Mode Switch Mode Primary Root Secondary Root Primary Root Secondary Root LAN LAN Active/Active Border Ports Border Ports Blocking FI-A FI-B FI-A FI-B Server Ports Server Ports Recommendation: End Host Mode 3

32 Disjoint L2 Upstream Backup Production EHM built on the premise that the L2 upstream is NOT disjoint. Incoming broadcast/multicast received only on uplink for ALL VLANs External LAN Border Ports Designated Bcast Receiver Fabric InterConnect Recommendation: Switch Mode 32

33 Application Specific Scenarios Certain application like MS-NLB (Unicast mode) have the need for unknown unicast flooding which is not done in EHM Certain network topologies provide better network path out of the Fabric Interconnect due to STP placement and HSRP L3 hop. Switch Mode is catch all for different scenarios. Recommendation: Switch Mode 33

34 Upstream Connectivity - Storage

FLOGI FDISC SAN End Host NPV Mode N-Port Virtualisation Forwarding SAN A N_Proxy 600-A 600-B SAN B vfc vfc 2 vfc vfc 2 vhba 0 NPIV VSAN F_Proxy N_Port Server VSAN F_Port vhba NPIV F_Proxy vhba 0

35 FLOGI FDISC SAN End Host NPV Mode N-Port Virtualisation Forwarding SAN A N_Proxy 600-A 600-B SAN B vfc vfc 2 vfc vfc 2 vhba 0 NPIV VSAN F_Proxy N_Port Server VSAN F_Port vhba NPIV F_Proxy vhba 0 F_Port VSAN N_Port N_Proxy vhba Server 2 VSAN Fabric Interconnect operates in N_Port Proxy mode (not FC Switch mode) simplifying managment SAN switch sees Fabric Interconnect as an FC End Host with many N_Ports and many FC IDs assigned Server facing ports function as F-proxy ports Server vhba pinned to an FC uplink in the same VSAN. Round Robin selection. Provides multiple FC end nodes to one F_Port off an FC Switch Eliminates the FC domain on UCS Fabric Interconnect One VSAN per F_port (multi-vendor) F_Port Trunking and Channeling with MDS, 5K 35

36 SAN End Host NPV Mode N-Port Virtualisation Forwarding with MDS, Nexus 5000 F_ Port Channel & Trunk SAN A NPIV VSAN,2 N_Proxy 600-A 600-B SAN B vfc vfc 2 vfc vfc 2 F_Proxy F_Port NPIV VSAN,2 F_Port Channeling and Trunking from MDS or Nexus 5000 to UCS FC Port Channel behaves as one logical uplink FC Port Channel can carry all VSANs (Trunk) UCS Fabric Interconnects remains in NPV end host mode Server vhba pinned to an FC Port Channel vhba 0 N_Port vhba Server VSAN vhba 0 vhba Server 2 VSAN 2 Server vhba has access to bandwidth on any link member of the FC Port Channel Load balancing based on FC Exchange_ID Per Flow 36

37 SAN FC Switch Mode Direct Attach FC & FCoE Storage to UCS UCS Fabric Interconnect behaves like an FC fabric switch FC FCoE SAN Storage ports can be FC or FCoE Light subset of FC Switching features Select Storage ports Set VSAN on Storage ports Default zoning per VSAN No zoning configuration inputs in UCSM MDS MDS N_Port VSAN VSAN 2 TE_Port F_Port 600-A FC Switch 600-B FC Switch vfc vfc 2 vfc vfc 2 F_Port Connection to an external FC switch is reqd: Zoning configured and pushed to UCS from MDS Fabric Interconnect uses a FC Domain ID vhba 0 N_Port vhba Server VSAN vhba 0 vhba Server 2 VSAN 2 37

Direct Attach IP Storage to UCS NAS direct attached to Appliance port Active/Standby Appliance Port A U A U 600-A 600-B veth veth 2 veth veth 2 vnic 0 Volume A NAS C vnic

Controller head Each NAS controller port owns the I/O for a given Volume Other port provides failover Controller interfaces active/standby for a given volume when attached to

38 Direct Attach IP Storage to UCS NAS direct attached to Appliance port Active/Standby Appliance Port A U A U 600-A 600-B veth veth 2 veth veth 2 vnic 0 Volume A NAS C vnic Server Accessing Volume A Volume B vnic 0 LAN vnic Server 2 Accessing Volume A Uplink Port IP Storage attached to Appliance Port NFS, iscsi NAS appliance with single Controller head Each NAS controller port owns the I/O for a given Volume Other port provides failover Controller interfaces active/standby for a given volume when attached to separate 600s Controller interfaces Active/Active when each handling their own volumes Sub-optimal forwarding possible if not careful Insure vnics are accessing Volumes local to its fabric 38

39 Upstream Connectivity - Ethernet

End Host Mode Individual Uplinks Dynamic Re-pinning of failed uplinks 600 A Fabric A All uplinks forwarding for all VLANs GARP aided upstream convergence No STP Sub-second re-pinning

40 End Host Mode Individual Uplinks Dynamic Re-pinning of failed uplinks 600 A Fabric A All uplinks forwarding for all VLANs GARP aided upstream convergence No STP Sub-second re-pinning No server NIC disruption veth 3 VLAN 0 Sub-second re-pinning L2 Switching VNIC 0 veth VNIC stays up VNIC 0 Server 2 MAC A Pinning Switching vswitch / NK ESX HOST VM VM 2 MAC B MAC C 40

41 End Host Mode Port Channel Uplinks Recommended: Port Channel Uplinks No disruption No GARPs needed 600 A Fabric A More Bandwidth per Uplink Per flow uplink diversity No Server NIC disruption Fewer GARPs needed Faster bi-directional convergence Fewer moving parts RECOMMENDED veth 3 Sub-second convergence VLAN 0 L2 Switching VNIC 0 veth NIC stays up Server 2 VNIC 0 MAC A Pinning Switching vswitch / NK ESX HOST VM VM 2 MAC B MAC C 4

vpc Domain More Bandwidth per Uplink No Server NIC disruption Switch and Link resiliency Per flow uplink

42 End Host Mode vpc Uplinks vpc uplinks hide uplink & switch failures from Server VNICs No disruption No GARPs Needed! vpc Domain More Bandwidth per Uplink No Server NIC disruption Switch and Link resiliency Per flow uplink diversity No GARPs Faster Bi-directional convergence Fewer moving parts vpc RECOMMENDED 600 A Fabric A veth 3 VLAN 0 L2 Switching VNIC 0 veth NIC stays up VNIC 0 Server 2 Pinning Switching vswitch / NK ESX HOST VM VM 2 MAC B MAC C 42

43 Inter-Fabric Traffic Example () EHM 600 A 600 B Dynamic VNIC VM L2 Switching PTS ESX HOST EHM VNIC 2 VM 2 Cisco UCS VM-FEX Dynamic Dynamic VNIC Primary fabric A Backup fabric B Dynamic VNIC2 Primary fabric B Backup fabric A VM on VLAN 0 VM2 on VLAN 0 VM to VM2 traffic: ) Leaves Fabric A 2) Gets L2 switched 3) Enters Fabric B 43

44 Inter-Fabric Traffic Example (2) VNIC 0 on Fabric A VNIC on Fabric B VM Pinned to VNIC0 VM4 Pinned to VNIC VM on VLAN 0 VM4 on VLAN 0 EHM 600 A L2 Switching EHM 600 B VM to VM4: ) Leaves Fabric A 2) L2 switched upstream 3) Enters Fabric B VNIC 0 VM ESX HOST vswitch / NK Mac Pinning VNIC VM 2 VNIC 0 VM 3 ESX HOST 2 vswitch / NK Mac Pinning VNIC VM 4 44

45 Singly Attached Uplinks 7K 7K2 EHM 600 A 600 B EHM. Traffic destined for a vnic on the Red Uplink enters 7K 2. Same scenario vice-versa for Green 3. All Inter-Fabric traffic traverses Nexus 7000 peer link 45

46 Recommended Topology vpc uplinks to L3 aggregation switch 7K vpc Domain keepalive 7K2 vpc peer-link EHM 600 A 600 B EHM 46

47 Recommended Topology w/o vpc With 4 x 0G (or more) uplinks per 600 Port Channels Connect 600 s with End Host Mode to Aggregation L3 switch EHM 600 A 600 B EHM All UCS uplinks forwarding No STP influence on the topology End Host Mode 47

48 Adapter Offerings

Failover 0GbE/FCoE Support for native drivers and utilities 2w

49 Gen Compatibility Adapters NIC and HBA ASICs from: Qlogic, Emulex, Intel Dual 0GbE/FCoE ports Cisco Menlo ASIC IEEE DCB VN-Tag Fabric Failover 0GbE/FCoE Support for native drivers and utilities 2w Customer certified stacks 0GbE FC 4Gbps FC PCIe Bus vnic Fabric Failover 49

50 Gen2 Compatibility Adapters Emulex M72KR-E (CNA) QLogic M72KR-Q (CNA) Single Emulex ASIC design Low power 3w 8 Gbps FC Emulex drivers, Eth & FC Single QLogic ASIC design Lowest power 4.5w 8 Gbps FC QLogic drivers, Eth & FC No vnic Fabric Failover No vnic Fabric Failover 50

51 Gen2 Cost Adapters Broadcom BCM577 Intel M6KR-I iscsi acceleration iscsi offload (HBA) iscsi boot (future) TCP offload engine (TOE) Low cost 0GE No vnic Fabric Failover SR-IOV compatible iscsi acceleration PXE Boot VMDq IEEE DCB FCoE software (future) No vnic Fabric Failover 5

52 Cisco Virtual Interface Card (VIC) (Palo) Converged Network Adapter FCoE in hardware Single-OS and VM deployments Virtualise in hardware PCIe compliant Up to 58 distinct PCIe devices (H/W capable of 28) Ethernet vnic and FC vhba 2 nd Tier Fabric Extender User Definable vnics Eth 0 FC 2 0GbE/FCoE FC 3 Eth 58 8w PCIe x6 52

Cisco Virtual Interface Card (VIC) (Palo) For virtualisation environments Bypass vswitch to deliver VN-Link in hardware Tight integration with VMware

53 Cisco Virtual Interface Card (VIC) (Palo) For virtualisation environments Bypass vswitch to deliver VN-Link in hardware Tight integration with VMware vcenter vnic as Hardware DVS-port QoS (8) COS based queues vnic bandwidth guarantees User Definable vnics Eth 0 FC 2 0GbE/FCoE FC 3 Eth 58 8w PCIe x6 53

54 Cisco VIC: VM-FEX Logical View vfc Fabric Interconnect A veth veth 2 veth 3 veth 4 veth 5 Fabric Interconnect B veth 6 veth 7 veth 8 vfc 2 IOM A IOM B Cisco VIC vhba 0 VM VM 2 VM 3 VM 4 VM 5 VM 6 VM 7 VM 50 vhba 54

55 UCS Generation - 2 Hardware

56 End Host End Host Next Gen UCS Components and Capabilities Additions to the UCS Fabric Portfolio 6248UP Fabric Interconnect 2x Fabric Capacity 40% Latency Reduction Unified Ports 2208XP IO Module VIC 280 2x Blade Chassis Bandwidth 60 Gb/s per Chassis 4x Blade Server Bandwidth Dual 40 Gb/s per Card 256 virtual Interfaces Production Vlan 0,20 UCS Version 2.0 Platform Features Public Vlan 3,32 Backup Vlan 40,4 L2 Disjoint Networks More Flexible Designs Reduced Networking HW iscsi Boot Support in UCSM Increased Customer Choice VM-FEX for RedHat KVM Additional Hypervisor Support 56

Unified Computing Fabric Feature details PID:

UCS 6248UP chassis comes with 32 fixed Unified

Ports Dual power supplies standard for both AC

57 UCS 6248UP Fabric Interconnect Customer benefits Highest density & performance for Unified Computing Fabric Feature details PID: UCS-FI-6248UP Double the port density in RU UCS 6248UP chassis comes with 32 fixed Unified Ports and expansion module slot 6 Unified Ports Dual power supplies standard for both AC (at FCS) and DC -48V (July) Redundant front to back airflow 57

specific protocol bandwidth bottlenecks Use-cases Flexible LAN & storage convergence based on business

58 UCS 6248: Unified Ports Dynamic Port Allocation: Lossless Ethernet or Fibre Channel FC Native Fibre Channel Benefits Simplify switch purchase - remove ports ratio guess work Increase design flexibility Remove specific protocol bandwidth bottlenecks Use-cases Flexible LAN & storage convergence based on business needs Service can be adjusted based on the demand for specific traffic 58 Eth Lossless Ethernet: /0GbE, FCoE, iscsi, NAS

59 UCS 6248: Unified Ports Dynamic Port Allocation: Lossless Ethernet or Fibre Channel Ports on the base card or the Unified Port GEM Module can either be Ethernet or FC Only a continuous set of ports can be configured as Ethernet or FC Ethernet Ports have to be the st set of ports Port type changes take effect after next reboot of switch for Base board ports or power-off/on of the GEM for GEM unified ports. Base card 32 Unified Ports GEM 6 Unified Ports Eth FC Eth FC 59

slot* *Requires VIC 280 for full server bandwidth Increased support for 8 egress CoS queues Lower latency Customer

60 UCS 2208XP I/O Module Feature details PID: UCS-IOM-2208XP Double the uplinks 8x 0GE uplinks from each IOM/FEX Total 60 Gbps per chassis Quadruple the downlinks 32x 0GE or 4x 0GE from each IOM/FEX to each blade slot* Total: 80 Gbps per server slot* *Requires VIC 280 for full server bandwidth Increased support for 8 egress CoS queues Lower latency Customer benefits Double the uplink bandwidth to the Fabric Quadruple the downlink bandwidth to the server slots Lower latency and better QoS 60

61 FAIL OK UCS 508 N0-PAC-550W SLOT SLOT 3 SLOT 5 SLOT 7 PS PS2 FAIL OK N0-PAC-550W FAN STAT FAN FAN2 FAN STAT STAT FAIL OK N0-PAC-550W OK FAIL OK FAIL OK FAIL OK FAIL PS PS2 FAIL OK N0-PAC-550W FAN STAT FAN FAN2! SLOT 2 SLOT 4 SLOT 6 SLOT 8 FAN STAT STAT FAIL OK UCS 508 N0-PAC-550W SLOT SLOT 3 SLOT 5 SLOT 7 PS PS2 FAIL OK N0-PAC-550W FAN STAT FAN FAN2 FAN STAT STAT FAIL OK N0-PAC-550W OK FAIL OK FAIL OK FAIL OK FAIL PS PS2 FAIL OK N0-PAC-550W FAN STAT FAN FAN2! SLOT 2 SLOT 4 SLOT 6 SLOT 8 FAN STAT STAT IOM to Fabric Interconnect Port Pinning Server-to-Fabric Port Pinning Configurations 60 Gb (Discrete Mode) 60 Gb (Port Channel Mode) Slot Slot 2 Slot 3 Slot 4 Slot 5 Slot 6 Slot 7 Slot 8 Slot Slot 2 Slot 3 Slot 4 Slot 5 Slot 6 Slot 7 Slot to to to

UCS 280 VIC Customer benefits Dual 4x 0 GE (80 Gb per host) UCS 2208 IOM UCS 2208 IOM Feature details Dual 4x 0 GE port-channels to a single server slot Host

both M8KR and 280 VIC FabricFailover supported Eth hash inputs : Source MAC Address,Destination MAC Address,Source Pprt, Destination Port,Source IP

62 UCS 280 VIC Customer benefits Dual 4x 0 GE (80 Gb per host) UCS 2208 IOM UCS 2208 IOM Feature details Dual 4x 0 GE port-channels to a single server slot Host connectivity PCIe Gen2 x6 HW Capable of 256 PCIe devices OS restriction apply PCIe virtualisation OS independent (same as M8KR) Single OS driver image for both M8KR and 280 VIC FabricFailover supported Eth hash inputs : Source MAC Address,Destination MAC Address,Source Pprt, Destination Port,Source IP address,destination IP address and VLAN FC Hash inputs: Source MAC Address Destination MAC Address,FC SID and FC DID and OXID Side A UCS 280 VIC 256 PCIe devices Side B 62

63 UCS Fabric Component Interoperability Complete hardware inter-operability between Gen and Gen 2 Fabric Interconnect IOM Adapter Supported Min Software version required UCS M8KR UCSM.4() or earlier UCS M8KR UCSM UCS UCS280 VIC UCSM UCS UCS280 VIC UCSM UCS UCS M8KR UCSM UCS UCS M8KR UCSM UCS UCS280 VIC UCSM UCS UCS280 VIC UCSM UCS

End Host Mode Disjointed L2 Domains UCS ver 2.

VLANs to uplinks (NO overlapping VLANs) Basing pinning decision on border

Vlan 40,4 Allocating a designated broadcast/multicast receiver on a per VLAN

64 End Host Mode Disjointed L2 Domains UCS ver 2.0 and beyond Hardware/ FI independent The ability to selectively assign VLANs to uplinks (NO overlapping VLANs) Basing pinning decision on border port and vnic VLAN membership Production Vlan 0,20 Public Vlan 3,32 Backup Vlan 40,4 Allocating a designated broadcast/multicast receiver on a per VLAN rather than global basis End Host End Host Max 3 disjoint Layer 2 domains supported Recommendation: End Host Mode 64

65 Q & A

Complete Your Online Session Evaluation Complete your session evaluation: Directly from your mobile device by visiting www.ciscoliveaustralia.

on your own computer to access the Cisco Live onsite portal Don t forget to activate your Cisco Live Virtual account for access to all session

66 Complete Your Online Session Evaluation Complete your session evaluation: Directly from your mobile device by visiting and login by entering your username and password Visit one of the Cisco Live internet stations located throughout the venue Open a browser on your own computer to access the Cisco Live onsite portal Don t forget to activate your Cisco Live Virtual account for access to all session materials, communities, and on-demand and live activities throughout the year. Activate your account at any internet station or visit 66

67 67

UCS Networking 201 Deep Dive

UCS Networking 201 Deep Dive UCS Networking 20 Deep Dive BRKCOM-2003 Brad Hedlund bhedlund@cisco.com Manish Tandon mtandon@cisco.com Agenda Overview / System Architecture Physical Architecture Logical Architecture Switching Modes