Návrh serverových farem

Transcription

1 Návrh serverových farem DCTECH4 Martin Diviš Consulting Systems Engineer Sponsor Sponsor Sponsor Sponsor Logo Logo Logo Logo CIscoEXPO 1

2 Agenda Introduction 5k/2k update Basic Concepts on Teaming vpc and 7k/5k/2k design best practices VLAN considerations vpc technology vpc designs in the access 2

3 Datacenter Components of a Nexus-based Design High Level View details in the rest of the presentation Aggregation Layer N7000 MDS Core Access La ayer 10 GigE/FCOE N5000 FCOE Edge N2000 NEXUS 1000v VM VM VM VM VM VM NEXUS 1000v VM VM VM VM VM VM blade1 slot 1 blade2 slot 2 blade3 slot 3 blade4 slot 4 blade5 slot 5 blade6 slot 6 blade7 slot 7 blade8 slot 8 NEXUS 1000v VM VM VM VM VM VM blade1 slot 1 blade2 blade3 slot 2 slot 3 blade4 slot 4 blade5 slot 5 blade6 slot 6 blade7 slot 7 blade8 slot 8 blade1 slot 1 blade2 slot 2 blade3 slot 3 blade4 slot 4 blade5 slot 5 blade6 slot 6 blade7 slot 7 blade8 slot 8 blade1 slot 1 blade2 slot 2 blade3 slot 3 blade4 slot 4 blade5 slot 5 blade6 slot 6 blade7 slot 7 blade8 slot 8 blade1 slot 1 blade2 slot 2 blade3 slot 3 blade4 slot 4 blade5 slot 5 blade6 slot 6 blade7 slot 7 blade8 slot 8 blade1 slot 1 blade2 slot 2 blade3 slot 3 blade4 slot 4 blade5 slot 5 blade6 slot 6 blade7 slot 7 blade8 slot 8 1G and 10GE Rack Mount Servers 1G and 10GE Blade Servers Pass-Thru HP/IBM/Dell N4K - DCB Blade Switch IBM/Dell 10GE Blade (HP) UCS Compute Pod 3

4 Agenda Introduction 5k/2k update Basic Concepts on Teaming vpc and 7k/5k/2k design best practices VLAN considerations vpc technology vpc designs in the access 4

5 NX-OS 4.1(3)N1(1) Support for 512 HW (but SW allows 507 maximum) VLANs (minus number of VSANs) Supports 12 Fabric Extenders Supports 16 Hardware Ethernet port channels (12 Ethernet and 4 Fiber Channel supported concurrently as well as just 16 Ethernet Portchannels and zero FC port-channels) Supports vpc, and specifically up to 480 Host vpcs Fabric Ports 5020 = 52 Fabric Ports & 16 Port Channels 5010 = 26 Fabric Ports & 16 Port Channels 5

6 Scalability for Port-Channels Consumes 1 HW Port-channel of the 16 available does NOT consume HW resources does NOT consume HW resources 2-Ports vpcs 4+ Ports vpcs As many as the number of ports on the 5k mgmt0 mgmt0 5k01 5k02 5k01 5k02 primary 5k01 Peer-link secondary 5k02 fabric links Max 16 HW-Port Channel eth2/1,2/2 eth2/3,2/4 eth2/1 eth2/2 vpc FEX100 HIF 2 ports FEX120 HIF vpc 6

7 Nexus 2000 straight-through with vpc Scalability for Host vpc n5k01 n5k02 max 24 FEXes = 1152 max 480 vpcs (each vpc has 2 ports) 7

8 Nexus 2000 dual-homed 5k01 5k02 vpc Primary Po10 vpc Secondary max 12 FEXes 8

9 Agenda Introduction 5k/2k update Basic Concepts on Teaming vpc and 7k/5k/2k design best practices VLAN considerations vpc technology vpc designs in the access 9

10 NIC Teaming Utility Green Light: teaming was successful A New Adapter appears 10

11 Teaming Options Automatic: this option figures out what is connected upstream and automatically configures the best teaming option 802.3ad dynamic: This option requires LACP support and in this configuration vpc since the NICs are split across FEXes Switch Assisted Load Balancing: translation = static port-channeling Transmit Load Balancing: 1 Port RX all ports in the team TX Network Fault Tolerance: Translation: Active/Standby 802.3ad requires vpc and LACP Switch Assisted Load Balancing requires vpc only TLB works ALWAYS and provides increased bandwidth in the TX direction NFT works always and provides not bandwidth benefits to the server 11

12 Agenda Introduction 5k/2k update Basic Concepts on Teaming vpc and 7k/5k/2k design best practices VLAN considerations vpc technology vpc designs in the access 12

13 Catalyst 6500 range compared with NXOS Range In Catalyst IOS the range of reserved VLANs is: 0, 4095, reserved by the cat6k ation/guide/vlans.html#wp In NXOS the range of reserved VLANs is: VLANs 3968 to 4047 and 4094 are reserved for internal use in each VDC The software allocates a group of 80 VLAN numbers for those features, like multicast and diagnostics, that need to use internal VLANs for their operation. 507 VLANs usable. os/layer2/configuration/guide/cisco_nexus_7000_series_nx- OS_Layer_2_Switching_Configuration_Guide_Release_4.2_chapter3.html#co n_ Be mindful of this when creating MST regions encompassing Catalyst and Nexus as you have to create region mapping for the least common denominator 13

14 VLANs on the Nexus 5000 FCoE considerations VSANs use VLAN hardware table resources FCoE requires a VLAN and a VSAN that you bind the VLAN to. Hence for each FCoE VSAN you should count using 2 VLANs Enabling FCoE burns two internal VSAN/VLAN resources 14

15 Useful Commands tc-nexus5k01# show resource vlan Used Unused Avail tc-nexus5k01# show vsan usage 2 vsan configured configured vsans:1,11 vsans available for configuration:2-10, , tc-nexus5k01# show vlan internal usage VLAN DESCRIPTION Multicast 4032 Online diagnostics vlan Online diagnostics vlan Online diagnostics vlan Online diagnostics vlan Reserved 4094 Reserved 15

16 Agenda Introduction 5k/2k update Basic Concepts on Teaming vpc and 7k/5k/2k design best practices VLAN considerations vpc technology vpc designs in the access 16

17 vpc Terminology (most of vpc s concepts equally apply to N7k and N5k) Fault Tolerant or peer keepalive link vpc peer link mgmt0 mgmt0 mgmt0 vrf Primary Secondary vpc member port vpc Peer Keepalive Peer Link/ MCT vpc Member Port Indicates that the behavior described equally applies to 7k or 5k 17

18 vpc Domain vpc domain is identified by a configured ID, and after successful establishment of peer-link adjacency, vpc domain is operationally enabled. MAC-address derived from domain-id is used for link-specific protocol operations (LACP lag-id for vpc, designated bridge-id for STP) vpc primary vpc domain vpc secondary Indicates that the behavior described equally applies to 7k or 5k 18

19 Double-sided vpc between Nexus 7000 and Nexus 5000 DESIGN 1 Make sure to leverage LACP domain-id needs to differ between the N7k vpc and the N5k vpc Spanning-Tree root is defined on one of the 2 N7ks N5k priorities are unmodified vpc on the N7k Max 16 Ports vpc on the N5k N7k01 N7k N5k01 N5k

20 Virtual Port-Channel vpc Peer Link Peer Link carries both vpc data and control traffic between peer switches Carries any flooded and/or orphan port traffic Carries STP BPDUs IGMP updates, etc. Carries Cisco Fabric Services messages (vpc control traffic) to perform, among other things, the synchronization of the MAC address table Must carry all host VLAN s Minimum 2 x 10GbE ports 7k01 5k01 7k02 5k02 losing the peer link is undesirable vpc Peer Link vpc Peer Link 5020 (config)# interface port-channel (config-if)# switchport mode trunk 5020 (config-if)# switchport trunk allowed <BETTER TO ALLOW ALL VLANS> 5020 (config-if)# vpc peer-link 5020 (config-if)# spanning-tree port type network 20

21 vpc Control Plane FT Link vpc FT (fault-tolerant) link is an additional mechanism to detect liveness of the peer. can use any L3 port. By default, will use management network. used only when peer-link is down does NOT carry any state information For Simplicity the rest of the slides show FT link as if it was directly connected between 7ks/5ks Peer-link FT link (can be routed) 21

22 vpc Primary Goal core1 core2 core1 core2 vpc peer link almost unutilized 5k01 5k02 5k01 5k02 acc1 acc2 acc3 acc1 acc2 acc3 22

23 vpc forwarding Rule To achieve the optimal usage of peer-link goal, learning on the Peer Link is such that the MAC addresses learned on a peer s vpc port are learned on local vpc ports instead of Peer-link This is done by disabling learning in the hardware on Peer-link ports. The MAC addresses learned on a switch s non vpc ports are notified to the peer switch via CFSoE in address update messages indicating the interface on which it is learned. When the update message is received from the peer, the local switch will update the local MAC table with the destination of The local vpc port if it was learned on an vpc port on the peer switch The Peer-link port if it was learned on a non vpc port (Orphan Port) 23

24 Virtual Port-Channel vpc Control Plane Consistency Check Both switches in the vpc Domain maintain distinct control planes System configuration must also be kept in sync Currently a manual process with an automated consistency check to ensure correct network behavior Two types of consistency checks Type 1 Will put interfaces into suspend state to prevent invalid forwarding of packets Type 2 Error messages to indicate potential for undesired forwarding behavior NX01 NX02 Consistency Checks Indicates that the behavior described equally applies to 7k or 5k 24

25 How does STP work with vpc? For vpc and vpc ports only the operational primary processes, generates BPDUs Each switch has its own priority. The primary can be root or secondary root for a VLAN The secondary can be root or secondary root for a VLAN If the secondary is root, the primary generates BPDUs for the secondary and indicates that the secondary is the root. But the secondary doesn t generate BPDUs for vpc ports. non-vpc ports function in the same way as without vpc 2/9 BPDU Root is Nx02 Operational Primary eth2/1 Po51 NX01 Switch3 Operational Secondary NX02 2/10 2/9 2/10 eth2/2 non-vpc eth2/1 eth2/2 Switch4 Root for VLAN 50 Indicates that the behavior described equally applies to 7k or 5k 25

26 Agenda Introduction 5k/2k update Basic Concepts on Teaming vpc and 7k/5k/2k design best practices VLAN considerations vpc technology vpc designs in the access 26

27 Nexus FEX topology support chart 27

28 Peer Keepalive vpc with FEX Straight Through Peer Link/ MCT vpc Member Port FT link (can be routed) mgmt network primary 5k01 mgmt0 Peer-link secondary 5k02 mgmt0 LACP is supported fabric links FEX100 HIF FEX101 HIF 2 ports vpc Port-Channels can have only 1-port per each FEX for a max of 2-ports 2-GigE ports host port channel 28

29 What if the Server has 4 NICs? FT link (can be routed) mgmt network primary 5k01 mgmt0 Peer-link fabric links secondary 5k02 mgmt0 FEX100 HIF 2 ports FEX101 HIF vpc vpc Create two different Teams 2-GigE ports host port channel 29

30 TLB Teaming with FEX ST primary 5k01 FT link (can be routed) mgmt0 Peer-link secondary 5k02 mgmt0 mgmt network vpc or non-vpc it doesn t matter fabric links FEX100 HIF FEX101 HIF This is not vpc, this is 4 GigE TLB configuration 4 ports team Create ONE Team with as many ports as needed (in this case 4, 4 Gbps TX and 1 Gbps RX) 4-GigE ports teaming TLB 30

31 vpc Failure Scenarios Peer-link failure with FEX straight-through FT communication path When the peer link fails, the secondary vpc peer switch suspends all its vpc member ports 5k01 vpc Primary Po10 5k02 vpc Secondary vpc secondary detects primary switch is alive through peer keepalive link See orphaned servers 31

32 vpc Failure Scenarios Peer-link failure with FEX Active/Active With FEX a/a when the peer-link fails all FEXes continue forwarding traffiv via the primary Nexus k01 vpc Primary Po10 5k02 vpc Secondary 32

33 vpc Failure Scenarios Primary Failure with FEX active/active FT communication path 5k01 5k02 If the Primary Nexus 5k fails, 5k02 becomes operational primary vpc Primary Po10 vpc Primary the FEX traffic is reassigned to 5k02 33

34 Otázky a odpovědi Vaše otázky zaslané pomocí SMS na číslo zodpovíme dnes od 17:45 v místnosti LEO CIscoEXPO 34

35 Dotazník Prosíme, vyplňte dotazník s hodnocením prezentace Děkujeme! CIscoEXPO 35

36 36