Best Practices come from YOU Cisco and/or its affiliates. All rights reserved.

Transcription

1

2 Best Practices come from YOU 2

3 Apple iphone4 launched in June

4 Antennagate 4

5 IPHONE4 Best Practices from CUSTOMERS 5

6 vpc Best Practices and Design on NXOS Nazim Khan, CCIE#39502 (DC/SP) Technical Marketing Engineer, Data Center Group

7 Session Goals Best Practices and Designs for vpc virtual port-channel Nexus 2000 (FEX) will only be addressed from vpc standpoint Fabricpath Overview VPC+ Overview vpc with VXLAN based networks vpc : Get it Right the very First time 7

8 Session Non-Goals vpc troubleshooting Details of vpc+ Details of Fabricpath and VXLAN ACI with or without vpc FCoE 8

9 Agenda Feature Overview Configuration Best Practices Design Best Practices Fabricpath / vpc+ VxLAN Scalability Reference Material 9

10 Data Center Technology Evolution MPLS, OTV, LISP MPLS, OTV, LISP FabricPath with vpc+ FEX with vpc VPC STP 2010 VXLAN ACI

11 Why vpc in 2015? 11

12 vpc is Foundation 12

13 Role of vpc in the Evolution of Data Center vpc launched in 2009 Deployed by almost 95% of Cisco customers Used to redundantly connect network entities at the Unified Fabric edge of the Fabric Dual-homed servers (bare metal, blades, etc.) Network services (Firewalls, Load Balancers, etc.) 13

14 Agenda Feature Overview Concepts and Benefits Terminology Configuration Best Practices Design Best Practices Fabricpath / vpc+ VxLAN Scalability Reference Material 14

15 vpc Feature Overview vpc Concept & Benefits S1 S2 STP vpc Physical Topology vpc Logical Topology S3 No Blocked Ports, More Usable Bandwidth, Load Sharing Fast Convergence 15

16 Feature Overview vpc Terminology vpc Peer Layer 3 Cloud vpc Domain Peer-Link vpc Peer Keepalive Link Orphan Port S1 CFS S2 vpc vpc Member Port Orphan Device S3 17

17 vpc Failure Scenario vpc Peer-Keepalive Link up & vpc Peer-Link down vpc peer-link failure (link loss): VPC system checks active status of the remote vpc peer via peer-keepalive link (heartbeat) P vpc Peer-keepalive S If both peers are active, then Secondary vpc peer will disable all vpcs to avoid Dual-Active Data will automatically forward down remaining active port channel ports vpc_plink Suspend secondary vpc Member Ports Failover gated on CFS message failure, or UDLD/Link state detection Orphan devices connected to secondary peer will be isolated SW3 vpc1 Keepalive Heartbeat vpc2 SW4 P S Primary vpc Secondary vpc

18 Agenda vpc Configuration Best Practices Building a vpc domain Domain-ID Peer-Link Peer-Keepalive Link Spanning-Tree Peer-switch Auto-recovery Object tracking vpc shutdown Maintenance Mode 19

19 vpc Configuration Best Practices Building a vpc domain Configuration Steps 1. Define domains 2. Establish Peer Keepalive connectivity S1 S2 3. Create a Peer link CFS 4. Create vpcs 5. Make Sure Configurations are Consistent (Order does Matter!) S3 20

20 vpc Configuration Best Practices vpc Domain-ID vpc Domain 10 The vpc peer devices use the vpc domain ID to automatically assign a unique vpc system MAC address S1 S2 You MUST use unique Domain id s for all vpc pairs defined in a contiguous layer 2 domain vpc Domain 20! Configure the vpc Domain ID It should be unique within the layer 2 domain NX-1(config)# vpc domain 20 S3 S4! Check the vpc system MAC address NX-1# show vpc role <snip> vpc system-mac : 00:23:04:ee:be:14 S5 21

21 vpc Configuration Best Practices vpc Peer-Link S1 S2 S1 S2 S3 S3 vpc Peer-link should be a point-to-point connection Peer-Link member ports can be 10/40/100GE interfaces Peer-Link bandwidth should be designed as per the vpc vpc imposes the rule that peer-link should never be blocking 22

22 vpc Configuration Best Practices vpc Peer-Keepalive link Recommendations (in order of preference): Preference Nexus 7X00 / 9X00 series 1 Dedicated link(s) (1GE/10GE LC) Nexus 6000 / 5X00 / 3X00 series mgmt0 interface 2 mgmt0 interface Dedicated link(s) (1GE/10GE LC) 3 L3 infrastructure L3 infrastructure 23

23 vpc Configuration Best Practices vpc Peer-Keepalive link Dual Supervisors For Your Reference When using dual supervisors and mgmt0 interfaces to carry the vpc peer-keepalive, DO NOT connect them back to back between the two switches vpc_pkl Management Switch Management Network vpc_pkl Only one management port will be active a given point in time and a supervisor switchover may break keepalive connectivity vpc1 vpc_pl vpc2 Use the management interface when you have an outof-band management network (management switch in between) Standby Management Interface Active Management Interface 24

24 vpc Configuration Best Practices Spanning Tree (STP) STP is running to manage loops outside of vpc domain, or before initial vpc configuration! S1 S2 S3 S4 S5 All switches in Layer 2 domain should run either Rapid-PVST+ or MST Do not disable spanning-tree protocol for any VLAN Always define the vpc domain as STP root for all VLAN in that domain 25

25 vpc Configuration Best Practices vpc Peer-Gateway Allows a vpc switch to act as the active gateway for packets addressed to the peer router MAC Keeps forwarding of traffic local to the vpc node and avoids use of the peer-link Allows Interoperability with features of some NAS or load-balancer devices S1 S3 S4 S2 N7k(config-vpc-domain)# peer-gateway 26

26 vpc Configuration Best Practices vpc Peer-switch Without Peer-switch STP for vpcs controlled by vpc primary. vpc primary send BPDU s on STP designated ports vpc secondary device proxies BPDU s to primary Primary vpc BPDUs Secondary vpc With Peer-switch Peer-Switch makes the vpc peer devices to appear as a single STP root BPDUs processed by the logical STP root formed by the 2 vpc peer devices Primary vpc Secondary vpc N7k(config-vpc-domain)# peer-switch 27

27 vpc Configuration Best Practices vpc auto-recovery Operational Primary P S P S P S1 S2 S1 S2 S1 S2 S3 S3 S3 1. vpc peer-link down : S2 - secondary shuts all its vpc member ports 2. S1 down : vpc peer-keepalive link down : S2 receives no keepalives 3. After 3 keepalive timeouts, S2 changes role and brings up its vpc P S vpc Primary vpc Secondary 28

28 vpc Configuration Best Practices vpc auto-recovery Auto-recovery addresses two cases of single switch behavior Peer-link fails and after a while primary switch (or keepalive link) fails Both VPC peers are reloaded and only one comes back up For Your Reference How it works If Peer-link is down on secondary switch, 3 consecutive missing peer-keepalives will trigger auto-recovery After reload (role is none established ) auto-recovery timer (240 sec) expires while peer-link and peer-keepalive still down, autorecovery kicks in Switch assumes primary role VPCs are brought up bypassing consistency checks Nexus(config)# vpc domain 1 Nexus(config-vpc-domain)# auto-recovery 29

29 vpc Configuration Best Practices Why Object-Tracking? Modules hosting peer-link and uplink fail on the vpc primary Peer-Link is down and vpc Secondary shut all its vpc Auto-Recovery does not kick in as peer-keepalive link is active Primary S4 S1 S5 S2 Secondary Traffic is black holed S3 30

30 vpc Configuration Best Practices Object-tracking vpc object tracking, tracks both peer-link and uplinks in a list of Boolean OR Object Tracking triggered when the track object goes down Suspends the vpcs on the impaired device. Traffic forwarded over the remaining vpc peer. S4 S5! Track the vpc peer link track 1 interface port-channel11 line-protocol! Track the uplinks track 2 interface Ethernet1/1 line-protocol track 3 interface Ethernet1/2 line-protocol S1 S2! Combine all tracked objects into one.! OR means if ALL objects are down, this object will go down track 10 list boolean OR object 1 object 2 object 3! If object 10 goes down on the primary vpc peer,! system will switch over to other vpc peer and disable all local vpcs vpc domain 1 track 10 S3 31

31 vpc Configuration Best Practices vpc Shutdown Isolates a switch from the vpc complex Primary Secondary Isolated switch can be debugged, reloaded, or even removed physically, without affecting the vpc traffic going through the non-isolated switch vpc S1 S2 switch# configure terminal switch(config)# vpc domain 100 switch(config-vpc)# shutdown S3 This Feature is currently supported only on Nexus 5X00 and 600X series 32

32 vpc Configuration Best Practices Maintenance Mode Primary vpc Secondary 1. vpc Primary enters maintenance mode via CLI 2. Running configuration is saved, key show command output is collected and saved 3. Change priority to highest value (65635) 4. Admin down all vpcs and vpc peer-link 5. Advertise state as self-isolated over peer keepalive link This Feature is currently supported only on Nexus 5X00 and 600X series

33 Agenda vpc Design Best Practices Mixed Hardware across vpc Peers Dynamic Routing over VPC vpc and Multicast vpc as Data Center Interconnect (DCI) FHRP with vpc Hybrid topology (vpc and non-vpc) vpc and Network Services vpc Fex Supported Topologies 34

34 Design Best Practices Mixed Hardware across vpc Peers : Line Cards Always use identical line cards on either sides of the peer link and VPC legs! Examples vpc Primary vpc Secondary vpc Primary vpc Secondary S1 N7000 vpc Peer-link F2E F2E S2 N7700 S1 vpc Peer-link M1 M2 S2 F3 F3 vpc vpc 35

35 Design Best Practices Mixed Hardware across vpc Peers : Chassis & Supervisors N7000 and N7700 in same vpc Construct -Supported VDC type should match on both peer device vpc peers can have mixed SUP version* (SUP1, SUP2, SUP2E) N5X00 and N600X in same vpc Construct Not Supported vpc Primary vpc Secondary vpc Primary vpc Secondary S1 N7000 vpc Peer-link S2 N7700 S1 N5X00 vpc Peer-link S2 N600X *Recommended only for short period such as migration 36

36 Design Best Practices Dynamic Routing over VPC Don t attach routers to VPC domain via L2 port-channel Common workarounds: A. Individual L3 links for routed traffic B. Static route to FHRP VIP A B SVI 1 IP Y VIP A SVI 1 IP Z VIP A SVI 1 IP Y VIP A L3 ECMP S1 S2 S1 S2 S1 S2 SVI 1 IP Z VIP A SVI 1 IP Y VIP A SVI 1 IP Z VIP A SVI 2 IP X Router SVI 2 IP X Router SVI 2 Router IP X Static Route to VIP A 37

37 Design Best Practices vpc and Multicast S1 Source S2 vpc supports PIM-SM only vpc uses CFS to sync IGMP state Sources in vpc domain both vpc peers are forwarders Duplicates avoided via vpc loop-avoidance logic Sources in Layer 3 cloud Active forwarder elected on unicast metric vpc Primary elected active forwarder in case metric are equal Source Receivers 38

38 ACCESS AGGR CORE vpc - Data Center Interconnect(DCI) Multi-layer vpc for Aggregation and DCI DC 1 DC 2 N Network port vpc domain 11 Long Distance Dark Fiber vpc domain 21 E Edge or portfast - Normal port type - N R N - R - - N N E E F F F F E E N N - - R - N R - N CORE AGGR B F R BPDUguard BPDUfilter Rootguard 802.1AE (Optional) - - R R vpc domain 10 vpc domain R R - E B E - B ACCESS Server Cluster Server Cluster 39

39 Design Best Practices vpc as Data Center Interconnect (DCI) PROS vpc is easy to configure and it provides robust and resilient interconnect solution CONS Maximum of only two Data Centers can be interconnected Layer 3 peering between Data Centers cannot be done through vpc and separate links are required 40

40 Design Best Practices vpc -Data Center Interconnect (DCI) vpc Domain id for vpc layers should be UNIQUE BPDU Filter on the edge devices to avoid BPDU propagation STP Edge Mode to provide fast Failover times No Loop must exist outside the vpc domain No L3 peering between Nexus 7000 devices (i.e. pure layer 2) 41

41 FHRP with vpc HSRP / VRRP/ GLBP Active/Active FHRP in Active/Active mode with vpc No requirement for aggressive FHRP timers Best Practice : Use default FHRP timers FHRP Active : Active for shared L3 MAC L3 L2 S1 FHRP Standby : Active for shared L3 MAC S2 42

42 FHRP with vpc Backup Routing Path Point-to-point dynamic routing protocol adjacency between the vpc peers to establish a L3 backup path to the core through PL in case of uplinks failure Define SVIs associated with FHRP as routing passive-interfaces in order to avoid routing adjacencies over vpc peer-link A single point-to-point VLAN/SVI (aka transit vlan) will suffice to establish a L3 neighbor L3 L2 Use one transit vlan to establish L3 routing backup path over the vpc peerlink in case L3 uplinks were to fail, all other SVIs can use passive-interfaces P P S3 OSPF/EIGRP VLAN 99 OSPF/EIGRP S4 P P Alternatively, use an L3 point-to-point link between the vpc peers to establish a L3 backup path S1 Primary vpc Secondary vpc S2 P Routing Protocol Peer 43

43 Hybrid topology (vpc and non-vpc) Bridge Priority VLAN 1 4K VLAN 2 8K STP Root VLAN 1 S1 vpc Primary STP Root VLAN 1 VLAN 2 vpc Peer-link vpc Secondary STP Root VLAN 2 peer-switch VLAN 1 vpc1 (blocked) S3 S4 VLAN 2 (blocked) supports hybrid topology where vpc and non-vpc are connected to the same vpc domain Need additional configuration parameters : spanning-tree pseudo-information S2 Bridge Priority VLAN 1 8K VLAN 2 4K If previously configured global spanning tree parameters and subsequently configure spanning tree pseudo information parameters, then pseudo information parameters take precedence over the global parameters. 44

44 vpc and Network Services Services Chassis w. Services VDC Sandwich Two Nexus 7000 Virtual Device Contexts to sandwich services between virtual switching layers Layer-2 switching in Services Chassis with transparent services vpc running in both VDC pairs to provide portchannel for both inside and outside interfaces to Services Chassis Design considerations: Access switches requiring services are connected to subaggregation VDC Access switches not requiring services be connected to aggregation VDC If Peering at Layer 3 is required between vpc layers an alternative design should be explored (i.e. using STP rather than vpc to attach service chassis) or using static routing Agg Layer Sub-Agg Layer Agg Layer Sub-Agg Layer 46

45 Nexus 2000 (FEX) Straight-Through Deployment with VPC Port-channel connectivity from the server Two Nexus switches bundled into a vpc pair Suited for servers with Dual NIC and capable of running Port-Channel S1 Fabric Links S2 HIF Fex 100 HIF Fex 101 * This design is currently not supported on N9500 series VPC 48

46 Nexus 2000 (FEX)Active-Active Deployment with VPC Fabric Extender connected to two Nexus 5X00 / 6000 Suited for servers with Single NIC or Dual NIC not having port-channel capability. Scale implications of less FEX per system and less VPC * This design is currently not supported on N7000 / N7700 and N9X00 Fex 100 HIF S1 Fabric Links S2 Nexus 6000 / 5000 Fabric Extender dual homed to redundant Nexus switches HIF Fex

47 Nexus 2000 (FEX) - Enhanced VPC Port-channel connectivity to dual-homed FEXs From the server perspective a single access switch with port-channel support each line card supported by redundant supervisors Ideal design for a combination of single NIC and Dual NIC servers with portchannel capability Scale implications of less FEX per system and less VPC Fex 100 HIF S1 Fabric Links S2 Fex 101 Nexus 6000 / 5000 Fabric Extender dual homed to redundant Nexus 5000 HIF * This design is currently not supported on N7000 / N7700 and N9X00 50

48 vpc : Get it Right the very First time 51

49 Agenda Feature Overview vpc Configuration Best Practices vpc Design Best Practices Fabricpath / vpc+ vpc in VxLAN network Scalability Reference Material 52

50 FabricPath: an Ethernet Fabric Shipping on Nexus 7x00, Nexus 600x and Nexus 5x00 FabricPath Eliminates Spanning tree limitations High resiliency, fast network re-convergence Any VLAN, Anywhere in the Fabric Connect a group of switches using an arbitrary topology With a simple CLI, aggregate them into a Fabric N7K(config)# interface ethernet 1/1 N7K(config-if)# switchport mode fabricpath 53

51 VPC vs VPC+ Architecture of vpc and FabricPath with vpc+ CE FP CE Port FP Port CE VLAN s FP VLAN s vpc vpc+ Physical architecture of vpc and vpc+ is the same from the access edge Functionality/Concepts of vpc and vpc+ are the same Key differences are addition of Virtual Switch ID and Peer Link is a FP Core Port vpc+ is not supported on Nexus 9X00 & Nexus 3X00 Series 54

52 VPC+ Virtual Switch ID S10 S20 S30 S40 S100 S200 A B S100 S200 S300 FabricPath S300 Mac address flapping on S300 Single path to A 1/1 S300: CE MAC Address Table 1/2 MAC IF A B 1/2 A S100 S200 Classical Ethernet B 55 55

53 VPC+ Virtual Switch S10 S20 S30 S40 S100 S200 A B S1 S300 FabricPath S300 A consistently associated to S1 Multipathing to A A S1 virtual S300: CE MAC Address Table MAC IF B 1/2 A S1 1/2 B Classical Ethernet Refer BRKDCT-2081 Cisco Fabric Path Technology & Design 56 56

54 Dynamic Routing over vpc+ Layer 3 devices can form routing adjacencies with both the vpc+ peers over vpc The peer link ports and VLAN are configured in FabricPath mode. N55xx, N56xx, N6000 support this design with IPv4/IPv6 unicast and PIM-SM multicast Fabricpath Core vpc This design is not supported on N7X00 P P P N55xx, N56xx, N6000 Router/ Firewall Fabricpath Link Dynamic Peering Relationship Routing Protocol Peer P 57

55 Agenda Feature Overview vpc Configuration Best Practices vpc Design Best Practices Fabricpath / vpc+ VxLAN Scalability Reference Material 58

56 Why VXLAN? Problems being addressed: VLAN scale VXLAN extends the L2 segment ID field to 24-bits, potentially allowing for up to 16 million unique L2 segments over the same network Layer 2 segment elasticity over Layer 3 boundary VXLAN encapsulates L2 frame in IP-UDP header High Level Technology Overview: MAC-in-UDP encapsulation. Leverages multicast in the transport network to simulate flooding behavior for broadcast, unknown unicast and multicast in the same segment Leverage ECMP to achieve optimal path usage over the transport network 59

57 VXLAN Packet Format Outer Mac Header Outer IP Header UDP Header VXLAN Header Original L2 Frame FCS FCS MAC Addr. Dst. Src. MAC Addr. VLAN Type 0x8100 VLAN ID Tag Ether Type 0x0800 IP Header Misc Data Protocol 0x11 Header Checksum Outer Src. IP Outer Dst. IP UDP Src. Port VXLAN Port UDP Length Checksum 0x0000 VXLAN RRRR1RRR Reserved VNID Reserved For Your Reference 14 Bytes (4 bytes optional) 20 Bytes 8 Bytes 8 Bytes VXLAN is a Layer 2 overlay scheme over a Layer 3 network. VXLAN uses Ethernet in UDP encapsulation VXLAN uses a 24-bit VXLAN Segment ID (VNI) to identify Layer-2 segments 60

58 VXLAN Terminology VTEP Virtual Tunnel End Point Transport IP Network VTEP IP Interface VTEP IP Interface Local LAN Segment Local LAN Segment End System End System End System End System VXLAN terminates its tunnels on VTEPs (Virtual Tunnel End Point). VTEP has two interfaces : 1. Bridging functionality for local hosts 2. IP identification in the core network for VXLAN encapsulation / de-encapsulation. 61

59 vpc VTEP When vpc is enabled an anycast VTEP address is programmed on both vpc peers Symmetrical forwarding behavior on both peers provides Multicast topology prevents BUM traffic being sent to the same IP address across the L3 network (prevents duplication of flooded packets) vpc peer-gateway feature must be enabled on both peers vpc VTEP vpc VTEP VLAN VXLAN VXLAN header is not carried on the vpc Peer link 62

60 VXLAN & VPC VPC Configuration Map VNI to VLAN Source Interface individual IP is used for single attached Hosts anycast IP is used for VPC attached Hosts VTEP1 vlan 10 vn-segment interface loopback 0 ip address <VTEP individual IP orphan) ip address <VTEP anycast IP per VPC domain> secondary! interface nve1 source-interface loopback0 member vni mcast-group VXLAN Tunnel Interface vtep 1 vtep 2 vtep 3 vtep 4 VTEP2 vlan 10 vn-segment interface loopback 0 ip address <VTEP individual IP - orphan> ip address <VTEP anycast IP per VPC domain> secondary! interface nve1 source-interface loopback0 member vni mcast-group H VLAN 10 (vpc) H VLAN 10 (vpc) 63

61 VXLAN & VPC VPC Configuration For Your Reference VTEP1 vlan 10 vn-segment VTEP3 vlan 10 vn-segment interface loopback 0 ip address /32 ip address /32 secondary! Interface nve1 source-interface loopback0 member vni mcast-group interface loopback 0 ip address /32 ip address /32 secondary! Interface nve1 source-interface loopback0 member vni mcast-group VTEP2 vlan 10 vn-segment vtep 1 vtep 2 vtep 3 vtep 4 VTEP4 vlan 10 vn-segment interface loopback 0 ip address /32 ip address /32 secondary! Interface nve1 source-interface loopback0 member vni mcast-group interface loopback 0 ip address /32 ip address /32 secondary! Interface nve1 source-interface loopback0 member vni mcast-group H VLAN 10 (vpc) H VLAN 10 (vpc) 64

62 VXLAN & VPC Dual attached Host to dual attached Host (Layer-2) Host 1 (H1) and Host 2 (H2) are dual connected to a VPC domain As H1 is behind a VPC interface, the anycast VTEP IP is the source for the the VXLAN encapsulation vtep 1 vtep 2 vtep 3 vtep 4 As H2 is behind a VPC interface, the anycast VTEP IP is the target H VLAN 10 (vpc) H VLAN 10 (vpc) 65

63 Agenda Feature Overview vpc Configuration Best Practices vpc Design Best Practices Fabricpath / vpc+ VxLAN Scalability Reference Material 66

64 vpc Scalability For Latest Scalability numbers please refer to the scalability limits pages for the platform Nexus 7X00 Nexus 5X00 Scalability_700N11_chapter_01.html Nexus 600X Scalability_602N12_chapter_01.html Nexus

65 Agenda Feature Overview vpc Configuration Best Practices vpc Design Best Practices Fabricpath / vpc+ VxLAN Scalability Reference Material 68

66 Reference Material For Your Reference vpc Best Practices Design Guide: vpc design guides: vpc and VSS Interoperability white Paper: VXLAN Overview : Fabrcipath whitepaper : 69

67 Key Take-Aways vpc in 2015 VXLAN, ACI, Fabricpath Fabricpath Eliminates Spanning-Tree * High resiliency vpc+ for legacy switches, servers, hosts vpc Design & Best Practices Optimal vpc performance with recommended deployment techniques vpc Benefits No Blocked Ports High availability Fast Convergence VXLAN L2 segment scalability VTEP redundancy with vpc 70

68 Related Cisco Live Milan 2015 Events Technical Breakout Sessions Session-ID BRKDCT-2404 BRKDCT-2334 BRKDCT-3378 BRKAPP-9000 Session Name VXLAN deployment models - A practical perspective Real World Data Center Deployments and Best Practice Building simplified, automated and scalable DataCenter network with Overlays (VXLAN/FabricPath) Introduction to Application Centric Infrastructure 71

69 Call to Action Visit the World of Solutions for Cisco Campus Data Center and Cloud Walk in Labs Technical Solution Clinics Meet the Engineer (Right after this session, until 30th Jan 2015) Lunch time Table Topics DevNet zone related labs and sessions Recommended Reading: for reading material and further resources for this session, please visit 72

70 Complete Your Online Session Evaluation Please complete your online session evaluations after each session. Complete 4 session evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt. All surveys can be completed via the Cisco Live Mobile App or the Communication Stations 73

71