Migration Use Cases for Catalyst 6500 Supervisor 2T

Transcription

1

2 Migration Use Cases for Catalyst 6500 Supervisor 2T Faraz Siddiqui, Network Consulting Engineer

3 Objectives for Understand the architectural building blocks of Supervisor 2T, hardware and software dependencies Identifying Migration use cases of Supervisor 2T and step by step migration walkthrough (what is happening during each step) Provide best practices, configuration and reference material for Migration process and VSS technology More sessions on Catalyst 6500 BRKARC-3465 Cisco Catalyst 6500 Switch Architecture TECCRS-2065 Cisco Catalyst 6500 Technical Deep Dive BRKCRS-3143 Troubleshooting Cisco Catalyst 6500 Series Switches BRKCRS-2468 Cisco Catalyst Virtual Switching System (VSS) 3

4 Verify Plan Execute 4

5 Presentation Legend Single Supervisor 720 Supervisor 2T VSS Sup720 VSS Sup 2T Dual Supervisor 720 Dual Supervisor 2T Access Switch Blue: Layer 3 Ethernet link Red: Layer 2 Ethernet link Acronyms Used VSS Virtual Switching System Sup Supervisor 2T 2 Terabit Switching HSRP Hot Standby Redundancy Protocol STP Spanning Tree Protocol VSL Virtual Switch Link MEC Multi Chassis Etherchannel VLAN Virtual LAN L2/L3 Layer 2 and Layer 3 ECMP Equal Cost multi-path CFC Centralized Forwarding Card DFC Distributed Forwarding Card PFC Policy Feature Card MSFC Multi-layer Switch Feature Card X-Bar Cross Bar Switch Fabric For Your Reference 5

6 Agenda Current Network Challenges network design with spanning Tree, User downtime, VSS Solution Supervisor 2T Architecture Overview architecture building blocks, hardware and software requirements Migration Use cases description of Standalone, HA and VSS, test traffic profile Migration Walk Through approach, migration Steps, STP and HSRP interaction, traffic re-routing Results Summary and Best Practices convergence summary, verification of Sup2T, VSS verification 6

7 Current Network Challenges

8 Business Continuity Challenges: STP Loops and Slow Routing Convergence Productivity Loss User Downtime (seconds) Complex Config to Manage HSRP Active 10 HSRP Standby 15 VLAN 15 VLAN 10 Traditional With Spanning Tree Switch 1 Switch 2 10GE X Si Access Switch or ToR or Blades Si Access Switch or ToR or Blades VLAN 10 HSRP Active 15 HSRP Standby 10 VLAN 15 X The Challenge Complex network design Typical Deployment Scenario Single active uplink per VLAN 50% bandwidth utilization only Spanning Tree loops First Hop Routing Protocol Convergence Routing Reconvergence

9 Business Continuity Enhanced Availability and Simplified Network Design with VSS VSS Physical Logical View Si 10GE Si LACP or PagP LACP or PagP LACP LACP Access Switch or Access Switch or Server Server ToR or Blades ToR or Blades The Solution 4 Tbps Virtual Switching System Double Bandwidth Utilization With Active-Active Multi- Chasis Etherchannel (LACP / PagP) Simplified Network Design Spanning Tree and First Hop Redundancy Protocols Eliminated Minimized Traffic Disruption Subsec Stateful and Graceful Recovery (SSO / NSF)

10 Agenda Current Network Challenges network design with spanning Tree, User downtime, VSS Solution Supervisor 2T Architecture Overview architecture building blocks, hardware and software requirements Migration Use cases description of Standalone, HA and VSS, test traffic profile Migration Walk Through approach, migration Steps, STP and HSRP interaction, traffic re-routing Results Summary and Best Practices convergence summary, verification of Sup2T, VSS verification 10

11 Catalyst 6500 Supervisor 2T

12 Supervisor 2T Architecture Overview Deployment at Core & Distribution layers 2Tbps switching capacity (4Tbps with VSS) Line-rate encryption (MACSec) New hardware and software features 12

13 Supervisor 2T Glance New MSFC5 with single Dual-Core CPU & single IOS image New USB based console support Cisco TrustSec (CTS) on ALL Uplink ports New 26 Channel 2T Switch Fabric which provides 80Gbps per slot 10G Uplinks New Connectivity Management Processor (CMP) New PFC4 featuring improved performance & scalability, along with new & enhanced hardware features 13

14 Supervisor 2T Block Diagram Fabric Connector Local-Bus Shared Bus Connector Crossbar Switch Fabric 26 x 40G Fabric Channels MSFC5 Fabric Intf 1 Fabric Intf 0 Bus Replication ASIC PFC4 Layer 3/4 forwarding Engine Layer 2 forwarding Engine Fabric Replication ASIC DRAM Bootdisk Connectivity MGMT processor 1GE FDX Port ASIC 0 Port ASIC 1 CTS ASIC CTS ASIC Compact Flash Serial Port MGMT Port USB Port Front Panel SFP-1 SFP-2 X2-1 SFP-3 X2-2 14

15 Policy Feature Card 4 Introduction PFC4 - Default PFC (EARL8) FIB & Netflow at 256K entries PFC4 PFC4XL - Upgrades FIB & Netflow Table to 1M entries Scalability Increased MAC Table (128K) L2 Bridge Domains (16K) L3 Logical Interfaces (128K) Increased Forwarding (60Mpps) Increased Throughput (80Gbps) IP Routing IPv6 Tunneling in FIB Unicast RPF for IPv6 IPv6 Multicast in FIB 512K Multicast Routes IGMPv3 / MLDv2 Snooping Virtualization Native (H)VPLS MPLS Aggregate Labels (16K) Multi-point EoMPLS L2oGRE VRF-based NAT & FnF QoS & Security Cisco TrustSec & SGACL s Increased ACL TCAM (256K) Increased ACL Labels (16K) Per-Port / Per-VLAN QoS Distributed Policers (512) Monitoring Flexible Netflow (FnF) Egress Netflow L2 (per VLAN) Netflow TCP Flags Per-Protocol Counters 15

16 Policy Feature Card 4 Earl 8 Overview Contains CEF IPv4, IPv6 prefixes & MPLS entries Contains Layer 2 rewrite information & pointers Collection of ADJ statistics for each active flow Contains the Ingress ACL entries (128K) Contains table of exception cases & action to take FIB TCAM Adjacency Table Adjacency Statistics Classification ACL Table #1 Exception Table Layer 3 / 4 Forwarding Engine Netflow Hash Table Netflow Data Table Netflow Statistics RPF Map Table Classification ACL Table #2 LIF Map Table Contains location of flow in Netflow Table Contains several key packet fields for flow Collection of NF statistics for each active flow Table of Src-Port info for Multicast & urpf Contains the Egress ACL entries (128K) Contains Logical Interface Mapping info 128K CAM containing L2 MAC address table Collection of ACL hit statistics & other info 128K MAC Table ACE Counters Layer 2 Forwarding Engine Fabric Replication ASIC Bus Backplane LIF Table LIF Statistics Contains the actual LIF Database entries Contains LIF Usage statistics 16

17 Policy Feature Card 4 EARL8 Processing The forwarding engine ASIC has 2 processing 60Mpps: Headers From L2 Engine 1. Input Forwarding Engine (IFE) 2. Output Forwarding Engine (OFE) As each packet header enters the L3 Forwarding ASIC, the IFE pipeline will perform L3 Lookup and Ingress Security, QoS & Netflow processing The header is merged with IFE results and then passed to the OFE pipeline, which will perform Egress Security, QoS & Netflow processing, to generate final result. Ingress ACL Ingress NetFlow IFE Process OFE Process L3 Lookup Ingress QoS Headers To L2 Engine Rewrite Result Generation Egress QoS *also applies to each DFC4 Egress NetFlow Egress ACL 17

18 Multilayer Switch Feature Card 5 Introduction Single Dual Core processor Combines the functionalities of the Switch Processor (SP) & the Route Processor (RP) Single Bootdisk filesystem Enhanced CPU Performance 2GB or 4GB DDR3 DRAM Connectivity Management Processor (CMP) On-Board Failure Logging (OBFL) Mini Protocol Analyzer (MPA) 18

19 Multilayer Switch Feature Card 5 Block Diagram Ethernet Out of Band Channel 100 Mbps HDX To Base-Board Inband Channel 1GE FDX OBFL Flash 4 MB NVRAM 4MB Rommon 4 MB 2 x 2 GB DDR2 Memory Control Plane CPU 1.5 GHz I/O ASIC Rommon 32 MB Core 0 Core 1 CMP CPU 266 MHz 256 MB Memory Bootdisk MUX Front Panel Compact Flash Serial Port Network Management port 10/100/100 Auto-MDI USB 2.0 Host Type A Type B 19

20 Multilayer Switch Feature Card 5 "Lights Out" Management with CMP The Connectivity Management Processor (CMP) supports new capabilities that will aid Network Administrators in managing the system: RP Image Recovery - TFTP boot of the system RP File Transfer - Image copy via TFTP Remote RP Reset - Hard or Soft reset RP Console Logging USB Support - Booting via Approved USB flash - USB serial console access Removes the need for a separate Telnet Server for console access Has unique GOLD tests 20

21 Multilayer Switch Feature Card 5 Accessing the CMP When the system comes online, RP initially owns the console. Use the following key sequence to switch between two consoles: (Ctrl-C, Shift-M) three times to switch to CMP console (Ctrl-R, Shift-M) three times to switch to RP console Sup2T# Sup2T#M Sup2T#M Sup2T# Sup2T-cmp login: root Password: Cisco CMP Software TAC support: Copyright (c) , Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained herein are owned by other third parties and are used and distributed under license. Some parts of this software may be covered under the GNU Public License or the GNU Lesser General Public License. A copy of each such license is available at and Sup2T-cmp# Ctrl-C, Shift-M, Ctrl-C, Shift-M, Ctrl-C, Shift-M CMP suffix added to the prompt Enter root as default login Enter default as default password 21

22 2T Switch Fabric Introduction Integrated 2Tbps Switch Fabric 26 Channels to support the 6513-E Dual Queues (Hi/Lo) per fabric channel Redundant channel to Standby for faster traffic convergence, during SSO switchover... Provides backplane interconnects Fabric traces are distributed across each slot Each Fabric trace can operate at either 20Gb/sec or 40Gb/sec Mixing 6700 & 6900 (20G & 40G) modules does not affect speeds of other modules 22

23 2T Switch Fabric High Availability If using VS-S720-10G or VS-SUP2T-10G with a redundant Standby Supervisor, then two (2) fabric channels are connected "back to back": Standby Supervisor uplink connection to Active Supervisor Active Supervisor bus connection to Standby Supervisor The Standby Supervisor is in DFC Mode, with its Bus connection disabled With Sup2T the redundant Standby Supervisor enables it s redundant channels for WS GE and WS GE modules, for "hitless" failover... Line Card Slot 1 Active Supervisor Line Card Slot 13 Standby Supervisor 23

24 720 vs 2T Fabric Hardware Data-Plane Feature Sup720 Sup720-10G Sup2T Number of Channels 18 20* 26 Aggregate Bandwidth 720 Gbps 720 Gbps 2 Tbps Channel Speeds (bps) 8G / 20G 8G / 20G 20G / 40G Fabric Redundancy Yes Yes Yes SSO Fabric Hot Synch No Yes* Yes Redundant Channels No No Yes Fabric Priority (QoS) Single Fabric Hi / Lo Priority Single Fabric Hi / Lo Priority Hi Priority Fabric Lo Priority Fabric Clear Block Support Yes Yes Yes Switching Modes Header Size) (DBUS Bus, Truncated, Compact Bus, Truncated, Compact Truncated, Compact Requires E-Series No No* Yes NOTE: Compact switching mode provides optimal Fabric performance... 24

25 Upgrading the Install Base to Sup2T Sup Series w/ CFC Sup2T Supported 6700 Series 1G w/ DFC G w/ DFC G Fiber T Copper G Fiber 6100 Series Legacy Services Modules WS-F6K-DFC4-A WS-F6K-DFC4-A WS-F6K-DFC4-E WS-F6K-DFC4-E G Supported Supported 25

26 Distributed Forwarding DFC Interoperability with PFC PFC3A PFC3B PFC3BXL PFC3C PFC3CXL PFC4 PFC4XL DFC3A Compatible Operate as PFC3A Operate as PFC3A Operate as PFC3A Operate as PFC3A X X DFC3B Operate as DFC3A Compatible Operate as PFC3B Operate as PFC3B Operate as PFC3B X X DFC3BXL Operate as DFC3A Operate as DFC3B Compatible Operate as PFC3B & DFC3B Operate as PFC3BXL X X DFC3C Operate as DFC3A Operate as DFC3B Operate as PFC3B & DFC3B Compatible Operate as PFC3C X X DFC3CXL Operate as DFC3A Operate as DFC3B Operate as DFC3BXL Operate as DFC3C Compatible X X DFC4 X X X X X Compatible Operates as PFC4 DFC4XL X X X X X Operates as DFC4 Compatible 26

27 Catalyst 6500 E Series Chassis Enhanced ( E ) Series chassis offer: higher bandwidth higher power capacity, better signal integrity to support Supervisor 2T 3, 4, 6, 9 & 13-slot versions Classic Data Bus traces/connectors Crossbar Fabric traces/connectors Redundant Power supplies Enhanced Fan for system cooling 6509-V-E chassis offers redundant fan trays & air filtration 27

28 Sup720 Fabric /6513-E The 720Gbps Switch Fabric has 18 channels which are distributed across the available slots (6503, 6504, 6506 & 6509 each get 2 (dual) fabric channels, per slot) but what about the 6513 & 6513-E? How do we split 18 channels across 13 slots? SWITCH FABRIC Sup / 6513-E Fabric-Channel Assignment Slots 1-8 each get a single fabric channel Slots 9-13 each get dual fabric channels Total fabric channels 8 x 1 = 8 5 x 2 = = 18 28

29 Sup2T Fabric /6513-E The 2Tbps Switch Fabric has 26 channels which are distributed across the available slots (6503-E, 6504-E, 6506-E, 6509-E & 6509-V-E already get 2 (dual) fabric channels, per slot) but what about the 6513-E? SWITCH FABRIC Sup2T E Fabric-Channel Assignment Slots 1-13 each get dual fabric channels 13 x 2 = 26 NOTE: This is now possible due to the additional fabric channel traces (physical connectors & wires) on both the Supervisor2T -AND E Hence, Supervisor 2T (non-e) chassis combination will NOT be supported... 29

30 Sup720 vs. Sup2T- Switching L2 (IPv4 / IPv6) Scaling Feature Sup720 Sup2T MAC Address Table 3A/B: 64K 3C: 96K 128K CAM Hash Table Single Bank Dual Bank L2 Bridge Domains 4K (VLAN) 16K (BD) Adjacency Entries 1M 1M MST Virtual Ports 100K 120K R/PVST Virtual Ports 12K 16K DAI, DHCP Snooping & SourceGuard Entries 8K 12K Policy-Based Forwarding (PBF) 32K 64K EFP (Ethernet Flow Point) N/A 32K EVC (Ethernet Virtual Connection) N/A 4K L2oGRE Tunnels N/A 1K * Available in future IOS software releases 30

31 Sup720 vs. Sup2T- Routing IPv4 Route Scaling Feature Sup720 Sup2T FIB TCAM (non XL) FIB TCAM (XL) 256K Entries 1M Entries 256K Entries 1M Entries TCAM Entry Size 144 bits 288 bits BGP Prefixes / Peers 750K / 1K 1M / 2K OSPF Prefixes / Peers 20K / 50 30K / 75 EIGRP Prefixes / Peers 20K / 50 30K / 75 RIPv2 Prefixes / Peers 10K / 10 50K / 50 ARP Entries 30K 100K FHRP Instances 500 1K NAT / PAT Entries 256K (Ingress Only) 512K Ingress / 512K Egress Policy Routing (PBR) Entries 2K 4K IP GRE Tunnels 1K 5K ECMP Load Sharing 16 paths 16 paths 31

32 Agenda Current Network Challenges network design with spanning Tree, User downtime, VSS Solution Supervisor 2T Architecture Overview architecture building blocks, hardware and software requirements Migration Use cases description of Standalone, HA and VSS, test traffic profile Migration Walk Through approach, migration Steps, STP and HSRP interaction, traffic re-routing Results Summary and Best Practices convergence summary, verification of Sup2T, VSS verification 32

33 Supervisor 2T Migration Use Cases

34 Migration Use cases Single/Dual Supervisor Standalone to VSS VSS to VSS Migrate single/dual Sup720 in the pair of Catalyst 6500 series non-e chassis with legacy hardware to single Sup2T in pair of E-chassis with supported linecards Typical deployment in campus and datacenter Core layer Migrate single Sup720 in pair of Catalyst 6500 series non-e chassis with legacy hardware to single Sup2T in pair of E-chassis with supported linecards Convert the standalone Sup2T to VSS mode Typical deployment in campus Core/Distribution and datacenter Distribution layer Migrate Sup720 deployed as VSS in pair of Catalyst 6500 series non-e chassis with legacy hardware to Sup2T in VSS mode with supported linecards Typical deployment in campus Core/Distribution and datacenter Distribution layer 34

35 Single Supervisor Topology and traffic details Single Sup720 deployed in pair of Non-E chassis at distribution layer Vlans are divided in group of Red and Green Dist-1 is configured as HSRP Primary/STP root for Red vlans Dist-2 is configured as HSRP Primary/STP root for Green vlans Spirent traffic generator is used to inject 5000 mac addresses, 100 VLANs, 5000 simulated transmit nodes (Layer 2), 50 SVIs at each core, 50 HSRP groups, 5000 ARP entries (Layer 3) Port Channel is connected between Catalyst pair at Layer2/Layer 3 boundary Layer 3 termination at distribution layer End-hosts are connected to access switch Dist-1 Dist-2 35

36 Dual Supervisors Topology and traffic details Dual Sup720s deployed in pair of Non-E chassis at distribution layer (HA or SSO mode) Vlans are divided in group of Red and Green Dist-1 is configured as HSRP Primary/STP root for Red vlans Dist-2 is configured as HSRP Primary/STP root for Green vlans Spirent traffic generator is used to inject 5000 mac addresses, 100 VLANs, 5000 simulated transmit nodes (Layer 2), 50 SVIs at each core, 50 HSRP groups, 5000 ARP entries (Layer 3) Port Channel is connected between Catalyst pair at Layer2/Layer 3 boundary Layer 3 termination at distribution layer End-hosts are connected to access switch Dist-1 Dist-2 36

37 Virtual Switch System Topology and traffic details Sup 720 deployed in pair of Non-E chassis at distribution layer Vlans are divided in group of Red and Green Dist-1 and Dist-2 acting as one logical switch (STP root) and vlans are load balanced across both links of MEC Access switch is connected to VSS through MEC VSL port channel is connected between Catalyst pair at Layer2/Layer 3 boundary Spirent traffic generator is used to inject 5000 mac addresses, 100 VLANs, 5000 simulated transmit nodes (Layer 2), 50 SVIs at each core, 50 HSRP groups, 5000 ARP entries (Layer 3) Layer 3 termination at distribution layer End-hosts are connected to access switch Dist-1 Dist-2 37

38 VSS Introduction

39 Current Network Challenges Enterprise Campus Traditional Campus Multi-Layer Design L3 Core Extensive routing topology, Routing reconvergence L2/L3 Distribution FHRP, STP, Asymmetric routing, Policy Management Access Single active uplink per VLAN (PVST), L2 reconvergence 39

40 Current Network Challenges Data Center Traditional Data Center Multi-layer design FHRP, HSRP, VRRP Spanning Tree Policy Management L2/L3 Core Single active uplink per VLAN (PVST), L2 reconvergence, excessive BPDUs L2/L3 Distribution Dual-Homed Servers to single switch, Single active uplink per VLAN (PVST), L2 reconvergence L2 Access 40

41 Catalyst 6500 Virtual Switching System Overview Traditional 10GE VSS (Physical View) 10GE VSS (Logical View) Si Si Si Si 802.3ad or PagP 802.3ad 802.3ad or PagP 802.3ad Access Switch or ToR or Blades Simplifies operational Manageability via Single point of Management, Non-loop design, minimize reliance on STP, eliminate FHRP etc Scales system capacity with Active-Active Multi-Chassis Etherchannel (802.3ad/PagP), no blocking links due to Spanning Tree Minimizes traffic disruption from switch or uplink failure with Deterministic subsecond Server Access Switch or ToR or Blades Server Stateful 2013 Cisco and and/or Graceful its affiliates. All rights Recovery reserved. (SSO/NSF) Access Switch or ToR or Blades Server 41

42 Virtual Switching System Enterprise Campus VSS Distribution Design L3 Core Reduced routing neighbors, Minimal L3 reconvergence L2/L3 Distribution No FHRPs No Looped topology Policy Management Access Multiple active uplinks per VLAN, No STP convergence 42

43 Virtual Switching System Data Center VSS Data Center Design Single router node, Fast L2 convergence, Scalable architecture Dual Active Uplinks, Fast L2 convergence, minimized L2 Control Plane, Scalable L2/L3 Core L2 Distribution Dual-Homed Servers, Single active uplink per VLAN (PVST), Fast L2 convergence L2 Access 43

44 Virtual Switching System Architectural Concepts Active Virtual Switch Domain Control Plane Standby Hot Virtual Switch Link Data Plane Switch 1 Switch 2 44

45 Virtual Switching System Architecture Virtual Switch Link (VSL) The Virtual Switch Link joins the two physical switch together - it provides the mechanism to keep both the chassis in sync VS Header L2 Hdr L3 Hdr Data CRC Virtual Switch Active Virtual Switch Link Virtual Switch Standby 45

46 Virtual Switching System Architecture Initialization The initialization process consists of 3 main steps: 1 2 Link Bringup to determine which ports form the VSL Link Management Protocol (LMP) used to track and reject Unidirectional Links, Exchange Chassis ID and other information between the 2 switches LMP RRP LMP RRP 3 Role Resolution Protocol (RRP) used to determine compatible Hardware and Software versions to form the VSL as well as determine which switch becomes Active and Hot Standby from a control plane perspective 46

47 Virtual Switching System Architecture VSLP Ping A new ping mechanism has been implemented in VSS mode to allow the user to objectively verify the health of the VSL itself. This is implemented as a VSLP Ping Switch1 VSLP Ping VSLP Ping VSL VSLP Ping VSLP Ping Switch2 The VSLP Ping operates on a per-physical interface basis and parameters such as COUNT, DESTINATION, SIZE, TIMEOUT may also be specified vss#ping vslp output interface tengigabitethernet 1/5/4 Type escape sequence to abort. Sending 5, 100-byte VSLP ping to peer-sup via output port 1/5/4, timeout is 2 seconds:!!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 12/12/16 ms 47

48 Virtual Switching System Architecture VSL Configuration Consistency Check After the roles have been resolved through RRP, a Configuration Consistency Check is performed across the VSL switches to ensure proper VSL operation. The following items are checked for consistency: Virtual Switch Switch Virtual Domain ID Switch Virtual Switch ID Switch Priority Switch Preempt VSL Port Channel Link ID VSL Port state, interfaces Power Redundancy mode Power Enable on VSL cards Note that if configurations do not match, the Hot-Standby Supervisor will revert to RPR mode, disabling all non-vsl interfaces 48

49 Virtual Switching System Unified Control Plane One active supervisor in each chassis with inter-chassis Stateful Switchover (SSO) Active supervisor manages the control plane functions such as protocols (routing, EtherChannel, SNMP, telnet, etc.) and hardware control (Online Insertion Removal, port management) Active/Standby supervisors run in synchronized mode (boot-env, running-configuration, protocol state, and line cards status gets synchronized) CFC or DFC Line Cards CFC or DFC Line Cards CFC or DFC Line Cards SF RP PFC VSL CFC or DFC Line Cards CFC or DFC Line Cards CFC or DFC Line Cards SF RP PFC Active Supervisor Standby HOT Supervisor CFC or DFC Line Cards CFC or DFC Line Cards CFC or DFC Line Cards SSO Synchronization CFC or DFC Line Cards CFC or DFC Line Cards CFC or DFC Line Cards 49

50 Virtual Switching System Dual Active Scenario VSL is the heart of the VSS functionality Protecting VSL link bundle is the best practice design Use one port from Supervisor and other from line cards to form a VSL bundle Use diverse fiber path for each VSL links Manage traffic forwarded over VSL link by avoiding single homed devices In case of loss of all members of the VSL bundle, the standby supervisor will go active, creating dual active condition Dual active leads to Two independent routers with same control plane information e.g. IP address, router ID etc. Error disabling of access-layer due to two STP BPDU sent with different source MAC 50

51 Virtual Switching System Dual Active Forwarding Planes Both forwarding planes are active Standby supervisor and all linecards including DFC s are actively forwarding VSS# show switch virtual redundancy My Switch Id = 1 Peer Switch Id = 2 <snip> Switch 1 Slot 5 Processor Information : Current Software state = ACTIVE <snip> Fabric State = ACTIVE Control Plane State = ACTIVE Switch 2 Slot 5 Processor Information : Current Software state = STANDBY HOT (switchover target) <snip> Fabric State = ACTIVE Data Plane Active Switch1 Si Si Switch2 Data Plane Active Control Plane State = STANDBY 51

52 Virtual Switching System Architecture Multichassis EtherChannel (MEC) Etherchannels can now be extended across the two physical chassis Standalone VSS Both LACP and PAGP Etherchannel protocols and Manual ON modes are supported Regular Etherchannel on single chassis Multichassis EtherChannel across 2 VSS-enabled chassis 52

53 Virtual Switching System Architecture EtherChannel Hash for MEC Etherchannel hashing algorithms are modified in VSS to always favor locally attached interfaces Blue Traffic destined for the Server will result in Link 1 in the MEC link bundle being chosen as the destination path Link 1 Link 2 Orange Traffic destined for the Server will result in Link 2 in the MEC link bundle being chosen as the destination path 53

54 Etherchannel Concepts Etherchannel Hash Distribution The default hashing algorithm will redistribute all the Result Bit Hash values across the available ports when there is a change. This affects all traffic traversing the Etherchannel RBH (for MEC) 2 Link Bundle Example Link 1 Link 2 Flow 1 Flow 2 Flow 3 Flow 4 Flow 5 Flow 6 Flow 7 Flow 8 RBH (for MEC) 3 Link Bundle Example Link 1 Link 2 Link 3 Flow 1 Flow 2 Flow 4 Flow 5 Flow 7 Flow 8 Flow 3 Flow 6 Links 1,2 Links 3,4 Links 1,2,3 Links 4,5,6 54

55 Etherchannel Concepts Etherchannel Hash Distribution Adaptive Adaptive Hash Distribution Enhancement allows for the addition or removal of links in a bundle without affecting all of the traffic in an Etherchannel. Note in the below example, only Flow 7 and 8 are affected by the addition of an extra link to the Channel RBH (for MEC) 2 Link Bundle Example Link 1 Link 2 Flow 1 Flow 2 Flow 3 Flow 4 Flow 5 Flow 6 Flow 7 Flow 8 RBH (for MEC) 3 Link Bundle Example Link 1 Link 2 Link 3 Flow 1 Flow 2 Flow 3 Flow 4 Flow 5 Flow 6 Flow 7 Flow 8 vss#conf t Enter configuration commands, one per line. End with CNTL/Z. vss(config)#port-channel hash-distribution adaptive vss(config)# ^Z vss# 55

56 Virtual Switching System Architecture VSL Initialization 1 Initialization 1 Initialization 2 Pre-Parse Config 2 Pre-Parse Config 3 Bring up VSL Linecards and 3 Bring up VSL Linecards and VSL Ports VSL Ports 4 Run VSLP 4 Run VSLP 5 Run RRP 5 Run RRP 6 Inter-chassis SSO 6 Inter-chassis SSO 7 Continue System Bootup 7 Continue System Bootup 56

57 Virtual Switching System Resilient VSL Configuration Protecting VSL bundle is of the highest priority. VSL bundle is a special purpose EtherChannel however all the best practices of designing and configuring of any general EtherChannel applies to VSL bundle Redundancy of VSL is important to avoid dual ACTIVE condition and instability of VSS Diversify VSL bundle on two separate hardware just like any resilient EtherChannel design VSL link hardware selection also affect the QOS configuration on the rest of the ports on supervisors. 57

58 VSL Design Link Diversification (Dual-Sup Design Option #1) CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard VSS Active CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard Ten 1/1/1 Ten 2/1/1 Ten 1/5/4 Ten 2/5/4 CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard VSS Standby CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard Minimum of two links provides protection from port and SFP failures Separate linecard provides protection from certain interface failures on a single Supervisor Diverse physical paths protect from physical layer outages Requires a VSL-capable linecard 58

59 VSL Design Link Diversification (Dual-Sup Design Option #2) CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard VSS Active CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard Ten 1/5/4 Ten 2/5/4 Ten 1/5/5 Ten 2/5/5 CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard VSS Standby CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard CFC or DFC Linecard Minimum of two links provides protection from port and SFP failures Diverse physical paths protect from physical layer outages No additional VSL-capable linecards are required (Minimal Cost) 59

60 Agenda Current Network Challenges network design with spanning Tree, User downtime, VSS Solution Supervisor 2T Architecture Overview architecture building blocks, hardware and software requirements Migration Use cases description of Standalone, HA and VSS, test traffic profile Migration Walk Through approach, migration Steps, STP and HSRP interaction, traffic re-routing Results Summary and Best Practices convergence summary, verification of Sup2T, VSS verification 60

61 Verify Plan Execute 61

62 Supervisor 2T Migration Walk through

63 Migration Tips Distribution switches MUST BE Spanning-tree ROOT Si 10GE Si Plan the migration with identified backup strategy Make sure to save the configs at each step to disk0: or bootflash: Use console connection during Migration process (if possible), telnet or ssh connections can be lost. It is a best practice to move the HSRP(Layer 3) first to redundant switch followed by spanning tree root for optimal results and convergence Access Switch or ToR or Blades Download the Sup2T supported image in advance to external compact flash before migration Use root guard at the edge ports to protect external switch introducing superior BPDUs, e.g. temporary connectivity Use Spanning tree portfast on all the access ports connected to servers and hosts 63

64 Software Recommendation For Your Reference Platform IOS version Minimum Recommended Supervisor (SXI3) 12.2(SXJ) Supervisor 2T 12.2 (SY) 15.1(SY1) VSS cannot be formed between Sup 720 running 12.2 SX and Sup 2T running 12.2(SY) or 15.0 (SY) 15.1 train is the long lived release Catalyst 6500 with Sup 720 Minimum Recommended Cisco IOS Release Catalyst 6500 with Sup 2T Recommended Cisco IOS Release 64

65 Case 1 : Single/Dual Supervisor 720 Migration 65

66 Case 1: Standalone Supervisor Migration Current Network Traditionally, traffic is load-balanced among distribution switches using vlan loadsharing and HSRP configuration L3 Core STP & HSRP Active Red Vlan Dist-1 Dist-2 STP & HSRP Active Green Vlan Distribution Access 66

67 Case 1 : Pre Migration Checks Verifying STP and HSTP states on Dist-1 Switch Dist-1#sh spanning-tree vlan 10 VLAN0010 Spanning tree enabled protocol rstp Root ID Priority 8192 Address 0017.df3f.e80a This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority 8192 Address 0017.df3f.e80a Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Desg FWD P2p Dist-1#sh spanning-tree vlan 20 VLAN0020 Spanning tree enabled protocol rstp Root ID Priority 8192 Address Cost 1 Port 1665 (Port-channel1) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority Address 0017.df3f.e814 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Root FWD P2p Dist-1#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Active local Vl P Standby local How to read Dist-1 is root bridge for vlan 10 and secondary root for vlan 20 Dist-1 is HSRP active for group 10 67

68 Case 1: Migration of Dist-2 Switch Step-1 Shift the HSRP Primary to Dist-1 Make Dist-1 switch HSRP primary for Green vlans Neighboring devices will detect this change and switch all traffic to Dist-1 switch Dist-2(config)#int vlan 20 Dist-2(config-if)#standby 20 priority 100 *Apr 20 02:00:15.047: %HSRP-5-STATECHANGE: Vlan20 Grp 20 state Active -> Speak *Apr 20 02:00:26.515: %HSRP-5-STATECHANGE: Vlan20 Grp 20 state Speak -> Standby STP & HSRP Active Dist-1 Red Vlan Green Vlan Dist-2 Dist-2#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Standby local Vl P Standby local Dist-1# *Apr 20 02:01:19.559: %HSRP-5-STATECHANGE:Vlan20 Grp 20 state Standby -> Active Dist-1#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Active local Vl P Active local Green vlans traversing through the Dist-2 will be affected due to HSRP change for 4 secs, largely depends on the HSRP timers 68

69 Case 1: Migration of Dist-2 Switch Step-2 Move Spanning root primary to Dist-1 Make Dist-1 switch STP root for Green vlans Neighboring devices will detect this change and switch all traffic to Dist-1 switch Shutdown Dist-2 physical interfaces to completely remove Dist-2 switch from the network Dist-1(config)#spanning-tree vlan 20 root primary Dist-1(config)#end STP & HSRP Active Dist-1 Red Vlan Green Vlan Dist-2 Dist-1#sh spanning-tree vlan 20 VLAN0020 Spanning tree enabled protocol rstp Root ID Priority 8192 Address 0017.df3f.e814 This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority 8192 Address 0017.df3f.e814 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Red vlans traversing through the Dist-2 will be affected, due to STP root change, for 30 secs, largely depends on the STP mode Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Desg FWD P2p 69

70 Case 1 : New Supervisor 2T insertion Step-3 Insertion of Sup 2T and configuration Remove Sup720 and all incompatible linecards from the chassis Replace non-e with E-series chassis and insert Supervisor 2T Boot Sup2T compatible image from rommon, copy the saved configuration from compact flash to running Validate the configs for Sup2T Dist-2 with Sup2T will bootup as HSRP/STP secondary for all vlans as configured The traffic will still be flowing through the Dist-1 STP & HSRP Active Dist-1 Red Vlan Green Vlan rommon>boot disk0:s2txx_new_sup2t_image Boot the new Supervisor 2T image from compact disk in rommon prompt,copied from cisco.com <omit output> Dist-2#copy disk0:saved_config system:running_config Dist-2 L3 Core STP & HSRP Secondary Distribution Access

71 Case 1: Migration of Dist-2 Switch Step-4 Un-shut the interfaces on Sup2T After config validation un-shut Dist-2 physical interfaces and port-channel between two peers Dist-2 will become HSRP/STP secondary There will be no impact on the traffic flowing through Dist-1 till this step Dist-2(configs)#int range gi2/48,gi2/3 4, int po 1 Dist-2(configs-if-range)#no shut Dist-2#show interfaces gi2/4 To Core Block GigabitEthernet2/4 is up, line protocol is up (connected) Hardware is C6k 1000Mb 802.3, address is 001f.6cf6.527c (bia 001f.6cf6.527c) - Omit Output - STP & HSRP Active Dist-1 Red Vlan Green Vlan L3 Core Dist-2 STP & HSRP Secondary Distribution Access Dist-2#show interfaces gi2/48 To Access Switch GigabitEthernet2/48 is up, line protocol is up (connected) Hardware is C6k 1000Mb 802.3, address is 001f.6cf6.528f (bia 001f.6cf6.528f) - Omit Output - Dist-2#show interfaces Po 1 To Primary HSRP Switch Port-channel1 is up, line protocol is up (connected) Hardware is EtherChannel, address is 588d.09e6.81ab (bia 588d.09e6.81ab) - Omit Output - 71

72 Case 1 : Post Migration Checks Verifying STP and HSTP states on Dist-2 Switch after migration Dist-2#sh spanning-tree vlan 20 VLAN0020 Spanning tree enabled protocol rstp Root ID Priority 8192 Address 0017.df3f.e814 Cost 1 Port 1665 (Port-channel1) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority 8192 Address Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Root FWD P2p Dist-2#sh spanning-tree vlan 10 VLAN0010 Spanning tree enabled protocol rstp Root ID Priority 8192 Address 0017.df3f.e80a Cost 1 Port 1665 (Port-channel1) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority Address a Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Root FWD P2p Dist-2#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Standby local Vl P Standby local How to read Dist-2 is now secondary root bridge for vlan 10 and vlan 20 Dist-2 is HSRP standby for group 10 and 20 after migration 72

73 Case 1 : Verification Supervisor 2T Verification Dist-2#show version Cisco IOS Software, s2t54 Software (s2t54- IPBASEK9-M), Version 15.1(1)SY, RELEASE SOFTWARE (fc5) Technical Support: Copyright I by Cisco Systems, Inc. Compiled Tue 27-Sep-11 02:02 by prod_rel_team ROM: System Bootstrap, Version 12.2(50r)SYS2, RELEASE SOFTWARE (fc1) Dist-2uptime is 51 minutes Uptime for this control processor is 51 minutes System returned to ROM by power on System image file is "bootdisk:s2t54-ipbasek9- mz.spa sy.bin" Last reload reason: power-on - Omit Output - Cisco WS-C6509-E (M8572) processor (revision) with K/262144K bytes of memory. Processor board ID SMG0929N81U CPU: MPC8572_E, Version: 2.1, (0x80E80021) CORE: E500, Version: 3.0, (0x ) CPU:1500MHz, CCB:600MHz, DDR:600MHz L1: D-cache 32 kb enabled I-cache 32 kb enabled Last reset from power-on Dist-2# show module Mod Ports Card Type Model Serial No DCEF2T 8 port 10GE WS-X G SAL16095SXR 2 48 CEF port 10/100/1000mb Ethe WS-X6748-GE-TX SAL1208GW5C 5 5 Supervisor Engine 2T 10GE w/cts (Acti VS-SUP2T-10G SAL16020SSN Mod MAC addresses Hw Fw Sw Status b to 442b cf (50r)SYL 15.1(1)SY Ok 2 001f.6cf to 001f.6cf6.52f (14r)S5 15.1(1)SY Ok 5 588d.098a.b517 to 588d.098a.b (50r)SYS 15.0(1)SY Ok Mod Sub-Module Model Serial Hw Status Distributed Forwarding Card WS-F6K-DFC4-E SAL16095R3F 1.2 Ok 2 Centralized Forwarding Card WS-F6700-CFC SAL1207GEH3 4.0 Ok 5 Policy Feature Card 4 VS-F6K-PFC4 SAL16010C7B 1.1 Ok 5 CPU Daughterboard VS-F6K-MSFC5 SAL16020TKS 1.3 Ok Mod Online Diag Status Pass 2 Pass 5 Pass Verify the Sup2T insertion and software version using show module and show version 73

74 Case 1 : Secondary Switch Migration Convergence result during Dist-2 Upgrade - a brief 4 sec traffic drop was seen for Green vlans during HSRP convergence - during Spanning tree root primary change, 30 sec traffic disruption was seen for Green vlans due to STP re-calculations 74

75 Case 1: Migration of Dist-1 Switch Step-5 Shift the HSRP Primary to Dist-2 Configure Dist-1 switch as HSRP secondary for all the vlans, Dist-2 will become HSRP primary Neighboring devices will detect and switch all traffic to Dist-2 switch Dist- 1 L3 Core STP & HSRP Active Dist-2 Red Vlan Green Vlan Dist1(config)#int vlan 20 Dist-1(config-if)#standby 20 priority 90 *Apr 20 02:00:15.047: %HSRP-5-STATECHANGE: Vlan20 Grp 20 state Active -> Speak *Apr 20 02:00:26.515: %HSRP-5-STATECHANGE: Vlan20 Grp 20 state Speak -> Standby Dist1(config)#int vlan 10 Dist-1(config-if)#standby 10 priority 90 *Apr 20 02:00:20.047: %HSRP-5-STATECHANGE: Vlan10 Grp 10 state Active -> Speak *Apr 20 02:00:35.515: %HSRP-5-STATECHANGE: Vlan10 Grp 10 state Speak -> Standby Dist-1#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Standby local Vl P Standby local Dist-2# *Apr 20 02:01:19.559: %HSRP-5-STATECHANGE:Vlan20 Grp 20 state Standby -> Active *Apr 20 02:01:19.559: %HSRP-5-STATECHANGE:Vlan20 Grp 10 state Standby -> Active Dist-2#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Active local Vl P Active local Distribution Access Red and Green vlans traversing through the Dist-1 will experience a brief outage due to HSRP change for 4 secs, largely depends on the HSRP timers 75

76 Case 1: Migration of Dist-1 Switch Step-6 Move Spanning root primary to Dist-2 Adjust the priorities to make Dist-2 STP root primary and Dist-1 STP root secondary for all vlans Neighboring devices will detect and switch traffic to Dist-2 switch Shutdown Dist-1 physical interfaces to completely remove Dist-1 switch from the network Dist-2(config)#spanning-tree vlan 10,20 priority 4096 Dist-2(config)#end Dist-2#sh spanning-tree vlan 20 VLAN0020 Spanning tree enabled protocol rstp Root ID Priority 4096 Address 0017.df3f.e814 This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority 4095 Address 0017.df3f.e814 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Desg FWD P2p Dist- 1 L3 Core STP & HSRP Active Dist-2 Red Vlan Green Vlan Distribution Access Vlans traversing through the Dist-1 will be affected, due to STP root change, for 30 secs, largely depends on the STP mode 76

77 Case 1 : New Supervisor 2T insertion Step-7 Insertion of Sup 2T and configuration Repeat the steps from step 3 step 5 to upgrade Sup2T in Dist-1 switch Verify that Supervisor 2T come up with supported software image Verify that Red vlan traffic is taking Dist-1 path and Green vlan traffic is taking Dist-2 path Verify all the L3-routing is converged All the unsupported linecards will remain in power denied state Both the chassis in distribution pair have now been migrated to new generation Supervisor 2T Same steps has to be followed for Dual Supervisors 720 in HA mode Red vlans traversing through the Dist-2 will be affected, due to STP root change and HSRP convergence, for 34 secs

78 Case 1 : Secondary Switch Migration Convergence result during Dist-1 Upgrade - a brief 4 sec traffic drop was seen for all vlans during HSRP convergence - during Spanning tree root primary change, 30 sec traffic disruption was seen for all vlans due to STP re-calculations 78

79 Case 1 : Final State Design L3 Core STP & HSRP Active Red Vlan Dist-1 Dist-2 STP & HSRP Active Green Vlan Distribution Access 79

80 Case 2 : Standalone to Virtual Switch System (VSS) Migration with Sup720 80

81 VSS Migration Tips VSS (Physical View) VSS domain switch MUST BE spanning-tree ROOT Make sure to save the configs at each step to disk0: or bootflash: Use console connection during Migration process Si 10GE Si It is a best practice to move the HSRP(Layer 3) first to redundant switch followed by spanning tree root for optimal results and convergence 802.3ad or PagP 802.3ad Remove HSRP configs and assigns the same virtual IP addresses to vlan SVIs. Download the Sup2T supported image in advance to external compact flash before migration Access Switch or ToR or Blades Server Use root guard at the edge ports to protect external switch introducing superior BPDUs, e.g. temporary connectivity Use Spanning tree portfast on all the access ports connected to servers and hosts Do not use loop guard as it will disable the entire MEC channel on fault detection 81

82 Case 2: Standalone Supervisor Migration in VSS Current Network Traditionally, traffic is load-balanced among distribution switches using vlan loadsharing and HSRP configuration L3 Core STP & HSRP Active Red Vlan Dist-1 Dist-2 STP & HSRP Active Green Vlan Distribution Access 82

83 Case 3: Standalone to VSS Migration Migration to VSS Multi Step Process L3 Core Migration Steps between Distribution and core 1. Configure MEC 2. Remove Routing Statements which are not needed. L2/L3 Distribution Migration Steps between Distribution and Access-layer 1. Modify FHRP Configuration 2. Configure Multichassis Ethrechannel Access 3. Move L2 Trunk configuration to MEC interfaces 4. Move Policies to MEC if needed 5. Keep Spanning-Tree Enabled 83

84 Case 2 : Pre Migration Checks Verifying STP and HSTP states on Dist-1 Switch Dist-1#sh spanning-tree vlan 10 VLAN0010 Spanning tree enabled protocol rstp Root ID Priority 8192 Address 0017.df3f.e80a This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority 8192 Address 0017.df3f.e80a Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Desg FWD P2p Dist-1#sh spanning-tree vlan 20 VLAN0020 Spanning tree enabled protocol rstp Root ID Priority 8192 Address Cost 1 Port 1665 (Port-channel1) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority Address 0017.df3f.e814 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Root FWD P2p Dist-1#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Active local Vl P Standby local How to read Dist-1 is root bridge for vlan 10 and secondary root for vlan 20 Dist-1 is HSRP active for group 10 84

85 Case 2: Migration of Dist-2 Switch Step-1 Shift the HSRP Primary to Dist-1 Make Dist-1 switch HSRP primary for Green vlans Neighboring devices will detect this change and switch all traffic to Dist-1 switch Dist-2(config)#int vlan 20 Dist-2(config-if)#standby 20 priority 100 *Apr 20 02:00:15.047: %HSRP-5-STATECHANGE: Vlan20 Grp 20 state Active -> Speak *Apr 20 02:00:26.515: %HSRP-5-STATECHANGE: Vlan20 Grp 20 state Speak -> Standby STP & HSRP Active Dist-1 Red Vlan Green Vlan Dist-2 Dist-2#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Standby local Vl P Standby local Dist-1# *Apr 20 02:01:19.559: %HSRP-5-STATECHANGE:Vlan20 Grp 20 state Standby -> Active Dist-1#sh standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Vl P Active local Vl P Active local Green vlans traversing through the Dist-2 will be affected due to HSRP change for 4 secs, largely depends on the HSRP timers 85

86 Case 2: Migration of Dist-2 Switch Step-2 Move Spanning root primary to Dist-1 Make Dist-1 switch STP root for Green vlans Neighboring devices will detect this change and switch all traffic to Dist-1 switch Shutdown Dist-2 physical interfaces to completely remove Dist-2 switch from the network Dist-1(config)#spanning-tree vlan 20 root primary Dist-1(config)#end STP & HSRP Active Dist-1 Red Vlan Green Vlan Dist-2 Dist-1#sh spanning-tree vlan 20 VLAN0020 Spanning tree enabled protocol rstp Root ID Priority 8192 Address 0017.df3f.e814 This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority 8192 Address 0017.df3f.e814 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 480 Red vlans traversing through the Dist-2 will be affected, due to STP root change, for 30 secs, largely depends on the STP mode Interface Role Sts Cost Prio.Nbr Type Gi8/2 Desg FWD P2p Peer(STP) Po1 Desg FWD P2p 86

87 Case 2 : New Supervisor 2T insertion Step-3 Insertion of Sup 2T and configuration Remove Sup720 and all incompatible linecards from the chassis Replace non-e with E-series chassis and insert Supervisor 2T Boot Sup2T compatible image from rommon, copy the saved configuration from compact flash to running Validate the configs for Sup2T Dist-2 with Sup2T will bootup as HSRP/STP secondary for all vlans as configured The traffic will still be flowing through the Dist-1 STP & HSRP Active Dist-1 Red Vlan Green Vlan rommon>boot disk0:s2txx_new_sup2t_image Boot the new Supervisor 2T image from compact disk in rommon prompt,copied from cisco.com <omit output> Dist-2#copy disk0:saved_config system:running_config L3 Core STP & HSRP Secondary Dist-2 Distribution Access

88 Case 2 : Verification Supervisor 2T Verification Dist-2#show version Cisco IOS Software, s2t54 Software (s2t54- IPBASEK9-M), Version 15.1(1)SY, RELEASE SOFTWARE (fc5) Technical Support: Copyright I by Cisco Systems, Inc. Compiled Tue 27-Sep-11 02:02 by prod_rel_team ROM: System Bootstrap, Version 12.2(50r)SYS2, RELEASE SOFTWARE (fc1) Dist-2uptime is 51 minutes Uptime for this control processor is 51 minutes System returned to ROM by power on System image file is "bootdisk:s2t54-ipbasek9- mz.spa sy.bin" Last reload reason: power-on - Omit Output - Cisco WS-C6509-E (M8572) processor (revision) with K/262144K bytes of memory. Processor board ID SMG0929N81U CPU: MPC8572_E, Version: 2.1, (0x80E80021) CORE: E500, Version: 3.0, (0x ) CPU:1500MHz, CCB:600MHz, DDR:600MHz L1: D-cache 32 kb enabled I-cache 32 kb enabled Last reset from power-on Dist-2# show module Mod Ports Card Type Model Serial No DCEF2T 8 port 10GE WS-X G SAL16095SXR 2 48 CEF port 10/100/1000mb Ethe WS-X6748-GE-TX SAL1208GW5C 5 5 Supervisor Engine 2T 10GE w/cts (Acti VS-SUP2T-10G SAL16020SSN Mod MAC addresses Hw Fw Sw Status b to 442b cf (50r)SYL 15.1(1)SY Ok 2 001f.6cf to 001f.6cf6.52f (14r)S5 15.1(1)SY Ok 5 588d.098a.b517 to 588d.098a.b (50r)SYS 15.0(1)SY Ok Mod Sub-Module Model Serial Hw Status Distributed Forwarding Card WS-F6K-DFC4-E SAL16095R3F 1.2 Ok 2 Centralized Forwarding Card WS-F6700-CFC SAL1207GEH3 4.0 Ok 5 Policy Feature Card 4 VS-F6K-PFC4 SAL16010C7B 1.1 Ok 5 CPU Daughterboard VS-F6K-MSFC5 SAL16020TKS 1.3 Ok Mod Online Diag Status Pass 2 Pass 5 Pass Verify the Sup2T insertion and software version using show module and show version 88

89 Case 2 : Secondary Switch Migration Convergence result during Dist-2 Upgrade - a brief 4 sec traffic drop was seen for Green vlans during HSRP convergence - during Spanning tree root primary change, 30 sec traffic disruption was seen for Green vlans due to STP re-calculations 89

90 Case 2 - Conversion to VSS Convert Sup2T in Dist-2 to run in VSS mode Supervisor uplink interfaces are utilized to form a VSL link Dist - 1 Dist - 2 T5/4 T5/5 VSL Link Bundle T5/4 T5/5 Port-Channel 1 Port-Channel 2 Switch Virtual Domain #100 90

91 Case 2 - Conversion to VSS Step- 4 Configuration for the conversion takes the following path Dist-2(config)#switch virtual domain 100 Domain ID 100 config will take effect only after the exec command 'switch convert mode virtual' is issued Dist-2(config-vs-domain)#switch 1 Dist-2(config-vs-domain)#mac-address use-virtual Dist-2(config)#udld enable Dist-2(config)#spanning-tree mode rapid-pvst Dist-2(config)#spanning-tree vlan priority Configure Switch Virtual Domain Configure Switch id Configure virtual mac address Enable udld Dist- 2 Dist-2(config-red)#int po 1 Dist-2(config-if)#switch virtual link 1 WARNING: Interface Port-channel1 placed in restricted config mode. All extraneous configs removed! WARNING: Interface TenGigabitEthernet5/4 placed in restricted config mode. All extraneous configs removed! WARNING: Interface TenGigabitEthernet5/5 placed in restricted config mode. All extraneous configs removed! Dist-2(config)#int range tengigabitethernet 5/4 5 Dist-2(config-if-range)#shutdown Dist-2(config-if-range)#channel-group 1 mode on Configure spanning tree for all vlans Configure VSL port-channel Assign the Sup2T uplinks to VSL port-channel Verify that the switch is still working in standalone mode Dist-2# show switch virtual Switch Mode : Standalone Not in Virtual Switch mode due to: Domain ID is configured but invalid SWITCH_NUMBER 0 setting. This implies an incomplete or failed Virtual Switch conversion process. 91

92 Case 2 - Conversion to VSS Step- 5 Convert the mode to virtual Dist - 2 Dist-2#switch convert mode virtual This command will convert all interface names to naming convention "interface-type switch-number/slot/port, save the running config to startup-config and reload the switch. NOTE: Make sure to configure one or more dual-active detection methods once the conversion is complete and the switches have come up in VSS mode. Do you want to proceed? [yes/no]: Converting interface names Building configuration [OK] Saving converted configuration to bootdisk: Destination filename [startup-config.converted_vs ]? AT THIS POINT THE SWITCH WILL REBOOT SWITCH CONSOLE OUTPUT After reload < snip > *Apr 20 04:59:53.999: %PFREDUN-6-ACTIVE: Initializing as ACTIVE processor for this switch *Apr 20 05:00:04.843: %VSL_BRINGUP-6-MODULE_UP: VSL module in slot 5 switch 2 brought up *Apr 20 05:02:06.363: %VSLP-5-RRP_PEER_TIMEOUT: VSLP peer timer expired without detecting peer. Resolving role as Active *Apr 20 05:02:06.415: %VSLP-2-VSL_DOWN: < snip.> VSL links down and not ready for any traffic The most important command Switch will reload after proceeding with this command How to read the output Switch will bootup as ACTIVE with VSL link in shutdown state *Apr 20 05:03:59.795: %DIAG-SW2-6-DIAG_OK: Switch 2 Module 1: Passed Online Diagnostics *Apr 20 05:03:59.987: %SATVS_IBC-SW2-5-VSL_DOWN_SCP_DROP: VSL inactive - dropping cached SCP packet: (SA/DA:0x0/0x4, SSAP/DSAP:0x0/0x1, OP/SEQ:0x1030/0x8, SIG/INFO:0x1/0x21, esa: ) 92

93 Case 2 - Conversion to VSS VSS# sh switch virtual role Switch Switch Status Preempt Priority Role Session ID Number Oper(Conf) Oper(Conf) Local Remote LOCAL 1 DOWN FALSE(N ) 110(110) ACTIVE 0 0 VSS Switch - 1 In dual-active recovery mode: No How to read the output Checking the local switch Configured switch id is 1 VSL status is down No Pre-empt configuration Configured and Operational priority is 110, derived during bootup Since there is no other peer, the switch boots up as ACTIVE switch Dist-2 is now converted to Sup 2T VSS in Active role 93

94 Case 2 - Conversion to VSS Step- 6 Pre-configure VSS Switch-1 Now that Dist-2 is successfully converted and It is operating in VSS mode, perform below steps to pre configure VSS Switch-1 (Dist-2) 1. Pre-configure MEC (Multi Chassis Ether Channel) using Switch-1 local interfaces, Switch-2 will be Dist-1 once it is converted, interfaces can be added to MECs after it s conversion to VSS at later steps 2. Move HSRP Virtual IP address to Vlan interfaces 3. Remove HSRP config, (active and standby chassis will be using active chassis burnt-in macaddress and Vlan ip address. HSRP is no longer required ) 4. Turn On NSF-SSO (Non-Stop forwarding) feature for routing protocol 5. VSS simplifies the routing configuration 6. Modify STP configuration such that VSS switch-1 be the root for all vlans Pre-configuration steps can also be performed after converting Dist-2 to VSS as well. Pre-configuration helps to reduces amount of packet loss during migration. 94

95 Case 2 - Conversion to VSS Pre-configure VSS Switch-1 L3 Core TGig2/2 TGig2/1 TGig1/2/2 TGig1/1/21 STP & HSRP Active Red Vlan Green Vlan Dist-1 Gig1/2 Gig1/3 Gig1/1 Gig1/1/2 VSS Gig1/1/3 Dist-2 Gig1/1/1 Distribution Access 95

96 Case 2 Configuration Migration Step- 6 Configuration Migration: Pre-Configure MEC Choose a unique port channel (MEC) id for each of the neighbor device that is dual homed to VSS VSS Active Configure MEC Move Interface configuration to MEC Traditional config VSS Active VSS(config)#int gig 1/1/1 MEC to Core VSS(config-if)#no ip add VSS(config-if)#int po20 VSS(config-if)#ip add VSS(config-if)no shut VSS(config-if)#int gig 1/1/1 VSS(config-if)#channel-group 20 mode desirable interface TenGigabitEthernet1/2/1 ip address interface GigabitEthernet1/1/2 switchport switchport trunk encapsulation dot1q switchport trunk allowed vlan 10,20 VSS(config-if)#int po10 MEC to Access VSS(config-if)#switchport VSS(config-if)#switchport trunk encap dot1q VSS(config-if)#switchport trunk allowed vlan 10,20 VSS(config-if)no shut VSS(config-if)#int gig 1/1/2 VSS(config-if)#switchport VSS(config-if)# channel-group 10 mode desirable 96

97 Case 2 Configuration Migration Step- 6 Configure port channel in VSS neighbor device VSS neighbor device interfaces will be down at this moment, port channel can be configured without interfering traffic flowing through Dist-1 Core Configure Layer-3 port channel Configure Layer-2 port channel Access Core(config-if)#int gig 1/1 Core(config-if)#no ip address Core(config-if)#int po20 Core(config-if)# ip address Core(config-if)#no shut Core(config-if)#int gig 1/1 Core(config-if)#channel-group 20 mode desirable Access(config)#int po10 Access(config)#switchport Access(config)# switchport trunk encapsulation dot1q Access(config)#switchport mode trunk Access(config)#no shut Access(config)#int range gig 1/1 Access(config-if#channel-group 10 mode desirable 97

98 Case 2 Configuration Migration Step- 6 Configuration Migration : Remove Gateway Protocol End devices are still pointing their arp entries to HSRP mac-address, till the entry times out or re-arp would update their cache. Temporarily connectivity may be lost during this time. VSS Active VSS Active Remove HSRP Standby config Move HSRP Standby IP Address to the Vlan interfaces Traditional config interface Vlan10 ip address standby 10 ip standby 10 priority 110! interface Vlan20 ip address standby 20 ip standby 20 priority 110 VSS(config)#interface Vlan10 VSS(config-if)# no standby 10 ip VSS(config-if)# no standby 10 pri 110 VSS(config-if)#ip address VSS(config)#interface Vlan20 VSS(config-if)# no standby 20 ip VSS(config-if)# no standby 20 pri 110 VSS(config-if)# ip address

99 Case 2 Configuration Migration Step- 6 Configuration Migration : Update Routing Protocol config Enable NSF/SSO and Remove Routing statements that are no longer needed with VSS VSS Active Previous L3 interfaces are merged as MEC, hence some routing statement are not needed.. VSS#sh run beg ospf router ospf 10 log-adjacency-changes network area 0 network area 0 network area 0 network area 0 VSS(config)#router ospf 10 VSS(config-router)# nsf VSS(config-router)# no network area 0 Core Previous L3 interfaces are merged as MEC, hence some routing statement are not needed.. Core#sh run beg ospf router ospf 1 log-adjacency-changes network area 0 network area 0 Demo-Core(config)#router ospf 1 Demo-Core(config-router)# nsf Demo-Core(config-router)#no network area 0 99

100 Case 2 Sup 2T Migration in VSS mode Step 7 : Verify VSS Switch-1 connectivity VSS Switch-1 is configured to forward traffic while we disconnect Dist-1 and migrate to VSS mode Verify VSS Switch-1 Configuration and connectivity 1. Enable VSS switch-1 interfaces STP & HSRP TGig2/2 TGig2/1 TGig1/2/2 TGig1/1/21 L3 Core 2. Very L2 connectivity to access switches 3. Verify L3 connectivity to core switches 4. After VSS switch-1 connectivity verification, shutdown Dist-1 interfaces to switch traffic over to VSS. Active Red Vlan Green Vlan Dist-1 Gig1/2 Gig1/3 Gig1/1 Gig1/1/2 VSS Gig1/1/3 Gig1/1/1 Dist-2 Distribution secs, traffic loss is expected during this step when all flows move from Dist-1 to VSS-Switch-1 Access 100

101 Case 2 - Conversion to VSS Convert Sup2T in Dist-1 to run in VSS mode Supervisor uplink interfaces are utilized to form a VSL link Dist - 1 Dist - 2 T5/4 T5/5 VSL Link Bundle T5/4 T5/5 Port-Channel 1 Port-Channel 2 Switch Virtual Domain #

102 Case 2 - Conversion to VSS Repeat the same set of Steps from 1 7 Dist-1(config)#switch virtual domain 100 Domain ID 100 config will take effect only after the exec command 'switch convert mode virtual' is issued Dist-1(config-vs-domain)#switch 2 Dist-1(config-vs-domain)#mac-address use-virtual Dist-1(config)#udld enable Dist-1(config-red)#int po 1 Dist-1(config-if)#switch virtual link 1 Configure Switch Virtual Domain Configure Switch id Configure virtual mac address Enable udld Dist- 1 WARNING: Interface Port-channel1 placed in restricted config mode. All extraneous configs removed! WARNING: Interface TenGigabitEthernet5/4 placed in restricted config mode. All extraneous configs removed! WARNING: Interface TenGigabitEthernet5/5 placed in restricted config mode. All extraneous configs removed! Dist-1(config)#int range tengigabitethernet 5/4 5 Dist-1(config-if-range)#shutdown Dist-1(config-if-range)#channel-group 1 mode on Dist-1# show switch virtual Switch Mode : Standalone Not in Virtual Switch mode due to: Domain ID is configured but invalid SWITCH_NUMBER 0 setting. This implies an incomplete or failed Virtual Switch conversion process. Configure VSL port-channel Assign the Sup2T uplinks to VSL port-channel Verify that the switch is still working in standalone mode 102

103 Case 2 Sup 2T Migration in VSS mode Step- 8 Sup 2T VSS Migration Completion Do a no shut on VSL link between Switch1-VSS (Dist-2) and Switch-2 VSS (Dist-1), Switch2-VSS switch will reload and resume the STANDBY role Dist-1(config)#interface po 2 Dist-1(config-if)#no shut *Apr 20 05:22:26.587: %VSLP-SW2-5-RRP_MSG: Use 'redundancy reload shelf' to bring this switch to its preferred STANDBY role *Apr 20 05:22:26.587: %DUAL_ACTIVE-SW2-1-RECOVERY: Dual-active condition detected: Starting recovery-mode, all non-vsl and non-excluded interfaces have been shut down Dist-1(recovery-mode)#redundancy reload shelf System configuration has been modified. Save? [yes/no]: yes Building configuration [OK] Reload this shelf [confirm] Preparing to reload this shelf *Apr 20 05:23:42.083: %RF-SW2-5-RF_RELOAD: Shelf reload. Reason: Admin reload CLI *Apr 20 05:23:42.083: %VSLP-SW2-3-VSLP_LMP_FAIL_REASON: Te2/5/4: Disabled by Admin self reload *Apr 20 05:23:42.083: %VSLP-SW2-3-VSLP_LMP_FAIL_REASON: Te2/5/5: Disabled by Admin self reload *Apr 20 05:23:42.087: %VSLP-SW2-2-VSL_DOWN: All VSL links went down while switch is in ACTIVE role no shut on VSL port channel 1 and port channel 2 on both VSS switches Both VSS switches will detect this change using VSL control messages Switch-1 is already working in VSS as ACTIVE switch, use redundancy reload shelf command to bring the Dist-1 in preferred STANDBY state after the Dist-1 comes back up, do a no shut on MEC port channels to load balance the traffic over to remaining channel members 103

104 Case 2 Sup 2T Migration Completion in VSS mode Supervisor 2T Migration is completed in VSS mode, at this point both switches are active and traffic will be load-balanced on all MEC member interfaces L3 Core VSS Distribution Access 104

105 Case 3 VSS mode Verification VSS redundancy Output Apr 20 05:25:19.015: %PFREDUN-6-ACTIVE: Initializing as ACTIVE processor for this switch *Apr 20 05:25:29.851: %VSL_BRINGUP-6-MODULE_UP: VSL module in slot 5 switch 2 brought up *Apr 20 05:26:03.419: %VSLP-5-RRP_ROLE_RESOLVED: Role resolved as STANDBY by VSLP *Apr 20 05:26:03.419: %VSL-5-VSL_CNTRL_LINK: New VSL Control Link 2/5/4 *Apr 20 05:26:05.483: %VSLP-SW2_STBY-5-VSL_UP: Ready for control traffic *Apr 20 05:26:13.699: SW2_STBY: Bring up standby supervisor as a DFC *Apr 20 05:26:13.775: %PFREDUN-SW2_STBY-6-STANDBY: Initializing for SSO mode < snip > Press RETURN to get started! *Apr 20 05:26:52.663: %PFREDUN-SW2_STBY-6-STANDBY: Ready for SSO mode Dist-2-sdby#sh switch virtual link VSL Status : UP VSL Uptime : 2 minutes VSL SCP Ping : Pass VSL ICC Ping : Pass VSL Control Link : Te2/5/4 Dist-2-sdby#sh switch virtual role RRP information for Instance 2 Switch Switch Status Preempt Priority Role Local Remote Number Oper(Conf) Oper(Conf) SID SID LOCAL 2 UP FALSE(N) 100(100) STANDBY 0 0 REMOTE 1 UP FALSE(N) 100(100) ACTIVE How to read the output Checking at the local switch as it appears first Configured switch ids are 2 and 1 for local and remote respectively VSL status is UP No Pre-empt configuration Configured and Operational priorities are 100, decided during bootup Local switch is working as STANDBY and remote switch is working as ACTIVE Hostname is automatically adjusted as Dist-2 because Dist-2 migrated first and boots up as ACTIVE VSS switch Dist-1 boots up as STANDBY VSS switch 105

106 Case 2 VSS mode Verification VSS redundancy Output Dist-2 #show switch virtual redundancy My Switch Id = 1 Peer Switch Id = 2 Last switchover reason = none Configured Redundancy Mode = sso Operating Redundancy Mode = sso Switch 1 Slot 5 Processor Information : Current Software state = ACTIVE Uptime in current state = 2 hours, 41 minutes Image Version = Cisco IOS Software, s2t54 Software (s2t54- ADVIPSERVICESK9-M), Version 15.1(SY1), RELEASE BOOT = Fabric State = ACTIVE Control Plane State = ACTIVE Switch 2 Slot 5 Processor Information : Current Software state = STANDBY HOT (switchover target) Uptime in current state = 2 minutes BOOT = disk0:s2t54-advipservicesk9- mz.spa.15.1(sy1).bin,1;,1; CONFIG_FILE = BOOTLDR = Configuration register = 0x2102 Fabric State = ACTIVE Control Plane State = STANDBY MEC and VSL Port Channel status Dist-2#sh etherchannel summary Flags: D - down P - bundled in port-channel I - stand-alone s - suspended H - Hot-standby (LACP only) R - Layer3 S - Layer2 U - in use N - not in use, no aggregation f - failed to allocate aggregator M - not in use, no aggregation due to minimum links not met m - not in use, port not aggregated due to minimum links not met u - unsuitable for bundling d - default port w - waiting to be aggregated Number of channel-groups in use: 3 Number of aggregators: 3 Group Port-channel Protocol Ports Po1(RU) - Te1/5/4(P) Te1/5/5(P) 2 Po2(RU) - Te2/5/4(P) Te2/5/5(P) 5 Po5(SU) LACP Gi1/1/2(P) Gi1/1/3(P) 106

107 Case 2 : Sup2T VSS Migration Traffic convergence result During Traffic switchover to VSS-Switch-1 with Sup2T, 50secs disruption for all the vlans were observed due to re-arping for SVI mac addresses and STP convergence 107

108 Case 3 : Sup720 to Sup2T Migration in Virtual Switch System (VSS) mode 108

109 Case 3: Migration of Sup 720 to Sup2T in VSS mode Step-1 Switch traffic over to Dist-1 Dist-1 and Dist-2 switches are configured as VSS ACTIVE and STANDBY pair respectively L3 Core Traffic is hashed on both the switches from access block and core block using MEC port channels Shutdown the MEC members connected to Dist-2, the core and access block will rehash the traffic flow to the remaining active link in MEC port channel towards Dist- 1 Dist-1 VSS Dist-2 Distribution After traffic redirection shutdown the VSL Port channel on both active and standby pair Hot-standby failover does not introduce control plane convergence because it is not actively responsible for managing various protocols and their updates Access a brief packet loss (4 secs) is expected when traffic is being rehashed to the remaining MEC members 109

110 Case 3: Migration of Standby VSS Switch Step-1 Switch traffic over to Dist-1 Dist-2(config)# int range ethernet 2/2/48,gig2/2/3,Gig 2/2/4,Gig 2/2/11 Dist-2(config-if-range)#shut %DUAL-5-NBRCHANGE: EIGRP-IPv41: Neighbor (GigabitEthernet2/2/3) is down: interface down Dist-2(config-if-range)#interface range te 2/5/4-5 Dist-2(config-if-range)#shut WARNING: You are shutting down one or more VSL interfaces. If all VSL interfaces are down, connectivity between active and standby switch (if present) will be lost and would also result in two active switches. Traffic disruption will occur, and possible configuration mismatch between the switches can happen. Do you want to proceed? [yes/no]: yes *Aug 31 17:18:41.469: %VSLP-SW1_SP-3-VSLP_LMP_FAIL_REASON: Te1/5/4: Link down Shutdown all the MEC members on Dist-2 (VSS STANDBY switch) Shutdown the VSL port channel between VSS pair This step will leave the Dist-1 (ACTIVE VSS switch) in simplex mode forwarding all the traffic from remaining MEC members *Aug 31 17:18:41.961: %VSLP-SW1_SP-3-VSLP_LMP_FAIL_REASON: Te1/5/5: Link Dist-1#sh down switch virtual role *Aug 31 17:18:41.961: %VSLP-SW1_SP-2-VSL_DOWN: Last VSL interface Te1/5/5 RRP information went down for Instance 1 *Aug 31 17:18:41.981: %VSLP-SW1_SP-2-VSL_DOWN: All VSL links went down while switch is in ACTIVE role Valid Flags Peer Preferred Reserved *Aug 31 17:18:42.125: SW1_SP: Switch 2 Physical Slot 5 - Module Type LINE_CARD Count Peer Peer removed *Aug 31 17:18:42.133: SW1_SP: Switch 2 Physical Slot 1 - Module Type LINE_CARD TRUE V removed *Aug 31 17:18:42.264: %PFREDUN-SW1_SP-6-ACTIVE: Standby processor removed Switch or Switch Status Preempt Priority Role Local Remote reloaded, changing to Simplex mode Number Oper(Conf) Oper(Conf) SID SID *Aug 31 17:18:42.368: SW1_SP: Switch 2 Physical Slot 2 - Module Type LINE_CARD removed LOCAL 1 UP FALSE(N ) 100(100) ACTIVE 0 0 *Aug 31 17:18:43.944: %SATVS_IBC-SW1_SP-5-VSL_DOWN_SCP_DROP: VSL inactive - dropping cached SCP packet: (SA/DA:0x4/0x4, SSAP/DSAP:0x19/0x0, OP/SEQ:0x2C/0x96D4, Peer 0 represents the local switch SIG/INFO:0x1/0x501, esa: ) Flags : V Valid In dual-active recovery mode: No 110

111 Case 3: Migration of Standby VSS Switch Step-2 Replacing Sup720 with Sup 2T in Dist-2 Standby VSS switch Copy the running configuration on compact flash Remove Sup720 and all incompatible linecards from the chassis L3 Core Replace non-e with E-series chassis and insert Supervisor 2T Dist-1 VSS Dist-2 Boot Sup2T compatible image from rommon, copy the saved configuration from compact flash to running Distribution Validate the configs for Sup2T and convert it in VSS mode. Access Configure the VSL and MEC port-channels, put them in shutdown state. Traffic is still hashed towards Dist-1 switch through MEC members 111

112 Case 3: Migration of Active VSS Switch Step-3 Switch traffic over to Dist-2 VSS cannot be formed between two mismatched supervisors and software images Traffic flow from access and core block has to be redirected to Dist-2 Switch using MEC members This step requires change in spanning tree primary root to Dist-2 switch, shutting down MEC member and VSL port channel on Dist-1 and no shut on Dist-2 MEC member interfaces along with VSL port channel all in one attempt This step will result in traffic disruption (30 secs for re-arping) When the traffic flow is re-directed to Dist-2 switch remove Sup720 from Dist-1 Repeat the steps of replacing Sup 720 to Sup 2T L3 Core VSS Dist-1 Dist-2 Distribution Access Recommendation Script this step, for less traffic disruption 112

113 Case 3 Sup 2T Migration in VSS mode Step- 4 Bringing up Dist-1 Switch Do a no shut on VSL link between Switch1-VSS (Dist-2) and Switch-2 VSS (Dist-1), Switch2-VSS switch will reload and resume the STANDBY role Dist-1(config)#interface po 2 Dist-1(config-if)#no shut *Apr 20 05:22:26.587: %VSLP-SW2-5-RRP_MSG: Use 'redundancy reload shelf' to bring this switch to its preferred STANDBY role *Apr 20 05:22:26.587: %DUAL_ACTIVE-SW2-1-RECOVERY: Dual-active condition detected: Starting recovery-mode, all non-vsl and non-excluded interfaces have been shut down Dist-1(recovery-mode)#redundancy reload shelf System configuration has been modified. Save? [yes/no]: yes Building configuration [OK] Reload this shelf [confirm] Preparing to reload this shelf *Apr 20 05:23:42.083: %RF-SW2-5-RF_RELOAD: Shelf reload. Reason: Admin reload CLI *Apr 20 05:23:42.083: %VSLP-SW2-3-VSLP_LMP_FAIL_REASON: Te2/5/4: Disabled by Admin self reload *Apr 20 05:23:42.083: %VSLP-SW2-3-VSLP_LMP_FAIL_REASON: Te2/5/5: Disabled by Admin self reload *Apr 20 05:23:42.087: %VSLP-SW2-2-VSL_DOWN: All VSL links went down while switch is in ACTIVE role no shut on VSL port channel 1 and port channel 2 on both VSS switches Both VSS switches will detect this change using VSL control messages Switch-1 is already working in VSS as ACTIVE switch, use redundancy reload shelf command to bring the Dist-1 in preferred STANDBY state after the Dist-1 comes back up, it will sync up the configs from active VSS switch (Dist-2) do a no shut on MEC port channels to load balance the traffic over to remaining channel members 113

114 Case 3 Sup 2T Migration completed in VSS mode Sup 2T VSS Migration Completion Supervisor 2T Migration is completed in VSS mode, at this point both switches are active and traffic will be load-balanced on all uplink interfaces L3 Core VSL Distribution Access 114

115 Case 3 VSS mode Verification VSS redundancy Output Dist-2 #show switch virtual redundancy My Switch Id = 1 Peer Switch Id = 2 Last switchover reason = none Configured Redundancy Mode = sso Operating Redundancy Mode = sso Switch 1 Slot 5 Processor Information : Current Software state = ACTIVE Uptime in current state = 2 hours, 41 minutes Image Version = Cisco IOS Software, s2t54 Software (s2t54- ADVIPSERVICESK9-M), Version 15.1(SY1), RELEASE BOOT = Fabric State = ACTIVE Control Plane State = ACTIVE Switch 2 Slot 5 Processor Information : Current Software state = STANDBY HOT (switchover target) Uptime in current state = 2 minutes BOOT = disk0:s2t54-advipservicesk9- mz.spa.15.1(sy1).bin,1;,1; CONFIG_FILE = BOOTLDR = Configuration register = 0x2102 Fabric State = ACTIVE Control Plane State = STANDBY MEC and VSL Port Channel status Dist-2#sh etherchannel summary Flags: D - down P - bundled in port-channel I - stand-alone s - suspended H - Hot-standby (LACP only) R - Layer3 S - Layer2 U - in use N - not in use, no aggregation f - failed to allocate aggregator M - not in use, no aggregation due to minimum links not met m - not in use, port not aggregated due to minimum links not met u - unsuitable for bundling d - default port w - waiting to be aggregated Number of channel-groups in use: 3 Number of aggregators: 3 Group Port-channel Protocol Ports Po1(RU) - Te1/5/4(P) Te1/5/5(P) 2 Po2(RU) - Te2/5/4(P) Te2/5/5(P) 5 Po5(SU) LACP Gi1/1/2(P) Gi1/1/3(P) 115

116 Agenda Current Network Challenges network design with spanning Tree, User downtime, VSS Solution Supervisor 2T Architecture Overview architecture building blocks, hardware and software requirements Migration Use cases description of Standalone, HA and VSS, test traffic profile Migration Walk Through approach, migration Steps, STP and HSRP interaction, traffic re-routing Results Summary and Best Practices convergence summary, verification of Sup2T, VSS verification 116

117 Migration Results Summary

118 Migration Results Traffic and impact on Summary Green Vlans moved from Dist-2 switch to Dist-1 switch, 34 Single/Dual Supervisor Migration seconds Traffic impact on all the vlans moved from Dist-1 switch to Dist-2 switch, 34 seconds Traffic impact on Red Vlans moved from Dist-2 switch to Dist-1 switch, 34 seconds Standalone to VSS Migration Traffic impact on Green Vlans moved from Dist-2 switch to Dist-1 switch, 34 seconds Traffic impact on all the vlans moved from Dist-1 switch to Active VSS switch, 50 seconds Traffic hashed to remaining Dist-1 MEC members, 4 seconds VSS to VSS Migration Traffic impact of shutting down MEC members on Standby VSS switch, to re-hash the traffic on Active VSS switch, 4 seconds Traffic impact of all the vlans re-directed from Dist-1 switch to Dist-2 VSS switch, 30 seconds 118

119 Q & A

120 References Supervisor 2T Migration White Paper Supervisor 2T Architecture Cisco Virtual Switching System Design Guide Migrate Standalone Cisco Catalyst 6500 Switch to Cisco Catalyst 6500 Virtual Switching System High Availability Campus Network Design: Routed Access Layer using EIGRP or OSPF Enterprise Campus 3.0 Architecture: Overview and Framework For Your Reference 120

121 Recommended Reading for Please visit the Cisco Book Store in the World of Solutions and browse through the extensive range of Cisco Press titles. 121

122 Call to Action Visit the Cisco Campus at the World of Solutions to experience Cisco innovations in action Get hands-on experience attending one of the Walk-in Labs Schedule face to face meeting with one of Cisco s engineers at the Meet the Engineer center Discuss your project s challenges at the Technical Solutions Clinics 122

123 Complete Your Online Session Evaluation Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Cisco Daily Challenge points for each session evaluation you complete. Complete your session evaluation online now through either the mobile app or internet kiosk stations. Maximize your Cisco Live experience with your free Cisco Live 365 account. Download session PDFs, view sessions on-demand and participate in live activities throughout the year. Click the Enter Cisco Live 365 button in your Cisco Live portal to log in. 123

124 Thank you

125 For Your Reference More commands for configuration and verification 125

126 switch virtual domain 10! Must configure unique domain ID VSS Global Configuration switch mode virtual switch 1 priority 110! Not needed, helps in operational mgmt switch 2 priority 100! Not needed, helps in operational mgmt dual-active exclude interface GigabitEthernet1/5/3! Connectivity to VSS during dual active mac-address use-virtual! Required for consistent MAC address dual-active detection pagp trust channel-group 202!Enhanced PAgP based dual-active detection redundancy! Default SSO Enabled main-cpu auto-sync running-config mode sso interface Port-channel1!Unique portchannel number for SW 1 description VSL Link from Switch 1 no switchport no ip address switch virtual link 1!Defines switch ID for SW 1 mls qos trust cos no mls qos channel-consistency interface ten 1/5/4 channel-group 1 mode on! EC mode is ON - EtherChannel Managemeent Protocol off interface ten 1/1/1 channel-group 1 mode on udld enable vtp domain campus-test vtp mode transparent spanning-tree mode rapid-pvst no spanning-tree optimize bpdu transmission spanning-tree extend system-id spanning-tree vlan priority 24576! STP Root port-channel load-balance src-dst-mixed-ip-port! Enhanced hash algorithem interface Port-channel2! Unique port-channel number for SW 1 description VSL Link from Switch 2 no switchport no ip address switch virtual link 2!Defines switch ID for SW 2 mls qos trust cos no mls qos channel-consistency interface ten 2/5/4 channel-group 2 mode on! EC mode is ON - EtherChannel Managemeent Protocol off interface ten 2/1/1 channel-group 2 mode on Interface Gigabitethernet1/8/23 description Access Switch switchport switchport trunk encapsulation dot1q swtichport trunk native vlan 202 switchport trunk allowed vlan 2,102 <snip> channel-protocol pagp (lacp is an option) channel-group 202 mode desirable MEC Interface Port-channel202 description Access Switch switchport switchport trunk encapsulation dot1q swtichport trunk native vlan 202 switchport trunk allowed vlan 2,102 Interface Gigabitethernet2/8/23 description Access Switch switchport switchport trunk encapsulation dot1q swtichport trunk native vlan 202 switchport trunk allowed vlan 2,102 <snip> channel-protocol pagp (lacp is an option) channel-group 202 mode desirable 126

127 Supervisor 2T Migration Best Practices Supervisor Engine 2T added significant value in the areas of MACsec encryption, improved ACL capabilities, and IPv4/IPv6/MPLS/VPLS/VSS throughput performance. Need to follow the right migration strategy Supervisor 2T is installed only on E-series chassis, make necessary arrangements before migration Check the hardware and software compatibility before Sup2T migration Move the Layer 3 (HSRP) function first and then Layer 2 (Spanning Tree) function to the primary distribution switch Make VSS switch as spanning primary root for all the vlans It is a requirement to use 10G interfaces for VSL trunk Set the VSS switch priority and number Make sure to use matching channel protocol between MEC members It is recommended to upgrade the supervisors during change management windows to avoid any production traffic loss. Perform the migration in steps, which should include hardware, software, and the actual migration plan. Verify configuration at each step, and make necessary adjustments for changed cli 127