Rozšiřitelnost a vysoká dostupnost v L2 sítích

Transcription

1 Rozšiřitelnost a vysoká dostupnost v L2 sítích Techtorial Jiří Tesař Systems Engineer CCIE #14558 jitesar@cisco.com Sponsor Logo Sponsor Logo Sponsor Logo Sponsor Logo CIscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 1

2 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mlacp Conclusions 2 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

3 802.1ah Terminologies BEB: Backbone Edge Bridge encapsulates customer frames for transmission across backbone. B-BEB: B type BEB contains a B-component, supports bridging in the provider backbone based on B-MAC and B-TAG info. I-BEB: I type BEB contains an I-component for bridging in the customer space, including customer MAC, service VLAN IDs. B-TAG: Backbone VLAN Tag an S-TAG used in conjunction with backbone MAC addresses. I-TAG: Service Instance Tag - encapsulates customer addresses and contains the Service Instance identifier (I-SID). I-SID: Service Instance identifier - A field of the Service Instance tag which identifies the service instance of the frame. S-TAG: A field defined in the 802.1ad Q-in-Q encapsulation which identifies the Service VLAN (S-VLAN). 3 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

4 802.1ah Terminologies and Interconnections CE MPLS Core Peer 802.1ah PBBN Hierarchical 802.1ah PBBN MPLS Core PE B BEB B BEB P I I I M B BEB B I BEB B 802.1ah PBBN B B B BCB (PB) B PE/ BEB B B B IB BEB B B BEB C S I I I BEB 802.1ad / Q-in-Q PBN PB S I BEB C S S S S 802.1Q Q C C PEB PB PEB Q Q C S S C C C CE CE CE CE CE CE Legend: C S I B M C-Tagged Interface S-Tagged Interface I-Tagged Interface B-Tagged Interface MPLS Interface I BEB I type Backbone Edge Bridge B BEB B type Backbone Edge Bridge IB BEB IB type Backbone Edge Bridge PE/ BEB MPLS PE and Backbone Edge Bridge PB Provider Bridge (S Bridge) PEB Provider Edge Bridge (C + S) Bridge Q 802.1Q C Bridge CE Customer Equipment 4 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

5 IEEE 802.1ah Provider Backbone Bridges (PBB) I-TAG: Contains 24 Bits to Identify a Service Instance B-DA B-SA B-TAG TPDNRI C-DA C-SA C-TAG L2 PDU FCS Second MAC-Header B-TAG: Equals S-TAG P802.1ah (Provider Backbone Bridges) Encapsulation Example Service Scalability Define a new Service Instance Identifier 24 Bits wide (taking the place of the former VLAN ): I-SID Domain Isolation, MAC-Address Scalability Encapsulate Customer MAC-frames at the edge of the network into a Provider MAC- Frame : New MAC-Header with B-TAG Backward Compatibility to 802.1ad Packet header of Provider Backbone Bridges (PBB, P802.1ah) and Provider Bridges (PB, P802.1ad) look the same 802.1ah assumes existing L2 control plane mechanisms such as spanning tree; however these are not required 5 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

6 802.1ah I-TAG B-DA B-SA B-TAG TPDNRI C-DA C-SA C-TAG L2 PDU FCS Octets Ether-Type (0x88-e7) I-PCP I-DEI NCA Res1 Res2 I-SID C-SA C-DA Bits ah Frame Format Settled Priority Code Point (I-PCP) Drop Eligible Indicator (I-DEI) No Customer Addresses (NCA) Reserved 1 (Res1) Reserved 2 (Res2) Backbone Service Instance Identifier (I-SID) 6 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

7 Ethernet Encapsulation Evolution C-DA: Customer dest addr C-SA: Customer src addr C-TAG: Customer tag S-TAG: Service tag C-DA C-SA Service Instances (VID) 2 12 =4,096 C-DA C-SA C-TAG Service Instances (VID) 2 12 =4,096 C-DA C-SA S-TAG C-TAG Service Instances (I-SID) 2 24 =16,777,216 B-DA B-SA B-TAG I-SID C-DA C-SA S-TAG C-TAG I-TAG B-DA: Backbone dest addr S-SA: Backbone src addr I-TAG: Service instance tag VID: VLAN identifier (part of C-/S-/ B-TAG) I-SID: Backbone service instance identifier (part of I-TAG) PB: Provider Bridges PBB: Provider backbone bridges 802.1Q/ad service Instances (2 12 ) Payload FCS Payload FCS Payload FCS Payload FCS 802.1ah service Instances (2 24 ) 7 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

8 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mlacp Conclusions 8 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

9 ah Line Card Support ES+ ES+ UNI NNI NNI UNI Ingress IB-BEB ES+ or Any DFC BCB ES+ or Any DFC Egress IB-BEB 802.1ah Imposition/Disposition is done on UNI facing ES+ cards NNI Facing Line card 1. Any DFC card ah LC Requirements 2. Adds B-VID Ingress LC Egress LC CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Native 802.1ah ES+ Any DFC card (Recommend ES+) 802.1ah + MPLS ES+ ES+, ES20, SIP600, SIP400 (Recommend ES+) 9

10 7600 VLAN Local Significance Support Interface Types ES+ ES20 SIP400 67xx EVC Dot1q Yes Yes Yes N/A EVC QinQ Yes Yes Yes N/A Sub-interface Dot1q Yes No Yes No Sub-interface QinQ Yes Yes Yes N/A VLAN Local Significance does means VLAN is terminated in the NPU => VLAN lookup, rewrites, etc are performed in NPU Same VLAN tag can be used on multiple ports VLAN tag leaving the port is different to VLAN allocated in internal Database VLAN Local Significance does NOT mean More than 4000 VLANs are supported for Layer 3 termination 10 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

11 Flexible Forwarding Model P2P XCONNECT EVC to L3/VRF L3 MPLS C-BRIDGE P2P XCONNECT VFI XCONNECT EFPs B-BRIDGE Local Connect L2 L2 BRIDGED L2 BRIDGED L2 EFPs TRUNK 11

12 Flexible Ethernet Edge Example ES Access port AS core interface, L2 trunk or L3 MPLS CS service instance 1 ethernet encapsulation dot1q 20 second-dot1q 10 rewrite ingress tag pop 1 sym bridge-domain 10 c-mac 802.1ah (PBB or.1ah over VPLS service instance 2 ethernet encapsulation dot1q rewrite ingress tag push dot1q 101 xconnect en mpls E-LINE (VPWS) Local connect service instance 3 ethernet encapsulation dot1q 101 second-dot1q 10 rewrite ingre tag translate 2-to bridge-domain 200 Interface vlan 200 xconnect vfi myvpls E-LAN (VPLS or Local bridging) Service instance or Ethernet Flow Point CiscoEXPO service instance 4 ethernet encapsulation dot1q 102 rewrite ingress tag pop 1 bridge-domain 201 Interface vlan 201 ip address ip vrf myvrf 2010 Cisco Systems, Inc. All rights reserved. Cisco Public L3 termination 12

13 IEEE 802.1ah Control Plane Model I-Component C-MAC Lookup Function MAC Relay IB-BEB B-Component B-MAC Lookup Function MAC Relay EFP (Physical) I-EFP (Virtual) B-EFP (Virtual) Switch Port (Physical) CIP PIP CBP PBP B-MAC Tagging/ I-SID Insertion B-VLAN Re-write/ I-SID Validation Ingress EFP (802.1ah UNI) MAC Tunnel Egress switchport (NNI) int gig1/1 service instance 15 ethernet encapsulation dot1q 9 second-dot1q 8 bridge-domain 10 c-mac ethernet mac-tunnel virtual abc.com bridge-domain 100 service instance 31 ethernet encapsulation i-sid bridge-domain 10 c-mac int gig1/2 switchport switchport mode trunk switchport allowed vlan

14 802.1ah on ES+ NPU Overview 802.1ah is implemented on ES+/7600 for first time on a Cisco platform 802.1ah utilizes both PFC/DFC ASIC and NPU to perform the 2 required layer2 switching decisions for dot1ah Dot1ah on ES+/7600 follows the IB Backbone Edge Bridge model PFC/DFC represents the B-component of the IB-BEB and switches the packet towards the provider backbone port or NNI 802.1ah (NPU) represents the I-component as well as the provider instance port (tunnel engine) and switches the packet towards the customer instance port or UNI PFD/DFC learns Backbone MAC addresses or B-MACs and floods on Backbone VLANs or B-VLANs 802.1ah (NPU) learns Customer MAC addresses or C-MACs and floods on Customer bridge-domains or C-BDs 14 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

15 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mlacp Conclusions 15 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

16 7600 PBB IB-BEB Logical Flow 7600 Ingress Egress C-MAC1 ISID-1 B-MAC1 Port, 802.1q or 802.1ad (QinQ) EFP Service instances C-MAC2 VLAN local Significance per Port C-MAC3 ISID-2 ISID-3 EFP or switchports with the B-VLANs VLAN tag translation and manipulation B-MAC2 C-MAC4 ISID q/qinq/ 802.1ad PBN PBBN AS 16

17 7600 PBB IB-BEB Configuration ELAN Service Implementation Step 1 Ingress EFP configuration (UNI) interface TenGigabitEthernet3/1 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP - ELAN Service encapsulation dot1q 100 second-dot1q rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-h-qos-parent l2protocol forward bridge-domain 100 c-mac Step 2 Mac-in-Mac tunnel configuration ethernet mac-tunnel virtual 1 description ** IB-BEB - Mac Tunnel 1 bridge-domain 1000 service instance 1 ethernet description ** ELAN Service - ISID encapsulation dot1ah isid bridge-domain 100 c-mac Step 3 Egress EFP configuration (NNI) interface TenGigabitEthernet3/2 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP - ELAN Service encapsulation dot1q 100 second-dot1q rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-h-qos-parent l2protocol forward bridge-domain 100 c-mac interface TenGigabitEthernet3/3 dot1q tunneling ethertype 0x88A8 service instance 1 ethernet description ** B-VLAN - MAC Tunnel 1 encapsulation dot1q 1000 rewrite ingress tag pop 1 symmetric service-policy output vz-core-queuing bridge-domain 1000 or Egress switchport configuration (NNI) interface TenGigabitEthernet3/3 switchport switchport mode trunk switchport allowed vlan

18 7600 MPLS ah IB-BEB Logical Flow Ingress Egress (ES +40) C-MAC1 ISID-1 VPWS (P2P) Pseudowire B-MAC1 Port, 802.1q or 802.1ad (QinQ) EFP Service instances C-MAC2 C-MAC3 VLAN tag translation and manipulation ISID-2 ISID-3 B-MAC2 MPLS Interface/ Sub-interfaces (H)-VPLS Pseudowire(s) MPLS Transport Network C-MAC4 ISID-4 VFI VPWS (P2P) Pseudowire 802.1q/qinq/ 802.1ad PBN AS 18

19 7600 MPLS + PBB IB-BEB Configuration VPWS ah Service Implementation Step 1 Step 2 Ingress EFP configuration (UNI) Mac-in-Mac tunnel configuration interface TenGigabitEthernet3/1 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP VPWS Service encapsulation dot1q 100 second-dot1q rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-h-qos-parent l2protocol forward bridge-domain 100 c-mac ethernet mac-tunnel virtual 1 description ** IB-BEB - Mac Tunnel 1 bridge-domain 1000 service instance 1 ethernet description ** VPWS Service - ISID encapsulation dot1ah isid bridge-domain 100 c-mac Step 3 VPWS configuration interface Vlan1000 description ** IB-BEB VPWS Service xconnect encapsulation mpls 19

20 7600 MPLS + PBB IB-BEB Configuration VPLS ah Service Implementation Step 1 Ingress EFP configuration (UNI) interface TenGigabitEthernet3/1 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP VPLS Service encapsulation dot1q 100 second-dot1q rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-h-qos-parent l2protocol forward bridge-domain 100 c-mac interface TenGigabitEthernet3/2 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP - VPLS Service encapsulation dot1q 100 second-dot1q rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-h-qos-parent l2protocol forward bridge-domain 100 c-mac Step 2 Mac-in-Mac tunnel configuration ethernet mac-tunnel virtual 1 description ** IB-BEB - Mac Tunnel 1 bridge-domain 1000 service instance 1 ethernet description ** VPWS Service - ISID encapsulation dot1ah isid bridge-domain 100 c-mac Step 3 VPLS configuration l2 vfi Vz-MAC-Tunnel-1 manual vpn id 3000 neighbor encapsulation mpls neighbor encapsulation mpls interface Vlan1000 description ** IB-BEB VPLS Service xconnect vfi Vz-MAC-Tunnel-1 manual 20

21 Flexible Ethernet Edge for.1ah The Cisco implementation will provide for the services mandated by 802.1ah, and will extend them to support all the following offerings: S-Tagged Service Multiplexed: Each S-VID maps to an I-SID. It is possible to retain or pop the S- TAG. (Retention of S-TAG is an extension of 802.1ah) Bundled (same as 802.lah): Multiple S-VIDs map to an I-SID. The S-TAG must be retained C-Tagged Service (extension of 802.1ah) Multiplexed: Each C-VID maps to an I-SID. It is possible to retain or pop the C- TAG. Bundled: Multiple C-VIDs map to an I-SID. The C-TAG must be retained. S/C-Tagged Service (extension of 802.1ah) Multiplexed: Each S-VID/C-VID pair maps to an I-SID. It is possible to retain or pop the S-TAG only or both S-TAG/C-TAG pair. Bundled: Multiple S-VID/C-VID pairs maps to an I-SID. The S-TAG/C-TAG pair must be retained. Port Based Service (same as 802.1ah): All frames are mapped to the same I-SID. All tags, if any, are retained. 21 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

22 PBB IB-BEB Packet Flow CE Side PBBN Side S-tagged Service - Multiplexed C-DA C-SA S-TAG C-TAG 0x800 S-tagged Service - Bundled C-DA C-SA S-TAG C-TAG 0x800 Data Data or B-DA B-SA B-TAG I-TAG C-DA C-SA S-TAG C-TAG 0x800 B-DA B-SA B-TAG I-TAG C-DA C-SA C-TAG 0x800 Data B-DA B-SA B-TAG I-TAG C-DA C-SA S-TAG C-TAG 0x800 Data Data C-tagged Service - Multiplexed C-DA C-SA C-TAG 0x800 Data C-tagged Service - Bundled C-DA C-SA C-TAG 0x800 Data or B-DA B-SA B-TAG I-TAG C-DA C-SA C-TAG 0x800 Data B-DA B-SA B-TAG I-TAG C-DA C-SA 0x800 Data B-DA B-SA B-TAG I-TAG C-DA C-SA C-TAG 0x800 Data S/C-tagged Service - Multiplexed C-DA C-SA S-TAG C-TAG 0x800 Data S/C-tagged Service - Bundled or or B-DA B-SA B-TAG I-TAG C-DA C-SA S-TAG C-TAG 0x800 B-DA B-SA B-TAG I-TAG C-DA C-SA C-TAG 0x800 Data B-DA B-SA B-TAG I-TAG C-DA C-SA 0x800 Data Data C-DA C-SA S-TAG C-TAG 0x800 Data B-DA B-SA B-TAG I-TAG C-DA C-SA S-TAG C-TAG 0x800 Data Port Based Service C-DA C-SA B-DA B-SA B-TAG I-TAG C-DA C-SA 22

23 MAC Address Scalability in H-VPLS H-VPLS IP/MPLS H-VPLS with PBB IP/MPLS IP/MPLS IP/MPLS c-mac c-mac : : : :: : : : : : :: : : c-mac c-mac c-mac : : : :: : : : : : :: : : c-mac c-mac c-mac : : c-mac b-mac b-mac : b-mac b-mac b-mac : b-mac c-mac c-mac : : c-mac No customer MAC addresses on N-PE nodes N-PEs only learn backbone MAC addresses imposed by U-PEs 23

24 Deployment Scenario: H-VPLS extension ah CE C-VLAN 802.1Q VPWS/ H- VPLS w/ 802.1ah VPWS/VPLS VPWS/ H- VPLS w/ 802.1ah VPWS upe/ IB-BEB 802.1ad/Q-in-Q S-VLAN CE CE CE 802.1ad/ Q-in-Q I-SID upe/ IB-BEB VPWS npe VPWS VSI VSI VPWS/VPLS IP/MPLS Core npe VPWS BEB I-SID upe/ IB-BEB S-VLAN CE CE 802.1ad/Q-in-Q CE upe/ IB-BEB npe VSI npe I-SID CE CE S-VLAN E-Line Service E-LAN Service upe/ IB-BEB S-VLAN CE MPLS Access Aggregation Core Transport Aggregation Access 24

25 Scalability Scalability Factor Scalability Number Total number of EVCs in the system Total number of EVCs per linecard Total number of ISIDs in the system Total C-MAC addresses per LC 16M (32000 per NPU) Total number of EVCs per ISID per NPU 110 Total number of EVCs per ISID for a two port Excalibur 220 Total number of EVCs per ISID for a four port Excalibur 440 Total B-bridge-domains per chassis 4094 Total I-SIDs or MAC-Tunnels Total entries in a C-MAC table CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

26 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mlacp Conclusions 26 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

27 Test: Verify PCP Propagation and Queuing Behavior for ELAN Service Using UNI Ingress 2R3C Policer Marking ES1 IXIA Ten3/1 Ten3/3 L2 Link IXIA Sniffer Ten1/0/0 Ten1/0/1 Ten3/3 L2 Link Ten3/1 ES3 IXIA ES2 IXIA Ten3/2 C-DA C-SA S-Tag, CoS=1 C-Tag, CoS =2 C payload FCS IB BEB 2R3C Policer 1. On PE1-IB_BEB Ten3/4 MPLS Link Queuing, etc.. P- BCB C-Tag, CoS =2 C payload Ten3/4 MPLS Link 27 B-DA B-SA B-Tag, CoS=3,4,5 I-Tag, CoS=3,4,5 C-DA C-SA Ingress card frames are remarked to CoS 5,4,3 by 2R3C Policer FCS PE1- PE2- IB-BEB C-DA Ten3/2 C-SA S-Tag, CoS=3,4,5 C-Tag, CoS =2 C payload Egress card frames are sent to separate queues based on Policer marking for Queuing, etc. 2. Show end-2-end remarked CoS mapping in PBB frame and Egress PE2-IB_BEB Sniffer capture to show COS is mapped from S-TagI-TagB-Tag and back to egress S-Tag FCS ES4 IXIA

28 References IEEE 802.1ah - Provider Backbone Bridges, Draft 4.2, April VPLS Interoperability with Provider Backbone Bridges, draftsajassi-l2vpn-vpls-pbb-interop-04-txt, March Extensions to VPLS PE model for Provider Backbone, Bridging draft-balus-sajassi-l2vpn-pbb-vpls-00.txt, March Provider Backbone Bridging and MPLS: Complementary Technologies for Next-Generation Carrier Ethernet Transport, S. Salam and A. Sajassi, IEEE Communications Magazine, Vol. 46, No. 3, March The Evolution of Carrier Ethernet Services Requirements and Deployment Case Studies, L. Fang, N. Bitar, R. Zhang, and M. Taylor, IEEE Communications Magazine, Vol. 46, No. 3, March

29 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mlacp Conclusions 29 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

30 L2 Convergence EVC L2 Convergence - SRD and before MST (BPDU relay, switchport, EVC) REP (switchport) PW Redundancy (MPLS aggregation) Flex-Link Etherchannel/LACP (single-homed devices) Interface-Backup (single-homed devices) New features in SRE MST AG simplify MST based deployments REP support for EVC added mlacp dual homed LACP CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 30 30

31 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mlacp Conclusions 31 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

32 Why MST Access Gateway? Avoid running full Spanning Tree protocol on NPE STP is challenging to troubleshoot Terminate multiple Ethernet access rings running MST on NPE without running full STP Each ring can have its own independent topology Isolate topology changes/mac flushes localized to each ring Improve scalability No full STP processes on NPE routers Maintain existing STP topologies on the access networks Access nodes just speak regular MSTP/RSTP Platform Support ASR9K since FCS SRE 32 32

33 MST AG operation SRE MST AG ports send preconfigured BPDU s with root or zero cost to root information towards access network. Access network sees a loop because of root reachability from both NPE s. Both NPE s can send the same information or arbitrarily can be set as best and second best bridge via priority or cost setting for load balancing purposes Root bridge can be one of the NPE s or arbitrarily set non-existent bridge address MST AG ports are always in Designated state and are forwarding L2 domain runs regular MST protocol. All convergence operations and port state transitioning happen in the access network

34 MST AG TCN Propagation NPE s snoop and relay TCN from BPDU received from access network NPE s trigger MAC withdrawal to neighbors TCN is forwarded only to the port within the same MST AG group thus providing L2 domains isolation 34 34

35 Access Network Failure Scenarios Failure Scenarios 1,2 and 3 cause a primary data path disruption UPE-2 BPDU on Atlernate Port is has now the best BPDU port transitions to root port role and forwarding state and data path is restored TCN propagates across L2 domain and is relayed between NPE s NPE s trigger MAC withdrawal 35

36 Root Port Recovery When root bridge recovers it starts sending best BPDU towards the access network and convergence to the original path occurs Sending of the best BPDU has to be delayed to allow core convergence; e.g. if the router was reloaded 7600 router runs STP state machine on MST AG when the port is coming from down to up state. The ports is going through LST-LRN-FWD states. To disable this behavior spanning tree port fast has to be configured on MST AG ports. BPDU s are sent immediately upon port recovery which can cause traffic black-hole if core has not converged. EEM can be used to delay port-up event under certain scenarios. ASR 9K does not run spanning tree state machine and defines a dedicated timer to delay the best BPDU generation. 36

37 NPE isolation failure scenario N-PE isolation occurs if all core facing interfaces are not available resulting in VPLS, Psedowires or L3 connectivity failure N-PE isolation failure is not propagated into access interfaces therefore STP topology remains unchanged, this results in traffic blackhole as access network continues forwarding towards isolated PE An uplink tracking feature is under consideration for future releases Current solution is based on EEM when router isolation is discovered the access interfaces from redundant networks can be shut down which triggers MST convergence. Upon recovery, timer can be set to delay access links recovery and avoid immediate BPDU sending to the access network. 37

38 Using EEM for Uplink Tracking Backbone uplink on NPE-1 is going down event manager applet Backbone-DN event syslog pattern "%LINK-3-UPDOWN: Interface GigabitEthernet1/40, changed s" action 1.0 cli command "enable" action 1.1 cli command "conf t" action 1.2 cli command "int g1/31" action 1.3 cli command "sh Backbone uplink on NPE-1 is going up event manager applet Backbone-UP event syslog pattern "Interface GigabitEthernet1/40, changed state to up" action 1.0 cli command "enable" action 1.1 cli command "conf t" action 1.2 cli command "int g1/31" action 1.3 cli command "no sh 38

39 Special PW failure scenario Special PW failure can be result of PE isolation or a miss-configuration Unlike MST, R-L2GP special PW failure does not cause a loop and therefore does not cause permanent traffic loss because BPDU forwarding topology remains unchanged and is not affected by this failure TCN will not be relayed between the two NPE s MAC flush may not happen in a part of L2 domain which may cause temporary traffic loss until MAC aging occurs. Bidirectional traffic will be restored immediately. MAC withdrawal will still be generated by the NPE receiving TCN 39

40 EVC STP Modes Comparison STP mode RPVST/ PVST MST MST AG EVC Support NO BPDU relay only SRD SRE VLAN-STP mapping Per VLAN Single MST Region VLAN to Instance mapping applies to all ports Single MST Region VLAN to Instance mapping applies to all ports Dynamic Port State N/A always FWD YES NO Designated ports, always forwarding Dynamic BPDU N/A YES NO preconfigured BPDU s TCN isolation MAC Withdrawal YES NO YES TCN forwarded between ports within the same L2GP group NO YES YES Complexity Medium High Low Prone to miss-configuration 40 40

41 MST AG configuration steps 1. Configure MST parameters MST AG reuses global MST configuration template to construct BPDU s. To insure proper MST function, parameters like name, revision and timers should match on other bridges. Note: due to single domain support the same MST parameters will be used on all MST AG groups. In particular IST to VLAN mapping. spanning-tree mode mst spanning-tree mst configuration name c7600 revision 1 instance 1 vlan spanning-tree mst hello-time 1 spanning-tree mst forward-time 4 spanning-tree mst max-age

42 MST AG configuration steps 2. Configure MST AG Pseudo-Information (NPE-1) spanning-tree pseudo-information transmit 1 remote-id 2! use the number of pseudo-information of the peer router mst 0-1 root e.f7f6.6040! root bridge and priority that will be send in BPDU on MST AG ports 3. Assing MST AG Pseudo-Information to a port interface GigabitEthernet1/32 no ip address spanning-tree portfast trunk spanning-tree pseudo-information transmit 1! the port will send preconf. BPDU s as per MST global and pseudo-inf. gr. 1 conf. service instance 3500 ethernet encapsulation dot1q 3500 rewrite ingress tag pop 1 symmetric bridge-domain

43 MST AG configuration steps Configure Special PW for TCN relay This is the same configuration step as for MST BPDU relay interface Vlan1 no ip address xconnect vfi BPDU end NPE-1#sh run sec BPDU l2 vfi BPDU manual vpn id 1 forward permit l2protocol all neighbor encapsulation mpls Configure Service Instances and Bridge Domains Configure all Egde Ports explicitly with portfast feature avoid LRN/LSTN states when bridge is converging 43 43

44 MST AG configuration validation NPE-1# sh spanning-tree mst 1 ##### MST1 vlans mapped: Bridge address 001e.f7f priority (32768 sysid 1) Root this switch for MST1 Interface Role Sts Cost Prio.Nbr Type Gi1/1 Desg FWD Edge P2p Gi1/32 Desg FWD P2p R-L2GP PW :1 Desg FWD P2p R-L2GP NPE-1#sh spanning-tree pseudo-information 1 configuration Pseudo id 1, type transmit: remote_id 2 mst_region_id 0, port_count 1, update_flag 0x0 mrecord 0x1A6BE02C, mrec_count 2: msti 0: root_id e.f7f6.6040, root_cost 0, update_flag 0x0 msti 1: root_id e.f7f6.6040, root_cost 0, update_flag 0x0 NPE-1# sh spanning-tree pseudo-information 1 interfaces Pseudo id 1: GigabitEthernet1/

45 Other useful commands on RP sh spanning-tree mst configuration! sh spanning-tree details! sh vlan id 3500! on SP deb spanning-tree pseudo-information! debug spanning-tree bpdu! deb spanning-tree mstp tc! deb spanning-tree mstp flush!! 45 45

46 MST AG Restrictions Supported on ES+ and ES20 Applicable to EVC with Bridge Domain only No xconnect, connect or subinterface support No EVC untagged, priority tagged or default encapsulation support Native VLAN is used for BPDU forwarding Single MST region support All MST AG groups share MST Instance - VLAN mapping, name and revision No MST boundary function (for RPVST/PVST/RSTP interoperability) 46 46

47 7600 MST AG Scale Feature Scale Comment STP Regions 1 All R-L2GP groups have to use common MST configuration; name, version, timers, IST-VLAN mapping MST instances R-L2GP groups Ports in R-L2GP group 64 As above 256 No limit All ports in a chassis can be assigned to a single R-L2GP group 50,000 vport limit per chassis 47 47

48 ARS9K MST Access Gateway Interface gig 0/0/0/10.1 l2 encap untagg spanning-tree ring-termination ring1 preempt delay { until <hh:mm:ss> for <n> { hours minutes seconds } } interface GigabitEthernet0/0/0/10.1 name cisco revision 1 bridge-id instance 0 I m the root root-id priority 4096 root-priority 4096! instance 1 vlan-ids 101,103,105,107 root-id priority 8192 root-priority 4096! instance 2 I m the root vlan-ids 102,104,106,108 root-id priority 4096 root-priority 4096 Access switch configuration interface GigabitEthernet1/1/1 switchport mode trunk spanning-tree mst 0,2 cost MST root for instance 0,2 VFI VFI MST root for instance 1 Access switch configuration interface GigabitEthernet1/1/1 switchport mode trunk spanning-tree mst 1 cost Interface gig 0/0/0/10.1 l2 encap untagg spanning-tree ring-termination ring1 preempt delay { until <hh:mm:ss> for <n> { hours minutes seconds } } interface GigabitEthernet0/0/0/10.1 name cisco revision 1 bridge-id VFI instance 0 root-id priority 8192 VFI root-priority 4096! instance 1 I m the root vlan-ids 101,103,105,107 root-id priority 4096 root-priority 4096! instance 2 vlan-ids 102,104,106,108 root-id priority 8192 root-priority

49 Conclusions MST AG provides an appealing option to operate STP networks to service providers: Maintaining access networks without modification Lower maintenance complexity on N-PE s no full spanning tree support Lower troubleshooting complexity on the network STP isolation for L2 aggregation domains separated by VPLS core Deterministic root location Improvements from MST/EVC: TCN isolation between access domains More robust implementation, special PW failure does not cause traffic black-hole 49 49

50 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mlacp Conclusions 50 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

51 mlacp SRE Standby POA Virtual LACP Peer DHD Inter-chassis Communication EtherChannel with mlacp Active POA mlacp provides a good mechanism for multi-chassis resiliency DHD is attached to a group of Points of Attachments which look like a single node mlacp appears to DHD as a single 802.3ad LACP POA work in active/standby mode ICC exchanges redundancy information between chassis Links to standby PoA are in hot-standby state 51

52 ICCP Overview ICCP is implemented according to the standard draft-martini-pwe3- iccp-00.txt ICCP is an extensible Protocol to synchronize event/states between multiple chassis which are part of the redundant group. ICCP is a reliable protocol which runs over TCP ICCP PDUs are exchanged between Peers to keep the application state consistent across Routers. Control Messages to setup, notify and exchange heartbeats. Data Messages to exchange the application state consistent across the chassis. Ex: LACP Parameters ICCP failure detection ICC Heartbeat Slow (~ 30 sec) /32 Next-hop Tracking Depends on IGP timers BFD ~ msec 52

53 Pseudowire Redundancy in SRE VCCV over Primary and Backup PW Preferential Forwarding status bit according to draft-ietf-pwe3-redundancy-bit Upon Receipt of PW switchover status request, The receiver should clear the preferential status forwarding bit and activate the PW. Back up Pseudowire will be preprovisioned in the data plane.but forwarding is disabled. Supported with Scale EoMPLS configuration only. VPLS Redundancy is supported only with mlacp configuration. Supported on ES40,ES20 only. 53

54 mlacp with two sided VPWS/VPLS redundancy MPLS SRE DHD1 VPWS VPLS L2 Standby L1 Active PE1 PE3 Standby PW 1 PW 3 PW 2 PW 4 Two PEs form one virtual group on each site, one PE is primary the other is backup PE s send primary/backup information during PW signaling PW with both sides status <active> are established, others are hot standby MPLS uplinks, attachment circuits and PW status tracking Message exchange within virtual group (for mlacp it is ICC) with redundancy status PW will be active between PE s with active access circuits only Single active path through VPLS domain between PE virtual group Similar model applies to REP access E Standby Active Active Active Active Standby Standby 54 PE2 PE4 L4 L3 Active Standby DHD2

55 Pseudowire Redundancy Two-way Prim. PE1 IP/MPLS Primary Pseudowire PE2a Prim. CE1 CE2 ICCP ICCP LACP Back. IP or MPLS Redundant Pseudowires PE2b Back. LACP ICCP = Inter-Chassis Control Protocol LACP = Link Aggregation Control Protocol Failures within MPLS network are protected by MPLS FRR Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy (Note: both sides of the PW are protected) Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for LACP and PW redundancy Synchronization of state (active/standby) between the ACs and PWs 55

56 Pseudowire Redundancy Two-way Prim. PE1 IP/MPLS Primary Pseudowire PE2a Prim. CE1 CE2 ICCP ICCP LACP Back. IP or MPLS Redundant Pseudowires PE2b Back. LACP ICCP = Inter-Chassis Control Protocol LACP = Link Aggregation Control Protocol Failures within MPLS network are protected by MPLS FRR Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy (Note: both sides of the PW are protected) Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for LACP and PW redundancy Synchronization of state (active/standby) between the ACs and PWs 56

57 Pseudowire Redundancy Two-way Prim. PE1 IP/MPLS Primary Pseudowire PE2a Prim. CE1 CE2 ICCP ICCP LACP Back. IP or MPLS Redundant Pseudowires PE2b Back. LACP ICCP = Inter-Chassis Control Protocol LACP = Link Aggregation Control Protocol Failures within MPLS network are protected by MPLS FRR Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy (Note: both sides of the PW are protected) Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for LACP and PW redundancy Synchronization of state (active/standby) between the ACs and PWs 57

58 mlacp CLI Interchassis Redundancy Group redundancy interchassis group 1 member ip ! this is IP-address of opposite end of direct link between NPE-1/2 backbone interface GigabitEthernet1/40 mlacp system-priority 100 mlacp node-id 0! monitor peer [bfd route-watch] Pseudowire Class to reflect or decouple AC and PW status pseudowire-class HS-PW encapsulation mpls status peer topology dual-homed!this command reflects AC circuit status on all PWs 58 58

59 mlacp CLI (cont) Port-Channel Definition interface Port-channel1 description mlacp no ip address lacp fast-switchover lacp max-bundle 1 mlacp lag-priority 100 mlacp interchassis group 1 service instance 3701 ethernet encapsulation dot1q 3701 xconnect pw-class HS-PW backup peer pw-class HS-PW 59 59

60 Platform Specifics for mlacp mlacp is SSO Aware mlacp is only supported with EVC configuration. No support for Subinterfaces, Access subinterfaces, Switchport configurations. mlacp is only supported with ES20,ES40 mlacp is not supported with EVC Routed Pseudowire configuration Exception is inter-poa Routed PW use case for VRRP/HSRP Number of chassis part of redundancy group is ah supports only 1 member link on the Port-Channel. mlacp configuration should be active-standby with 1 member link. ASR9K target for mlacp is in release

61 Attachment Circuit Redundancy Options summary STP based solution can t provide sub second convergence time and is difficult to support REP is simple, spanning tree free protocol and can coexist with spanning tree topologies. REP integration with EVC in SRE. Etherchannel / LACP provides a good link redundancy scheme for single homed devices, supported with EVC starting from SRC mlacp will provide good redundancy scheme for dual homed devices. Hot-standby PW synchronization with mlacp and REP will be required 61

62 Reference 1. LDP Specification - RFC Pseudowire Setup and Maintenance Using the Label Distribution Protocol (LDP) - RFC PWE3 - RFC Inter-Chassis Communication Protocol (ICCP) to synchronize multi-chassis LACP and PW redundancy state - draft: pwe3-iccp 5. Pseudowire Virtual Circuit Connectivity Verification (VCCV) - RFC Bidirectional Forwarding Detection (BFD) for the Pseudowire Virtual Circuit Connectivity Verification (VCCV) - draft-ietf-pwe3-vccv-bfd Pseudo Wire (PW) OAM Message Mapping - draft-ietf-pwe3-oam-msg-map CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

63 Questions & Answers 63 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

64 64 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

65 Backup Slides CIscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 65

66 Pseudowire Operation - Creation IGP = transport infrastructure Targeted LDP for L2VPN PW creation: PW Label Withdrawal It will result in the Label Mapping Message being advertised only if the attachment circuit is active PW Status TLV Mapping for primary and and backup, but using TLV Status for detection 66 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

67 PW Status 1. Until the SRC: when the AC associated with a PW is down (or being held down for PW redundancy) labels advertised to peers are withdrawn. 2. RFC4447 specifies extensions for LDP which allow PW status to be carried in notification messages to peers. This diverges LDP label mappings from the AC status notification and allows labels to be retained through AC status changes: CiscoEXPO - as soon as the xconnect is provisioned, - and until the xconnect is unprovisioned or AC interface shutdown. 3. The router can send pseudowire status to a peer router, even when the attachment circuit is down interface Loopback0! ip address !!! pseudowire-class atomstatus! encapsulation mpls! status!!! interface GigabitEthernet10/5! xconnect pw-class atomstatus! 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Router# show mpls l2transport vc detail Last remote LDP TLV status rcvd: AC DOWN(rx,tx faults) 67

68 Virtual Circuit Connectivity Verification Pseudowire VCCV Control channel between a pseudowire's ingress and egress points over which connectivity verification messages can be sent Encapsulated using PWE3, follows data paths Control Channel (CC) Types in-band, out-of-band, Connectivity Verification (CV) Types LSP Ping [RFC4379], ICMP Ping [RFC0792], BFD Can additionally carry fault detection status between the endpoints of the PW Translated into the native OAM status codes used by the native access technologies 68 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public

69 Pseudowire - Detection, Notification 1. VCCV-BFD Connectivity Verification fault detection only 1. LDP status TLV mechanism for AC and PW status and defect notification 1. PW OAM Message Mapping specifies the mapping and notification of defect states between a Pseudo Wire and the Attachment Circuits (AC) of the end-to-end emulated service 69 CiscoEXPO 2010 Cisco Systems, Inc. All rights reserved. Cisco Public