High Availability for 2547 VPN Service

Size: px

Start display at page:

Download "High Availability for 2547 VPN Service"

Gregory Craig
5 years ago
Views:

1 Fast Service Restoration High Availability for 2547 VPN Service Emil Gągała JNCIE PLNOG, Kraków,

2 ACKLOWLEDGEMENTS Many thanks to Yakov Rekhter, Hannes Gredler for their contributions to the development of this technology Special thanks to Yimin Shen, Minto Jeyananth & Wen Lin who are driving the technical details in JNPR and protocol drafts in IETF. 2 Copyright 2011 Juniper Networks, Inc.

5 High Availability Quiz % availability means: A)15.36 minutes downtime in year B) 5.26 minutes downtime in year C) 2.53 minutes downtime in year Non Stop Routing needs support on neighboring routers True or false? It is possible to achieve with LDP FRR behavior True or false? 5 Copyright 2011 Juniper Networks, Inc.

7 The Purple Line MPLS as a transport for all services IP Services Plane SERVICES Pri ivate services Lea ased Lines, Frame Rel lay ATM, POTS Eth hernet, ATM, FR PW Ws (VPLS/VPWS) DT TV IMS (services delivered to IPenabled mobile handsets) VPNs IP Vo oip Vo oip Peering Internet (search, e- commerce, advertising, video, IM, over-the-top ) IPT TV/VoD TRANSPORT OTN SW Infrastructure Control Plane MPLS Data Plane (P2P, P2MP, MP2P, MP2MP) Ethernet Framing OTN Muxing (G.709, FEC, OAM) DWDM Fiber 7 Copyright 2011 Juniper Networks, Inc.

8 MPLS AS A TRANSPORT Unified transport plane for services Well tested fast restoration (FRR, LFA) Ease of service placement (with Seamless MPLS) Nice scaling characteristics 8 Copyright 2011 Juniper Networks, Inc.

9 MPLS FOR SERVICES Purple line is moving up MPLS is a transport layer for services And a lot of services are MPLS-based Virtual networks using BGP VPNs Circuit transport using BGP PWs and LDP PWs Mobile backhaul using PWs IPTV using MPLS Multicast But the service layer is fragile Failure restoration of MPLS services is still not 50ms Service layer needs to be robust to move the purple line 9 Copyright 2011 Juniper Networks, Inc.

10 Securing the Edges Protecting L3VPN services Protecting LDP PW services Protecting BGP PW services Protecting VPLS Protecting Hosts Summary PE1 L3VPN Cloud PE2 VPN A/Site 1 Host-A Host-B PE4 PE2 PLR PE3 PE1 10 Copyright 2011 Juniper Networks, Inc. VPN A/Site 2

12 Goals: High service availability 2547 VPN as the service both IPv4 and IPv VPN service Service disruption time less than 50 msec in the presence of failures within the service provider infrastructure 12 Copyright 2011 Juniper Networks, Inc.

13 2547 VPN Service Failures Decomposition Core failures (e.g., PE-P link, P-P link, P router) Existing MPLS FRR link/node local protection mechanisms allows to provide sub-50msec connectivity recovery Ingress PE router failure, ingress CE-PE link failure CE detects primary PE router failure (or CE-PE link failure) Could be accomplished using L2 OAM or BFD between CE and (ingress) PE router CE re-routes traffic towards the backup (ingress) PE router local protection Allows to provide sub-50msec connectivity recovery Egress PE-CE link failure Egress PE detects PE-CE link failure Could be accomplished using L2 OAM or BFD between (egress) PE and CE Egress PE re-routes traffic towards the backup (egress) PE router local protection Allows to provide sub-50msec connectivity recovery Egress PE router failure Not covered by the existing MPLS FRR local protection schemes Sub-50msec connectivity recovery using local protection is the focus of this presentation 13 Copyright 2011 Juniper Networks, Inc.

14 Digression: global vs local protection for egress PE failure (1) Global Protection (using IGP to propagate failure notification) P router adjacent to (egress) PE detects PE failure, and advertises it into IGP (ISIS/OSPF) IGP (ISIS/OSPF) is used to propagate failure notification to other (ingress) PEs Using OSPF/ISIS flooding procedures Other (ingress) PEs adjust their forwarding tables, once they receive the failure notification via ISIS/OSPF At this point connectivity is restored Connectivity recovery depends on propagating failure notification in ISIS/OSPF Connectivity recovery time can not be less than the time it takes to propagate and process failure notification in ISIS/OSPF Propagation time involves ISIS/OSPF control plane processing delay on all the intermediate nodes (several control plane hops) Several 100s of msec Connectivity recovery time is dependent of (OSPF/ISIS) routing convergence speed 15 Copyright 2011 Juniper Networks, Inc. Local Protection P router adjacent to (egress) PE detects PE failure P router adjacent to PE adjusts its forwarding table P router becomes Point of Local Repair (PLR) At this point connectivity is restored Connectivity recovery does not depend on propagating failure notification in ISIS/OSPF Connectivity recovery time does not depend on ISIS/OSPF propagating and processing failure notification all the way to the ingress PEs Connectivity recovery time can be comparable to the time it takes for PLR to detect PE failure 50 msec Connectivity recovery time is independent of routing convergence speed

15 Digression: global vs local protection for egress PE failure (2) Local protection is the fastest and the most scalable way to provide connectivity recovery Restoring connectivity does not require propagating any control plane information from PLR to other nodes Connectivity recovery time is independent of routing convergence speed Actions/changes required to restore connectivity upon failure detection are fully localized to the router closest to the failure The router that detects the failure becomes Point of Local Repair (PLR) Enables connectivity recovery time under 50 msec That is precisely why we focus on local protection as a way to achieve high service availability 16 Copyright 2011 Juniper Networks, Inc.

16 Local vs. Global repair Local-repair complements Global-repair Local-repair keeps traffic flowing while Global-repair gets things right Variation of Make before break link break, local-repair start local repair stop global repair start global repair stop 20-40ms ms 18 Copyright 2011 Juniper Networks, Inc.

17 2547 VPN Operations (Background) P routers maintain no VPN state (VPN state is present only on PEs) P routers maintain state only for inter- PE tunnels/lsps (e.g., T1, T2) /16 VPN A/Site 1 VPN B/Site 1 PE5 VRF-B: PE4 T PE5 Dest: 10.2/16, Tunnel T1 (PE1), Label 70 Dest: 10.2/16, Tunnel T2 (PE2), Label 50 P2 T IBGP BGP: RD2, 10.2/16, RT-B, Next-Hop=PE2, Label 50 P3 IBGP Route Reflector PE2 P1 BGP: RD1, 10.2/16, RT-B, Next-Hop=PE1, Label 70 IBGP 19 Copyright 2011 Juniper Networks, Inc. PE VPN B/Site 2 (multi-homed to PE1 and PE2) VPN B/Site 3 PEs connected to a multi-homed site of a given VPN use different RDs (but the same RT) when originating VPN-IP routes for the destinations within the site Results in several VPN-IP routes with the same IP prefix, same RTs, but different RDs and Next-Hop. PEs connected to other sites of that VPN import all these routes, creating Equal Cost Multi-Path (ECMP) for the destinations within the multi-homed site PE3 10.2/16 VPN A/Site 2 (multi-homed to PE1 and PE3)

19 Big picture (1) VPN A/Site 1 VPN B/Site 1 PE4 Inter-PE LSP Goal: In the presence of PE1 failure provide 50 msec connectivity recovery time for traffic from VPN A/Site 1 to VPN A/Site 2 that used to go via PE1 PE2 PE3 PLR PE1 10.2/ /16 VPN B/Site 2 (multi-homed to PE1 and PE2) VPN B/Site 3 VPN A/Site 2 (multi-homed to PE1 and PE3) Goal: In the presence of PE1 failure provide 50 msec connectivity recovery time for traffic from VPN B/Site 1 to VPN B/Site 2 that used to go via PE1 How: by using local protection - penultimate hop P router acts as PLR and re-routes this traffic via PE3 21 Copyright 2011 Juniper Networks, Inc. How: by using local protection - penultimate hop P router acts as PLR and re-routes this traffic via PE2 PLR can not accomplish this on its own, as doing this would require VPN-related state on PLR, yet PLR (being P router) does not maintain any VPN-related state

20 PROBLEMS TO BE SOLVED #1 Point of Local Repair (PLR) has no label state for service routes #2 The backup node has to correctly interpret labels used by the service LSPs PLR has label state only for transport LSP PLR needs to divert the transport LSP to some other node As a result, all Service LSPs carried over the outer LSP will be re-routed to that other node as well The backup node has to know all incoming-label -> FEC mappings advertised by the protected node for all the service LSPs The backup node should use this mapping for the forwarding of service LSPs 22 Copyright 2011 Juniper Networks, Inc.

21 Big picture (2) introducing Protector VPN A/Site 1 VPN B/Site 1 PE4 Protector PE2 PLR PE3 VPN B/Site 2 PE1 Make PLR re-route to Protector the traffic that used to go via PE1 Make Protector maintain VPN routes for VPN A and VPN B 23 Copyright 2011 Juniper Networks, Inc / VPN B/Site /16 VPN A/Site 2 This way Protector will re-route via PE3 traffic from VPN A/Site 1 to VPN A/Site 2, and via PE2 traffic from VPN B/Site 1 to VPN B/Site 2

22 STEP 1: PLR DETECTS (EGRESS) PE FAILURE VPN A/Site 1 VPN B/Site 1 PE4 Protector Step 1: PLR detects (egress) PE failure E.g., PLR detects PE1 failure PE2 24 Copyright 2011 Juniper Networks, Inc. PLR PE3 VPN B/Site 2 PE1 10.2/ VPN B/Site /16 VPN A/Site 2 Could be accomplished using L2 OAM or BFD between PLR and (egress) PE Further details are outside the scope of this presentation

23 STEP 2: PLR REDIRECTS TRAFFIC TO PROTECTOR VPN A/Site 1 VPN B/Site 1 PE4 Protector PE2 25 Copyright 2011 Juniper Networks, Inc. PLR PE3 VPN B/Site 2 PE1 VPN A/Site /16 Step 2: PLR redirects to Protector the traffic that used to go (via PLR) to the (failed) PE E.g., PLR sends to Protector traffic that used to go via PLR to PE1: from VPN A/Site 1 to VPN A/Site 2, from VPN B/Site 1 to VPN B/Site 2 More details later VPN B/Site /16

24 STEP 3: PROTECTOR FORWARDS TRAFFIC TO APPROPRIATE (EGRESS) PE VPN A/Site 1 VPN B/Site 1 PE4 Protector 26 Copyright 2011 Juniper Networks, Inc. PE2 PLR PE3 VPN B/Site 2 PE1 10.2/16 Step 3: Protector forwards the traffic received from PLR to the appropriate other (egress) PEs E.g., Protector sends via PE3 traffic from VPN A/Site 1 to VPN A/Site 2 E.g., Protector sends via PE2 traffic from VPN B/Site 1 to VPN B/Site 2 More details later VPN A/Site VPN B/Site /16

26 Step 2: PLR redirecting traffic to Protector How? (1) VPN A/Site 1 VPN B/Site 1 PE4 Inter-PE LSP to PE3 BGP: RD3, 10.2/16, RT-A, Next-Hop= , Label 60 BGP: RD2, 10.2/16, RT-B, Next-Hop= Label 70 PE2 PLR PE1 OSPF: metric 1 29 Copyright 2011 Juniper Networks, Inc / /16 VPN B/Site 2 (multi-homed to PE1 and PE2) Context Identifier VPN A/Site 2 (multi-homed to PE1 and PE3) On Protected PE (PE1): Configure (additional) IP address identifies forwarding context (PE) that has to be protected Context Identifier Advertise this Context Identifier into OSPF/IS-IS with small metric (e.g., 1) Use this Context Identifier as BGP Next-Hop for VPN-IP routes originated by Protected PE (PE1) Creates association between Context Identifier and a set of routes to be protected Inter-PE transport LSP used by these routes is associated with Context Identifier

27 Step 2: PLR redirecting traffic to Protector How? (2) OSPF: metric 2^24 VPN A/Site /16 VPN B/Site 2 Protector BGP: RD3, 10.2/16, RT-A, Next-Hop= , Label 60 BGP: RD2, 10.2/16, RT-B, Next-Hop= Label 70 PE (multi-homed to PE1 and PE2) PE1 Context Identifier VPN B/Site 1 On Protector: PE4 Inter-PE LSP to PE3 PLR OSPF: metric 1 Configure IP address that is used as Context Identifier on Protected PE Creates coupling between Protected PE and Protector Advertise this address into OSPF/IS-IS with large metric (e.g., 2^24) 30 Copyright 2011 Juniper Networks, Inc /16 VPN A/Site 2 (multi-homed to PE1 and PE3)

28 Step 2: PLR redirecting traffic to Protector How? (3) OSPF: metric 2^24 VPN A/Site 1 VPN B/Site 2 Protector Bypass LSP to BGP: RD3, 10.2/16, RT-A, Next-Hop= , Label 60 BGP: RD2, 10.2/16, RT-B, Next-Hop= Label 70 PE2 PE1 10.2/ (multi-homed to PE1 and PE2) Context Identifier VPN B/Site 1 PE4 Inter-PE LSP to PE3 PLR OSPF: metric /16 VPN A/Site 2 (multi-homed to PE1 and PE3) On PLR use MPLS FRR procedures to create a Bypass LSP from PLR to Protector Bypass LSP terminates on Protector Basic LFA FRR may not be sufficient (except for particular network topology cases) setting up Bypass LSP is likely to require RSVP-TE Direct consequence of the inability of basic LFA FRR to provide full coverage use RSVP-TE LSP to extend coverage 31 Copyright 2011 Juniper Networks, Inc.

29 Step 2: PLR redirecting traffic to Protector How? (4) VPN A/Site /16 VPN B/Site 2 VPN B/Site 1 PE4 OSPF: metric 2^24 Protector Bypass LSP to (T-P) Inter-PE LSP to (T1) PE3 BGP: RD3, 10.2/16, RT-A, Next-Hop= , Label 60 BGP: RD2, 10.2/16, RT-B, Next-Hop= Label 70 PE2 PLR PE1 OSPF: metric 1 32 Copyright 2011 Juniper Networks, Inc /16 (multi-homed to PE1 and PE2) Context Identifier Stitching inter-pe LSP and Bypass LSP VPN A/Site 2 (multi-homed to PE1 and PE3) When PLR detects PE1 failure, PLR stitches inter-pe LSP and Bypass LSP Using MPLS FRR procedures Results in sending to Protector the traffic that used to go via PLR to PE1: E.g., from VPN A/Site 1 to VPN A/Site 2 E.g., from VPN B/Site 1 to VPN B/Site 2

30 Step 3: Protector Forwarding How? bgp.l3vpn: From PE1 (Protected Route): RD2 10.2/16, RT-B, Label 70, N-H From PE2 (Backup Route): RD1 10.2/16, RT-B, Label 50, N-H _ _.mpls.0: 70 swap to 50, push T2 Protector (protects PE1) PE4 BGP: RD1, 10.2/16, RT-B, Next-Hop= , Label 50 T T-P BGP: RD2, 10.2/16, RT-B, Next-Hop= , Label 70 PE2 PE1 10.2/ VPN B/Site 2 (multi-homed to PE1 and PE2) Context Identifier VPN A/Site 1 VPN B/Site 1 On Protector: T PE3 PLR 33 Copyright 2011 Juniper Networks, Inc /16 VPN A/Site 2 (multi-homed to PE1 and PE3) Put L3VPN routes whose BGP Next Hop matches the context identifier for which we are protector into bgp.l3vpn E.g., RD2 10.2/16, RT-B, Label 70, N-H Identify matching backup routes and put them into bgp.l3vpn: Exact matching Route Target Exact matching IP Prefix part of VPN-IP NLRI (not RD, as RDs may be different) E.g., RD1 10.2/16, RT-B, Label 50, N-H is backup for RD2 10.2/16, RT-B, Label 70, N-H Splice MPLS label information from bgp.l3vpn matching routes into LFIB ( context.mpls.0)

32 PREPARING PROTECTION FOR PRIMARY PE Primary PE cli changes [edit protocols BGP] protocols { replace: bgp { group INTERNAL { type internal; local-address ; family inet-vpn { unicast { egress-protection { context-identifier { ; neighbor ; Egress-protection stanza allows BGP to rewrite protocol nh to for family inet-vpn (afi1/safi128). This feature may be used for many future capabilities, e.g. iso-vpn, vpls, labeled-unicast for interprovider-vpn s. Hence the need to define the egressprotection per bgp-family. Once configured under [edit protocols bgp group <> family inet-vpn unicast], it acts as GLOBAL setting for all VRF s and gets inherited to the local configured VRF s. This can be overwritten, see next slides [edit routing-instances C1 egress-protection] stanza If egress-protection is configured on the vrf-level, then it is NOT required to have it configured under [edit protocols bgp group <> family inet-vpn unicast] 42 Copyright 2011 Juniper Networks, Inc.

33 PRIMARY PE MUST ADVERTISE CONTEXT-ID INTO LDP/IGP TO ALLOW OTHER PE S RESOLVE THE PROTOCOL NH protocols { replace: mpls { interface all; interface fxp0.0 { disable; egress-protection { context-identifier { primary; As the primary PE will have all its mpbgp-updates send with a protocol nh of , this context-identifier must be reachable by other PE s. The egress-protection knob under [edit protocols mpls] enforces advertisement of into IGP and LDP (same is needed on the protector) 43 Copyright 2011 Juniper Networks, Inc.

34 PROTECTOR ATTRACTING TRAFFIC protocols { replace: mpls { interface all; interface fxp0.0 { disable; egress-protection { context-identifier { protector; context-identifier { metric 2000; protector; The protector can protect multiple context-id s This stanza lets the protector advertise the context-identifier into LDP and ISIS with a default of max-metric -1 to attract traffic in case the primary PE fails. Metric is configurable, see snippet. See next slide for results 44 Copyright 2011 Juniper Networks, Inc.

35 PROTECTOR LABEL-MIRRORING AND SWAP-TABLES Defining the route-targets to listen on. Any route-updates for the given VRF are now being processed by the Protector. - Protector learns the prefixes (and VPN-lables) advertised by Primary PE and the backup PE - Protector learns if any backup-pe exists offering the same prefixes with different RD. - Protector learns as well VPN-labels as advertised from Backup PE - As result of learning VPN labels from backup PE and primary PE, the protector can now populate the mpls swap tables - Config next slide 45 Copyright 2011 Juniper Networks, Inc.

36 CREATING THE PROTECTOR Enabling the VPN-label mirroring for given vrf-targets protocols { bgp { group internal { replace: family inet-vpn { unicast { egress-protection { keep-import PROTECTOR-COMMUNITY; replace: policy-options { policy-statement LB { term 1 { then { load-balance per-packet; policy-statement PROTECTOR-COMMUNITY { term a { from community [ COMM_1 COMM_2 ]; then accept; community COMM_1 members target:100:1; community COMM_2 members target:100:2; 46 Copyright 2011 Juniper Networks, Inc.

37 HOW DOES PROTECTOR IDENTIFY & FORWARD TRAFFIC TO CORRECT BACKUP PE? Incoming packet arriving on bypass from PLR mpls.0 Protector Pop bypass label _ _.mpls.0 Particular Forwarding Context within Protector Swap primary PE VPN label with backup PE VPN label Push Backup PE tunnel label Outgoing packet Bypass LSP identifies the PE being protected Based on bypass LSP label, protector PE knows to lookup packet in a special MPLS table MPLS context table identifies the egress PE the protector is protecting Based on VPN label, protector identifies VPN Protector subsequently sends packet to backup PE, using VPN label advertised by backup PE 47 Copyright 2011 Juniper Networks, Inc.

38 ROUTE DETAILS ON PROTECTOR protector> show route table mpls.0 label (S=0) *[MPLS/0] 00:34:50 > to table mpls.0 LSP label points to context table, identifying Primary PE protector> show route table mpls.0 label *[Egress-Protection/170] 2d 08:09:19 > Swap 80, Push Primary PE VPN label (45) being swapped with Backup PE VPN label (80) Traffic tunneled over transport LSP (label ) to backup PE 52 Copyright 2011 Juniper Networks, Inc.

40 SUMMARY END-POINT PROTECTION PE FAILURE Junos tail-end-protection allows FRR/LFA rerouting in case a primary egress PE fails <50ms recovery time As recovery is quick, there is no urgent need to speedup global convergence Tailend-protection simply reroutes to a protector. Protector swaps VPN labels and forwards to a applicable backup-pe 59 Copyright 2011 Juniper Networks, Inc.

42 Implications on the overall connectivity recovery time for 2547 VPN service: system-wide perspective Ingress PE failure, ingress CE-PE link failure connectivity recovery time could be under 50 msec (local protection) PE-P link, P-P link, P node failure connectivity recovery time could be under 50 msec (local protection) Egress PE-CE link failure connectivity recovery time could be under 50 msec (local protection) Egress PE failure with global protection - connectivity recovery time is several 100s of msec Overall connectivity recovery time several 100s of msec Egress PE failure with local protection (as described in this presentation) - connectivity recovery time could be under 50 msec Overall connectivity recovery time could be under 50 msec Your chain is as strong as your weakest link 61 Copyright 2011 Juniper Networks, Inc.

43 In conclusion This presentation outlines a scheme that provides local protection against egress PE router failure Without imposing any constrains on network topology Applicable to both IPv4 and IPv VPN service Similar approach can be applied to provide local protection in the presence of ASBR failures Without imposing any constrains on network topology Useful for supporting 2547 VPN inter-as option (b) and (c) When BGP is used as an inter-area routing and label distribution protocol ( seamless MPLS ) similar approach can be applied to provide local protection in the presence of ABR failures Without imposing any constrains on network topology The scheme outlined in this presentation fills a crucial missing piece required to provide high availability 2547 VPN service 62 Copyright 2011 Juniper Networks, Inc.

MPLS Egress Protection Framework draft-shen-mpls-egress-protectionframework-02

MPLS Egress Protection Framework draft-shen-mpls-egress-protectionframework-02 Yimin Shen (yshen@juniper.net) Minto Jeyananth (minto@juniper.net) Bruno Decraene (bruno.decraene@orange.com) Updates New