SDN-based Automated Peering Optimization Challenges and Solutions

Transcription

1 SDN-based Automated Peering Optimization Challenges and Solutions Reda Laichi: Hamid Ould-Brahim: NANOG 74, Vancouver, October 2018

2 Agenda Defining the Problem SDN and Automation Use Cases Summary and References Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 2

3 Internet traffic reality 2000 à 2018 From web browsing to social media, video streaming and online gaming Traffi c New episodes ios update Viral video Time Internet traffic became much more versatile, dynamic and unpredictable Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 3

4 Problem Space Objectives & applicability... What do we want to address? Better bandwidth management, automatic congestion resolution, better traffic symmetry Better SLAs, & application Performance (Latency, Packet Loss) Reducing transit peering cost, addressing OPEX & CAPEX What traffic are we interested in? Inbound Traffic Outbound Traffic Balanced What part of the network? Peering Only Peering & Internal Network Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 4

5 Existing Limitations... Routing mechanism - Border Gateway Protocol (BGP) Unaware of link capacity & real-time utilization Packet loss and congestion No real-time path performance indication High latency No end-to-end performance indication Sub-optimum overall performance Visibility Lack or limited visibility Limited traffic engineering and steering Multiple teams involved (network operations, peer engineers, OAM probing, EMS alarms ) Slow communication Complex, manual processes Error-prone configurations Reactive model Organization and tools Inadequate for sudden realtime event changes Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 5

6 Agenda Defining the Problem SDN and Automation Use Cases Summary and References Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 6

7 What is being done today? Webscale players leading the way... Webscale players Gaming companies Existing route controllers/appliances Source: delivery.acm.org, 2017, Taking the Edge off with Espresso: Scale, Reliability and Programmability for Global Internet Peering Source: Latency Source: research.fb.com, 2017, Engineering Egress with Edge Fabric Steering Oceans of Content to the World Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 7

8 The Approach: Ten Thousand Foot View... Closed-Loop Automation for Peering Intent Cost Latency Bandwidth Automated Semi-automated Automation & Optimization Insight Action Real-time visibility Network control & Monitoring Information Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 8

9 Complexity of the automation problem? Stats, BGP routes, flow stats, protocol, destination AS, prefixes, application-based? Rates of sampling & collection Challenge: Existing operational environment and its complexity, and taking into account the human dimension Ping, TWAMP, TCP/UDP, HTTP, in-band? Internal lsp-ping Bi-directionality Return-path Peering Analytics & Telemetry OAM - Probing Automatic Real-time Events Detection Approach? Software Control Steering, flows, Routes, Policies Network Data & Monitoring topology, path, router/interface discovery Leveraging recent technology developments Filters, BGP Policies, Route Injection, BGP FlowSpec, Openflow, BGP SR TE Policies & route coloring, RIB/ FIB API, RSVP-TE/SR, PCEP, Netconf BGP-LS, EPE extensions, OSPF, ISIS, BMP, netconf The other challenge is in system integration, open interfaces and multi-vendor...at scale Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 9

10 Agenda Defining the Problem SDN and Automation Use cases Summary and References Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 10

11 Use Case 1: Local Peer Engineering Congestion example for outbound traffic Problem: 100s of peering partners, which link is best and is least congested? Approach: Congestion Threshold Target Alternate Traffic Offloading Interface Heatmap BR1 SDN 2 Analytics 1 Stats per source subnet or per destination subnet or per destination AS, per traffic category or per specific APP 1 2 Automatic link congestion detection Real-time stat/data collection and correlation (grpc, Netconf/SNMP) Interface & Flow Stats (IPFix) Determine optimal alternate peer based on bandwidth availability Match selected traffic flows at local ASBR based on IP flow/route/as/5 tuple, or application match (requires transaction-based steering) Redirect to next hop using FlowSpec, openflow, filters/netconf, RIB/FIB API, BGP policies) Content source PE1 PE2 ASBR (BR4) DC/Hosting Content ASBR (BR1) ASBR (BR2) ASBR (BR3)! Peer Transit 1 Transit 2 IXP AS521 AS522 AS523 Internet End users Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 11

12 Use Case 2: Egress peer engineering Problem: 100s of peering partners, which peer and egress link is best and is least congested? Approach: 2 1 Automatic link congestion detection Real-time stat/data collection and correlation (grpc, Netconf/SNMP) Interface & Flow Stats (IPFix) Determine optimal alternate ASBR & alternate peer based on topology and bandwidth availability. Steer selected traffic directly to LSP /Tunnel (filters, openflow, flowspec, segment routing BGP colored route Content source PE1 Establish an LSP/ Tunnel to alternate that meets traffic offloading profile 3 2 SDN Analytics ASBR (BR1)! 1 Peer Transit 1 Transit 2 Stats per source subnet or per destination subnet or per destination AS, per traffic category or per specific APP End users 3 Auto-create/use existing tunnel/path to alternate ASBR (can be PCE initiated) Steer selected IP flows at the edge of the network across the newly/existing tunnel and encode the egress peer link label/segment ID. PE2 ASBR (BR4) DC/Hosting Content ASBR (BR2) ASBR (BR3) IXP AS521 AS522 AS523 Internet Note: Colored BGP route to BGP-SR TE Policy tunnel can be used for steering Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 12

13 Use Case 3: Locale and EPE with Latency-based Steering Performance-based Optimization End-to-End Problem: 100s of peering partners, BGP best path on default peer causing high latency for top traffic or selected application/branch destination traffic. Approach: Real-time probing from each source PE and each alternate ASBRs of discovered top/vip destinations Determine optimal alternate ASBR based on end-to-end latency/performance data using path computation based on abstract topology Auto-create the LSP/ tunnel to alternate ASBR Auto-steer selected traffic to the LSP/ tunnel to alternate ASBR/ASBR+Egress peer link Steer selected traffic directly to LSP /Tunnel (filters, openflow, flowspec, segment routing BGP colored route) Content source Compute end-to-end based on Total latency information Establish an LSP/Tunnel to alternate that meets traffic offloading profile 3 2 PE1 PE2 ASBR (BR4) DC/Hosting Content SDN Analytics ASBR (BR1) ASBR (BR2) ASBR (BR3) 1! Peer Transit 1 Transit 2 IXP In addition to stats, collect latency measurement from PE-ASBR (inband using LSP ping) and ASBR- Destination point per selected prefix using Ping, TWAMP, TCP/UDP, HTTP Latency degradation AS521 AS522 AS523 Internet End users Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 13

14 Use Case 4: Controlling Inbound Traffic The example of Auto-pilot mode Problem: The links from transit providers are congested due to high incoming traffic. Automate traffic shifting per BGP communities to alternate TP. Inject BGP communities or BGP routes to influence traffic shifting (depending on BGP implementation/config.) 2 SDN Analytics Collects interface stats and flow stats per destination AS/destination prefix (IPFIX) Build BGP community real-time utilization Solution/Approach: Integrates and automate analytics data showing top BGP community traffic. Customers/Users 2 1 TP1 Inbound traffic Monitors bandwidth availability down to Customer devices and performs BGP route extraction and analysis (using BMP) Extract topology information, LSPs, telemetry PE1 ASBR (BR1) TP2 TP3 Using BGP Policies or route injection, selectively shift customer incoming traffic to alternate peer links or to even to an alternate transit provider Optionally steer selected customer prefixes to specific LSPs ASBR (BR2) Automation hierarchical policies: Automate within the same router, other routers same site, completely different region (slice) Need to factor in the BGP convergence time TP4 Internet Content source Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 14

15 Use Case 4: Controlling Inbound Traffic Example of Operator Triggered Change Problem: The links from transit providers are congested due to high incoming traffic. Peering Policy Intent: Shift top traffic per destination AS/ prefix to South/East region and pick and choose which prefix to steer to alternate tunnel once traffic is shifted. Solution/Approach: Slice the peering/network in terms of automation/ optimization zones. Performs BGP route extraction and analysis (using BMP) Extract topology information (BGP-LS, IGP), discovers LSPs. Using BGP Policies or route injection, selectively shift customer incoming traffic to alternate transit provider by injecting the change to the PEs. Optionally steer selected customer prefixes to specific LSPs Policies set once (match on community set MED...) End users Add BGP communities or BGP routes to influence traffic shifting (depending on BGP implementation/config.) PE1 North Region/West South Region/East Automation/optimization slices 2 2 SDN Analytics 1 ASBR (BR1) ASBR (BR2) Peer Transit 1 Transit 2 Integrates and automate analytics data showing customers destination AS traffic is in the top-5. Collects interface stats and flow stats per destination AS/destination prefix (IPFIX) Transit 3 Select prefixes and steer to selected LSPs/tunnels. Use BMP to learn all the routes Inbound traffic Internet Content source Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 15

16 Use Case 4: Controlling Inbound Traffic Behind The Scene Example of Operator Triggered Change From Transit provider 2/Slice6011 to transit provider 2/ Slice6012 policy-statement ebgp {... from { } protocol bgp; as-path CUSTOMER1-ASN; then { } community add TP2_SLICE6011_SUPPRESSED; community add TP1_SUPPRESSED; community add TP2_SLICE6012_USED; community add TP2_USED; accept; Automatic Injection community TP1_SLICE6011_SUPPRESSED members 1000:12227; community TP1_SUPPRESSED members [ 1000: :12222 ]; community TP2_SLICE6012_USED members 1000:12176; community TP2_USED members [ 1000: :12171 ]; Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 16 Deployed on Border Router: community ASPATH_PREPEND_1_TP1_NORTH members 1000:12201; term 40 { from { protocol bgp; community ASPATH_PREPEND_1_TP1_SOUTH; } then { as-path-prepend 1000;.

17 Use Case 5: The Case of Destination AS Optimization Problem: Hosting provider with congestion uplinks and available alternate network (metro) but limited bandwidth used congestion with no BGP reconfiguration Approach: Build one steering transaction of a set of traffic steering rules and redirect to alternate Next Hop/LSP To ASBR BR4 SDN Analytics 2 1 Stats per destination AS, per traffic category or per specific APP 1 2 Real-time traffic utilization monitoring per destination AS. Determine best next hop and steer selected AS to that next hop. PE1 Service Provider ASBR (BR1) Peer Transit 1 Transit 2 Uplinks End users Requires next hop tracking capabilities PE2 ASBR (BR2) IXP AS521 AS522 Internet ASBR (BR4) DC/Hosting Content ASBR (BR3) AS523 Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 17

18 Summary and Other Considerations Tracking changes (uniform data retention consideration) Culture and operational change considerations Think about Revertive actions (Undo button)! Peering simulation & predictive analysis on multi-dimensional data-set Traffic engineering/steering on multi-dimensional data-set Network control changes at large scale and real-time or near real-time telemetry at high frequency. Keep what BGP is best at and complement existing functionality Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 18

19 References BGP-LS PCEP Segment Routing Architecture Insight Driven Automated Network white paper Reda Laichi & Hamid Ould-Brahim, NANOG 74, Oct. 2018, 19

20 Questions? Reda Laichi: Hamid Ould-Brahim: 20