WAN Architectures and Design Principles

Transcription

1

2 WAN Architectures and Design Principles Stephen Lynn Consulting Systems Architect CCIE 5507 (R&S/WAN/Security) CCDE

3 Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot# 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

4 Agenda WAN Technologies & Solutions WAN Transport Technologies WAN Overlay Technologies WAN Optimization Wide Area Network Quality of Service WAN Extension into the Cloud WAN Architecture Design Considerations WAN Design and Best Practices SD-WAN Summary

5 WAN Transport Technologies

6 The WAN Technology Continuum Early Networking Early-Mid 1990s Mid 1990s-Late 2000s Today Flat/Bridged Experimental Networks Multiprotocol Business Enabling Large Scale Mission Critical Global Scale IP Ubiquity Business Survival Architectural Lessons Protocols required for Scale & Restoration Architectural Lessons Route First, Bridge only if Must Architectural Lessons Redundancy Build to Scale Planning? 1960 Internet Protocol X.25 Frame-Relay 1980 BGP GRE IPv DMVPN 4G/LTE Future ARPAnet TCP/IP RIP (BSD) OSPF, ISDN, ATM Tag Metro-Ethernet Switching SDN 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6

7 The Challenge Build a network that can adapt to a quickly changing business and technical environment Realize rapid strategic advantage from new technologies IPv6: global reachability Cloud: flexible diversified resources Internet of Things Fast-IT What s next? Adapt to business changes rapidly and smoothly Mergers & divestures Changes in the regulatory & security requirements Changes in public perception of services 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7

8 Tier 3 Tier 2 Tier 1 Network Design Modularity West Theater Global IP/MPLS Core East Theater West Region In-Theater IP/MPLS Core East Region Internet Cloud Public Voice/Video Mobility Private IP Service Metro Service Public IP Service Metro Service 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8

9 Hierarchical Network Principle Use hierarchy to manage network scalability and complexity while reducing routing algorithm overhead Hierarchical design used to be Three routed layers Core, aggregation, access Only one hierarchical structure end-to-end Hierarchical design has become any design that Splits the network up into places, or regions Separates these regions by hiding information Organizes these regions around a network core hub and spoke at a macro level 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9

10 MPLS L3VPN Topology Definition Spoke Site 1 Spoke Site 2 Spoke Site Y Hub Site (The Network) SP-Managed MPLS IP WAN Equivalent to Spoke Site X Spoke Site N Spoke Site 1 Spoke Site 2 Spoke Site X Spoke Site Y Spoke Site N MPLS WAN is provided by a service provider As seen by the enterprise network, every site is one IP hop away Equivalent to a full mesh, or to a hubless hub-and-spoke 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10

11 Virtual Routing and Forwarding Instance (VRF) Provides Network Virtualization and Path Isolation VRF VRF VRF Virtualization at Layer 3 forwarding Associates to Layer 3 interfaces on router/switch Each VRF has its own VRF-Lite Forwarding table (CEF) Routing process (RIP, OSPF, BGP) Hop-by-hop MPLS VPN Multi-hop VRF VRF VRF! PE Router Multiple VRFs ip vrf blue rd 65100:10 route-target import 65100:10 route-target export 65100:10 ip vrf yellow rd 65100:20 route-target import 65100:20 route-target export 65100:20! interface GigabitEthernet0/1.10 ip vrf forwarding blue interface GigabitEthernet0/1.20 ip vrf forwarding yellow 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11

12 Wide Area Network Design Trends Single Provider Design Enterprise will home all sites into a single carrier to provide L3 MPLS VPN connectivity. Pro: Simpler design with consistent features Con: Bound to single carrier for feature velocity Con: Does not protect against MPLS cloud failure with Single Provider Dual Providers Design Enterprise will single or dual home sites into one or both carriers to provide L3 MPLS VPN connectivity. Pro: Protects against MPLS service failure with Single Provider Pro: Potential business leverage for better competitive pricing Con: Increased design complexity due to service implementation differences (e.g. QoS, BGP AS Topology) Overlay Network Design Overlay tunneling technologies with encryption for provider transport agnostic design Pro: Can use commodity broadband services for lower cost higher bandwidth service Pro: Flexible overlay network topology that couples from the physical connectivity Con: Increased design complexity Con: Feature differences between providers could force customer to use least common denominator features. Con: Additional technology needed for SLA over commodity transport services 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12

13 Single Carrier Site Types (Non-Transit) Dual Homed Non Transit AS Only advertise local prefixes (^$) CE3 CE4 CE5 Typically with Dual CE routers BGP design: AS 200 ebgp to carrier ibgp between CEs Redistribute cloud learned routes into site IGP CE1 CE2 Single Homed Non Transit C1 Site IGP AS C2 Advertise local prefixes and optionally use default route Cisco and/or its affiliates. All rights reserved. Cisco Public 13

14 Dual Carrier: Transit vs. Non Transit To guarantee single homed site reachability to a dual homed site experiencing a failure, transit sites had to be elected. CE3 CE4 CE5 Transit sites would act as a BGP bridge transiting routes between the two provider clouds. Transit AS AS 100 AS 200 To minimize latency costs of transits, transits need to be selected with geographic diversity (e.g. from the East, West and Central US.) CE1 Site IGP CE2 C1 C2 Prefix X Prefix Y Prefix Z 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14

15 Single vs. Dual Carriers Single Provider Pro: Common QoS support model Pro: Only one carrier to tune Pro: Reduced head end circuits Pro: Overall simpler design Con: Carrier failure could be catastrophic Con: Do not have another carrier in your pocket Dual Providers Pro: More fault domains Pro: More product offerings to business Pro: Ability to leverage vendors for better pricing Pro: Nice to have a second vendor option Con: Increased Bandwidth Paying for bandwidth twice Con: Increased overall design complexity Con: May be reduced to common denominator between carriers Resiliency Drivers vs. Simplicity 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15

16 Metro Ethernet Service (L2VPN) E-Line (Point-to-Point) Replaces TDM private line and Frame-Relay or ATM L2VPN Point-to-point EVCs offer predictable performance for applications One or more EVCs allowed per single physical interface (UNI) Supports hub & spoke topology E-LAN (Point-to-Multipoint) Offers point to multipoint connectivity Transparent to VLANs and Layer 2 control protocols 4 or 6 classes of QoS support Supports service multiplexing (Ex. Internet access and corporate VPN via one UNI) 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16

17 MPLS (L3VPN) vs. Metro Ethernet (L2VPN) MPLS Layer 3 Service Routing protocol dependent on the carrier Layer 3 capability depends on carrier offering QoS (4 classes/6 classes) IPv6 capability Transport IP protocol only Highly scalable and ideal for large network MetroE Layer 2 Service Flexibility of routing protocol and network topology independent of the carrier Customer manages layer 3 QoS Capable of transport IP and non-ip traffic. Routing protocol determines scalability in point-to-multipoint topology 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17

18 WAN Overlay Technologies

19 Types of Overlay Service Layer 2 Overlays Layer 2 Tunneling Protocol Version 3 (L2TPv3) Layer 2 payloads (Ethernet, Serial, ) Pseudowire capable Other L2 overlay technologies OTV, VxLAN Layer 3 Overlays IPSec Encapsulating Security Payload (ESP) Strong encryption IP Unicast only Generic Routing Encapsulation (GRE) IP Unicast, Multicast, Broadcast Multiprotocol support Other L3 overlay technologies MPLSomGRE, LISP, OTP 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19

20 Tunnelling GRE and IPSec Transport and Tunnel Modes IP HDR IP Payload GRE packet with new IP header: Protocol 47 (forwarded using new IP dst) IP HDR GRE IP HDR IP Payload 20 bytes 4 bytes IPSec Transport mode IP HDR ESP HDR 20 bytes 30 bytes IP Payload Encrypted Authenticated 2 bytes ESP ESP Trailer Auth IPSec Tunnel mode IP HDR ESP HDR 20 bytes 54 bytes IP HDR IP Payload Encrypted Authenticated ESP 2 bytes Trailer ESP Auth 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

21 Locator / ID Separation Protocol (LISP) LISP Mapping System is analogous to a DNS lookup DNS resolves IP addresses for queried URL Answers the WHO IS question Host [ Who is ]? DNS Server [ Address is , 2610:D0:110C:1::3 ] DNS Name -to- IP URL Resolution LISP resolves locators for queried identities Answers the WHERE IS question LISP Router [ Where is 2610:D0:110C:1::3 ]? LISP Map System [ Locator is , ] LISP ID -to- Locator Map Resolution This Topic Is Covered in Detail in BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 21

22 LISP Overview - Terminologies EID (Endpoint Identifier) is the IP address of a host just as it is today RLOC (Routing Locator) is the IP address of the LISP router for the host EID-to-RLOC mapping is the distributed architecture that maps EIDs to RLOCs ITR Ingress Tunnel Router Receives packets from site-facing interfaces Encap to remote LISP sites, or native-fwd to non-lisp sites ETR Egress Tunnel Router Receives packets from core-facing interfaces De-cap, deliver packets to local EIDs at site xtr-1 ETR Provider A /8 Provider X /8 xtr-1 ETR ITR ITR S ETR ITR xtr-2 packet flow Provider B /8 Provider Y /8 packet flow ETR ITR xtr-2 D LISP Site 1 LISP Site Cisco and/or its affiliates. All rights reserved. Cisco Public 22

23 LISP Operation Example LISP Data Plane - Unicast Packet Forwarding 2 PI EID-prefix 2001:db8:1::/48 S LISP Site 1 xtr :db8:1::1 -> 2001:db8:2::1 ETR ITR ETR ITR xtr Map-Cache Entry EID-prefix: 2001:db8:2::/48 Locator-set: , priority: 1, weight: 50 (D1) , priority: 1, weight: 50 (D2) Provider A / > :db8:1::1 -> 2001:db8:2::1 Provider B /8 4 5 This policy controlled by the destination site > :db8:1::1 -> 2001:db8:2::1 Provider X /8 Provider Y / PI EID-prefix 2001:db8:2::/48 LISP Site :db8:1::1 -> 2001:db8:2::1 xtr-1 ETR ITR ETR ITR xtr-2 D 7 1 DNS entry: D.abc.com AAAA 2001:db8:2:: Cisco and/or its affiliates. All rights reserved. Cisco Public 23

24 LISP Use Cases IPv6 Transition Efficient Multi-Homing IPv4 Core v6 service xtr v6 v4 v6 IPv4 Internet PxTR v6 IPv6 Internet LISP Site LISP routers Internet IPv6-over-IPv4, IPv6-over-IPv6 IPv4-over-IPv6, IPv4-over-IPv4 Virtualization/Multi-tenancy IP Portability Ingress Traffic Engineering Without BGP Data Center/ VM Mobility Legacy Site Legacy Site Legacy Site Data Center 1 Internet Data Center 2 LISP Site IP Network PxTR Mapping DB LISP routers VM move LISP routers West DC East DC VM a.b.c.1 VM a.b.c.1 Large Scale Segmentation Cloud / Layer 3 VM Move 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

25 Cisco Site to Site VPN Technologies Comparison Features DMVPN FlexVPN GET VPN Infrastructure Network Network Style Public or Private Transport Overlay Routing IPv4/IPv6 dual Stack Large Scale Hub and Spoke with dynamic Any-to-Any Public or Private Transport Overlay Routing Converged Site to Site and Remote Access Private IP Transport Flat/Non-Overlay IP Routing Any-to-Any; (Site-to-Site) Failover Redundancy Active/Active based on Dynamic Routing Dynamic Routing or IKEv2 Route Distribution Server Clustering Transport Routing COOP Based on GDOI Scalability Unlimited Client/Srv Unlimited Client/Srv 8000 GM total 4000 GM/KS IP Multicast Multicast replication at hub Multicast replication at hub Multicast replication in IP WAN network QoS Per Tunnel QoS, Hub to Spoke Per SA QoS, Hub to Spoke Per SA QoS, Spoke to Spoke Transport QoS Policy Control Locally Managed Centralized Policy Management Locally Managed Technology Tunneled VPN Multi-Point GRE Tunnel IKEv1 & IKEv2 Tunneled VPN Point to Point Tunnels IKEv2 Only Tunnel-less VPN Group Protection IKEv Cisco and/or its affiliates. All rights reserved. Cisco Public 25

26 Dynamic Multipoint VPN (DMVPN) Branch spoke sites establish an IPsec tunnel to and register with the hub site IP routing exchanges prefix information for each site BGP or EIGRP are typically used for scalability With WAN interface IP address as the tunnel source address, provider network does not need to route customer internal IP prefixes Data traffic flows over the DMVPN tunnels When traffic flows between spoke sites, the hub assists the spokes to establish a site-to-site tunnel Per-tunnel QOS is applied to prevent hub site oversubscription to spoke sites ISR Branch 1 SECURE ON-DEMAND TUNNELS ASR 1000 IPsec VPN Hub Traditional Static Tunnels Branch 2 DMVPN On-Demand Tunnels Static Known IP Addresses ISR Dynamic Unknown IP Addresses Branch n ISR 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

27 DMVPN How it Works Spokes build a dynamic permanent GRE/IPsec tunnel to the hub, but not to other spokes. They register as clients of the NHRP server (hub) and register their NBMA address Active-Active redundancy model two or more hubs per spoke All configured hubs are active and are routing neighbors with spokes Routing protocol routes are used to determine traffic forwarding A spoke will initially send a packet to a destination (private) subnet behind another spoke via the hub, and the hub will send it an NHRP redirect. The redirect triggers the spoke to send an NHRP query for the data packet destination address behind the destination spoke The destination spoke initiates a dynamic GRE/IPsec tunnel to the source spoke (it now knows its NBMA address) and sends the NHRP reply. The dynamic spoke-to-spoke tunnel is built over the mgre interface When traffic ceases then the spoke-to-spoke tunnel is removed Physical: Tunnel0: /24 Dual DMVPN Design Single mgre tunnel on Hub, two mgre tunnels on Spokes /24 Physical: (dynamic) Tunnel0: Tunnel1: /24 Physical: Tunnel0: Physical: (dynamic) Tunnel0: Tunnel1: / Cisco and/or its affiliates. All rights reserved. Cisco Public 27

28 DMVPN Network Designs Spoke-to-hub tunnels Spoke-to-spoke tunnels 2547oDMVPN tunnels Hub and spoke Spoke-to-spoke VRF-lite Increase in Scale Server Load Balancing Hierarchical 2547oDMVPN 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28

29 Any-to-Any Encryption Before and After GETVPN Public/Private WAN Before: IPSec P2P Tunnels Private WAN After: Tunnel-Less VPN WAN Multicast Scalability an issue (N^2 problem) Overlay routing Any-to-any instant connectivity can t be done to scale Limited QoS Inefficient Multicast replication Scalable architecture for any-to-any connectivity and encryption No overlays native routing Any-to-any instant connectivity Enhanced QoS Efficient Multicast replication 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29

30 Group Security Functions Key Server Validate Group Members Manage Security Policy Create Group Keys Distribute Policy/Keys Group Member Group Member Encryption Devices Route Between Secure/ Unsecure Regions Multicast Participation Key Server Routing Members Routing Member Forwarding Replication Routing Group Member Group Member Group Member 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30

31 Group Security Elements Group Policy Key Servers KS Cooperative Protocol Key Encryption Key (KEK) Traffic Encryption Key (TEK) RFC3547 RFC6407: Group Domain of Interpretation (GDOI) Group Member Routing Members Group Member Group Member Group Member 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31

32 GETVPN - Group Key Technology Operation Example Step 1: Group Members (GM) register via GDOI (IKE) with the Key Server (KS) KS authenticates and authorizes the GM KS returns a set of IPsec SAs for the GM to use Step 2: Data Plane Encryption GM exchange encrypted traffic using the group keys The traffic uses IPSec Tunnel Mode with address preservation Step 3: Periodic Rekey of Keys KS pushes out replacement IPsec keys before current IPsec keys expire; This is called a rekey GM1 GM1 GM1 GM2 GM9 GM2 GM9 GM2 GM9 GM3 GM4 GM5 GM6 KS GM8 GM7 GM3 GM4 GM5 GM6 KS GM8 GM7 GM3 GM4 GM5 GM6 KS GM Cisco GM7 and/or its affiliates. All rights reserved. Cisco Public

33 GETVPN Virtualization Deployment Model GETVPN Segmented WAN CE PE PE CE MPLS VPN LISP with GETVPN CE PE PE CE GET Encrypted LISP LISP over GETVPN 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33

34 MACsec Layer 2 Encryption for WAN Ethernet is not only in the campus and data center any longer Ethernet is growing rapidly as a WAN & Metro transport service WAN/Metro SP offerings are replacing existing T1, ATM/FR, and SONET OC-x options Ethernet services apply to many areas of the WAN/MAN: WAN links for core, edge, remote branch PE-CE links (leveraging L3 VPN services) Metro-E service hand-offs (P2P, P2MP) Cisco s goal is to integrate MACsec as part of new Ethernet interface/lc development moving forward on routers leveraged in WAN, MAN, Branch Market Transition Ethernet WAN Transport 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

35 What is WAN MACsec? Secure Ethernet Link(s) over Public Ethernet Transport Data Center Remote Campus/DC MKA Session Service Provider Owned Routers/Bridges Public Carrier Ethernet Service Leverage public standard-based Ethernet transport Central Campus/DC Data Center Optimize MACsec + WAN features to accommodate running over public Ethernet transport Target line-rate encryption for high-speed applications Inter DC, MPLS WAN links, massive data projects Targets 100G, but support 1/10/40G as well MACsec MKA Session MACsec Secured Path / MKA Session MACsec Capable Router MACsec Capable PHY SP Owned Ethernet Transport Device 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36

36 What is WAN MACsec? New Enhancements to 802.1AE for WAN/Metro-E Transport AES-256 (AES/GCM) support 1/10/40 and 100G rates Target Next Generation Encryption (NGE) profile that currently leverages public NSA Suite B Standards Based MKA key framework (defined in 802.1X-2010) within Cisco security development (Cisco NGE ) Ability to support 802.1Q tags in clear Offset 802.1Q tags in clear before encryption (2 tags is optional) Vital Network Features to Interoperate over Public Carrier Ethernet Providers 802.1Q tag in the clear Ability to change MKA EAPoL Destination Address type Ability to change MKA Ether-type value Ability to configure Anti-replay window sizes System Interoperability Create a common MACsec integration among all MACsec platforms in Cisco and Open Standards 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37

37 MACsec Deployment Model IPv4/v6 Ring Topology IPv4/v6 IPv4/v6 P-t-P E-LINE P-t-MP E-LAN IPv4/v6 CE2 Point-to-point dark fiber Metro E Routers peer per VLAN subinterface per PW IPv4/v6 IPv4/v6 CE4IPv4/v6 Ethernet Sub-interface with 802.1q support CE3 IPv4/v6 Clear-Tag Feature: Leverage 802.1Q tag support for full/partial-mesh over Metro-Ethernet 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38

38 WAN Optimization

39 The WAN Is the Barrier to Branch Application Performance Applications are designed to work well on LAN s High bandwidth Low latency Reliability Round Trip Time ~ 0ms Client LAN Switch Server WANs have opposite characteristics Low bandwidth High latency Packet loss Round Trip Time ~ Many milliseconds Client LAN Switch WAN LAN Switch Server WAN Packet Loss and Latency = Slow Application Performance = Keep and manage servers in branch offices ($$$) 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40

40 TCP Behavior Return to maximum throughput could take a very long time! Window Size Packet loss Packet loss Packet loss Packet loss TCP Slow start Congestion avoidance Time (RTT) RFC TCP Extensions for High Performance 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41

41 WAAS TCP Performance Improvement Transport Flow Optimization (TFO) overcomes TCP and WAN bottlenecks Shields nodes connections from WAN conditions Clients experience fast acknowledgement Minimize perceived packet loss Eliminate need to use inefficient congestion handling WAN LAN TCP Behavior Window Scaling Large Initial Windows Congestion Mgmt Improved Retransmit LAN TCP Behavior 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42

42 WAAS Advanced Compression DRE and LZ Manage Bandwidth Utilization Data Redundancy Elimination (DRE) Persistent LZ Compression Application-agnostic compression Up to 100:1 compression WAAS 4.4: Context Aware DRE Benefits Session-based Application-agnostic compression compression Up Up to to 10:1 100:1 compression compression Works WAAS even 4.4: during Context cold Aware DRE DRE cache LZ WAN LZ DRE Synchronized Compression History DRE 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43

43 Comparing TCP and Transport Flow Optimization Cisco TFO Provides Significant Throughput Improvements over Standard TCP Implementations Window Size TFO TCP Slow start Congestion avoidance Time (RTT) TFO is using RFC2018, RFC1323, RFC3390 and BIC-TCP Cisco and/or its affiliates. All rights reserved. Cisco Public 44

44 WAN Quality of Service

45 Quality of Service Operations How Does It Work and Essential Elements Classification and Marking Queuing and Dropping Post-Queuing Operations Classification and Marking: The first element to a QoS policy is to classify/identify the traffic that is to be treated differently. Following classification, marking tools can set an attribute of a frame or packet to a specific value. Policing: Determine whether packets are conforming to administratively-defined traffic rates and take action accordingly. Such action could include marking, remarking or dropping a packet. Scheduling (including Queuing and Dropping): Scheduling tools determine how a frame/packet exits a device. Queuing algorithms are activated only when a device is experiencing congestion and are deactivated when the congestion clears Cisco and/or its affiliates. All rights reserved. Cisco Public 47

46 Enabling QoS in the WAN Traffic Profiles and SLA Requirements Voice SD Video Conf Telepresence Data Smooth Benign Drop sensitive Delay sensitive UDP priority Bursty Greedy Drop sensitive Delay sensitive UDP priority Bursty Drop sensitive Delay sensitive Jitter sensitive UDP priority Smooth/bursty Benign/greedy Drop insensitive Delay insensitive TCP retransmits Bandwidth per Call Depends on Codec, Sampling-Rate, and Layer 2 Media Latency 150 ms Jitter 30 ms Loss 1% Bandwidth (30-128Kbps) One-Way Requirements SD/VC has the Same Requirements as VoIP, but Has Radically Different Traffic Patterns (BW Varies Greatly) Latency 150 ms Jitter 30 ms Loss 0.05% Bandwidth (1Mbps) One-Way Requirements HD/VC has Tighter Requirements than VoIP in terms of jitter, and BW varies based on the resolutions Latency 200 ms Jitter 20 ms Loss 0.10% Bandwidth (5.5-16Mbps) One-Way Requirements Traffic patterns for Data Vary Among Applications Data Classes: Mission-Critical Apps Transactional/Interactive Apps Bulk Data Apps Best Effort Apps (Default) 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48

47 Scheduling Tools LLQ/CBWFQ Subsystems Packets In 1 Mbps VoIP Policer 5 Mbps RT-Interactive Policer IOS Interface Buffers LLQ CBWFQ Scheduler policy-map WAN class VOICE priority percent 10 class VIDEO priority percent 23 class CRITICAL-DATA bandwidth percent 15 random-detect dscp-based class DATA bandwidth percent 19 random-detect dscp-based class SCAVENGER bandwidth percent 5 class NETWORK-CRITICAL bandwidth percent 3 service-policy MARK-BGP class class-default Tx-Ring Packets Out bandwidth percent 25 random-detect CBWFQ 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49

48 Traffic Shaping Line Rate Shaped Rate Without Traffic Shaping With Traffic Shaping Traffic Shaping Limits the Transmit Rate to a Value Lower Than Line Rate Policers typically drop traffic Shapers typically delay excess traffic, smoothing bursts and preventing unnecessary drops Very common with Ethernet WAN, as well as Non- Broadcast Multiple-Access (NBMA) network topologies such as Frame-Relay and ATM 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50

49 Hierarchical QoS For Subrate Service H-QoS Policy on WAN Interface, Shaper = CIR Policy-map PARENT class class-default shape average service-policy output CHILD Interface gigabitethernet 0/1 service-policy output PARENT Two Levels MQC Policy-map CHILD class VOICE priority percent 10 class VIDEO priority percent 23 class CRITICAL-DATA bandwidth percent 15 random-detect dscp-based class DATA bandwidth percent 19 random-detect dscp-based class SCAVENGER bandwidth percent 5 class NETWORK-CRITICAL bandwidth percent 3 service-policy MARK-BGP class class-default bandwidth percent 25 random-detect Service Level Video Voice Best Effort Scavenger Critical Data Network Critical 150 Mbps Gig 0/ Cisco and/or its affiliates. All rights reserved. Cisco Public 51

50 ToS ToS ToS ToS ToS GRE/IPSec QoS Consideration ToS Byte Preservation ToS byte is copied to the new IP Header IP HDR IP Payloaad GRE Tunnel IP HDR GRE HDR IP HDR IP Payload IPSec Tunnel mode IP HDR ESP HDR IP HDR IP Payload ESP Trailer ESP Auth 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53

51 WAN Extension into the Cloud

52 Cloud Migration Trend Secure Agile Exchange at Colocations Facilities Customers Secure Agile Exchange Cloud Customers SaaS Colocation Centers Employees Employees Secure Agile Exchange Private Data Center Partners DMZ Private Data Center Applications Partners Public Cloud BRKDCT-2409 Building The Secure Agile Hybrid Cloud Network 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55

53 Connectivity Options into AWS Cloud Corporate DC AWS Managed VPN Internet Cisco ISR/ASR VGW Private Network CSR 1000V AWS Direct Connect Corporate DC CSR 1000V Colocation Facility Cisco ISR/ASR 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56

54 AWS Virtual Private Cloud (VPC) Overview Represents logically isolated network Separate IP range, Subnet, Routes, and Security Overlap IP address allowed NO Support for Multicast/Broadcast traffic, VLANs, and Transit Routing Network Controls Available to you: Route Tables, aka VRFs Internet Gateway (IGW), aka an Internet Router Security Groups and Network ACLs VPNaaS Internet GW Internet gateway forward traffic to outsides and in between VPCs Subnet Router forward traffic within the VPC Subnet Router 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58

55 AWS VPC Peering Transit Routing Limitations You have a VPC peering connection between VPC A and VPC B And between VPC A and VPC_C But you cannot route packets B X C directly from VPC B to VPC C through VPC A 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60

56 Inter-VPN Design using Transit VPCs with CSR1000v End to end routings between VPCs in all regions and other none AWS network App VPC Shared Services VPC 3 rd Party VPC Used for hierarchical designs Scaling beyond VPC peering limits Security/Monitoring Services in Transit VPC VPC peering VPC peering VGW Redundant CSRs for HA and fast convergence Availability Zone 1 Transit VPC Availability Zone 2 Can use VRFs to resolve recursive routing issues Direct Connect Corporate Network/DC 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 61

57 Transit VPC to Interconnect AWS Regions AZ 1 AZ 2 Transit VPC US-West-2 AZ 1 AZ 2 Transit VPC US-East-2 Corporate Network/DC 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

58 WAN Architecture Design and Best Practices

59 Cisco Validate Design MPLS WAN Technology Design Guide 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

60 WAN Aggregation Reference Design Data Center/ Campus Campus/ Data Center WAAS Service WAN Services/ Distribution Key Servers VPN Termination WAN Edge MPLS A MPLS B Internet 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69

61 Routing Topology at WAN Aggregation Core Layer Campus/ Data Center WAN Distribution Layer EIGRP AS 100 Summaries+ Default EIGRP AS = 100 ibgp EIGRP AS = 100 DMVPN Hub Routers EIGRP AS = 100 Internet Edge BGP AS = MPLS CE Routers BGP AS = Layer 2 WAN CE Router EIGRP AS = 200 MPLS A ebgp MPLS B Layer 2 WAN DMVPN 1 DMVPN 2 Internet 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

62 WAN Edge Connection Methods Compared Recommended Multi-Chassis EtherChannel VSS/3850 Stacks Si Shared LAN Si Si Layer 3 P-to-P Link Si WAN WAN WAN No Static Routes No First Hop Redundancy Protocols 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72

63 Optimize Convergence and Redundancy Multi-chassis EtherChannel IGP recalc Si Layer 3 P-to-P Link Si Channel Member Removed VSS/3850 Stacks Link redundancy achieved through redundant L3 paths Flow based load-balancing through CEF forwarding across Routing protocol reconvergence when uplink failed Convergence time may depends on routing protocol used and the size of routing entries Provide Link Redundancy and reduce peering complexity Tune L3/L4 load-balancing hash to achieve maximum utilization No L3 reconvergence required when member link failed No individual flow can go faster than the speed of an individual member of the link 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73

64 sec of lost voice Link Recovery Comparison ECMP vs. Multichassis EtherChannel ECMP convergence is dependent on the number of routes Si Layer 3 P-to-P Link Si MEC convergence is consistent, independent of the number of routes ECMP MEC Max 1.5 VSS/3850 Stacks Number of Routes Number of Routes - Sup720C 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

65 Seconds Redundancy vs. Convergence Time More Is Not Always Better In principle, redundancy is easy Any system with more parallel paths through the system will fail less often The problem is a network isn t really a single system but a group of interacting systems Increasing parallel paths increases routing complexity, therefore increasing convergence times 2.5 Routes Cisco and/or its affiliates. All rights reserved. Cisco Public 75

66 Best Practice Summarize at Service Distribution It is important to force summarization at the distribution towards WAN Edge and towards campus & data center Summarization provides topology change isolation. Campus/ Data Center Summary /16 Summarization reduce routing table size. interface Port-channel1 description Interface to MPLS-A-CE no switchport ip address ip pim sparse-mode ip summary-address eigrp MPLS A MPLS B Summaries + Default / / Cisco and/or its affiliates. All rights reserved. Cisco Public 76

67 Best Practice Preventing Routing Loops with Route Tag and Filter Mutual route redistribution between protocols can cause routing loops without preventative measures Use route-map to set tags and then redistribute based on the tags Routes are implicitly tagged when distributed from ebgp to EIGRP/OSPF with carrier AS Use route-map to block re-learning of WAN routes via the distribution layer (already known via ibgp) router eigrp 100 distribute-list route-map BLOCK-TAGGED-ROUTES in default-metric [BW] redistribute bgp route-map BLOCK-TAGGED-ROUTES deny 10 match tag IGP Domain (EIGRP/OSPF) Campus MPLS WAN BGP Domain route-map BLOCK-TAGGED-ROUTES permit Cisco and/or its affiliates. All rights reserved. Cisco Public 77

68 Dual Carriers with BGP as CE-PE Protocol Use ibgp for Path Selection Run ibgp between the CE routers to exchange prefixes associated with each carrier CE routers will use only BGP path selection information to select both the primary and secondary preferences for any destinations announced by the IGP and BGP Use IGP (OSPF/EIGRP) for prefix readvertisement will result in equal-cost paths at remote-site bn-br # sh ip bgp /21 BGP routing table entry for /21, version 71 Paths: (2 available, best #2, table default, RIB-failure(17)) Not advertised to any peer , (aggregated by ) from ( ) Origin IGP, localpref 100, valid, external, atomicaggregate 65402, (aggregated by ) (metric 51456) from ( ) Origin IGP, metric 0, localpref 100, valid, internal, atomic-aggregate, best MPLS A A ibgp Campus /21 ibgp B MPLS B / Cisco and/or its affiliates. All rights reserved. Cisco Public 78

69 Best Practice - Implement AS-Path Filter Prevent Branch Site Becoming Transit Network Dual carrier sites can unintentionally become transit network during network failure event and causing network congestion due to transit traffic Campus Design the network so that transit path between two carriers only occurs at sites with enough bandwidth Implement AS-Path filter to allow only locally originated routes to be advertised on the outbound updates for branches that should not be transit MPLS A MPLS B router bgp neighbor route-map NO-TRANSIT-AS out! ip as-path access-list 10 permit ^$! route-map NO-TRANSIT-AS permit 10 match as-path 10 A ibgp B 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79

70 Golden Rules Route Preference for EIGRP & OSPF For Your Reference EIGRP Internal EIGRP Admin Dist. 90 External EIGRP Admin Dist. 170 Metric Calculation metric = bandwidth + delay Bandwidth (in kb/s) Delay (in microseconds) OSPF Admin Dist. 110 Route Preference 1. Intra-Area 2. Inter-Area 3. External E1 (Internal + External Cost) 4. External E2 (External Cost) Cost Calculation Cost= Reference BW / Interface BW Default Reference BW = 100Mbps 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 80

71 MPLS + Internet WAN Prefer the MPLS Path over Internet Campus ebgp EIGRP AS100 ebgp routes are redistributed into EIGRP 100 as external routes with default Admin Distance 170 Running same EIGRP AS for both campus and DMVPN network would result in Internet path preferred over MPLS path MPLS A Internet EIGRP AS / Cisco and/or its affiliates. All rights reserved. Cisco Public 81

72 MPLS + Internet WAN Use Autonomous System for IGP Path Differentiation ebgp MPLS A Campus EIGRP AS100 Internet D EX /21 [170/28416] via EIGRP AS200 ebgp routes are redistributed into EIGRP 100 as external routes with default Admin Distance 170 Running same EIGRP AS for both campus and DMVPN network would result in Internet path preferred over MPLS path Multiple EIGRP AS processes can be used to provide control of the routing EIGRP 100 is used in campus location EIGRP 200 over DMVPN tunnels Routes from EIGRP 200 redistributed into EIGRP 100 appear as external route (distance = 170) Routes from both WAN sources are equal-cost paths. To prefer MPLS path over DMVPN use eigrp delay to modify path preference MPLS CE router# /21 router eigrp 100 default-metric Cisco and/or its affiliates. All rights reserved. Cisco Public 82

73 MPLS VPN BGP Path with IGP Backdoor Path ebgp as the PE-CE Routing Protocol Campus MPLS VPN as preferred path learned via ebgp EIGRP AS100 Secondary path via backdoor IGP link (EIGRP or OSPF) over tunneled connection (DMVPN over Internet) Default configuration the failover to backup path works as expected R1 ebgp MPLS A R2 Internet IGP Backup Link / Cisco and/or its affiliates. All rights reserved. Cisco Public 83

74 MPLS VPN BGP Path with IGP Backdoor Path After link restore, MPLS CE router receives BGP advertisement for remote-site route. Campus EIGRP AS100 Does BGP route get (re)installed in the route table? R1 R2 D EX /24 [170/3584]... R1# show ip route B /24 [20/0] via , 01:30:06 B /24 [20/0] via , 01:30:06 D EX /24 [170/3584] via , 00:30:06 ebgp MPLS A Internet IGP Backup Link B /24 [20/0] / Cisco and/or its affiliates. All rights reserved. Cisco Public 84

75 BGP Route Selection Algorithm BGP Prefers Path with: 1. Highest Weight 2. Highest Local Preference 3. Locally originated (via network or aggregate BGP) 4. Shortest AS_PATH 5. Lowest Origin type IGP>EGP>INCOMPLETE (redistributed into BGP) 6. Lowest Multi-Exit Discriminator (MED) 7. Prefer Externals (ebgp over ibgp paths) 8. Lowest IGP metric to BGP next hop (exit point) 9. Lowest Router ID for exit point For Your Reference 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85

76 BGP Prefers Path with Highest Weight Routes redistributed into BGP are considered locally originated and get a default weight of The ebgp learned prefix has default weight of 0 Path with highest weight is selected ASR1004-1#show ip bgp BGP routing table entry for /24, version 22 Paths: (3 available, best #3, table default) Advertised to update-groups: from ( ) Origin IGP, localpref 200, valid, external Local from ( ) Origin incomplete, metric , localpref 100, weight 32768, valid, sourced, best 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 86

77 Prefer the ebgp Path over IGP Set the ebgp weight > To resolve this issue set the weights on route learned via ebgp peer higher than neighbor weight ASR1004-1#show ip bgp BGP routing table entry for /24, version 22 Paths: (1 available, best #1, table default) Not advertised to any peer from ( ) Origin IGP, metric 0, localpref 100, weight 35000, valid, external, best ASR1004-1#show ip route... B /24 [20/0] via , 05:00: Cisco and/or its affiliates. All rights reserved. Cisco Public 87

78 Cisco SD-WAN

79 Cisco SD-WAN Solution Philosophy Application SLA Traffic Engineering Per-Segment Topologies Secure Perimeter Cloud Path Cloud Accel Transport Hub Analytics Application Policies Monitoring Routing Security Segmentation QoS Multicast Svc Insertion Survivability Operations Delivery Platform Broadband MPLS Cellular Transport Independent Fabric 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 89

80 Cisco SD-WAN Secure Extensible Network Orchestration Plane vorchestrator vbond MANAGEMENT Management Plane (Multi-tenant or Dedicated) ORCHESTRATION API Control Plane (Containers or VMs) vmanage CONTROL ANALYTICS Data Plane (Physical or Virtual) vsmart INTERNET MPLS 4G vedge Data Center Campus Branch Home Office 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 90

81 Cisco SD-WAN Solution Elements Orchestration Plane vmanage APIs Orchestration Plane Cisco vbond vbond vanalytics vsmart Controllers MPLS INET 4G 3 rd Party Automation vedge Routers Orchestrates connectivity First point of authentication (white-list model) Distributes list of vsmarts/ vmanage to all vedge routers Requires public IP Address [could be behind 1:1 NAT] Facilitates NAT traversal Highly resilient Cloud Data Center Campus Branch SOHO 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 91

82 Cisco SD-WAN Solution Elements Management Plane vmanage Management Plane Cisco vmanage vbond Cloud vanalytics vsmart Controllers MPLS INET 4G APIs 3 rd Party Automation vedge Routers Data Center Campus Branch SOHO Single pane of glass for Day0, Day1 and Day2 operations Multitenant with web scale Centralized provisioning Policies and Templates Troubleshooting and Monitoring Software upgrades GUI with RBAC Programmatic interfaces (REST, NETCONF) Highly resilient 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 92

83 Cisco SD-WAN Solution Elements Control Plane Control Plane vbond Cloud vanalytics vsmart Controllers MPLS vmanage INET 4G APIs 3 rd Party Automation vedge Routers Data Center Campus Branch SOHO Cisco vsmart Centralized brain of the operation Facilitates fabric discovery Dissimilates control plane information between vedges Distributes data plane and appaware routing policies to the vedge routers Implements control plane policies such as o Traffic Engineering o Service Chaining, o Multi-topology (hub/spoke, partial, or full mesh) 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 93

84 Cisco SD-WAN Solution Elements Data Plane vmanage Data Plane Physical/Virtual Cisco vedge vbond Cloud vanalytics vsmart Controllers MPLS INET 4G APIs 3 rd Party Automation vedge Routers Data Center Campus Branch SOHO WAN edge router Leverages traditional routing protocols like OSPF, BGP and VRRP Provides secure data plane with remote vedge routers Establishes secure control plane with vsmart controllers (OMP) Apply application aware routing policies Support Zero Touch Deployment Physical or Virtual form factor (VNF) support 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 94

85 Overlay Management Protocol (OMP) Unified Control Plane vsmart vsmart vsmart TCP based TLS/DTLS control plane protocol Runs between vedge routers and vsmart controllers and between the vsmart controllers Advertises control plane context, i.e. TLOCs, unicast/multicast destinations, service routes (L4- L7), BFD stats (TE and H-SDWAN) and Cloud onramp for SaaS probe stats (gateway) Distributes IPSec encryption keys, and data and app-aware policies (embedded NETCONF) VS vedge vedge Note: vedge routers need not connect to all vsmart Controllers 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 96

86 Secure Segmentation End-to-End Segmentation Interface VLAN VPN1 VPN2 Ingress vedge SD-WAN IPSec Tunnel VPN 1 VPN 2 VPN 3 Egress vedge VPN1 VPN2 Interface VLAN IP UDP ESP VPN Data Segment connectivity across fabric w/o reliance on underlay transport vedge routers maintain per-vpn routing table Labels are used to identify VPN for destination route lookup Interfaces and sub-interfaces (802.1Q tags) are mapped into VPNs 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 97

87 Fabric Operation Walk-Through OMP DTLS/TLS Tunnel IPSec Tunnel BFD OMP Update OMP Update vsmart Policies OMP Update: Reachability IP Subnets, TLOCs Security Encryption Keys Policy Data/App-route Policies OMP Update OMP Update vedge Transport 1 vedge TLOCs TLOCs BGP, OSPF, Connected, Static VPN1 A VPN2 B Transport 2 VPN1 C VPN2 D BGP, OSPF, Connected, Static Subnets Subnets 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 98

88 Cisco SD-WAN Migration Strategy Gateway/DC Site Deployment BGP/OSPF DC/Gateway Site Identify Gateway/DC Sites providing connectivity between SD-WAN and legacy sites Legacy sites talk to each other directly Internet SD-WAN Secure Fabric MPSL SD-WAN sites talk to each other directly Legacy router/connectivity is dropped in the DC/Gateway sites once migration is complete SD-WAN Sites Legacy/MPLS Sites 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 100

89 SD-WAN TCP Maximum Segment Size Adjustment Packet Fragmentation Avoidance for TCP Traffic Host Send MSS 1320B Signaled MSS 1460B Link IP MTU 1500 Bytes vedge Router MSS Adjust to 1320B IPSec Secure Fabric Signaled MSS 1320B Signaled MSS 1320B vedge Router MSS Adjust to 1320B Tunneling introduces additional overhead and increases packet size Send TCP MSS is min (local link IP MTU - 40B*, signaled MSS value) - Signaled in SYN packets vedge Router TCP MSS adjustment overrides signaled MSS value - Each side determines its own MSS value, not synchronized Link IP MTU 1500 Bytes Signaled MSS 1460B Application Servers Send MSS 1320B * 20B IP header + 20B TCP header. Does not include TCP options Cisco and/or its affiliates. All rights reserved. Cisco Public 101

90 Summary

91 Tier 3 Tier 2 Tier 1 Modern Hierarchical Global WAN Design West Theater Global IP/MPLS Core East Theater West Region In-Theater IP/MPLS Core East Region Internet Cloud Public Voice/Video Mobility Private IP Service Metro Service Public IP Service Metro Service 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 104

92 Key Takeaways Understand how WAN characteristics can affect your applications. Bandwidth, latency, loss A modular hierarchical network infrastructure is the foundation for a solid WAN architecture. Good WAN design have many well-utilized component Encryption is a foundation component of all WAN designs and can be deployed transparently. Understand how to build wide area network leveraging Internet transport with SD-WAN. Design a network with consistent behavior that provides predictable performance. More is not always better. Keep it simple! 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 105

93 Recommended Reading Abstract: Virtual Routing in the Cloud are key enablers of today s revolutionary shift to elastic cloud applications and low-cost virtualized networking. The book covers every essential building block, present key use cases and configuration examples, illuminate design and deployment scenarios, and show how the CSR 1000V platform and APIs can enable state-of-the-art software-defined networks (SDN). Drawing on extensive early adopter experience, they illuminate crucial OS and hypervisor details, help you overcome migration challenges, and offer practical guidance for monitoring and operations Cisco and/or its affiliates. All rights reserved. Cisco Public 106

94 Suggested Sessions BRKRST-2042 Highly Available WAN Design BRKRST-2309 Introduction to WAN MACsec BRKRST-2514 Application Optimization and Provisioning the IWAN BRKRST-3045 LISP A Next Generation Networking Architecture BRKCRS-2000 Intelligent WAN 2.0 Update BRKSEC-4054 Advanced Concepts of DMVPN 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 107

95 Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot# 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

96 Please complete your Online Session Evaluations after each session Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Complete Your Online Session Evaluation Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online Cisco and/or its affiliates. All rights reserved. Cisco Public

97 Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Tech Circle Meet the Engineer 1:1 meetings Related sessions 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 111

98 Thank you

99