BRKCRS-2112 Serviceability of SD-WAN Chandrabalaji Rajaram & Ali Shaikh
Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#brkcrs-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco SD-WAN learning Journey at Cisco Live Monday Tuesday Wednesday Thursday Friday BRKCRS-2110 Delivering Cisco Next Generation SD-WAN with Viptela BRKCRS-2111 Migration to Next-Gen SD-WAN Deep Dive Architecture and solution Migration and vqoe SP orchestration Serviceability TECCRS-20004 Cisco SD-WAN Technical Deep Dive BRKCRS-2113 Cloud-Ready WAN for IAAS and SAAS with Cisco Next-Gen SD- WAN BRKRST-2514 Next Gen SDWAN with application acceleration/optimization BRKRST-2557 SD-WAN and NFV Orchestration for Managed Service Providers BRKCRS-2112 Serviceability for Next Generation SD-WAN
Agenda SDWAN Components overiew Day 0 Deployment and troubleshooting Day N Deployment and troubleshooting System Maintenance Tech Support Demo
SDWAN Components Overview
SDWAN Components overview vmanage NMS vedge Cloud Router SDWAN Components vsmart Controller vedge Router vbond Orchestrator BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
SDWAN Components overview Orchestration Plane vmanage APIs Orchestration Plane Cisco vbond vanalytics 3 rd Party Automation Orchestrates Connectivity vbond First point of authentication vsmart Controllers (white-list model) MPLS 4G Facilitates NAT traversal INET vedge Routers Cloud Data Center Campus Branch SOHO BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
SDWAN Components overview Management Plane vmanage Management Plane Cisco vmanage APIs vbond vanalytics vsmart Controllers 3 rd Party Automation Single pane of glass Policies and Templates Troubleshooting and Monitoring MPLS 4G Programmatic interfaces INET vedge Routers Cloud Data Center Campus Branch SOHO BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
SDWAN Components overview Control Plane Control Plane vmanage APIs Cisco vsmart vbond vanalytics vsmart Controllers 3 rd Party Automation Handles all the Overlay-network routing Facilitates the DP encryption between vedges MPLS 4G Propagates the policies for INET vedge Routers handling DP traffic Cloud Data Center Campus Branch SOHO BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
SDWAN Components overview Data Plane Data Plane Physical/Virtual vbond vanalytics vmanage vsmart Controllers MPLS INET 4G APIs 3 rd Party Automation vedge Routers vedge vedge Cloud WAN edge router Provides secure data plane with remote vedge routers Implements data plane and application aware routing policies Cloud Data Center Campus Branch SOHO BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Analytics Cisco SD-WAN Cloud-Delivered Architecture Multitenant, Cloud-Operated and Cloud-Delivered REST API GUI vmanage vsmart Controllers Cloud Data Center Secure SD-WAN Fabric Private/Hosted/Managed Cloud MPLS 4G Data Center Secure Control Plane INET vedge Router Small Office Home Office Campus Branch BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Fabric Operation Walk-Through OMP DTLS/TLS Tunnel IPSec Tunnel BFD OMP Update OMP Update vsmart Policies OMP Update: Reachability IP Subnets, TLOCs Security Encryption Keys Policy Data/App-route Policies OMP Update OMP Update vedge Transport1 vedge TLOCs TLOCs BGP, OSPF, Connected, Static VPN1 A VPN2 B Transport2 VPN1 C VPN2 D BGP, OSPF, Connected, Static Subnets Subnets BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Secure Segmentation Interface VLAN Security Zoning Compliance Guest WiFi Multi-Tenancy Extranet Per-VPN Topology Full-Mesh Hub-and-Spoke Partial Mesh Point-to-Point BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Day 0 Deployment and troubleshooting
Zero Touch Provisioning vedge Appliance Zero Touch Provisioning Server Control and Policy Elements Assumption: DHCP on Transport Side (WAN) DNS to resolve ztp.viptela.com* 1 2 3 4 5 Full Registration and Configuration * Factory default config vedge Delivered as-a-service BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Zero Touch Provisioning vedge Cloud vmanage Control and Policy Elements 1 Cloud-Init VM Provisioning Tool 2 3 5 Full Registration and Configuration 4 Assumption: DHCP on Transport Side (WAN) * Factory default config vedge Cloud BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Building basic overlay network 1) Perform initial bring-up and do basic configuration. 2) Enable host or service-side interfaces and routing. 3) Enable overlay routing over OMP. 4) Check the automatic setup of the IPsec data plane. 5) Enforce policies. BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Checking Control connections Control Up: Total number of devices with the required number of operational control plane connections to a vsmart controller. Partial: Total number of devices with some, but not all, operational control plane connections to vsmart controllers. Control Down: Total number of devices with no control plane connection to a vsmart controller. BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Checking Control connections for a single device If the device has multiple interfaces, vmanage NMS displays a graphical topology of all control connections for each color. Click the arrow to the left to view the control connections for that TLOC color. Click the checkbox to the left to select and deselect control connections. BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Checking Data connections Down: Non-operational connections with other vedge routers in the network. Init: Connections that are reachable but not up yet. Up: Operational connections BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Checking OMP Summary OMP Summary of the vedge router BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Checking OMP Summary OMP Summary of the vedge router Field admin-state omp-uptime oper-state routes-installed routes-received tlocs-installed tlocs-received tlocs-sent Explanation Administrative state of the OMP session. It can be UP or DOWN. How long the OMP session has been up and operational. Operational status of the OMP session. It can be UP or DOWN. Number of routes installed over the OMP session. Number of routes received over the OMP session. Number of TLOCs installed that were learned over OMP sessions. Number of TLOCs received over OMP sessions. Number of TLOCs advertised over OMP sessions. BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Checking OMP Peers detail OMP Peers of the vedge router BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Checking OMP Peers detail OMP Peers of the vedge router Field Peer Type Explanation IP address of the connected Edge device. Type of SDWAN device State down The connection is not functioning. init The connection is initializing. up The connection is operating. Domain ID Identifier of the domain that the device is a member of. Site ID R/I/S routes-installed Identifier of the administrative site where the connect Edge device is located. Number of routes received, installed, and sent over the OMP session. Number of routes installed over the OMP session. BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Checking Device bring-up - Indicates control plane connections are successful - Indicates ZTP is disabled. Seen during SW upgrade only - Indicates control plane connection failure - Indicates that the reason for device bring-up failure is Unknown BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Troubleshooting Control connections
Possible causes for control connection failure Connectivity Issues Certificate Issues DTLS Connection Failure TLOC Disabled Transient Conditions Serial number(s) not present Certificate revoked/invalidated Certificate Verification Failed Org. Name Mismatch BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
DTLS connection failure Probable causes NH not reachable Def-GW not installed in RIB DTLS port not open in the Controllers Debugging steps: PING Def-GW Ping vbond if ICMP is allowed on the vbond Traceroute to vbond DNS Address BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
TLOC disabled Probable causes Clearing of Control Connections Changing the color on TLOC Change in System IP Change in any of the configs mentioned in the system block or in the tunnel properties BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Transient Conditions Following are some Transient conditions where the control connections flap. System-IP change on the vedge Tear-down msg. to vbond [control connection to vbond is transient] This can be verified using the show control connections output as shown below BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
Transient Conditions Following are some Transient conditions where the control connections flap. System-IP change on the vedge Tear-down msg. to vbond [control connection to vbond is transient] This can be verified using the show control connections output as shown below Disconnect vbond after register reply BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Transient Conditions Following are some Transient conditions where the control connections flap. System-IP change on the vedge Tear-down msg. to vbond [control connection to vbond is transient] This can be verified using the show control connections output as shown below BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Transient Conditions Following are some Transient conditions where the control connections flap. System-IP change on the vedge Tear-down msg. to vbond [control connection to vbond is transient] This can be verified using the show control connections output as shown below System-IP Changed BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Transient Conditions Following are some Transient conditions where the control connections flap. System-IP change on the vedge Tear-down msg. to vbond [control connection to vbond is transient] This can be verified using the show control connections output as shown below BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Serial Number(s) NOT present If the serial number is not present on the controllers for a given vedge, the control connections will fail Verify this by send to controllers option from vmanage and / or show controllers [ validvsmarts valid-vedges ]. Challenge response rejected by peer Peer Board ID Cert not verified BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Serial Number(s) NOT present.contd Serial Number is NOT present Challenge response rejected by peer BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
Certificate revoked/invalidated The certificate will be revoked in case of controllers or vedge serial number is invalidated vsmart Certificate revoked BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Certificate installation failed Certification verification failure is when certificate cannot be verified with the root cert installed. Fail to verify Peer Certificate BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Organization-name Mismatch For a given a overlay, the Org. Name has to match across all the controllers and vedges so that control connections can come up. If not, you will see Certificate Org. name mismatch as seen below in the show control connections output. Certificate Org name mismatch BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Day N Monitoring and troubleshooting
Health Status Check on vedge
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
Checking System Status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
Checking System status.contd BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Checking the Circuit-Utilization BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
Checking the Circuit-Utilization BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Checking Transport Quality WAN > TLOC status 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Checking Transport Quality WAN > TLOC status 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Checking Tunnel Quality WAN > Tunnel status BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Checking DPI stats BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Checking DPI stats BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
Checking DPI stats BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Checking DPI stats BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Checking DPI stats BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
Checking App flows BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
Checking Events BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
Service-side to Service-side Troubleshooting BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
Service-side to Service-side Troubleshooting BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
Service-side to Service-side Troubleshooting BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
Service-side to Service-side Troubleshooting BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
App route visualization BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
Simulate Flows BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Simulate Flows BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Debug Logs BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
System Maintenance
Configuration roll-back
Configuration roll-back using vmanage BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
Software Upgrade
Software Upgrade Upgrade software version of the vedge router NOTE: If the software upgrade is NOT successful and the device loses its connectivity after upgrade, it will automatically roll-back to the previous Software version BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Device Reboot
Device Reboot BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
Generic Alarms/Notifications
System Alarm - Types Major Alarm - RED One or more hardware components on the router has failed. One or more hardware components on the router has exceeded the temperature threshold. Minor Alarm - YELLOW Indicates a warning on the router that, if left unattended, might result in an interruption in router operation or degradation in router performance. BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
Checking Alarms BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
Tech-Support
Collecting Show Admin Tech Generate Show-admin Tech BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
Demo
Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#brkcrs-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Please complete your Online Session Evaluations after each session Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Complete Your Online Session Evaluation Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/. 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Tech Circle Meet the Engineer 1:1 meetings Related sessions BRKCRS-2112 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 94
Thank you