Maximizing Network Programmability and Automation with Open NX-OS

Transcription

1

2 BRKDCN-2025 Maximizing Network Programmability and Automation with Open NX-OS Nicolas Delecroix, Technical Marketing Engineer, INSBU

3 Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#brkdcn Cisco and/or its affiliates. All rights reserved. Cisco Public

4 Agenda Extending NX-OS Guest Shell NX-SDK Automating NX-OS YANG and OpenConfig Ansible Monitoring NX-OS Streaming Telemetry

5 Extending NX-OS

6 Guest Shell

7 Securely Run Custom On-Box Linux Apps NX-OS CLI Guest Shell: Secure Linux Container bootflash: Open Source Apps Your Custom Apps (C, Python, Go ) CentOS 7.0 rootfs BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 7

8 What Have Customers Been Doing with the Guest Shell? Monitoring agents distributed on every switch Existing collectors: OpenTSDB, Ganglia, Nagios, etc. For both standard Linux components (CPU, memory, interface counters), as well as NX-OS (routes, buffers,...) Custom collectors: ECMP load balancing, PTP accuracy Chef / Puppet agents Intrusion Detection DNSFlow agent to detect phishing activity Custom IDS agents Custom control plane: Quagga BGP Automatic configuration backup to a private Git repository Tcpdump Use the Nexus 9K as a PXE server! BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 8

9 Guest Shell Import Export: Deployment at Scale New! Shipping from 7.0(3)I7(1) September 2017 Define your custom environment in the Guest Shell, install custom apps, etc Then export it and import it on any other Nexus 9K BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 9

10 Guest Shell Export on the Development Switch EX-1# guestshell ~]$ ls -l total 6 -rwxrwxr-x 1 guestshell users 303 Apr buffer.py -rwxrwxr-x 1 guestshell users 946 Apr ptp_monitoring.py -rw-rw-r-- 1 guestshell users 15 Apr test.py [guestshell@guestshell ~]$ exit logout EX-1# BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 10

11 Guest Shell Export on the Dev Switch EX-1# guestshell ~]$ ls -l total 6 -rwxrwxr-x 1 guestshell users 303 Apr buffer.py -rwxrwxr-x 1 guestshell users 946 Apr ptp_monitoring.py -rw-rw-r-- 1 guestshell users 15 Apr test.py [guestshell@guestshell ~]$ exit logout EX-1# guestshell export rootfs package bootflash:my_guest_shell Access to the guest shell will be temporarily disabled while exporting it to target destination. Are you sure you want to continue? (y/n) [n] y 2017 Nov 30 15:38: EX-1 %VMAN-2-MOVE_STATE: Successfully exported virtual service 'guestshell+' ; Destination set to bootflash:/my_guest_shell EX-1# dir bootflash:my_guest_shell Nov 30 15:38: my_guest_shell BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 11

12 Guest Shell Import on the Production Switches EX-2# copy scp:// /home/img/my_guest_shell bootflash: vrf management my_guest_shell 100% 250MB 12.5MB/s 00:20 Copy complete EX-2# guestshell upgrade package bootflash:my_guest_shell Your root filesystem will be deleted and replaced with an upgraded root filesystem. Be sure to save your work. Are you sure you want to upgrade the guest shell? (y/n) [n] y Validating the provided rootfs EX-2# guestshell [guestshell@guestshell ~]$ ls -l total 6 -rwxrwxr-x 1 guestshell users 303 Apr buffer.py -rwxrwxr-x 1 guestshell users 946 Apr ptp_monitoring.py -rw-rw-r-- 1 guestshell users 15 Apr test.py [guestshell@guestshell ~]$ BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 12

13 NX-SDK

14 Tighter Integration of Custom Linux Apps with NX-OS New! Shipping from 7.0(3)I6(1) May NX-SDK: a simple, flexible and powerful tool for custom on-box applications to gain access to NX-OS infra functionalities. Custom on-box Linux applications can: Be written in Python or C++. Run natively just like any other native NX-OS feature. Startup and management is handled by NX-OS, similar to the native NX-OS features. Define their own custom CLIs (config and show commands), with callback handlers when their CLI gets executed. Generate custom syslogs, events & error history messages. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 14

15 NX-SDK Components On the switch: NX-OS Infra SDK API Library, for on-box apps to interact with NX-OS. Off the switch: off-box toolkit for app development. Python/C++ NX-SDK header and stub files for apps to interact with NX-OS. Framework to compile C++ applications. Not needed for Python. Framework to package those apps as RPMs. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 15

16 Nexus 9K Custom Applications (Python, C++, Go, Ruby) Native NX-OS Applications (L2, L3, Platform, etc.) Linux NX-OS NX-SDK API Library CLI Syslog, Event & Error history HA Event Manager Much More DME, RIB, timers, etc. 16

17 Some APIs Exported by NX-SDK For You to Use Gain access to NX-OS Infra functionalities static NxSdk *getsdkinst(int argc, char **argv); Gain access to NX-OS CLI virtual NxCliParser *getcliparser(); Create a new custom CLI config command virtual NxCliCmd *newconfigcmd(const char *cmd_name, const char *syntax); Create a new custom CLI show command virtual NxCliCmd *newshowcmd(const char *cmd_name, const char *syntax); Add app command handler callback for custom CLI: virtual void setcmdhandler(nxcmdhandler *handler);

18 NX-SDK Toolkit Custom Application Code (Python, C++, Go, Ruby) Use the toolkit to write the app and package it offline Linux Server Copy the app to the switch, and start it like a regular NX-OS feature Nexus 9K Custom Applications (Python, C++, Go, Ruby) Native NX-OS Applications (L2, L3, Platform, etc.) Linux NX-OS NX-SDK API Library CLI Syslog, Event & Error history HA Event Manager Much More DME, RIB, timers, etc.

19 Additional NX-SDK Benefits For custom or existing on-box monitoring agents: Automated startup and management of the collector by NX-OS. Cleanly and more efficiently extend the collector for NX-OS monitoring by using APIs, instead of CLI calls from the code. Improved performance via event notifications when accessing NX-OS data based on triggers. Network-operators role: Leverage NX-OS RBAC. Familiar NX-OS environment. Custom syslogs natively integrate with your existing syslog monitoring systems. Custom apps have their configuration in the NX-OS runningconfig/startup-config: one central repository. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 19

20 Code: Cisco and/or its affiliates. All rights reserved. Cisco Public

21 NX-SDK Enhancements in 7.0(3)I7(3) NX-OS Release Committed For Q1CY18 RIB integration Route lookup Route events. Get notified upon updates: route add, route remove, next-hop change Criteria for lookup and events: prefix, protocol, address family, VRF One surprise to be covered slightly later BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 21

22 NX-SDK Roadmap In Planning Go and Ruby support ARP / ND APIs MAC APIs Interface APIs Direct access to the NX-OS object database (DME). Event subscription for DME objects. Stateful apps: store state in the NX-OS object database (DME). Retrieve it after app restart or switch reload. NX-API support for custom show commands. App performance metrics. Program and process Control and Data Plane Policies. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 22

23 Automating NX-OS

24 Off-Box Automation with YANG and OpenConfig

25 Before We Get To YANG See Something Cool Here? New Nexus 9K NETCONF Capabilities BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 25

26 New NETCONF Capabilities for Greater Stability and Consistency Before 7.0(3)I6(1): only base capability was supported. Any changes get directly applied to running config. The candidate config temporarily holds any changes you make, without changing the running configuration. Validate the candidate config: the switch verifies the consistency of this candidate config, SW and HW resources availability, etc. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 26

27 New NETCONF Capabilities for Greater Stability and Consistency Commit: the candidate config becomes the actual running config. Confirmed-commit: the commit is final only when a confirm operation is performed, before the confirm timeout expires. If not confirmed, it rolls back to the previous config state. Useful to make sure you don t get locked out of the switch because of a management IP or ACL change. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 27

28 New NETCONF Capabilities for Greater Stability and Consistency If an error occurs during the config processing, it will roll back to the original state. All-or-nothing edit mode. Use case example: large ACL that could exceed the TCAM. If the TCAM gets exhausted in the middle of the ACL programming, we don t want to end up with half of the ACL active. We would rather roll back to a clean state. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 28

29 Model-Driven Programmability Stack Model-Driven Configuration BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 29

30 YANG Is a Language Used to Model Configuration BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 30

31 A Simple YANG Model YANG defines the basic blocks of syntax to define data in a tree hierarchy. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 31

32 A Simple YANG Model YANG defines the basic blocks of syntax to define data in a tree hierarchy. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 32

33 A Simple YANG Model YANG defines the basic blocks of syntax to define data in a tree hierarchy. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 33

34 Who Defines YANG Models? Model IEEE IETF OpenConfig Cisco Juniper URL More BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 34

35 Source: Cisco and/or its affiliates. All rights reserved. Cisco Public

36 OpenConfig Support for Nexus 3K and 9K Core Feature Set - Multi-Vendor Support Shipping Future Release In Planning Interfaces (L3) IPv4 and IPv6 for interfaces Static Route Route Policy VLAN VRRPv3 BGP Interfaces (L2) Inventory (memory usage, LC details, CPU details, modules) OSPF ACL AAA STP Models definitions on OpenConfig is also supported on IOS-XR and IOS-XE. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 36

37 Native NX-OS YANG Models For the Complete Feature Set ACL QoS Segment Routing VXLAN F&L and BGP EVPN ARP Interfaces VLAN IPv4 and IPv6 for interfaces RBAC Tunnels VRRPv2 and v3 OSPFv2 Shipping HSPR Syslog NTP Route Policy HSRP URIB SNMP L2RIB MPLS Static Route Streaming Telemetry Configuration Future Release In planning Incremental Models for: ACL QoS BGP Route Policy SNMP PIM Supported protocols for OpenConfig and Native models are NETCONF, RESTCONF, and grpc. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 37

38 Let s take a step back so we went from this: vlan 100 vn-segment To this: <System> <bd-items> <bd-items> <BD-list> <fabencap>vlan-100</fabencap> <accencap>vxlan-20100</accencap> <pctag>1</pctag> </BD-list> </bd-items> </bd-items> </System> Can we do better? 2 lines of CLI that we re familiar with 5 x more lines of XML that are fairly complex BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 38

39 Using APIs to Operationalize YANG

40 Beyond Payloads: Code APIs It s complicated and time-consuming to create YANG XML payloads manually. It also doesn t fit very well with automation via scripting. We would have to do text manipulation to insert the dynamic data into the payload. We want to generate a library of classes (code API) that follows exactly the YANG model definition. And also abstract the communication to the device (i.e., NETCONF, RESTCONF, etc) Cisco and/or its affiliates. All rights reserved. Cisco Public

41 YANG Development Kit (YDK) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 41

42 A YDK-Py Hello World Using OpenConfig BGP BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 42

43 Six Reasons Why YDK is Great We build our scripts by calling the APIs and this is independent of the encoding and of the transport. There is no manual NETCONF transport manipulation. Edit-config, etc. are all auto-generated. There is no manual YANG XML payload manipulation. We can focus on the YDK object model and the application logic. YDK APIs have been pre-generated for OpenConfig, IOS-XR, IOS-XE native models. NX-OS is in progress. Leveraging OpenConfig models enables network-as-code for multivendor environments. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 43

44 Ansible

45 Configuring a Device with Ansible Ansible Engine Modules Playbooks APIs Libraries Configure Goal: automate most DC assets with a minimal learning curve. Agent-less + push model easy to adopt. Nexus 9K: Ansible modules abstract the CLI. Switch or Server BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 45

46 Ansible Playbook: Sequence of Tasks (Plays) Example: Deploy NTP On All Our Servers Set of target devices (defined in a separate file) --- sudo - hosts: all become: yes tasks: below - name: Ensure NTP is installed yum: name=ntp state=present - name: Ensure NTP is running Arguments to the module service: name=ntpd state=started enabled=yes Ansible module to invoke: it will do the actual work For all those target devices, execute the tasks BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 46

47 Provision a VXLAN EVPN Segment with Ansible VLAN 200 L2VNI VLAN 200 e1/41 e1/41 N N BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 47

48 Create the VLAN and Map it to the VNI BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 48

49 Add the L2VNI to the EVPN Overlay BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 49

50 BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 50

51 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

52 BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 52

53 NX-OS Features Supported as Native Ansible Modules Ansible 2.4 / NX-OS 7.0(3)I7(3) AAA ACL BGP Checkpoint / Rollback Graceful Insertion and Removal (GIR) HSRP IGMP IGMP Snooping Interface Settings NTP NX-API NX-OS Upgrade OSPF PIM Port-Channel Patching / SMU SNMP Static Routes UDLD VLAN vpc VRF VRRP VTP VXLAN Flood & Learn VXLAN BGP EVPN BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 53

54 Monitoring NX-OS

55 Streaming Telemetry

56 Efficient Access to SW and HW State Monitor SW State (periodic, on change) BGP Ports VXLAN Transmit required information Monitor HW State (periodic, on change) BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 56

57 Three Principles of Nexus 9K Streaming Telemetry Push, not pull Performance Analytics-Ready Consumption Data-Model Driven Granularity BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 57

58 HTTP Telemetry Collector grpc Nexus 9K JSON Encoder GBP Encoder Telemetry Engine Policy Config CLI Output Object Store BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 58

59 Source: CLI JSON Output New! Shipping from 7.0(3)I6(1) May 2017 BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 59

60 Telemetry Source: Objects For Greater Granularity System Physical BGP ARP VRF eth1/1, eth1/2 BGP router ID, peers ARP entries VRF instances BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 60

61 Telemetry Source: Objects For Greater Granularity System Physical BGP ARP VRF eth1/1, eth1/2 BGP router ID, peers ARP entries VRF instances BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 61

62 HTTP Transport With JSON Encoding New! Shipping from 7.0(3)I6(1) May 2017 telemetry destination-group 1 ip address port 5000 protocol HTTP encoding JSON Get the sample collector: Programmability/blob/master/Streaming_Telemetry/HTTP_ Transport/http_receiver.py BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 62

63 NX-SDK Integration with Streaming Telemetry Committed for 7.0(3)I7(3) NX-OS Release (Q1CY18) 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

64 Demo: ECMP Load Monitoring with NX-SDK and Streaming Telemetry

65 10G link 4G flow 1. Detect ECMP bundles 2. If the bandwidth usage of a ECMP link member exceeds the threshold: Display a NX-OS syslog Create a streaming telemetry event EX EX-2 ECMP Bundle 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

66 10G link 4G flow 1. Detect ECMP bundles 2. If the bandwidth usage of a ECMP link member exceeds the threshold: Display a NX-OS syslog Create a streaming telemetry event EX EX-2 Code: Programmability/tree/master/NX-SDK ECMP Bundle 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

67 Use Case: VXLAN BGP EVPN Pre-built ELK collectors for VXLAN EVPN: and BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 67

68 ELK Collector VXLAN Programmable Fabric Dashboard PEER/ # EVPN ROUTES BGP PEERS AND STATE VNI TYPE AND STATE NVE PEERS AND # VNIs DEVICE AND VNI STATE 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

69 Streaming Telemetry Large Scale Validation Let s Load It Up! Spine Spine Spine Spine Leaf Leaf Leaf Leaf Leaf Leaf Leaf VXLAN EVPN leaf/spine topology L2VNIs, 900 L3VNIs, 900 VRFs, 2900 VLANs, 250 NVE Peers, 128 Multicast groups, 100K BGP EVPN routes. GBP/gRPC with receiver on bare-metal server. Config equivalent to 22K specific object paths, total data 14MB. Collection time + streaming time = 75 seconds. BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 69

70 DCNM Turnkey Telemetry Receiver Preview in DCNM , Official in Q2CY18 Select Metric Per Process details Real-time View BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 70

71 Streaming Telemetry Roadmap for Nexus 9K Shipping Future Release In Planning SW Streaming Telemetry CLI source DME models source JSON encoder GBP encoder grpc transport HTTP transport Custom receiver or opensource receiver Stream YANG models SW Streaming Telemetry integration with DCNM HW Telemetry NX-OS support Flow events ASIC-level statistics BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 71

72 Resources N9K NX-OSv: (3)I7(2) N9K NX-OSv in VIRL: DevNet sandbox: OpenConfig payloads and scripts: Programmability/tree/master/OpenConfig NX-YANG payloads and scripts: Programmability/tree/master/YANG/RESTCONF Ansible VXLAN BGP EVPN Playbooks: Demo: BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 72

73 Resources (continued) YDK: Streaming Telemetry Sample HTTP receiver: Programmability/blob/master/Streaming_Telemetry/HTTP_Transport/http_receiver.py Sample GPB receiver: Demo of manual receiver: Demo of Kibana receiver: Deployment guide for VXLAN EVPN: BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 73

74 Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#brkdcn Cisco and/or its affiliates. All rights reserved. Cisco Public

75 Please complete your Online Session Evaluations after each session Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Complete Your Online Session Evaluation Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at Cisco and/or its affiliates. All rights reserved. Cisco Public

76 Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Tech Circle Meet the Engineer 1:1 meetings Related sessions BRKDCN Cisco and/or its affiliates. All rights reserved. Cisco Public 76

77 Thank you

78