APIs Assist Troubleshooting in Manufacturing

APIs Assist Troubleshooting in Manufacturing Gabriel Zapodeanu Technology Solutions Architect, Cisco Systems gzapodea@cisco.com, @zapodeanu, github.com/gzapodea

Agenda Wireless Networks in Manufacturing Troubleshooting Wireless Connectivity REST APIs Detect, Diagnose and Alert (DDA) Lessons Learned Summary and Demo

Wireless Networks in Manufacturing Plant floor pervasive wireless coverage Infrastructure usually supports: Automated Guided Vehicles (AGV s) and AGC s Wireless enabled tools and forklifts, RFID tags Diagnostic and service tools Handheld scanners, tablets and wireless IP phones Remote expert capabilities via real time video collaboration Visibility and security of the wireless environment Wireless connectivity is critical for the manufacturing process Lack of the connectivity is very costly, may stop the manufacturing line 5

Mobility Infrastructure Components Typical infrastructure supporting a manufacturing plant floor: Redundant controllers in an HA configuration Access Points Catalyst and Industrial Ethernet switching Wi-Fi enabled clients and RFID tags Network management: DNA Center, APIC-EM Cisco Prime Infrastructure CMX Device APIs NETCONF/RESTCONF 6

Mobility Clients - AGVs and AGCs Credit: Harley Davidson Inc Automated Guided Vehicles (AGVs) and Automated Guided Carts (AGCs) are increasing popular on the plant floor Each AGV hosts: Single or multiple Programmable Logic Controllers (PLCs) and a layer 2 switch A wireless client to connect to the infrastructure Critical communication over wireless between PLC s and Human Machine Interfaces (HMIs) 7

Agenda Wireless Networks in Manufacturing Troubleshooting Wireless Connectivity REST APIs Detect, Diagnose and Alert (DDA) Lessons Learned Summary and Demo

Wireless Connectivity Troubleshooting Communication is lost between a PLC and HMI Plant Management alerts IT and Automation Engineering Typical troubleshooting process: Identify disconnected clients and record MAC/IP addresses Find out the disconnected AGV number Is it a HMI problem, or a PLC problem? Review PI alarms, Syslog, CMX notifications Get last known AGV location, if available Look for AP outages or coverage hole issues Validate access layer switch ports and uplinks status Verify Layer 2 and Layer 3 protocols Dispatch IT or Automation engineer for physical troubleshooting 9

Troubleshooting Challenges A slow process with many manual steps Require corelation of logs from multiple management systems IT and Automation engineers must be proficient with a multitude of technologies IT resources availability Time to respond to incident Collaboration tools not integrated 10

Agenda Wireless Networks in Manufacturing Troubleshooting Wireless Connectivity REST APIs Detect, Diagnose and Alert (DDA) Lessons Learned Summary and Demo

What is REST API? REST Representational State Transfer Stateless, client-server communications, HTTP protocol An architecture style for designing networked applications It is not a standard API Application Programming Interface Set of subroutine definitions, protocols, and tools for building application software Specifies how software components should interact with each other RESTful API s Use HTTP requests to Create/Read/Update/Delete (CRUD) operations 12

How to Make a REST API call REST is centered around the HTTP request and response model. Consuming an API is just as simple as making an HTTP request. Request Response Your Application Application Server 13

REST API Request URL: Application Server and the API resource Authentication: HTTP Basic, Custom, OAuth, none Custom Headers: HTTP Headers, example: Content-Type: application/json Request Body: JSON or XML containing data needed to complete request Method (CRUD) : POST - Creates a new resource GET - Retrieves/Read a resource PUT - Updates an existing resource DELETE - Deletes a resource. 14

REST API Response HTTP Status Codes 2xx Success - 200 OK, 201 Created 4xx Client Error - 400 Bad Request, 401 Unauthorized, 404 Not Found 5xx Server Error - 500 Internal Server Error Headers Content type JSON or XML, cache control, date, encoding Body Payload with requested data formatted in JSON or XML 15

REST API Request - Response url = http://api.ciscospark.com/v1/rooms header = {'content-type': 'application/json', 'authorization': SPARK_AUTH} payload = {'title': 'Cisco Live Room'} room_response = requests.post(url, data=json.dumps(payload), headers=header) Client Your Application Request Response Cisco Spark Response 200 / success { "id": "Y2lzY29zcGFyazovL3VzL1JPT ", "title": Cisco Live Room", "type": "group", "islocked": false, "lastactivity": "2017-01-09T01:03:28.522Z", "creatorid": "Y2lzY29zcGFyazovL3VzL1BFT1B ", "created": "2017-06-09T01:03:28.483Z" } The response value will be assigned to variable room_response 16

Agenda Wireless Networks in Manufacturing Troubleshooting Wireless Connectivity REST APIs Detect, Diagnose and Alert (DDA) APIs Assist Troubleshooting Detect Diagnose Alert Lessons Learned Summary and Demo

APIs Assist Troubleshooting Detect, Diagnose and Alert (DDA), an API based troubleshooting assist tool Troubleshooting could be automated by using API s to: Process notifications of disconnected clients Validate reachability of Access Points and Switches Verify switch ports and uplinks status Show on floor plan the last known location of the AGV Radio information for the AGV Alert IT and Automation engineers Collect relevant information Present all data gathered in a user friendly format Self healing steps could be included 18

High Level Architecture Functional Requirements End User Interaction with the application Input requests Receive notifications DDA Detect Diagnose Alert User Interface Location and Alerting Network Controller Spark/Tropo CMX APIC-EM / DNA C Update status Mobility services Floor maps and tracking Notifications Analytics Information about: Network Device Management PI, RESTCONF, NETCONF, DNA C Clients Network devices Topology physical and logical Path trace Alarms and logs Configuration management CLI templates deployment Job status report 19

Detect CMX will learn about disconnected clients from WLC CMX notifications - alerts for critical events Disconnected client MAC address is identified Database lookup will provide the IP address and AGV number 21

CMX API Request Find the number of active clients User defined function def all_client_number(): """ Build Code Documentation using PyDoc This function will find out how many wireless clients are visible in the environment REST API call to CMX - /api/location/v2/clients/count :param :return: The total number of clients, associated and not associated with the MDE SSID """ URL url = CMX_URL + 'api/location/v2/clients/count' Headers header = {'content-type': 'application/json', 'accept': 'application/json'} response = requests.get(url, headers=header, auth=cmx_auth, verify=false) Method response_json = response.json() clients_number = response_json['count'] return clients_number Parsing JSON Authentication 22

CMX API Response Find the number of active clients Response 200 / success Request Status Code { } "devicetype" : "Wireless_Client", "count" : 36, "devicequerystring" : null JSON Body Number of active clients: 36 23

Diagnose - CMX CMX will provide the last known location for the disconnected AGV X and Y coordinates on a map Access Point information for disconnected AGV provided by CMX X and Y coordinates on a map Radio connected to AGV Plant Floor map image file Wireless LAN Controller IP address 25

Diagnose - APIC-EM APIC-EM will supply: Access Point status and reachability Access switch status and reachability Switchport information and status Path trace between HMI and PLC 26

APIC-EM API Request Find out the APIC-EM device id for the network device with the hostname {device_name} def get_device_id(device_name, ticket): User defined function Headers url = EM_URL + '/network-device/ URL header = {'accept': 'application/json', 'X-Auth-Token': ticket} device_response = requests.get(url, headers=header, verify=false) Authentication Method device_json = device_response.json() device_list = device_json['response'] Parsing JSON for device in device_list: if device['hostname'] == device_name: device_id = device['id'] return device_id 27

APIC-EM API Response Find out the hostname of a network device mapped to the APIC-EM {device Id} { } "family" : "Switches and Hubs", "serialnumber" : "FCW2002D0HM", "type" : "Cisco Catalyst38xx stack-able ethernet switch", "role" : "ACCESS", "softwareversion" : "03.07.03E", "uptime" : "6 days, 18:58:12.60", "series" : "Cisco Catalyst 3850 Series Ethernet Stackable Switch", "hostname" : "NYC-SW", "collectionstatus" : "Managed", "interfacecount" : "37", "managementipaddress" : "172.16.33.2", JSON format "memorysize" : "536870912", "platformid" : "WS-C3850-24U-E", "reachabilitystatus" : "Reachable", "id" : "69c19860-e93e-41ca-b3a8-00fb1b5f0842 28

Diagnose - DNA Center DNA Center will supply: Access Point status and reachability Access switch status and reachability Switchport information and status Path trace between HMI and PLC Self Healing steps could be attempted Templates deployment 29

DNA Center API Request Retrieve all the information for the device with the DNA C {device_id} User defined function def get_device_info(device_id, dnac_jwt_token): URL Headers url = DNAC_URL + '/api/v1/network-device/?id=' + device_id header = {'accept': 'application/json', Cookie': dnac_jwt_token} device_response = requests.get(url, headers=header, verify=false) Method Authentication device_info = device_response.json() return device_info['response ] Parsing JSON 30

DNA Center API Response Retrieve all the information for the device with the DNA C {device_id} "response": [ { "location": null, "type": "Cisco Catalyst 9300 Switch", "role": "ACCESS", "lastupdated": "2018-01-30 05:37:24", "macaddress": "2c:ab:eb:37:c3:00", "serialnumber": "FCW2123L0N3", "softwareversion": "16.6.1", "hostname": "NYC-9300", "softwaretype": "IOS-XE", "uptime": "53 days, 10:15:04.14, "managementipaddress": "10.93.130.21, "reachabilitystatus": "Reachable, } ] Response Body 31

Diagnose PI, NETCONF, CMX DDA could collect: Logs from PI for disconnected client MAC address Number of other wireless clients connected to the same AP (CMX) Switch access port or uplinks interfaces operational data stats using NETCONF errors, interface status 32

Alert A new Spark room will be created Membership invitation to IT and Automation Engineers, NOC, Plant Management All data collected will be posted in Spark Notifications sent from Tropo to engineering: Voice calls 37

URL Spark API Request Create a room with the {room_name} def create_spark_room(room_name): User defined function """ This function will create a Spark room. API call to Spark - '/rooms' :param room_name: The new room title :return: Spark room id """ Request Body payload = {'title': room_name} url = SPARK_URL + '/rooms' Headers header = {'content-type': 'application/json', 'authorization': SPARK_AUTH} room_response = requests.post(url, data=json.dumps(payload), headers=header) Method Request Body room_json = room_response.json() room_number = room_json['id'] return room_number Parsing JSON 38

Spark API Response Create a room with the {room_name} Response 200 / success { "title" : "MDE Line Outage", "creatorid" : "Y2lzY29zcGFyazovL3VzL1BFT1BMRS8zZDJ ", "type" : "group", "created" : "2017-06-11T01:14:24.912Z", "lastactivity" : "2017-02-11T01:14:24.944Z", "id" : "Y2lzY29zcGFyazovL3VzL1JPT00vNmQyZWMxMDAtGB ", "islocked" : false } Created Room with the name: MDE Line Outage 39

DDA Roadmap Add common mitigation options, and status of other clients associated with the AP Get additional information about the access and uplink ports counters NETCONF/RESTCONF Use DNA Center Coverage hole detection Locate the skilled IT engineer in the proximity of the outage Machine learning for known, and unknown incidents 41

Agenda Wireless Networks in Manufacturing Troubleshooting Wireless Connectivity REST APIs Detect, Diagnose and Alert (DDA) Lessons Learned Summary and Demo

Lessons Learned Start with simple tasks to automate Think APIs first, CLI and GUI last Find something you can not do today, be creative by using API s Your application does not have to be perfect, we are not developers Expect to troubleshoot your code, ask for help, check communities Join programmability communities Remember to have fun! 43

Agenda Wireless Networks in Manufacturing Troubleshooting Wireless Connectivity REST APIs Detect, Diagnose and Alert (DDA) Lessons Learned Summary and Demo

Summary APIs assist troubleshooting will greatly improve time to repair Detect, Diagnose and Alert concepts could be used for: Costly outages Chronic problems Improve IT response Proactive using machine learning Integration possible with collaboration tools, remote expert and video, Manufacturing Execution Systems 45

DDA Python Code It is not intended for use in any production environment without significant testing, validation and re-write to meet your Enterprise Application Development Policies This code intent is for teaching purposes The use case code may be found here: https://github.com/gzapodea/devnet_2049 Included: DEVNET_2049_DDA_CL.py DEVNET_2049_DDA_CL.html - PyDoc generated documentation DEVNET_2049_DDA_CL.pdf - Application run output Manufacturing plant floor maps files from CMX, and annotated with the AP and AGV locations are included. 46

DDA Lab Demo pre-recorded

Cisco Spark How Questions? Use Cisco Spark to communicate with the speaker after the session 1. Find this session in the Cisco Live Mobile App 2. Click Join the Discussion 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#

Please complete your Online Session Evaluations after each session Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Complete Your Online Session Evaluation Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online.

Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Lunch & Learn Meet the Engineer 1:1 meetings Related sessions: BRKSDN-2935 - From Zero to Network Programmability in 90 minutes 51

Q & A

Thank you