Industrial Automation in Manufacturing Environments - Architecture and Use Cases Connected Factory

Transcription

1

2 Industrial Automation in Manufacturing Environments - Architecture and Use Cases Connected Factory Arun Siddeswaran, Sr. Manager, IoT Solutions Frank Baro, Sr. Solution Architect, Customer Experience

3 Source: Cisco and/or its affiliates. All rights reserved. Cisco Public 3

4 Agenda Connected Factory Architecture Cisco Reference Architecture Factory Network Factory Wireless Factory Security IoT Data Management Connected Factory in Practice Achieving Business Outcomes Factory Security Enabling Analytics Factory Wireless AGV Roaming Conclusion Recommended Resources 4

5 Connected Factory Reference Architectures

6 Connected Factory Reference Architectures Converged Plantwide Ethernet (CPwE) Tested, validated and documented reference architectures Developed from use cases - customer and application Tested for performance, availability, repeatability, scalability and security Comprised of Cisco and Rockwell Automation Validated Designs Built on technology and industry standards Future-ready network design Content relevant to both OT and IT Engineers Deliverables Recommendations, best practices, design and implementation guidance, documented test results and configuration settings Simplified design, quicker deployment, reduced risk in deploying new technology 6

7 Built on Industry Standards Purdue/IE62443 Reference Model Level 5 Level 4 , Intranet, etc. Enterprise Network Site Business Planning and Logistics Network Enterprise Security Zone Remote Gateway Services Patch Management AV Server Application Mirror Web Services Operations Application Server Firewall Firewall Industrial DMZ Level 3 FactoryTalk Application Server FactoryTalk Directory Engineering Workstation Remote Access Server Site Operations and Control Industrial Zone Level 2 Level 1 FactoryTalk Client Batch Control Operator Interface FactoryTalk Client Engineering Workstation Discrete Control Drive Control Continuous Process Control Operator Interface Safety Control Area Supervisory Control Basic Control Cell/Area Zone Level 0 Sensors Drives Actuators Robots Process 7

8 Converged Plantwide Ethernet (CPwE) Reference Architecture Wide Area Network (WAN) Data Center - Virtualized Servers ERP - Business Systems , Web Services Security Services - Active Directory (AD), Identity Services (AAA) Network Services DNS, DHCP Call Manager Identity Services Enterprise External DMZ/ Firewall Internet Enterprise Zone Levels 4-5 Physical or Virtualized Servers Patch Management AV Server Application Mirror Remote Desktop Gateway Server ASA 5500 ASA 5500 Plant Firewalls Active/Standby Inter-zone traffic segmentation ACLs, IPS and IDS VPN Services Portal and Remote Desktop Services proxy Industrial Demilitarized Zone (IDMZ) Physical or Virtualized Servers FactoryTalk Application Servers and Services Platform Network & Security Services IND, DNS, AD, DHCP, Identity Services (AAA), MSE Storage Array Level 3 - Site Operations (Control Room) Cisco Kinetic (IoT Platform) Wireless LAN Controller (WLC) Active Standby Identity Services Industrial Network Director Stealthwatch Core IE 5K (Distribution Switch) Distribution Switch Stack Access Sw itches Cell/Area Zone Levels 0 2 Access Sw itches Cell/Area Zone Levels 0 2 Industrial Zone Levels 0 3 (Plant-wide Network) Camera IW3700 (WGB) Phone SSID 5 GHz IE2K / IE3X, IE4K LWAP 2.4 GHz IE-1K IW3700 (WGB) IFW LWAP IE2K / IE3K / IE4K LWAP SSID 2.4 GHz IFW Industrial Ethernet Switch (IE2K,IE3X / IE4K) HMI Soft Starter IFW IE2K / IE3X / IE4K Safety Controller IE2K / IE3X / IE4K Safety I/O AP SSID 5 GHz IW3700 (WGB) Controller Controller Cell/Area Zone - Levels 0 2 Redundant Star Topology - Flex Links Resiliency Unified Wireless LAN (Lines, Machines, Skids, Equipment) Controller Drive I/O IE-1K Cell/Area Zone - Levels 0 2 Ring Topology - Unified Wireless LAN (Lines, Machines, Skids, Equipment) Servo Drive HMI Robot Cell/Area Zone - Levels 0 2 Linear/Bus/Star Topology Autonomous Wireless LAN (Lines, Machines, Skids, Equipment) Safety I/O

9 Connected Factory - Designed for Digital Manufacturing 9

10 Connected Factory Reference Architecture 10

11 Factory Network

12 Cell/Area Zone Overview Layer 2 Access Switch Layer 3 Distribution Switch IE5K Media and Connectors Legend: Level 2 HMI HMI Controller IE2K / IE3X / IE4K Controller Level 1 Controller IE2K / IE3X / IE4K IE2K / IE3X / IE4K VFD Layer 2 Interswitch Uplink-VLAN Trunk, Layer 2 Resiliency Level 0 Device (Drive) IE2K / IE3X / IE4K HMI Layer 2 Access Link-Single VLAN Assigned to Port Cell/Area Zone Distributed IO Controller Controller Cell/Area Zone Cell/Area Zone - Functional Area of a Production Facility. Considerations Include: Environmental constraints Range of device intelligence Time-sensitive applications 12

13 Typical Cell/Area Zone Traffic Flows IDMZ Engineering Laptop Manufacturing Zone Network Management CIP Implicit - Producers & Consumer >80% local IE5K Cyclical I/O traffic, UDP unicast and multicast <500 Bytes, Frequent 0.5 to 10 s of ms, typically 20 ms HMI IE2K /IE3X/ IE4K IE2K /IE3X/ IE4K HMI Controller IE2K /IE3X/ IE4K IE2K /IE3X/ IE4K Drive Cell/Area Zone Cell/Area Zone CIP Explicit - Informational control and administration Intra- and inter-cell/area zone traffic flow Non-critical administrative or data traffic using TCP ~1500 Bytes, infrequent Above 500 ms 13

14 Benefits of Managed Infrastructure Managed Switches Benefits Loop prevention and resiliency Security services Management services (Multicast and DHCP per port) Diagnostic information Segmentation services (VLANs) Prioritization services (QoS) Considerations More expensive Requires some level of support and configuration to start up Unmanaged Switches Inexpensive Simple to set up No loop prevention or resiliency No security services No diagnostic information No segmentation or prioritization services Difficult to troubleshoot, no management services 14

15 Industrial Network Topologies Cell/Area Zone Topology Options Star/Bus Linear Cell/Area Zone IE5K (Distribution Switch) Ring Resilient Ethernet Protocol (REP) Cell/Area Zone IE5K (Distribution Switch) Redundant Star Flex Links EtherChannel Cell/Area Zone IE5K (Distribution Switch) Cisco Catalyst 2955 HMI Controllers Access IE2K / IE3X / IE4K HMI Controllers Access IE2K / IE3X / IE4K Controller HMI HMI Access IE2K / IE3X / IE4K Controllers, Drives, and Distributed I/O Controllers, Drives, and Distributed I/O Controllers, Drives, Cell/Area and Distributed ZoneI/O Linear Ring Redundant Star Cabling Requirements Ease of Configuration Implementation Costs Bandwidth Redundancy and Convergence Disruption During Network Upgrade Readiness for Network Convergence Overall in Network TCO and Performance Worst OK Best 15

16 Performance Requirements Industrial Automation & Control System Applications Process Automation Discrete Automation Loss Critical Function Information Integration, Slower Process Automation Time-critical Factory Automation Multi-axis Motion Control Comm. Technology.Net, DCOM, TCP/IP Industrial Protocols, CIP, Profinet Hardware and Software solutions, e.g. CIP Motion, PTP Period 1 second or longer 1 ms to 100 ms 100 µs to 10 ms Industries Oil & Gas, chemicals, energy, water Auto, food and bev, electrical assembly, semiconductor, metals, pharmaceutical Utilities Subset of Discrete automation Applications Pumps, compressors, mixers; monitoring of temperature, pressure, flow Material handling, filling, labeling, palletizing, packaging; welding, stamping, cutting, metal forming, soldering, sorting Life/equipment safety, Synchronization of multiple axes: printing presses, wire drawing, web making, picking and placing Source: ARC Advisory Group 16

17 Network Resiliency Protocols Selection is Application Driven Resiliency Protocol Mixed Vendor Ring Redundant Star Net Conv >250 ms Net Conv ms Net Conv < 0~10 ms Layer 3 Layer 2 STP (802.1D) RSTP (802.1w) Process and Information MSTP (802.1s) PVST+ REP EtherChannel (LACP 802.3ad) Time Critical MRP (IEC )* Flex Links PRP/HSR (IEC 62439)* DLR (IEC & ODVA) Loss Critical StackWise HSRP VRRP (IETF RFC 3768) * Not part of CPwE 17

18 Industrial IoT Networking Portfolio Industrial Switching IoT Gateways Industrial Routing Cisco Resilient Mesh Low Power Wide Area Wireless IE 1K,2K,3K,4K,5K, CGS, 3x MNA, IR807, IR809, IR829, IR1101 ASR 902U/903U/920U, CGR 1000, CGR 2000 IR500, DevNet LoRaWAN IXM Gateway Industrial Wireless Industrial Security Embedded IoT Edge Computing Management & Automation AP1552, IW3702 ISA 3000 ESS, ESR IOx IC 3000 Field Network Director Industrial Network Director 18

19 Industrial Ethernet Switch Characteristics Features Cisco Industrial Ethernet (IE) Typical Non-Industrial Ethernet Switch Form Factor / Mounting Options Din Rail, Panel and Rack Mount Rack Mount Interface Options Port density 6-28 High port density PoE Density / Max Power Port density 6-28 High port density Power Supply Options AC and DC AC and DC DC input voltage range = 10 to 300 DC input voltage range = 36 to 72 Converged Access Yes, No (Wired plus Wireless) Mobility agent and Mobility controller Environmental Design Fanless (no moving parts) vs Fans Operating Temperature Range Ingress Protection (IP) Rating Fanless -40c to +60c IP30 (models up to IP67) Hardened for vibration, shock, Fans -5c to +45c IP XX (Not Specified, IP20 or less) Enterprise class certifications Industry Certifications surge, and noise immunity Swap Drive Removable Flash Yes No Dying Gasp - Upon loss of input power Yes No Alarm Ports Yes No Deterministic Ethernet Yes Supported by IE 4000 and IEEE TSN 5000 No

20 Industrial Ethernet Switch Characteristics Cont. Features Cisco Industrial Ethernet (IE) Typical Non-Industrial Ethernet Switch Industrial Protocols - Management EtherNet/IP CIP, Profinet, Modbus TCP Not available Industrial Protocols High Availability REP, MRP, Flexlink, PRP, HSR REP (slower convergence time), Flexlink IE Smart-port macros (Qty 32): QoS policies, IED, PTP, CIP, HMI No IE Smart-port macros Smart-port Macros etc Enterprise (qty 6) : global, desktop, Enterprise (qty 6): global, desktop, phone, switch, router, wireless phone, switch, router, wireless Device Manager Ease of use on device web server On device web server for device for device management management Network Management Industrial Network Director (IND) Prime Infrastructure / Cisco DNA-C Prime Infrastructure/Cisco DNA-C Typical Boot Time 30sec 2 min,20 sec 5 mins (single switch) L2 and L3 Images Yes, same hardware Yes, same hardware Precise Timing IEEE 1588 PTP IEEE C (Power Profile) Yes IEEE 1588, inc. Power Profile level of accuracy (50ns per hop) Option for GPS and IRIG-B on IE 5000, including Grand Master with Stratum 3E on board oscillator No

21 Factory Network Layer 2 NAT

22 Challenge - Ethernet Growing Pains Ethernet networks continue to grow: Each machine adds another 5-50 EtherNet/IP enabled devices Every line adds another 250-1,000 EtherNet/IP enabled devices How do I connect all these machines into a plant network to gain the advantages? 22

23 Solution- Layer 2 Network Address Translation (NAT) Outside Subnet (ex x) One to One (1:1) NAT Many Outside IP addresses (One per device wishing to be accessible from the Outside Subnet NAT Enabled Device Many Inside IP addresses (One per connected device) Inside Subnet (ex x) 23

24 Trunk Layer 2 NAT Design Scenarios Single-Cell, Single VLAN per Switch Line Controller OUTSIDE VLAN10 IE 5K (Distribution Switch) Inside to Outside NAT Table Outside to inside NAT Table Inside Outside Outside Inside INSIDE VLAN10 Inside Address IE2K / IE4K Machine 26

25 Layer 2 NAT Design Scenarios Cont. Multi-Cell, Single Switch, Multi-VLAN OUTSIDE INSIDE IP Address: X IP Address: X IE2K / IE4K Line Controller VLAN10 VLAN40 IE 5K (Distribution Switch) IE2K / IE4K INSIDE IP Address: X VLAN20 VLAN40 NAT IE2K / IE4K Work Station Multiple Instance of NAT per VLAN VLAN30 INSIDE IP Address: X IE2K / IE4K Machine 1 NAT Table Inside Outside Machine 2 NAT Table.3 Inside Outside Machine 3 NAT Table.3 Inside Outside

26 Factory Network Network Management for OT

27 Cisco Industrial Network Director Network Management, Simplified & Automated Native industrial protocol support Dashboard for monitoring system health, metrics, and traffic statistics Plug-and-Play Day-0 configuration Alarm management with real-time alerts of network events Plug-and-Play for Zero-Touch Switch Commissioning Improved Industrial Asset Visibility Network Troubleshooting with Automation Context APIs for Integration with Automation Systems 31

28 Cisco Plug and Play Zero-Touch Commissioning and Replacement Pre-provision configuration and software for automated network commissioning Help ensure consistent network design and security policy Swap hardware when switch fails and recover with automated configuration and software image replacement Open protocol based on XMPP and HTTP with publically available schema Cisco Industrial Network Director PnP-Server XML PnP Protocol Cisco Industrial Ethernet Switch Switch Configuration Software Image PnP-Agent 32

29 Factory Wireless

30 Wireless Overview Benefits of industrial wireless network Connection to hard-to-reach and restricted areas Integration of machines / skids Remote diagnostics Intelligent assets Lower installation and operational costs Cabling reduction, elimination of cable failures Equipment mobility New and more efficient machine designs 36

31 Wireless Overview Benefits of industrial wireless network Workforce mobility improves effectiveness Operators can trend/write back from a mobile device when they step away from machine Engineering and Maintenance can see and react to system alarming and production data from anywhere, anytime Industrial IT provide secure infrastructure and multi-platform support Equipment wireless IEEE Wireless connectivity for critical Industrial Automation and Control System (IACS) applications Asset Tracking Track assets to optimize cost and for safety 37

32 Wireless Overview Challenges of wireless communication IW3702 Half-duplex shared medium: Only one radio can transmit on a particular wireless channel A radio cannot transmit and receive at the same time on the same channel Higher latency, jitter and packet loss compared to wired Ethernet Media contention, collisions and interference Can be minimized but not eliminated AP/WGB - IW

33 Wireless Overview Challenges of wireless communication Wireless coverage area cannot be precisely defined Site survey is required Spectrum sharing and security concerns Signal quality may change over time Interference sources and obstructions Unauthorized transmissions Wireless advantages > challenges when WLAN is designed and maintained properly Used for appropriate applications 39

34 Factory Wireless Equipment to Equipment

35 Equipment to Equipment Use Cases Wireless Mobility Types Static equipment Permanent location Wire replacement for hard-to-reach places Examples: process control, condition monitoring, standalone OEM machines AP IW3702 Access IE2K / IE3X / IE4K WGB AP/WGB - IW

36 Equipment to Equipment Use Cases Wireless Mobility Types Nomadic equipment Stays in place while operating Moves to a new location in the shutdown state AP Access IE2K / IE3X / IE4K AP Examples: process skids, storage tanks, reactors, portable manufacturing equipment WGB AP/WGB - IW

37 Equipment to Equipment Use Cases Wireless Mobility Types Mobile equipment (no roaming) Changes position while operating Remains connected to the same AP Examples: rotary platforms, manufacturing machines with tracks, overhead cranes with small spans Access IE2K / IE3X / IE4K AP WGB AP/WGB - IW

38 Equipment to Equipment Use Cases Wireless Mobility Types Mobile equipment (fast roaming) Connects to multiple APs while operating Does not drop application connections Examples: AGVs, ASRS, overhead cranes, train cars, entertainment ride vehicles Access IE2K / IE3X / IE4K AP Site survey and architecture selection are critical WGB AP/WGB - IW

39 Unified WLAN Architecture Overview Identity Services Engine (ISE) Connected Mobile Experiences (CMX) Cisco Prime Infrastructure WLC Access IE2K / IE3X / IE4K Access IE2K / IE3X / IE4K WGB LWAP SSID1 5 GHz LWAP WGB (Roaming) LWAP WGB SSID2 5 GHz WGB LWAP SSID3 2.4 GHz AP/WGB - IW

40 Wireless Access Asset Tracking

41 Location Based Asset Tracking Asset Utilization Track supplies in transit Inventory accuracy of receivables Retrieve misplaced components, subassemblies, etc. Locate missing tools, test harnesses, etc. Vehicle location for smarter dispatch Material Flow Efficiency Wireless restocking trigger Choke point recording The right supplies get to the right place In-line rework Bar Code replacement Business Value Production throughput increase Improved equipment utilization Reduced scrap Labor efficiency 82% improvement in retrieval time results in increased throughput, and on time delivery was improved 13% Ops Manager, Semiconductor 48

42 Device Wireless Infrastructure Applications and Management Real Time Location Services (RTLS) Architecture Business Intellience Single Pane of Glass for Cockpit Dashboard Partner Applications Access Point Cisco Identity Services Engine (ISE)/ Cisco Prime TM Network Wireless LAN Controller Enterprise Network Access Point Mobility Services Engine Access Point Open Ecosystem Scalable Infrastructure Leverage Common Wireless Infrastructure Track Any Wi-Fi Device or Tag Chokepoint Integration Chokepoint 49

43 Factory Security IDMZ

44 Controlling Access to the Industrial Zone IEC Industrial Network Security Level 5 Level 4 , Intranet, etc. Enterprise Network Site Business Planning and Logistics Network Enterprise Security Zone Remote Gateway Services Application Mirror Patch Management Web Services Operations AV Server Application Server Firewall Firewall Web CIP Industrial DMZ Level 3 Level 2 Level 1 Application Server Client Batch Control Directory Operator Interface Discrete Control Engineering Workstation Client Drive Control Remote Access Server Engineering Workstation Continuous Process Control Site Operations and Control Area Supervisory Control Operator Interface Safety Control Basic Control Industrial Security Zone Cell/Area Zone Level 0 Sensors Drives Actuators Robots Process Logical Model Industrial Automation and Control System (IACS) Converged Multi-discipline Industrial Network No Direct Traffic Flow between Enterprise and Industrial Zone 56

45 IDMZ Replicated Data and Services Permit Secure Remote Access to Industrial Assets Permit Data from the Industrial Zone to Enterprise Stakeholders Wide Area Network (WAN) Physical or Virtualized Servers ERP, Active Directory (AD), AAA Radius Call Manager Firewall (Inspect Traffic) Physical or Virtualized Servers Patch Management Remote AV Server Desktop Application Mirror Gateway Remote Desktop Gateway Server Firewall (Inspect Traffic) Physical or Virtualized Servers FactoryTalk Application Servers & Services Network Services e.g. DNS, AD, DHCP, AAA Call Manager Storage Array Engineer Remote Access Level 3 Site Operations Web Reports Permit Remote Access Server Web Proxy Plant Manager Permit Block VantagePoint Untrusted Block Untrusted Access to Industrial Zone Block Distribution switch Core switches Firewalls (Active/Standby) Core switches WLC (Enterprise) ISE (Enterprise) ISE WLC (Active) WLC (Standby) Enterprise Zone Levels 4-5 Industrial Demilitarized Zone (IDMZ) Industrial Zone Levels 0-3 Block Untrusted Access to Enterprise Zone Levels 0-2 Cell/Area Zone Untrusted PAC FactoryTalk Client Access IE2K / IE3X / IE4K IO Drive MCC PAC LWAP PAC WGB 59

46 Factory Security Industrial Firewall ISA 3000

47 Industrial Firewall ISA 3000 Architecture & Management software Centralized Management Cisco Security Manager (CSM) Firewall FireSIGHT Management Center FirePOWER Local Management Firewall, ACL, NAT & VPN Adaptive Security Appliance (ASA) Firewall, ACL, NAT & VPN On Board the ISA 3000 Adaptive Security Device Manager (ASDM) Firewall & FirePOWER Management ISA 3000 Hardware FirePOWER Application & Threat Control IPS - Application & Threat control 61

48 Industrial Firewall ISA 3000 Architecture Positioning Enterprise External DMZ/ Firewall Internet Enterprise Zone: Levels 4-5 Industrial Demilitarized Zone (IDMZ) Physical or Virtualized Servers Patch Management AV Server Application Mirror Remote Desktop Gateway Server Industrial Zone: Levels 0-3 Authentication, Authorization and Accounting (AAA) Core Switches IDMZ Firewalls create a security boundary between the Enterprise and Industrial Zone Wireless LAN Controller (WLC) Active Level 3 Site Operations Distribution Switch Stack Standby FactoryTalk Client Level 2: Area Supervisory Control LWAP SSID 2.4 GHz Controller Controller Level 1 - Controller Industrial Firewall I/O Soft Starter MCC Level 0 - Process SSID 5 GHz I/O WGB Drive 62

49 Factory Security OT Intent-based Security for Industrial Networks

50 Cisco Identity Services Engine (ISE) Delivering Visibility, Context, and Control to Secure Network Access NETWORK / USER CONTEXT DEVICE PROFILING FEED SERVICE Who What When Where How REDUCE NETWORK UNKNOWNS AND APPLY THE RIGHT LEVEL OF SECURE ACCESS CONSISTENTLY ACROSS WIRED, WIRELESS and VPN Guest Access Contractor + Vendor (e.g. RBAC) Employee Access 64

51 Secure Access Consolidating access for employee/contractors/vendors Who? Employee Attacker Guest What? Personal Device Company Asset How? Wired Wireless VPN plant 1, zone 2 Headquarters When? Weekends (8:00am 5:00pm) PST 65

52 Operational challenges due to IT-OT dependency VISIBILITY Enforcing security in the process network requires Security systems to have visibility to plant floor Assets with the Context of observed behaviors INTENT Maintaining it effectively, requires dynamic security policy application triggered by OT intent without dependency on IT for day to day operations 66

53 Defining security policies without visibility is complex Security Platforms C a m e r a P r i n t e r L a p t o p P h o n e?????????? Enterprise Assets Industrial Assets 67

54 IoT Threat Defense C O N T E X T OT Platform IT Platform SGT dacl IE Switching V I S I B I L I T Y I N T E N T C O N T E X T pxgrid SXP IND ISE pxgrid NGFW C O N T E X T Quarantine StealthWatch 69

55 Visibility in Industrial Networks Security starts with Visibility Context Enhances Security Discover Industrial Assets using CIP, PROFINET, Modbus, BACNet Protocols Visualize connectivity between automation and networking assets Industrial Network Director pxgrid Identity Services Engine Who What When Where How Compliance Threat Vulnerability Bob Rockwell PLC 11:00 AM EST on April 10 th Extrusion, Zone-2, Cell-1 Wired Access Yes None CVSS score of 6 IND shares industrial asset identity with ISE over pxgrid this Visibility combined with Context, becomes a force-multiplier for Security 70

56 Industrial Asset Visibility with IND IND Asset Inventory ISE Profiler Attributes pxgrid Identity Services Engine iotmacaddress iotipaddress iotname iotvendor iotproductid iotserialnumber iotdevicetype iotswrevision iothwrevision iotprotocol iotconnectedlinks iotcustomattributes ISE profiling rules based on attributes like Make, Model, Serial Number, Device Type etc. instead of just IP address Custom Attributes allows IND to signal higher order information that is common to a group of assets 71

57 Use Case#1 - Cell Segmentation Segmentation Requirement Segment the industrial network OT user have the ability classify the assets into segments Security Policy Pre-Staging SGT 33 SGT 100 SGT 200 SGT 33 SGT 100 SGT 200 IT and OT decide on the segmentation policy IT configures ISE with Secure Group Tags (SGT), TrustSec policy to match rules Workflow during Asset Classification pxgrid 3 C O N T E X T IT User ISE 4 SGT 33 MES Level 3 1. OT user selects assets and groups them in IND as Cell-1 and Cell-2 2. OT user assigns a tag to C2-PLC 3. IND sends OT user intent and asset details to ISE in pxgrid 4. Profiling policy match in ISE results TrustSec policy distribution SGT 100 SGT 200 IND SGT 33 OT User 1 2 Level

58 Use Case#2 On-Demand Remote Access Remote Access Requirement Only specific asset in the machine must be accessible No dependency on IT Security Policy Pre-Staging 1. IT user pre-defines profiling rules in ISE to match custom attributes AnyConnect to check security posture, establish VPN, and collect application telemetry info Track user session in ISE along with SGT role. 3 SGT 777 S X P ASA AnyConnect RDP OEM DMZ 2. IT user pre-defines SGT firewall rules in ASA to allow remote Access Workflow during Maintenance Window 1. During machine maintenance, OT user changes asset attribute tag in IND which denotes intent to allow remote access IT User 1 2 C O N T E X T IND ISE Level 3 Level IND sends OT user intent and asset details to ISE in pxgrid, which results in asset reauthorization OT User SGT ISE distributes new TrustSec policy to Firewall and access switches to enable remote access 74

59 Use Case#3 Flow Based Anomaly Detection Requirement Group assets in communication trust zones and detect anomalous traffic behavior Easily detect the source of anomaly Security Policy Pre-Staging Assets grouped in IND by OT user, automatically creates Host Groups in StealthWatch H O S T G R O U P S Level 3 IT defines Alarms in StealthWatch for Host Group zone map violations C O N T E X T ISE Q u a r a n t i n e 3 Stealth Watch 2 IT User IT configures policies in ISE to quarantine devices on violations C o A Level 0-2 Workflow 1. Compromised Camera in Cell-2 initiates Port Scan 2. StealthWatch raises Recon Alarm, and zone map violation alarm OT User Cell-1 IND Port Scan 1 4 N E T F L O W Cell-2 3. StealthWatch sends quarantine request to ISE 4. ISE moves camera access port to isolated VLAN to quarantine 75

60 Industrial Network Security Framework CPwE - Holistic Defense-in-Depth Enterprise Zone: Levels 4-5 Industrial Demilitarized Zone (IDMZ) Physical or Virtualized Servers Patch Management AV Server Application Mirror Remote Desktop Gateway Server Industrial Zone: Levels 0-3 Identity Services Enterprise External DMZ/ Firewall Internet Core Switches Wireless LAN Controller (WLC) Active Control System Engineers Control System Engineers in Collaboration with IT Network Engineers (Industrial IT) IT Security Architects in Collaboration with Control Systems Engineers Level 3 Site Operations FactoryTalk Client Level 2 Area Supervisory Control Distribution Switch Stack Controller Controller Level 1 - Controller Access IE2K / IE3X / IE4K IFW I/O Soft Starter LWAP MCC Level 0 - Process Standby SSID 2.4 GHz SSID 5 GHz I/O WGB Drive 76

61 Cisco Kinetic IoT Platform & Vertical use cases

62 Voice of the customer We want business value from Data We want ownership, privacy and security of our Data We want fast start, and grow over time 78

63 To get value from data App App App App App 79

64 Customers are challenged! App App App App App App App App App 80

65 Cisco Kinetic Platform Sensors Sensors IoT Devices Protocol Translation SW Protocol Translation SW Protocol Translation SW Edge Compute SW Sensors Edge Compute SW Sensors Edge Compute SW IoT Gateway Business Business Intelligence Intelligence Business Applications Intelligence Applications Applications IoT Data Fabric Visualization SW Cisco Solutions+ (DGLux5) Gateway Management Module Edge & Fog Processing Module Data Control Module 81

66 Manage Secure Cisco Kinetic Platform Cloud Hosted Applications Vertical Industry Applications Visualize Move Data Data Control Compute Data Edge and Fog Cisco Kinetic Platform Extract Network: Connect Device Communication Application

67 Cisco Kinetic Energy Monitoring Example Manufacturing workflow apps API Cisco Kinetic Capabilities Real-time monitoring of energy consumption Consumption data by machine, line, area and building Automated policy to notify factory staff potential about peak power or issues before they occur Secure Data Delivery Cisco Kinetic Edge and Fog Processing Benefits Data Capture & Normalization Real-time visibility Policy automation to improve energy optimization: identify leaks, inefficient machines, peak loads Nitrogen Flow Meters Equipment Power Meters Compressed Air Flow Meters Thermal Sensors Easy-to-use, accurate cost reporting for compliance Simplified deployment 84

68 Cisco Kinetic Equipment Health Monitoring Example Manufacturing workflow apps Cisco Kinetic Capabilities Real-time monitoring of equipment state and condition Easy to read trending data Automated policy to notify factory staff potential issues before they occur Secure Data Delivery Cisco Kinetic EFM Benefits Real-time visibility Enables steps toward predictive maintenance Improved issue response time Simplified deployment and meter deployment Vibration Sensors Pressure Sensors Power Sensors PLCs 85

69 Cisco Kinetic Manufacturing Solutions DSLink DSLink DSLink DSLink DSLink Energy Monitoring Todd M. Edmunds Vibration Monitoring Vibration Monitoring Energy Monitoring Energy Reduction Equipment Health 86

70 Cisco Kinetic Manufacturing Solutions DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink IE DSLink 4K w/ IOx IOx MES / Historian Energy Monitoring PLCs PLCs PLCs Vibration Monitoring Vibration Monitoring IE MTConnect 4K w/ IOx IOx CNC / Milling Interface Energy Monitoring Energy Reduction Manufacturing Visibility Equipment Health Connected Machine Plant Efficiency 87

71 Cisco Kinetic DCM DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink On Prem Analytics IE DSLink 4K w/ IOx IOx MES / Historian Energy Monitoring PLCs PLCs PLCs Vibration Monitoring Vibration Monitoring FANUC ZDT IE MTConnect 4K w/ IOx IOx CNC / Milling Interface Energy Monitoring Energy Reduction Manufacturing Visibility Equipment Health Predictive Maintenance Connected Machine Plant Efficiency 88

72 Cisco Kinetic DCM DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink MRP On Prem Analytics Connected Logistics Wireless Location IE DSLink 4K w/ IOx IOx MES / Historian Energy Monitoring PLCs PLCs PLCs Vibration Monitoring Vibration Monitoring FANUC ZDT IE MTConnect 4K w/ IOx IOx CNC / Milling Interface Energy Monitoring Energy Reduction Asset Tracking Manufacturing Visibility Equipment Health Predictive Maintenance Connected Machine Plant Efficiency Supply Chain 89

73 Connected Factory in Practice Achieving Business Outcomes

74 Drivers for the Connected Factory Becoming an Insight-Driven Manufacturer Have the Ability to Accurately Track Machine Utilization (e.g. OEE) Facilitate the Use of Advanced Sensor Technologies and Enabling Predictive Maintenance Continuously Innovating Products, Services, and Relationships Create Connected Environments Inclusive of Partners (Internal and External ones) Becoming Agile While Maintaining Control of the Business We Want New Operational and Business Models 91

75 Connected Factory - Achieving Business Outcomes Reduce Costs (Optimize Operations) Increase Revenues (More Capabilities) Meet Responsibilities (Environmental, Safety, Regulatory) Production Automation Inventory Management Quality Control Cost Management Workforce Enablement Product Enhancement Applications Building Management Facilities Management SCADA Ind. Access & Control Safety Unified Application Layer (Any Device - Any Application) Security Real Time Location Services Manu. Execution Systems Ent. Resource Planning Reports Analytics Internet Collab. Networks Automation Network Unified Network Management Layer (Deployment + Service Management) Management Network Collaboration Network (IT) Devices Sensors Robots Supply Chain Tracking Personal Energy Devices Voice Video 92

76 Industrie 4.0 Technology Progress Smart Devices 18 th Century Steam 20 th Century Mass Production 70 s Robots Today Digitization/Cyber- Physical Cyber-physical systems monitor physical processes, create a virtual copy ( Digital Twin ) of the physical world, and make decision decentralized decisions Cyber-physical systems communicate and cooperate with each other and with humans in real time Internal and cross-organizational services are offered and used by participants of the value chain Includes soft topics like work/life balance 93

77 IoT, IIoT, Industrie 4.0 and the Connected Factory Connected Factory 94

78 Connected Factory in Practice Factory Security

79 Hope is NOT a Strategy And Not Doing Anything is Either Manufacturing is the most targeted category and small to medium manufacturers are the most targeted. 40 percent of manufacturing companies ended up affected by cyber incidents in the past 12 months, 38 percent of those that felt the effects indicated cyber breaches resulted in damages in excess of $1 million, 96

80 Industrie 4.0 Driving the Connected Factory Connect OEM North Enterprise-wide Systems Supplier Corporate Headquarters Other Plant Customer Plant-wide Systems Protect & Detect West Receiving Batching/ Blending Processing Packaging Material Handling South Lower Total Cost of Ownership Faster Time to Market Better Asset Optimization Broader Risk Management Control Room Shipping East Utilities Collect 97

81 Security is NOT a Product but a Process Where do I Begin? NIST Cybersecurity Framework MFG Profile 98

82 NIST Framework Core Functions and Categories FUNCTION IDENTIFY PROTECT DETECT RESPOND CATEGORIES - Asset Management - Business Environment - Governance - Risk Assessment - Risk Management Strategy - Access Control - Awareness and Training - Data Security - Information Protection Processes and Procedures - Maintenance - Protective Technology - Anomalies and Events - Security Continuos Monitoring - Detection Processes - Response Planning - Communications - Analysis - Migration - Improvements Know what you have & How critical it is to your org. Secure what you have Spot threats quickly Take action immediately RECOVER - Recovery Planning - Improvements - Communications Restore operations

83 Assess the Threats and Vulernabilities Targeted or Not Employee carelessness Employee(&former employee sabotage Internet Phishing Infected CD Infected PDF file Infected memory stick A printer Threats from Cloud Services and Internet Internet Cloud Systems Exfiltration attacks Threats from Infected HMI s or PLC s Enterprise Network IDMZ Web Server Supervisory Network SCADA Control System Network HM I PLCs App Threats Server from Unauthorized Control Uncontrolled Access Databas e Historian Histor ian Threats through Remote Access VP N Threats from Unauthorized Control Remote Facility Field Network PLCs 101

84 Develop the Transformation Current State Security through Obscurity Future State Flat and Open IACS Network Infrastructure Flat and Open IACS Network Infrastructure Structured and Hardened IACS Network Infrastructure 102

85 Strategic Factory Security Approach Secure Network Environment Phase 1 Factory(OT) Architecture IDMZ (IT OT Separation) Secure Remote Access to OT OT Network Segmentation Secure Visibility & Control Phase 2 OT Identity Base Network (ISE) OT Dedicated Security Appliances at Major Demarcation OT Network Security Monitoring Phased Factory Security Maturity Advanced Industrial Security Phase 3 Convergence of IT and OT Network Security Cyber-Security Overlays Enhance Protections Content 103

86 Factory Security Challenge Need to connect machines from the factory floor for visibility, but have Security by Obscurity posture. Need protect IT from OT and OT from IT. Reduced Risk Reduced Downtime Solution Factory Cyber Security Assessment Industrial DMZ Defense in Depth Framework Business Outcomes Reduced downtime Protect brand reputation Minimize cyber theft Increase Visibility to Factory Floor

87 Why Segmentation? manages attacks Securing Environment Segment infrastructure Protect inbound and outbound communications and each other Scalable software defined segmentation Separate systems and users based on role and policy. Reducing security complexity Identity based access Restrict connection to known systems and devices Profiling IoT Evaluate and determine characteristics and posture to see if a device is Misbehaving 105

88 Map out IDMZ Traffic Flow Requirements for the network services and application data flow Applications and protocols may have to be allowed A certain network services may be allowed to communicate directly while ICS applications use IDMZ assets to exchange data. 106

89 IDMZ Implementation- Current State Connected Factory - Holistic Defense-in-Depth Implement Purdue model with level segmentation via firewall with routing controls Proper configuration and maintenance on Firewalls and ACL s Build and commission a DMZ at level 3.5 for IT services, agents, patch management etc. Distribution Switch Stack Enterprise External DMZ/ Firewall Layer 3 Layer 2 Internet Controller Controller Level 1 - Controller I/O Soft Starter MCC Level 0 - Process 107

90 IDMZ Implementation- Interim Connected Factory - Holistic Defense-in-Depth Enterprise Zone: Levels 4-5 Industrial Demilitarized Zone (IDMZ) Identity Services Enterprise External DMZ/ Firewall Layer 3 Layer 2 Internet Industrial Zone: Levels 0-3 Build the New IDMZ and the Factory Core Distribution Switch Stack Core Switches Layer 3 Layer 2 Controller Controller Level 1 - Controller I/O Soft Starter MCC Level 0 - Process 108

91 IDMZ Implementation- Access Migration Connected Factory - Holistic Defense-in-Depth Enterprise External DMZ/ Firewall Internet Enterprise Zone: Levels 4-5 Industrial Demilitarized Zone (IDMZ) Identity Services Industrial Zone: Levels 0-3 Migrate Access/Distribution Factory Floor Switch to New Core Core Switches Layer 3 Layer 2 Add Static Routes on Enterprise Core to Factory Floor Subnets and Redistribute into Enterprise IGP Distribution Switch Stack IDMZ FW Permit Any/Any and Logging Controller Controller Level 1 - Controller I/O Soft Starter MCC Level 0 - Process 109

92 IDMZ Implementation- Server Migration Connected Factory - Holistic Defense-in-Depth Enterprise Zone: Levels 4-5 Industrial Demilitarized Zone (IDMZ) Physical or Virtualized Servers Patch Management AV Server Application Mirror Remote Desktop Gateway Server Identity Services Enterprise External DMZ/ Firewall Internet Industrial Zone: Levels 0-3 Level 3 Site Operations Distribution Switch Stack Core Switches Migrate Servers in to their proper zones IDMZ FW build policy and enforce Controller Controller Level 1 - Controller I/O Soft Starter MCC Level 0 - Process 110

93 Protect Critical Infrastructure: Through Network Segmentation Zone Definition

94 How TrustSec Simplifies Network Segmentation Traditional Segmentation TrustSec IDC Servers IDMZ Firewall / Switch Static ACL Routing Redundancy Factory Network Micro/Macro Segmentation Central Policy Provisioning Factory Network ISE DHCP Scope Address VACL Aggregation Layer No Topology Change No VLAN Change Policy VLAN Access Layer Access Layer Non-Compliant Machine Employee Supplier BYOD Machine Non-Compliant Employee Supplier BYOD Quarantine VLAN Machine VLAN Data VLAN Guest VLAN BYOD VLAN Employee Tag Machine VLAN Data VLAN Security Policy based on Topology High cost and complex maintenance Supplier Tag Non-Compliant Tag Use existing topology and automate security policy to reduce OpEx 112

95 Extensible - Scalable Segmentation Easily Separate Devices and Data Using the Network Assign role-based groups SGT_Contractor SGT_Factory Floor SGT_Employee Assign business-based groupings to provide consistent policy and access independent of network topology Guest 1 50 Employee 1 Employee 2 Employee 3 Temperature Device 1 Get up and running quickly Utilize a controller to support group design Guest 2 IP Camera Employee 4 50 Establish context-aware groups Guest 3 Guest 4 Temperature Device 2 IP Camera Leverage attributes such as location and device type to define group assignments SGT_ERP SGT_Cell ERP 1 ERP 2 Conveyor system 1 Conveyor system 2 113

96 SF Operator SF Device SF Development Vendor/Con SF Operator SF Device SF Development Vendor/Con Factory Device Segmentation Example Software-Defined Segmentation- TrustSec Switch automatically downloads all policies from ISE for only devices connected MES Server Data Center Historian SGACL Policy SF Operator SF Development SF Device DC FW Factory FW ISE TrustSec Policy (SGACL) configured and provisioned by ISE Vendor/Contactor Factory Backbone Traffic filtered even in same VLAN Shop Floor Device SW 2 (SGACL) SW 1 SF Operator SF Development SF Device Vendor/Contactor Engineering Workstation Vendor / Contractor 114

97 Factory Data Access Control using TrustSec Software-Defined Segmentation- TrustSec Data Center ASA Firewall Policy MES Server Historian SF Operator SF Operator Eng Work Stn SF Device MES Server MES MES Server Historian DC FW Factory FW (SGFW) ISE OS Type: Windows XP Embedded User: Frank AD Group: Shop Floor Device Group: Eng Workstation Security Group = Shop Flr Device SW 2 Factory Backbone SW 1 Access Privilege Authorization with Security Group OS Type: Windows 8.1 User: contractor123@acme.com AD Group: None Device Group: BYOD Laptop Security Group = Contractor Engineering Workstation Vendor / Contractor 115

98 Monitoring & Analysis Why Visibility Communication in both IT and OT Monitor Infrastructure communications Identify and alert on abnormal traffic flows Threat intelligence Knowledge of existing attacks and communication vectors Intrusion Prevention - Block attacks, exploitation and intelligence gathering 116

99 Have you been compromised? How and when would you know? You have already made a lot of investment in network and security yet threats are getting through. 117

100 Protecting IoT and OT devices Segmentation No endpoint agents Detect malicious behavior 118

101 Effective security depends on total visibility KNOW every host SEE every conversation Understand what is NORMAL Be alerted to CHANGE Respond to THREATS quickly Identify every asset on the network Manufacturing Set policies based on hosts as well as applications Datacenter Enterprise Model policies before enforcing them IoT Devices 119

102 Security Analytics with Stealthwatch Multilayered machine learning Combination of supervised and unsupervised techniques to convict advanced threats with high fidelity Behavioral modeling Behavioral analysis of every activity within the network to pinpoint anomalies Global threat intelligence (powered by Talos) Intelligence of global threat campaigns mapped to local alarms for faster mitigation Data collection Rich telemetry from the existing network infrastructure Stealthwatch Encrypted Traffic Analytics Malware detection without any decryption using enhanced telemetry from the new Cisco devices 120

103 Introduction into Data and Analytics: Insight Driven Operations

104 Data in Manufacturing - Two Distinct Viewpoints Manufacturing has always had Big Data. We have been collecting data with historians, and MES systems for decades. Big Data Analytics Selected data Manufacturing is an untapped market for Big Data. There is lots of data, lots of different types of data, and hardly any of it is being used for analysis today. Ethernet Switch Edge Compute Edge Compute Data with context & quality flag PLC Cisco Kinetic Data with modeling & logic applied I/O 122

105 Data Driving Decision Making Analytics MEASURE ANALYZE DECIDE ACT STREAMING DATA 125

106 Data Driving Design and Digital Twin Analytics DESIGN SIMULATE & OPTIMIZE PRODUCE REACT STREAMING DATA 126

107 Analytics Maturity Data into Action Data analytics applied to factory equipment and sensors can bring operational efficiencies and cost savings to manufacturing processes. Analytics Human Input Required Descriptive What happened? Data Diagnostic Why did it happen? Predictive What will happen? Decision Action Prescriptive What should I do? Decision Support Decision Automation 127

108 Data and Decision Time within the Purdue Model Level 5 Planning Decision: Month/Year Network: Enterprise Cloud Level 4 Level 3 Level 2 Level 1 Business Systems Decision: Days/Weeks Network: Enterprise Manufacturing Operation Management Decision: Seconds/Minutes/Hours Network: Plant/Enterprise Equipment and Process Control Decision: Sub-second Network: Plant Sensors, Instrumentation, and Data Collection Decision: Sub-second Network: Plant Enterprise Fog Edge (IE w/ IOx Kinetic Level 0 Production Assets 128

109 What problems are we solving for customers? Environmental Sensing Remote Visibility Efficiency through Process Automation Plant Hazard Awareness Pollution Security Condition Monitoring Preventive & Predictive Maintenance Asset Health Cost Reduction Efficiency Consistency Business Outcomes Business Outcomes Business Outcomes Safety Cost Avoidance Increased up time Compliance Reliability Faster and accurate decision 131

110 Cisco Kinetic DCM DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink DSLink MRP On Prem Analytics Connected Logistics Wireless Location IE DSLink 4K w/ IOx IOx MES / Historian Energy Monitoring PLCs PLCs PLCs Vibration Monitoring Vibration Monitoring FANUC ZDT IE MTConnect 4K w/ IOx IOx CNC / Milling Interface Energy Monitoring Energy Reduction Asset Tracking Manufacturing Visibility Equipment Health Predictive Maintenance Connected Machine Plant Efficiency Supply Chain 132

111 Factory Wireless Autonomous Guided Vehicle(AGV) Roaming

112 Factory Wireless Use Cases Wireless tooling Monitoring hard-to-reach and restricted areas PLCs and automated guided vehicles (AGVs) Key Enabling IW3702 Features Seamless roaming at low to moderate speeds Supports prioritized PROFINET traffic for industrial applications PRP (Parallel Redundancy Protocol) over wireless for high resilience

113 Factory Wireless WGB Roaming Evolution Basic WGB roaming Fast WGB roaming PRP enhanced roaming Low to moderate speed Limited Scanning of channels High speed v BSS Fast Transition on WGB RSSI smoothing filter Optimized rateshifting algorithm Highest speed PRP over wireless Dual radios enables always-bestconnected Roaming coordination prevents two radios from roaming at the same time 138

114 Parallel Redundancy Protocol (PRP) over Wireless Wireless Network Without PRP PRP Enabled Wireless Network PRP RedBox PRP RedBox Data Frame Each data transmission goes through single radio path RF interference, hand off results in packet loss Data Frame PRP over wireless creates redundant radio path for data transmission Zero recovery time in event of temporary failure PRP is defined in the International Standard IEC and designed to provide hitless redundancy (zero recovery time after failures) in networks 139

115 PRP over Wireless Redundancy Options Dual WGBs, Dual Radios - WLC 8.4 Single WGB, Dual Radios - WLC 8.5 5GHz 5GHz WGB WGB 2.4GHz 5GHz PRP Switch as RedBox WGB as RedBox External PRP switch as RedBox (redundancy box) performs packet duplication/duplication discard function Application examples: Industrial automation and AGV applications WGB as RedBox (redundancy box) performs packet duplication/duplication discard function Application examples: Autonomous vehicles and straddle carriers and mission critical applications etc. 140

116 Roaming Coordination Gi1.51 VLAN 51 Gi1.51 AP1 AP2 WGB1 Direct Wired Connection or through a Switch WGB2 2.4GHz 5GHz WGB Gi0/2 Gi0/1 Switch WGB sends an indication to the other WGB indicating it wants to start roam Other WGB shall wait for 100ms (configurable) by default if it also needs to roam Once the roam event on the WGB is complete or if the timeout expires, the other WGB is free to roam 141

117 Sample Topology for Dual WGBs PRP Function 801 Data frame AP1 Data frame SSID A (LAN_A) Aggregate Switch 801 Data frame 802 Data frame 5GHz SSID B (LAN_B) Data frame Data frame 802 5GHz PRP Switch AP2 Data frame Data frame Data frame WLC Client VLAN Infrastructure Side Mobile Client Side Infrastructure Side An aggregate switch in the infrastructure side carries the duplicated packets APs in flex connect mode The APs transmits/receives the redundant data traffic over different SSIDs, tag with different VLANs Mobile Client Side Each WGB associates to different SSIDs and locates in different VLANs WGB1 Data frame PRP Switch Data frame WGB2 Data frame Client VLAN Client VLAN: 800 LAN_A: 801 LAN_B: 802 Roaming Coordination WGBs are connected to provide roaming coordination function, preventing both WGBs from roaming at the same time 142

118 Conclusion: Measure Twice, Cut Once Connected Factories reference architectures - Simplified design, quicker deployment, reduced risk in deploying new technology to achieve business outcomes Factory Network: Secure, scalable and resilient network infrastructure Factory Wireless: Enables mobility, secure personnel access, equipment to equipment communication and asset tracking Factory Security: Defense-in-depth security for multiple layers of threat detection and prevention Cisco Kinetic: IoT Platform to Extract, Compute and Move data 144

119 Recommended Resources Reference Architectures Websites Design Zone Industry Solutions For your reference 145

120 Cisco Webex Teams Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How Find this session in the Cisco Events Mobile App Click Join the Discussion Install Webex Teams or go directly to the team space Enter messages/questions in the team space cs.co/ciscolivebot# 146

121 Complete your online session survey Please complete your Online Session Survey after each session Complete 4 Session Surveys & the Overall Conference Survey (available from Thursday) to receive your Cisco Live T- shirt All surveys can be completed via the Cisco Events Mobile App or the Communication Stations Don t forget: Cisco Live sessions will be available for viewing on demand after the event at ciscolive.cisco.com 147

122 Continue Your Education Demos in the Cisco Showcase Walk-in self-paced labs Meet the engineer 1:1 meetings Related sessions 148

123 Thank you