OTSDN What is it? Does it help? Dennis Gammel Schweitzer Engineering Laboratories, Inc. Funded by the U.S. Department of Energy and the U.S. Department of Homeland Security cred-c.org
Important Aspects of Critical OT Networks Determinism and low latency Precise time Fast fault detection, isolation, and recovery Cybersecurity defense in layers Monitoring, self-testing, and alarming Maintainability, testing and diagnostics High MTBF hardware cred-c.org 2
Message Delivery Performance Criteria Defined by International Standards IED performance requirements IEC 61850, IEC 60834, IEC 15802, IEEE 802.1 Latency specifications IEC 61850, IEC 60834, IEC 15802, IEEE 802.1 Speed IEC 61850 cred-c.org 3
Message Delivery Quality Criteria Defined by International Standards Dependability and security requirements IEC 61850, IEC 60834 Availability requirements IEC 61850, IEC 60834, IEEE 802.1 Reliability metrics IEC 61850, IEEE 1613, IEC 60870 cred-c.org 4
International Standards Dictate Protection Signal Exchange Acceptance Criteria Signal < 3 ms packet transit < 1 ms 99.99% of the time Signal <18 ms packet transit <15 ms 0.01% of the time Zero dropped GOOSE messages per year, <9 extra messages every 24 hours cred-c.org 5
Challenges With Traditional Ethernet Switching Designed for plug and play Conveniently does things we don t want Reactive failover Topology dependent performance Difficult to achieve 100% test coverage cred-c.org 6
Network Healing Using IEC 62439-1 RSTA C1 IED IED C2 S1 7 RSTA 1 2 2 7 RSTA 1 S2 2 S3 RSTA 1 7 IED Peer-to-peer RSTP informs RSTA cred-c.org 7
Introducing SDN Traditional Ethernet Switch Individual Control and Data Planes Software-Defined Networking (SDN) Switch Centralized Control Plane, Individual Data Plane Traditional Eth Switch Control Plane Data Plane Centralized Control Plane SDN Ethernet Switch Data Plane cred-c.org 8
Introducing SDN and OpenFlow Application Layer OAM Applications Network Visualization Control Plane Open Flow Data Plane Configuration Programming Network Operating System Simple Packet- Forwarding Hardware Simple Packet- Forwarding Hardware Simple Packet- Forwarding Hardware cred-c.org 9
How SDN Works Data plane inspects each Ethernet packet and performs one or more Match fields match rule based on first 4 layers of the Ethernet packet Instructions perform one or more programmed actions Counters increment counters and send counter data to centralized point cred-c.org 10
Multilayer Match Rules Forward Packets SDN Flow Match Rule Ethernet Header Layer 2 IP Header Layer 3 TCP / UDP Header Layer 4 Payload cred-c.org 11
OTSDN vs Traditional SDN Static vs Reactive Flows Traditional SDN uses reactive flows to dynamically respond and adapt to changes in the network and traffic Focus is on bandwidth utilization and latency rather than determinism Continuous learning and flow management Uncertain network performance at any given time SDN Controller performance bottleneck cred-c.org 12
Reactive IT SDN in Operation IT Flow Controller Rule SDN Switch Server Rule SDN Switch Rule SDN Packet Switch IED Packet cred-c.org 13
OTSDN vs Traditional SDN Static vs Reactive Flows OTSDN is uses static flows for proactive engineering of known network configuration Static flows can be used because all traffic is known Networks never have new traffic or devices without official change order New or unexpected traffic will be dropped Network state and performance is always known and as designed cred-c.org 14
Proactive OT SDN in Operation OT Flow Controller Rule SDN Switch Server Rule SDN Switch Rule SDN Switch IED Packet IED cred-c.org 15
Design Traffic Where Paths Are Based on Requirements and Applications Flow Controller Is Not Required for Network Operation SDN Switch GOOSE 2 Relay Relay SDN Switch GOOSE 1 SDN Switch Combined SCADA SDN Switch Engineering Access Rugged Computer cred-c.org 16
OTSDN - Cybersecurity at Every Network Hop Only allow traffic that is required and only to the places it is needed. No ARP Cache poisoning No Broadcast storms No BPDU attacks Hosts only see traffic for destined them and nothing else cred-c.org 17
No traffic injection from unexpected locations Locked down flows restrict what traffic is allowed on the network at every point Spoofing a device MAC/IP address is difficult Packets that match flow rules must originate from predetermined location. Any attempt to spoof a device from an alternate location raises alert and tracked cred-c.org 18
Traditional Intrusion Detection System External with Slow Action Response WAN IDS Knowledge Database Gateway Switch Alarms & events Network Sensor Analysis Engine Switch IED Switch IED Network Sensor Network Sensor Response / Action cred-c.org 19
OTSDN Intrusion Detection System Integrated With Fast Dynamic Response WAN OTSDN Switch Dynamic change of security policies IDS Application OTSDN Switch OTSDN Controller OTSDN Switch OTSDN Switch IED IED cred-c.org 20
Targeted IDS All needed traffic is engineered to go where it is needed Any unmatched traffic can be easily be discarded or sent to an IDS IDS will ONLY see the traffic that was not already engineered IDS will be burdened much less than watching all traffic More scrutiny can be given to this unwanted traffic cred-c.org 21
Targeted Deep Packet Inspection Focus DPI processing only where it is needed Individual Flow(s) from individual switch(es) can easily be sent to a DPI processor. The DPI process can determine if the packets should be allowed on the network. If allowed, send it back to the OTSDN switch for further processing, otherwise drop/log. Reduces burden on the DPI device by only processing the chosen stream of data. cred-c.org 22
Conclusion OTSDN is standard technology with different methodology Purpose engineered networks allow deny-by-default cybersecurity at every hop in the network Deterministic failover with traffic metrics New approach to IPS, IDS, and DPI Multipath capable / Application based circuits Controlled change management and network access cred-c.org 23
http://cred-c.org @credcresearch facebook.com/credcresearch/ Funded by the U.S. Department of Energy and the U.S. Department of Homeland Security