APPLYING VLAN INSERTION IN ICS/SCADA

Transcription

1 APPLYING VLAN INSERTION IN ICS/SCADA SUMMARY Overview Network segmentation is a critical, fundamental building block in today s modern process controls networks (PCN). Once thought to be an optional practice, the recent increase in attacks on ICS/ SCADA systems shows it is vital to network design and security implementation. Case studies on both the Target and the more recent Ukraine electrical grid attacks indicate that these attacks were containable even preventable had the proper network segmentation been in place. This document discusses ways to perform segmentation within an ICS/SCADA/PCN environment in a fashion that is minimally disruptive, with a primary focus on ICS infrastructures predating the best practice of network segmentation. Business Drivers Isolate plant control network (PCN), industrial control systems (ICS), and SCADA from the enterprise network, following NIST SP and IEC best practices. Isolate internal communications at a remote site. Facilitate safe and minimally disruptive migration of current systems to new infrastructure. Avoid changes in network topology and IP addressing. Improve the ability to respond to and remediate cyber incidents. Maintain a homogeneous network design while retaining heterogeneous systems. A compensating control for patch management. Satisfy compliance obligations with internal or governmental regulatory body. Remediating factor; two-factor authentication (2FA), auditing. Implement access control in control networks, and reduce the cyber footprint of control networks. Decrease the potential for zero-day exploit and malware exposure. Business Problem Noted in both NIST SP and IEC 62443, network segmentation of the controls network from the enterprise is an essential best practice. Now with the increased need for connectivity into the ICS/SCADA environments, network segmentation is vital to help ensure the protection of trade secrets, ensure uptime, and maintain safe operations. An operational challenge that many owner-operators of ICS/SCADA systems must address is how to implement network segmentation. Performing segmentation without impacting business operations is especially challenging for those with a significantly older infrastructure. As described in the Reference Blueprint for ICS/SCADA Systems, 1 ICS/PCN control networks were designed to provide high uptime and a quick system response. Operators adamantly avoid practices that could potentially impact these metrics and cause an extended outage. Network segmentation is a project with the potential to cause extended downtime should something become unaccounted for during implementation. Operators of ICS/SCADA perform system upgrades and enhancements on tightly defined maintenance windows. In some industries, maintenance windows are pre-assigned up to a year in advance. This is usually determined by the operation and projected run time of the system in place. While network restructuring and optimization is a possible activity during maintenance windows, the reality is that this activity is often not performed or is postponed as more critical issues take precedence. Operations then look at the next maintenance window to attempt the network restructure, and again, they often displace the priority by another more critical site need. The net effect is that legacy network architectures tend to be static for long periods of time if not the entire duration of the ICS lifecycle. The simultaneous fixing of high-priority issues and implementing of network segmentation is rarely done. As a general rule, operators of ICS/SCADA/ PCN never institute multiple major/extreme changes within the same maintenance window, unless necessary. The main reasons for avoiding several system changes in the same maintenance window include the following: Numerous changes to the control system require more time to validate that systems performance is still within specification. Multiple changes take more time to troubleshoot in the event something is not performing within tolerance. If the origin of the problem cannot be determined or resolved and the operators are forced to roll back to the last-known good configuration, all system upgrades must be undone. Unfortunately, to ensure safe operations, this rollback includes even the changes that may be functioning correctly. 1 Reference Blueprint for ICS/SCADA Systems Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 1

2 With multiple system changes, system validation times become extended, presenting the possibility of a prolonged outage. As with all updates or significant changes done within a control systems environment, the concern is that the modification can render the system inoperable, even after rolling back to the last known-good configuration. This is why testing of all changes under consideration is a best practice before implementation, allowing for the development of a back-out plan in the event something goes wrong. This type of testing and development is difficult to do with network segmentation because of the number of unknown variables. The choice to segment a network usually requires an upgrade of network equipment, IP-address reassignment of devices on one network to a different one, or both. To achieve the desired results with network segmentation, several key decisions must be made with specific elements in place. Routing of the Defined IP Address Space If IP-Address Reassignment Is Required Older ICS/SCADA systems do not use DHCP and rely on static IP addresses and LMHOST files to establish and maintain connectivity. A concern is that in the course of changing IP addresses to a new IP address range, critical devices get overlooked, resulting in a possible long-term outage until identified and corrected. Accurate inventory as to which devices are currently on the network may not exist, increasing the chance of outage due to an overlooked appliance. Even with an accurate list, the time required to change every IP address can be time-intensive, especially if the majority of devices are remote. Factoring in for the time, distance, personnel, and unforeseen complications encountered during the process, the cost of network segmentation has the potential to be very expensive. Necessary Hardware That Is Capable of Addressing Design and Site Requirements The exchange or upgrade of existing network equipment is just as risky as the reassignment of devices to a new IP-address space. The high uptime requirements, age and maintenance windows directly affect the operator s ability to upgrade network devices. In much older systems, it is not uncommon for devices to be attached to the same router, switch and bridge for years without ever being touched. Over time, the cables become worn and brittle; in many cases they can develop wear patterns to the point where no other cable or device can be used to attach the device to the network. If the device is a highly specialized piece of equipment, the option to change it out with a newer or different one may not be viable due to cost or availability. In addition to physical connectivity, there is also the concern over software code incompatibility with existing devices. Even though network teams attempt to mock up a site in the lab, the end results are still inconclusive because they are unable to account for every piece of equipment and how it reacts to the network change. Another reason site mockups are challenging is the fact that most labs network hardware and control devices are running in ideal conditions and on current software code. It is not unusual for newer versions of operating code (or hardware) to cause communication problems for older PCN devices. Operators realized that networking labs never truly match the environment s current run state and understand that network segmentation may have many unforeseen problems during implementation on an actual production network. Many control environments have the necessary components; however, due to the nature of these systems and their strict uptime requirements. Network segmentation falls under the category of a major systems change with potentially inconclusive results. The standard operating procedure in ICS/SCADA is to test and confirm all change before implementing to ensure system integrity. The option to check and verify is not something that easy to do with ICS/SCADA networks. The choice of an owner-operator (especially those with significantly older infrastructure) to apply network segmentation to the PCN ecosystem is not something that is entered into lightly. Depending on their runtime requirements, pre-existing network conditions, and number of devices in play, the decision to implement network segmentation can be very time-consuming and costly. These organizations need a way to implement network segmentation that is minimally intrusive to production and does not put safe operations at risk. Traditional Approaches Air Gap: These systems were initially deployed based on serial connectivity. At initial deployment, the only means of connecting to these devices was to be physically present on-site. As a need for information and technology progressed, commercial off-the-shelf products (COTS) were used to provide connectivity. Some owner-operators opted to deploy either packet switching or stateful port-based firewalls to restrict access to these critical assets. The implementation of these technologies works to link the controls network to the enterprise and, in turn, completely destroy the isolation that once existed, exposing these mission-critical assets to the same malware, ATPs, and zero-day exploits as their business counterparts. Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 2

3 Packet Filtering (Stateless Firewall): Often done in conjunction with network address translation (NAT), this is the process of permitting or denying packets at a defined network interface based on source and destination addresses, ports, or protocols. Owner-operators would deploy this technique with routers and then later on with firewalls. A rule-based technology, the filter examines the header of the packet and allows or denies access based on how the packet matches the rule. When deployed, owner-operators would configure rules to accept only packets from particular IPs they deemed as safe, dropping all others. At the time, this was the most secure way to protect the PCN, however the most inconvenient because of the fact that legitimate traffic could be dropped. Mac Address Filtering: This is a form of security access control that uses the 48-bit addresses assigned to a network card to determine access into the PCN environment. This leverages the fact that Mac addresses are uniquely assigned to individual networking devices owner-operators grant or deny access through the use of blacklisting and whitelisted different machines. This method does not identify individuals, so authorized personnel have to have a whitelist entry for each machine they work on to gain access to the system. Stateful Inspection Firewall: In the early 90s, most operators of ICS/SCADA began to make the switch from packet filters (stateless firewalls) to stateful firewalls in an effort to protect the production environment. Owner-operators found full-state firewalls gave them greater control over which traffic was allowed into the PCN environment than the stateless solutions. The stateful firewalls were also not vulnerable to simple attacks, like IP buffer overflow. The advantage of state firewalls is they keep track of all TCP connections made through the firewalls. Also, state firewalls are aware of unprivileged ports on either side of the firewall. They can block packets not using the correct IP address ports and monitor the connection state. A significant disadvantage found with stateful firewalls is they require more memory to track active links. They are much harder to administer than stateless firewalls. Most important was the discovery of protocols that cannot be controlled by stateful inspection of TCP and IP. Unidirectional Diodes: This is a network device appliance that allows traffic to flow in only one direction. Unidirectional diodes are frequently found between layer 1 and 0. Some owner-operators have deployed them between the higher levels to protect critical assets. The set of diodes are very efficient; however they do add a level of complexity to the network. Palo Alto Networks Approach Palo Alto Networks NGFW is the ideal solution to address the costly and time-consuming problem of network segmentation. Through the use of the VLAN Insertion, the Next-Generation Firewall (NGFW) not only resolves the issue of segmentation within the PCN ecosystem; the implementation of our solution will provide owner-operators with the additional functionality necessary to protect today s ICS/SCADA environments. This is all done in a manner that is minimally disruptive to the plant process and on one box. So What Is VLAN Insertion? VLAN insertion is the logical placement of one device between two others without the need for physical recabling of the original devices or introduction of additional switches. Use of the VLAN insertion provides a method to introduce (or remove) a device from the data path with minimal disruption. VLAN insertion allows the Palo Alto Networks NGFW to insert itself between other devices. By placing itself in between devices, the NGFW can emulate the physical architecture where the devices were once connected. The advantage of VLAN insertion is that the connections are logical and physical connections remain in place. This logical connection allows for devices to be dynamically inserted or removed without modifying the physical cable or the need to change the IP address. For owner-operators of ICS/SCADA systems, VLAN insertion provides a means to do network segmentation with little to no impact on the plant process. Using VLAN insertion, operators can now confidently go into PCNs and remote sites that have not been touched in years with little fear of disrupting operations due to someone removing the wrong cable. Most operators have migrated to managed switches capable of supporting VLANs. VLAN insertion leverages this switch technology attribute, providing operators with a way to achieve the necessary segmentation in a way that does not affect network connectivity or process control. Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 3

4 Two Ways to Apply VLAN Insertion There are two methods for applying VLAN insertion within a PCN. The first method is the direct introduction of the NGFW between the devices that require separation. Direct connections back to the NGFW require the use of multiple interfaces on the firewall and demand the relocation of cables. The second way to apply VLAN insertion is with dot1q trunking, so pre-existing cabling to devices can remain untouched. The only cables to be added are the ones connecting the NGFW into the switch fabric. Once the necessary configurations are made to both the NGFW and the switch fabric, the downtime seen by switching the device to its new VLAN is the time it takes the switch port to go back into forwarding. Depending on switch configuration, this time is, on average, 45 seconds or less. Once port forwarding is complete, the device or devices will be segmented off from other devices on the same network segment, and the operator has a means to enforce access control over devices on this VLAN. IP Network Space With either method of VLAN insertion, the need to re-ip devices or network space is eliminated. Again, by leveraging the managed switch infrastructure and the fact that the NGFW is placing itself logically between devices the need to re-ip an address space is not necessary, thanks to VLAN tag rewrites. Being placed logically between the devices allows the firewall to perform VLAN tag rewrites. Acting as a bridge/enforcement point, the NGFW is now able to allow or deny traffic based on the zone s VLAN tag. Whether it s one device that needs to be isolated to respond to a potential threat or an entire network space that needs protection from enterprise traffic, the NGFW is now the virtual conduit (VC) sitting between them, addressing the need for access. Virtual Conduits Virtual conduits provide operators with control over what enters or leaves a zone. For traffic to flow between zones, virtual conduits (or diodes) must be built between them. If a conduit does not exist and doesn t until it s built traffic cannot enter or leave a zone. So zones are initially secure from outside traffic upon creation. Virtual conduits are also directional, so traffic into or out of a zone must be identified and allowed. Once in an enclave, access to devices can be defined based on: Zone User ID: by individual or group membership, applying the practice of least-privileged access Source IP address Source IP port Destination IP address Destination IP port Application Zones block all traffic unless it meets the defined policy criteria. VLAN insertion not only accomplishes the goal of segmentation but also provides operators with a clear demarcation or enforcement point, making it easy to define roles and responsibilities by internal organizations, like IT and OT, if necessary. Building upon the concept of zones and conduits, VLAN insertion aligns with the concepts of Zero Trust network methodology. An additional benefit of the VLAN insertion is that owner-operators are now in a position for faster isolation and remediation of devices that become compromised. Not to mention that indicators of compromise (IoCs) will be easier to identify because of the way zonebased technology works. Other Benefits Other benefits derived by implementing VLAN insertion, besides reducing the attack surface of the ICS/SCADA/PCN ecosystem, include the ability to apply subscriptions, like malware, antivirus, web filter, vulnerability protection, file blocking and data filtering, on a per-zone basis. This provides operators the option of using the NGFW s subscription as a compensating factor for zones containing devices or services that require them. Now, instead of installing antivirus on HMIs and running the risk of degraded performance, operators can isolate all existing site HMI into their own unique zone and use the NGFW antivirus/malware subscription to protect all devices located in the enclave. Intrusion prevention system (IPS) and intrusion detection system (IDS) capabilities for industrial control systems built into the nextgeneration firewall. IPS/IDS functionality is configurable just to monitor and alert taking no action on events seen, providing owneroperators the ability to do deep packet inspection in a nonintrusive fashion. Next-generation firewalls utilize the concept of positive enablement, which makes policies easier to write because they are written based on what is allowed. Operators can now protect devices with weak or no authentication capabilities with Palo Alto Networks User-ID technology, identifying them from such sources as LDAP, Active Directory (AD), TACACS, Radius or even sparse syslog. This makes the concept of least-privileged access applicable to and sustainable on devices that lack proper security measures. Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 4

5 Implementation Overview Figure 1 is a logical diagram representing an example of two symbolic hosts, which are on the same IP subnet and VLAN. The hosts shown are symbolic, as they could be replaced with multiple hosts, routers or other IP devices /24 Figure 1: Initial connection of devices that need to be isolated from each other Figure 2 is the physical diagram representing an example of these same symbolic hosts, which are on the same IP subnet and VLAN. Figure 2: Actual connection between devices that need to be isolated from each other VLAN Insertion: Method 1: Two Interface The first method of VLAN insertion will logically place a Palo Alto Networks Next-Generation Firewall with two physical interfaces, using virtual wire, or Layer 2, between two hosts. Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 5

6 Figure 3 is a logical diagram representing an example of these same two symbolic hosts, which are on the same IP subnet but now are on two separate VLANs in order to route traffic between them through the firewall. VLAN A VLAN B Figure 3: Two-interface insertion using direct horizontal runs back to the NGFW Note: This is not IP-routing and does not require hosts to be re-ip-addressed or re-ip-subneted; this is a logical insertion of the firewall between these two hosts. To facilitate this insertion, the VLAN is changed from VLAN A to VLAN B. This is the second example of the first method, with a managed switch as the intermediary device. Figure 4 is a physical diagram representing an example of the same two hosts, which are on the same IP subnet but now are on two separate VLANs in order to route traffic between them through the firewall by utilizing a switch. VLAN A VLAN B VLAN B VLAN A Figure 4: Two-interface insertion utilizing existing switch fabric Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 6

7 To enable this configuration on the switch, VLAN B is created; however, VLAN A will now see VLAN B traffic, including spanning tree. In order to utilize this configuration without creating spanning tree loops, resulting in one of the two firewall ports going into the blocking state, either the loop prevention must be disabled or its update packet must be blocked or filtered out. VLAN Insertion: Method 2 Single L2 Interface The second method of VLAN insertion will logically place the Palo Alto Networks Next- Generation Firewall with a single, physical Layer 2 interface between two hosts. Figure 5 is a physical diagram representing an example of these same two symbolic hosts, which are on the same IP subnet but now are on two separate VLANs in order to route traffic between them through the firewall by utilizing a single firewall interface (with IEEE 802.1q tagging). VLAN A VLAN B TRUNK Note: This is not IP routing; everything is done on Layer 2. Neither host has changed IP address. No change in subnet. How this was done: was changed to a different VLAN from. NGFW physical interface was configured as Layer 2. Sub-interfaces defined on NGFW Layer 2 interface. Figure 5: One-interface insertion using dot1q Example Application in Typical Plant Network The following is an actual deployment done in an ICS/SCADA environment. The operator of the facility was given the mandate to isolate the plant control and remote SCADA networks from business machines and processes. The company s network team uses a private network address space for all its internal routing. Remote-Site Details: Flat network: all devices on /24. Site does use a managed switch solution. Site is mainly production, but there are systems that require internet access and other business services. Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 7

8 Customer s Issue: Despite network and security teams best efforts, plant HMI and Historian keep getting infected with malware. It has been discovered that these machines are continually being configured to surf both the intranet and internet by (a) local user(s). These machines have a common, shared password and user ID, making it difficult to determine which user(s) is responsible. INTERNET Layer 3 switch Business Computer HMI PLC Historian Figure 6: Logical diagram: existing network pre-vlan-insertion Desired Segmentation: Customer would like to have the business machines isolated from the rest of the plant. Users from the business machine granted access to plant HMI by network ID and just to the Historian. Customer would also like the HMI, PLCs, and remote field network separated so that controlled access can be enforced as needed on a per-device basis and that none of these device groups have intranet or internet access. Most importantly, the operator has a small maintenance window and cannot afford for the network to be down for an extended period of time. So no change of IP address space is allowed at this time. Procedure: How do we go about doing this? On the switches 2 : Determine which VLANs are in use and which are available. Define the VLANs on switches that will be used for VLAN insertion. Assign zones (enclaves) to each VLAN. Configure the TRUNK port to connect NGFW and place in shutdown. It is highly advised that this port be down and manually turned up when ready. Configure switches to account for route-loop prevention where needed. 2 Please reference your chosen vendor s documentation for directions on how to adjust loop prevention and how to configure and assign VLANs. Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 8

9 On the NGFW 3 Connect port from NGFW to assigned TRUNK port on switch (if it lights up, turn either the firewall port or the switch port off until necessary configurations are complete). Configure the interface as either a Layer 2 or vwire interface. Create sub-interfaces for each needed VLAN created on switch fabric. Create zones (enclaves) for each VLAN and assign to sub-interfaces. Create a new VLAN under the VLANs section and assign subinterfaces to it. Under Policies, define zone-access criteria and commit. Now that both the switch and the NGFW are configured, enable both devices trunk ports. Customer Observation after VLAN Insertion Benefits No IP address changes required Minimum downtime Increased network visibility Clear line of demarcation between IT and OT resources Ability to customize and apply subscriptions on individual zones Perform deep-packet inspection on a per-zone basis Ability to identify and isolate potential issues faster. Ability to scale up or down as needed INTERNET Layer 3 switch ZONE: SCADA 0 Business Computer ZONE: Plant-HMI HMI VLAN 9 TRUNK TRUNK TRUNK VLAN 6 VLAN 7 VLAN 8 PLC Historian ZONE: PLC ZONE: L3.5 Figure 7: Post-VLAN-Insertion Summary Next Steps Now recognized as the cornerstone for cybersecurity, network segmentation is a critical necessity in defending the ICS. The ever-increasing number of attacks against these systems is a major concern due to the services and processes these systems control. The ability to protect, monitor, and remediate any indication of malicious intent is not optional; it is required. A wise builder would never erect a multi-story structure on sand, and cybersecurity should not be constructed on a sand equivalent. The intentional or unintentional manipulation of these systems can have both financial and environmental impact that can last for years. Protecting these systems (ICS/SCADA/PCN) from malicious intent is essential, and they must be built on systems that are capable of scaling to the task. Now that you see how Palo Alto Networks NGFW can help you achieve network segmentation, visibility and security for your ICS/ SCADA environments, contact your local Sales Representative for more information. 3 For detailed directions on how to configure the next-generation firewall for VLAN insertion, please reference. No documentation follows. Palo Alto Networks Use Case l Applying VLAN Insertion in ICS/SCADA 9