Unified Multi-Critical Infrastructure Communication Architecture

Transcription

1 Unified Multi-Critical Infrastructure Communication Architecture Titus Okathe, Shahram Shah Heydari, Vijay Sood and Khalil El-Khatib University of Ontario Institute of Technology, Oshawa, Ontario, Canada Abstract Recent events have brought to light the increasingly intertwined nature of modern infrastructures. As a result much effort is being put towards protecting these vital infrastructures without which modern society suffers dire consequences. These infrastructures, due to their intricate nature, behave in complex ways. Improving their resilience and understanding their behavior requires a collaborative effort between the private sector that operates these infrastructures and the government sector that regulates them. This collaboration in the form of information sharing requires a new type of information network whose goal is in two parts to enable infrastructure operators share status information among interdependent infrastructure nodes and also allow for the sharing of vital information concerning threats and other contingencies in the form of alerts. A communication model that meets these requirements while maintaining flexibility and scalability is presented in this paper. Index Terms Critical Infrastructure, Publish/Subscribe, Information Sharing, Interdependency, Situational awareness M I. INTRODUCTION ODERN society depends on a number of vital goods and services which help to maintain its health, wealth and security. The collection of physical assets, processes and organizations that are responsible for the production and distribution of these essential goods and services are termed Critical Infrastructures (CIs)[1]. CIs encompass a number of different sectors which include electricity, oil and gas, water distribution and waste water management, transportation (airways, railways etc.), banking and finance, emergency services and government services. Individually, each sector comprises various organizations, processes and agencies that interact with one another and with other sector agents to create and distribute goods and services. These interactions result in the emergence of complex relationships, dependencies and interdependencies between the CI sectors [2]. The normal operation of CIs may be disrupted due to a number of reasons such as component failure, human error, terrorism, natural events etc. One type of failure that arises as a consequence of interdependencies is cascade failure. For example in July 19, 2001 a freight train derailed in the Howard Street Tunnel, Baltimore. As anticipated, there were disruptions to rail system traffic and automobile traffic as well as increased activity for emergency services. However, besides these effects there was a cascading degradation of infrastructure components. A water main above the tunnel broke due to the fire in the tunnel, resulting in localized flooding. Additionally, the flooding resulted in electricity blackout to about 1,200 downtown Baltimore residences. Also, fiber optic cables running through the tunnel were damaged resulting in disruptions to communication services (phone, cell phone, internet etc.) to major corporations. Furthermore, the effect was felt far away, as the disruptions in the rail system led to delays in the delivery of coal and limestone for steel in the Middle Atlantic States [3]. A dependency is said to exist between two infrastructures if the state of the latter is influenced by the state of the former or if the state of the latter is correlated with the state of the former [2]. When this dependency is bidirectional, it is referred to as interdependency. In practice most critical infrastructures are interdependent. Interdependency may be physical, cyber, geographic or logical. Physical interdependency exists between infrastructures that rely on the flow of material from one infrastructure to another such as between a gas power plant and the gas pipeline infrastructure. Cyber dependency arises when there is a reliance on information flow between the infrastructures, an example is between the supervisory control and data acquisition (SCADA) network of an electric grid and the electric grid itself. Geographic interdependency is the dependency due to proximity or co-location. This is especially true for road networks, railways, natural gas pipelines and underground electricity cables or telecommunication fiber. Finally, logical dependency is any dependency that does not fall under any of the other categories. Logical dependencies usually arise from human decisions and policies. In this paper the performance of a unified architecture for integrated status monitoring and management of multiple CIs is demonstrated. A publish-subscribe architecture that has already been proposed for Electric grid is extended here for multiple CI interconnection, and CAP (Common Alerting Protocol) is implemented as the method for data exchange in this new architecture. Simulation results are presented to show the performance of this scheme.

2 The rest of this paper is organized as follows. Section II presents a review of the current trends in CI protection and management (interdependency modelling, information sharing) and some of their challenges. Next, a new unified architecture for multi-ci management is proposed in Section III, followed by performance results in Section IV. Finally, conclusions and some future possible enhancements to the current model are presented. II. CI MANAGEMENT AND PROTECTION A. Interdependency Modeling A first step towards CI protection is to understand the vulnerabilities and risks posed to CIs as a result of interdependency. This is a complex task due to the sheer size of CI sectors and number of individual components and agencies involved. To improve the understanding of these risks, there has been much research into interdependency modelling. CI interdependency modelling has been approached in a number of ways. These approaches include continuous or discrete time step simulations and modeling techniques like Markov chains, Petri Nets, dynamic simulation, agent-based, etc. [3]. Each of the modelling techniques has its strength and weaknesses. For example, agent-based models usually do not capture specific details of each CI; however, they have the advantage of being simple and fast. The main objective of modelling the interdependencies in CIs is to allow stake holders to better prepare for contingencies and prevent cascade failures from happening or at least mitigate their effect. However, CI modelling is limited by data accessibility, model development (what to model, how much detail is sufficient), and model validation (does the model approximate real events sufficiently). Currently CIs are separately managed and this is not expected to change in the future. However, the models being developed will eventually be used in the management of CIs. Most of these models require inputs from multiple CIs to make accurate predictions. Furthermore, to aid in the development of better models the current models need to be tested. This will require integrating these models into current CI management, hence necessitating the collection of information about the state of interdependent CIs to ensure accurate predictions of CI risks. Therefore, different stakeholders in CI management will need to share data for improved CI management and security. Second, these establishments are often in competition with one another and exposing sensitive data could damage their business advantage. Nevertheless, there have been some government-mandated programs to bring together these private operators in public-private partnerships (PPP) to provide a platform to exchange CI information. For instance the European Programme for Critical Infrastructure Protection (EPCIP) established by the European commission proposed the development of a Critical Infrastructure Warning Information Network (CIWIN)[4]. The goal of this network was to provide a platform for the exchange of best practices among member states as well as provide an optional platform for disseminating rapid alerts. A similar initiative by the US Department of Homeland Security is the National Center for Infrastructure. A brief overview of MICIE and IRRIIS architectures is presented below. The MICIE (Tool for systemic risk analysis and secure mediation of data exchanged across linked CI information infrastructures) project was developed under the European FP7 programme as part of the EPCIP[5]. The goal of this project was to provide real time risk assessment for CI operators. The risk assessment measures the possibility that a given CI would be unable to provide its services with the required Quality of Service (QoS) due to undesirable events in the reference CI and/or in its interdependent CIs. Furthermore, it was designed to provide a means to discover distributed information relevant for the alerting system. It also aims at overcoming the disparity of the information from multiple CIs and providing a means to exchange this information securely over the internet. Therefore, a central component of the system is the secure mediation gateway (SMGW). The overall MICIE system architecture is shown in Figure 1 [6]. The SMGWs connects each infrastructure to a central communication network and provides a secure point to send and receive risk data from other interconnected CI information networks. It also helps in converting CI specific data into meta-data that is exchanged between heterogeneous CIs. B. Information Sharing In Multi-CI management, there are challenges limiting the amount of information that can be shared between different infrastructures as well as the granularity of such information. First, many CIs are privately held and their data types and communication protocols and systems may be proprietary. Figure 1: MICIE system architecture [5]

3 The data in the MICIE system is exchanged using a special data structure. The data structure defines the state of a Critical infrastructure by considering ten parameters describing its QoS. These parameters are: availability, confidentiality, integrity, authenticity, reliability, safety, accountability, non-repudiation, and auditability. This special data structure is defined as the Service Quality Descriptor (SQD) [7]. Another important part of the MICIE architecture is the prediction tool that calculates a prediction of the risk for its parent CI and provides the information to the CI operator. It operates by inputting real time status information of its CIs and information from the SMGW into its prediction model. The Integrated Risk Reduction of Information-based Infrastructure Systems (IRRIIS) was launched under the European Union 6 th Framework programme [8]. The main objectives of this project are identified as: determining a comprehensive set of public and private sector requirements based on scenarios and related data analysis; developing, integrating and testing communication components suitable for preventing and reducing cascading effects as well as support recovery and service continuity in critical situations; developing, incorporating, and validating newfangled and advanced modelling and simulation tools incorporated into a simulation environment for experiments and exercise; and validating the functions of middleware communication components using the simulation environment and the results of the scenario and data analysis[9]. The IRRIIS project developed a set of information models to adequately capture the physical behaviour of CI systems as well as their controls and their emergent interdependency. The information model in the IRRIIS is broken down into three levels. These are: The Generic Information Model (GIM. The GIM represents the top-level ontology of CIs. It assumes that all CIs, irrespective of their technical differences, have a common information model which for the purpose of dependency management sufficiently represents them and their dependencies. It provides the basis for communication between different CIs. The domain specific information model, represents a specialize GIM that takes into account the special requirements of the CI domain. It contains the specific type of components and their characteristics as specialized forms of the more general concepts in the GIM. The final level in the information model is the Instance level models that describes the actual physical critical infrastructures using the abstractions and relations defined in the domain specific information model. While these architectures provide generic designs for management and status data collection from CIs, an integrated multi-ci management environment adds a number of challenges that will have to be addressed. Questions such as the existence of different and sometimes incompatible communication technologies in different CIs, control and authorization of data transport between devices in different CIs, design of the overall data exchange system as centralized or peer-to-peer, ability to filter and customized data per request, and scalability, are some of the issues that must be addressed. The next section presents a new proposal to address some of these issues, based on extending a publisher-subscriber model that was previously proposed for the electrical grid, and to extend it for multiple, heterogeneous CI environments. The proposed architecture makes use of common XML-based data records for utility and infrastructures, and implements a hierarchical structure to improve scalability. III. UNIFIED ARCHITECTURE FOR MULTI-CI MANAGEMENT A. Architecture The proposed unified architecture is based on extending the Gridstat model, a QoS-managed middleware communication framework for management of Electrical grid [10]. The proposed system uses a publish/subscribe communication paradigm and an information processing network to move information from producers to consumers. It also groups a collection of information processing nodes (information routers) into a single unit controlled by a control entity (simply called a broker). The system brokers can be hierarchical to reflect geographic or business boundaries or support a peer-to-peer communication architecture. The GridStat architecture is shown in Figure 2. It consists of two planes, the management plane and the data plane. The management plane represents the control layer, and this plane allocates resources and adapts the network to the changing power system configurations and communication network failures. This plane is also responsible for the security policies in effect on the network. The data plane consists of components that are responsible for moving data from producers to consumers and follows a publish/subscribe communication paradigm.

4 subscription by the QoS broker. The proposed model defines a basic autonomous unit, which is a collection of routers controlled by a broker that services publishers and subscribers as shown in Figure 3. Figure 2: GridStat Architecture The QoS broker represents the active components in the management plane. They are arranged in a hierarchical manner. This hierarchical arrangement reflects the geographic and business boundaries or hierarchies of the physical system [11]. At the bottom of this hierarchy are the leaf brokers who are directly responsible for a set of commonly-controlled data plane entities. Leaf brokers establish the data paths through the network of status routers for each subscription according to the QoS requirements for the subscription. Data plane components include the status router, publishers and subscribers. The status router support a couple of mechanism designed to make the network more efficient. These include rate-filtering, multicast, packet combination, and operational modes. Rate-filtering allows the status routers to drop a packet if no subscribers exist downstream for the current packet. Multicast support means that data can be delivered from single producers to multiple consumers. Packet combination refers to the ability of the status routers to combine multiple packets at the middleware layer into a single network layer packet when such combination does not affect the QoS requirements of the subscriptions. Finally, operational modes are predefined network operation modes that allow the network to rapidly respond to changes in the underlying power grid. The Status routers get preloaded with multiple routing configurations for each mode and the subscribers can define a set of operation modes. Publishers represent the producers of data on the network and represents entities being monitored or controlled. The subscribers represent the monitoring entities or the controller entity on the network. Subscribers may represent the control centre or a control or business application. Publishers announce their intent to publish a specific type of data and the rate at which they would publish that data. On the other hand, subscribers subscribe by specifying a set of QoS requirements including latency, rate of delivery, and redundancy. If these requirements can be met by the current network resources a path is created through the network of status routers for the new Figure 3: Basic autonomous unit The broker manages the subscriptions and publications in the unit, and controls the behaviour of the information routers by writing their forwarding rules. Publishers represent an entity with data to share on the network. Each data could be published in a periodic or aperiodic manner. Periodic data could be status information or data readings from devices out in the field. Aperiodic data could be any data such as alerts generated by process monitoring applications or even people. The basic unit in the proposed model extends the Gridstat model in a number of ways. Unlike the GridStat system, the routers in the proposed method are able to deal with data that is not periodic. This is because the system uses a hybrid of type-based and content-based publish-subscribe. Each data packet is essentially an object encapsulating the publication. Subscribers define predicate algorithms in their subscriptions that define what type of data they wish to receive as well as the rate at which they want to receive the data if the data is a subtype of the periodic data type, otherwise only the type is specified and a predicate algorithm. The predicate algorithm performs an operation on the data properties, for instance to check if the value of a field is equal to a value, or if the data is of a specific object type, etc. When the broker receives a subscription request it checks if any publication currently on the network matches the request and if the network can support the new subscription, if so, it sends a forwarding rule to all the routers with a matching publisher attached, otherwise it notifies the subscriber that their request was unsuccessful. The forwarding rules encapsulates a matching algorithm that matches data packets to subscriptions. If the data packet matches a subscription it is forwarded to the subscriber, else it is dropped. The network therefore does not store data but

5 consumes all data as it is generated. Multiple autonomous units can be connected to form a much larger network. Figure 4 shows one configuration for an autonomous unit. The connection between units is established over a wide area network (WAN) via edge information routers called boundary routers for simplicity. These represent a gateway from the internal network to the external WAN. When a subscription request is made by a subscriber in one autonomous unit for data published in another autonomous unit, the request s source broker contacts the next higher broker. This broker, identified as a tier broker, contacts the other brokers under its domain, querying for the original subscription. If any of the brokers have the requested data, a path is created from the boundary router of the unit where the data is present to the boundary router of the unit that requested the data. Also paths are generated within each network from the publishers to the boundary routers and from boundary routers to the subscriber. Figure 4: Centrally controlled autonomous units This tiered system means that more general security policies can be set at higher levels in the broker chain, leaving specifics to the lower level broker similar to the domain name system (DNS) zones. B. Data Types One of the many challenges in managing multiple CIs is the differences in the data format used in utility networks. The Common Information Model (CIM) format was originally developed by the Electric Power Research Institute (EPRI) and is now maintained by the IEC under IEC61968 and IEC It provides an XML-based common definition of power system components to enable Energy Management System (EMS) from different vendors exchange data independent of their internal software design and or implementation platform. CIM defines a number of classes that models the components of a powers system, and can be extended for other CIs. Similarly the Common Alerting Protocol (CAP) also provides a vendor independent and open digital message format. It is primarily designed for the exchange of messages or alerts. CAP messages can define geographic location as well as provide multilingual and multi-audience messaging. It is readable by humans and can also be processed by machines as it is XML based. According to the CAP specification [12], the primary use of CAP Alert Message is to ensure consistency in the information transmitted over multiple alerting and public warning systems. Secondly, CAP provides a means to normalize warnings/alerts from different sources. Furthermore, it may also be used by sensor systems for reporting significant events. In our simulations we adopted the CAP for alerting purposes, however other options are also possible. The next section discusses some performance studies. IV. PERFORMANCE ANALYSIS The proposed multi-ci communication model was implemented in the OMNet++ simulator. The current model consists of modules for the brokers, the information routers, publishers (periodic and aperiodic) and subscribers (periodic or type). The information routers use a predicates, as discussed in the previous section, to test each data object to see if they match a given subscription. This would determine if the packet is routed or dropped. This predicate encapsulates the subscription requirements of the subscriber. The model consists of two basic types of traffic: control traffic and data traffic. The control traffic represents the control messages sent between the elements of the system and the broker. These messages implement a basic clientserver communication. They include queries sent by tier brokers to sub brokers, forwarding rules sent by the brokers to the information router for new subscriptions, subscription messages and publication messages. They form a small part of the network traffic as the bulk of the message through the network is data packets from the publishers to the subscribers. Figure 5: OMNet++ Test Network

6 Figure 6: Packet Arrival Rate for Routers Figure 7: Subscription setup duration Figure 8:End-to-End Delay for data packets Figure 5 shows a preliminary test network. There are 40 publishers and 4 subscribers connected to each network. Each of the networks consists of 50 routers. The router topology was generated using the Boston Representative Internet Topology Generator (BRITE). Simulation run was for 900 seconds and the results collected includes the total number of packets transmitted by the publishers, the total number of packets received by the subscribers, the average time between packets for information routers, the end to end delay for of data packets. In Figure 6, the mean packet arrival rate is shown in megabytes per second (MBps) for each router on the network. This provides an estimate of the amount of traffic the routers need to deal with. Another important parameter was the time it took to setup new subscriptions. This is shown in the Figure 7. The high durations for subscribers 3 and 4 are due to the fact that the requested data is not in the same network. The maximum and minimum delays are shown in Figure 8. In the test network, link delays are uniformly distributed between 100ms and 10ms. Hence a packet traversing from one end of the network through the WAN and to the other end the network could do over 10 hops. The delays reported here arise mainly from the propagation delays. V. CONCLUSION In this paper, a multi-critical infrastructure communication network to aid the information sharing and data dissemination across distributed connected infrastructures was presented. The architecture is simple yet interoperable among multiple CIs. Preliminary simulation results indicate the feasibility of such a network architecture. Further comparative and quantitative analysis between the peer-to-peer and tiered configurations needs to be done. REFERENCES [1] J. D. Moteff, "Critical Infrastructures: Background, Policy, and Implementation," [2] S. M. Rinaldi, J. P. Peerenboom, and T. K. Kelly, "Identifying, understanding, and analyzing critical infrastructure interdependencies," Control Systems, IEEE, vol. 21, pp , [3] P. Pederson, D. Dudenhoeffer, S. Hartley, and M. Permann, "Critical infrastructure interdependency modeling: a survey of US and international research," Idaho National Laboratory, pp. 1-20, [4] E. Commission, "Communication from the Commission on a European Programme for Critical Infrastructure Protection," ed. Brussels, [5] P. Capodieci, S. Diblasi, E. Ciancamerla, M. Minichino, C. Foglietta, D. Lefevre, et al., "Improving resilience of interdependent critical infrastructures via an on-line alerting system," in Complexity in Engineering, COMPENG'10., 2010, pp [6] F. Caldeira, M. Castrucci, M. Aubigny, D. Macone, E. Monteiro, F. Rente, et al., "Secure mediation gateway architecture enabling the communication among critical infrastructures," in Future Network and Mobile Summit, 2010, pp [7] M. Aubigny, C. Harpes, and M. Castrucci, "Risk ontology and service quality descriptor shared among interdependent critical infrastructures," in Critical Information Infrastructures Security, 2011, pp [8] H. Dellwing, S. Geretshuber, C. Schwaegerl, and O. Seifert, "Power system survivability increase with intelligent support tools," in Power & Energy Society General Meeting, PES '09. IEEE, pp [9] M. Aubigny, "Risk Modelling and Simulation for Critical Information Infrastructure Protection," Master Thesis, University of Luxembourg itrust consulting, [10] H. Gjermundrod, H. Gjermundrod, D. E. Bakken, C. H. Hauser, and A. Bose, "GridStat: A Flexible QoS-Managed Data Dissemination Framework for the Power Grid," Power Delivery, IEEE Transactions on, vol. 24, pp , [11] R. A. Johnston, C. H. Hauser, K. H. Gjermundrod, and D. E. Bakken, "Distributing Time-Synchronous Phasor Measurement Data Using the GridStat Communication Infrastructure," in System Sciences, HICSS '06. Proceedings of the 39th Annual Hawaii International Conference on, pp. 245b-245b. [12] O. Standard, "Common Alerting Protocol Version 1.2," 2010.