Architectural Support for Model-Driven Performance Prediction of Distributed Real-Time Embedded Systems of Systems

Transcription

1 Architectural Support for Model-Driven Performance Prediction of Distributed Real-Time Embedded Systems of Systems Vanea Chiprianov, Katrina Falkner, Claudia Szabo, and Gavin Puddy School of Computer Science University of Adelaide Abstract. Systems of systems (SoS) are large-scale systems composed of complex systems with difficult to predict emergent properties. One of the most significant challenges in the engineering of such systems is how to predict their non-functional properties such as performance, and more specifically, how to model non-functional properties when the overall system functionality is not available. In this paper, we define an approach to SoS performance prediction based on the modelling of system interactions and their impacts. We adopt an Event Driven Architecture to support this modelling, as it allows for more realistic and flexible performance simulation, which enables more accurate performance prediction. We introduce a generic architecture and present its instantiation in a software architecture for the performance prediction of defence SoS. Our architecture allows for loose coupling, interoperability, and adaptability and facilitates sustainable evolution of the performance model of the SoS. 1 Introduction Systems of systems (SoS) are large-scale concurrent and distributed systems that are comprised of complex systems [1]. SoS are complex systems themselves, and thus are distributed and characterized by interdependence, independence, cooperation, competition, and adaptation [2]. In the context of defence, SoS are concerned with interoperability and synergism of Command, Control, Computers, Communications, and Information (C4I) and Intelligence, Surveillance, and Reconnaissance (ISR) systems [3]. Defence SoS are characterized by long lifecycles, hard constraints on non-functional properties to meet the requirements of space, weight and power, and conformance to regulations and standards. These characteristics are reflected at the SoS level as well. These challenges imply it is necessary to explore the expected performance by investigating several alternatives to system architecture, incorporating performance and space, weight and power requirements within the analysis. As they are conducted at the architecture level, such investigations provide a coarse grain prediction about the performance of a defence SoS, and not precise predictions. Investigating non-functional properties of SoS comprise all the issues P. Avgeriou and U. Zdun (Eds.): ECSA 2014, LNCS 8627, pp , c Springer International Publishing Switzerland 2014

2 358 V. Chiprianov et al. associated with the investigation of non-functional properties of composing systems. Moreover, there are challenges related to the specific nature of SoS [1], [2]. The decentralized, distributed nature of SoS require an emphasis on interface architecting to foster collaborative functions among its composing independent systems. The heterogeneity of the composing systems require interoperability and integration approaches. All these factors increase the difficulty of analysing non-functional properties of SoS. One approach to allow the early checking of meeting non-functional performance requirements is performance prediction modelling. Inthispaper,weadopt the definition of software performance prediction introduced by [4]: the process of predicting (at early phases of the life cycle) and evaluating (at the end) based on performance models, whether the software system satisfies the user performance goals. Similarly, we require that our understanding of performance prediction be based upon the provision and evaluation of an existing performance model. Prediction of software performance has developed from early approaches based on abstract models to model-driven engineering [5] based approaches. Model-driven engineering techniques use Domain Specific Modelling Languages (DSMLs). A DSML is defined in this paper to be a language that offers expressive power focused on a particular problem domain through appropriate notation and abstractions. System execution modelling (SEM) [6], a recent development from research into measurement-based performance prediction, provides detailed early insight into the non-functional characteristics of a DRE system design. System execution modelling supports the evaluation of overall (software) system performance, incorporating component interactions and the performance impact of 3rd party software such as middleware. These approaches [6], [7], [8] support detailed performance modelling of software systems, thus enabling predictions of performance through execution of representative source code of behaviour and workload models deployed upon realistic hardware testbeds. While these approaches address the challenges of performance prediction of individual systems, cf. a review in Section 5, new mechanisms are needed to address the performance prediction of SoS. We review the requirements for such mechanisms, in Section 2. To fulfil them, we introduce in this paper a software architecture pattern for the SEM-based performance prediction of SoS, in Section 3. This generic software architecture provides a rich connection mechanism between performance models of individual, standalone, composing systems. Being based on event driven architecture, it allows for more realistic and flexible performance simulation, which enables more accurate performance prediction. We instantiate this generic architecture into a specific architecture for the performance prediction of defence SoS, in Section 4. 2 Requirements To determine the requirements for the modelling environment designed to support the performance prediction of SoS within defence DRE systems, we

3 Architectural Support for Model-Driven Performance Prediction 359 undertook extensive discussions with stakeholders from the defence industry. The requirements detailed below influence our proposed software architecture. 1. Loose coupling. The systems that form an SoS are independent, but also need to interoperate and interact. Towards this, a mechanism that allows the description of loosely coupled interactions is needed. 2. Interoperability of composing systems. Different formalisms or modelling languages may be used to model performance prediction across the SoS. Moreover, to simulate realistic interconnectivity conditions between systems, different simulators (e.g. network simulators) may be used. A means to interconnect all these heterogeneous performance prediction models is necessary. 3. Interaction specification. The mechanism allowing the specification of loose interactions between composing systems of the SoS needs to be precise enough so as to limit the emergence of unexpected interactions to the point that they can be analysed as part of the performance prediction process. This also implies that the mechanism should allow for repeatability. 4. Time and data distribution. Because of its distributed nature, the performance prediction model of an SoS needs time and data distribution mechanisms between its composing performance models. 5. Adaptability. Both at the composing system and the SoS level, architectural reconfigurations (e.g. different types of middleware) within the performance models are necessary in order to analyse different architectural alternatives. Thus, a mechanism to generate code for a specific architectural configuration is necessary. 6. Sustainable evolution. TheperformancemodelofanSoSneedstoaccommodate models of composing systems being added, removed, and changed. The addition of new models, conforming to new formalisms, is related to the interoperability challenge. Removing models may impact the interaction specifications. 3 Software Architecture for Performance Prediction of Systems of Systems The software architecture of our system addressing the requirements defined above is presented in Fig. 1, described in a formalism inspired from UML component diagrams. To predict the performance of a SoS, the performance of each of the composing systems needs to be predicted. This is due to the independent nature of the standalone composing systems of the SoS. Therefore, for each composing system, a Performance Model ofthesystem(pems)isnecessary. For each of these composing PeMS we use a SEM approach. In addition to the composing PeMS, a mechanism to specify the interaction of the loosely coupled interoperable composing systems is necessary. The performance models of a system are integrated in our architecture using an event-driven approach. We base our architecture on the Event Driven Architecture (EDA) [9]. In an EDA, a notable event is immediately disseminated to all interested parties (human or automated). The interested parties evaluate the event, and optionally take action. The creator of the event (event generator) has no knowledge of the event s subsequent processing, or of the interested

4 360 V. Chiprianov et al. Fig. 1. Generic Software Architecture for Performance Prediction of SoS parties (event sink). This makes EDA an extremely loosely coupled and highly distributed architecture. In terms of implementation, after an event has been triggered, a notification is produced and propagated to an event processing engine. The engine may order events according to a priority criteria, or may do other type of processing specified in the activity associated with the event. Next, it publishes the event notification on the event channel, which propagates it to all interested parties. The event sinks detect and decide whether to consume it. In addition to addressing the loose coupling requirement, EDA also addresses the sustainable evolution requirement. Since the EDA event generator knows nothing about the event sinks, this enables an open-ended extension approach, in which event generators do not need to be modified to include new event sinks. Therefore, adding, removing and changing PeMS is simplified. The EDA event channel can be enhanced with a time and data distribution management bus. Such a bus, as long as it is independent of technologies used to describe PeMS, and is distributed, enables the interoperability of the PeMS. The PeMS may be thought of as a component that provides an interface, and may use other interfaces, as shown in Fig. 1. It is described using a specific formalism. Independent of this formalism, we model the interactions with other PeMS using event generators and event sinks. The event generators of a PeMS produce event notifications that are sent to an Event Processing Engine, which orders them in a queue using a priority criteria. The engine processes the first event in the queue, and executes its associated activity. It next publishes the event notification on the Event Channel. Theeventchannelmayusedifferent patterns, such as Reactor or Proactor [10], and propagates the event notification to all interested parties. The event sinks of all other PeMS detect it and may decide on an action. Event generators and sinks are introduced in the PeMS in ways specific to their formalism. There may be multiple Event Processing Engines within the architecture. Each orders, in a queue, the events generated by several PeMS. The choice of which PeMS events should be ordered by a certain Engine may be decided through loading algorithms. Alternatively events may be grouped as Complex Events, that can be handled by the Complex Event Processing component of

5 Architectural Support for Model-Driven Performance Prediction 361 that Engine. All Event Processing Engines publish the event notifications directly on the Event Channel. This ensures a highly distributed, loosely coupled architecture that facilitates scalability and fault tolerance as the possibility of single point of failures or choke-points is reduced. We introduce a Scenario domain specific modelling language (DSML) to specify interactions. It is built on top of the SoS performance prediction software architecture, containing concepts specific to EDA, and thus generic with respect to the composing PeMS. It is used to describe interactions between composing PeMS. The Scenario DSML also contains entities that are specific to the type of SoS whose composing systems interactions are being described. For example, a defence SoS will contain Organisations with Units in an Environment, while an enterprise SoS may contain Actors interacting with Components, Devices, following certain Processes, etc. The Scenario DSML is presented in more detail in [11]. Complementary to it, we allow the user to interact with the model, to visualise its performance results. This visualisaton component can be extended to replace the Scenario DSML only with user commands. The Scenario DSML uses model-driven engineering and code generation techniques to facilitate adaptability. For example, from the scenario model, code can be generated to different middleware implementations of the event channel or bus. The adaptability requirement is met also inside the PeMS, as it is possible to reconfigure each model with a different middleware. In summary, the software architecture of our system answers the identified requirements, and is generic with respect to the composing PeMS, being agnostic of them, with the exception of event generators and sinks. 4 Specific Software Architecture for Performance Prediction of DRE Defence Systems of Systems We have instantiated the Software Architecture for Performance Prediction of Systems of Systems to the domain of Defence. As part of this instantiation, the performance model of a system is modelled as a system execution model, using the Component Workload Emulator Utilization Test Suite (CUTS) formalisms, as described in Section 4.1. To instantiate the Event channel, we chose the Data Distribution Service (DDS), due to its extensive support of non-functional properties through QoS policies that support various time and data management mechanisms. We adopt a global wall clock time management pattern. DDS uses apublisher-subscriber(observer)pattern. The Event Generator is modelled specifically for the SEM, as an Effector worker, which describes the behaviour of the model when it sends an output. As discussed above, the Effector worker is a mechanism to implement the Event Generator in a way specific to the formalism used to describe the performance model. Similar to the Event generator, the Event sink is modelled in a way specific to the performance model formalism wechose,i.e.,thesystemexecution model, as a Sensor worker. A Sensor Worker models the system execution model behaviour when it receives an input. To communicate with the Event channel,

6 362 V. Chiprianov et al. i.e. the DDS bus in our case, a DDS subscriber - a mechanism specific to DDS - is necessary as well. Complementary, to send information on the DDS bus, the Activity attached to an Event must have a DDS publisher mechanism. The Event Processing Engine is implemented in C++, containing a generic queue for all types of events. The priority criterion for ordering the events in the queue is based the conditions that guard the triggering of an event, e.g., conditions based on specific values of input parameters. These conditions are part of the Scenario DSML, described below. 4.1 Software Architecture for Performance Prediction of DRE Defence Standalone Systems For completeness, we include here a discussion on our performance analysis and prediction process for a standalone SEM, described using CUTS formalisms. It has five steps: Model; Execute; Predict; Evaluate; and Evolve [12]. A DRE system is first modelled from different points of view. The modelling step includes the modelling of the DRE system s performance constraints together with scenarios of exercising the system in different conditions. From these models, distributed code is generated for different platforms of interest. The generated code is executed in the second step and information about its execution on various platforms is captured and aggregated into performance metrics. In the evaluation step, the metrics are shown to the expert through context-specific visualisations, such that (s)he can decide if the model fulfils the performance requirements. Depending on the expert s decision, modifications may be proposed to the initial models. These modifications may explore several alternatives, and each may result in a new generation of alternatives in the evolution step. The process continues from the modelling step and stops only when the expert decides to do so. We defined an architecture and associated tools to implement this process [13]. Executing the SEM code within its indicated deployment produces Execution traces and Basic metrics about the system performance. 5 Related Work Predicting the performance of SoS has a number of approaches, e.g. a systematic review [14] identifies nine. For example, [15] presents a data-centric, capabilityfocused process for analysing architectures of SoS. An executable model of the architecture and the performance of the SoS is defined and its results are used to analyse and evaluate the performance of the SoS architecture. However, the SoS is treated like a standalone system; one model is defined for the entire SoS, not a model for each of its composing systems, as we do. Other approaches deal with developing metrics for performance measurement. For example, [16] adapts the notion of technical performance measure to SoS, proposing a hierarchical metric called SoS performance measure. However, it focuses strictly on defining a measurement metric, while our approach is much more complete.

7 Architectural Support for Model-Driven Performance Prediction 363 There are numerous works on model-based performance engineering, including comprehensive surveys [4], [17], [18] that explore the many approaches, methodologies, and case studies. Several researchers explore the potential for modelling performance based on a complete understanding of the system architecture. UML MARTE defines a UML profile, which provides for the inclusion of non-functional requirements (i.e. performance, reliability, scalability) as UML models, which can be analysed as part of the development process. Our approach is complementary to that provided by MARTE, in that we provide support for emulation of performance models above existing middleware and hardware to support early performance evaluation within multiple realistic deployment scenarios, in addition to integrated analysis and visualisation. 6 Conclusion and Perspectives In this paper, we proposed a software architecture for predicting the performance of SoS. We focused on SoS for which event-based modeling and simulation is pertinent. Based on the Event Driven Architecture, our architecture allows connecting heterogeneous performance models of composing systems (PeMS) using Event Channels. It is generic with respect to the composing PeMS, being agnostic of them, with the exception of event generators and sinks. We instantiated this generic architecture into an architecture for predicting the performance of Distributed Real-time Embedded defence SoS. The PeMS are assumed to be solved by simulation. The Performance Modelling process for each composing system of the SoS has five steps: Model; Execute; Predict; Evaluate; and Evolve. To implement this process we used System Execution Modelling tools and Modelling Languages and tools that we defined using a Model Driven Engineering approach, but other ways of defining the Performance Models can be envisaged and easily included, for example with UML MARTE. This shows the genericity of our architecture in including heterogeneous performance modeling formalisms. However, several avenues for future work still exist. In the generic software architecture, we are investigating alternatives for the Event Processing Engine(s) to allow the specification of various loading and other complex event processing criteria. In the specific architecture, we are looking into ways to integrate network and other simulators to provide for even mode detailed performance analysis. References 1. Jamshidi, M.: System of systems engineering - new challenges for the 21st century. IEEE Aerospace and Electronic Systems Magazine 23(5), 4 19 (2008) 2. Dagli, C.H., Kilicay-Ergin, N.: System of Systems Architecting, pp John Wiley & Sons, Inc. (2008) 3. Manthorpe, W.H.: The Emerging Joint System of Systems: A Systems Eng. Challenge and Opportunity for APL. J. Hopkins APL Tech. Digest 17, (1996) 4. Balsamo, S., Marco, A.D., Inverardi, P., Simeoni, M.: Model-based performance prediction in soft. dev.: a survey. IEEE Trans. on Soft. Eng. 30, (2004)

8 364 V. Chiprianov et al. 5. Beydeda, S., Book, M., Gruhn, V. (eds.): Model Driven Software Development. Spinger (2010) 6. Hill, J., Schmidt, D., Slaby, J.: System Execution Modeling Tools for Evaluating the Quality of Service of Enterprise Distributed Real-time and Embedded Systems. In: Designing Software-Intensive Systems: Methods and Principles, pp (2008) 7. Paunov, S., Hill, J., Schmidt, D., Baker, S., Slaby, J.: Domain-Specific Modeling Languages for Configuring and Evaluating Enterprise DRE System Quality of Service. In: 13th IEEE Intl Symp and Wksh on Eng. of Comp. Based Sys. (2006) 8. Hill, J., Schmidt, D., Edmondson, J., Gokhale, A.: Tools for continuously evaluating distributed system qualities. IEEE Software 27(4), (2010) 9. Michelson, B.M.: Event-driven architecture overview. Technical report, Patricia Seybold Group (2006) 10. Schmidt, D.C., Stal, M., Rohnert, H., Bushmann, F.: Pattern-oriented Software Architecture: Patterns for Concurrent and Networked Objects. Wiley (2000) 11. Falkner, K., Chiprianov, V., Falkner, N., Szabo, C., Puddy, G.: Modeling scenarios for the performance prediction of distributed real-time embedded systems. In: Military Communications and Inf. Systems Conf., Canberra, Australia, pp. 1 6 (2013) 12. Falkner, K., Chiprianov, V., Falkner, N., Szabo, C., Puddy, G.: A model driven engineering method for DRE defence systems performance analysis and prediction. In: Bagnato, A., Indrusiak, L.S., Quadri, I.R., Rossi, M.G. (eds.) Industry and Research Perspectives on Embedded System Design. IGI-Global (accepted, 2014) 13. Falkner, K., Chiprianov, V., Falkner, N., Szabo, C., Hill, J., Puddy, G., Fraser, D., Johnston, A., Rieckmann, M., Wallis, A.: Model-driven performance prediction of distributed real-time embedded defence systems. In: The 18th Intl Conf. on Engineering of Complex Computer Systems, Singapore, pp (2013) 14. Klein, J., van Vliet, H.: A Systematic Review of System-of-systems Architecture Research. In: The 9th Intl ACM Sigsoft Conf. on Quality of Software Architectures, QoSA 2013, pp ACM, New York (2013) 15. Ge, B., Hipel, K.W., Yang, K., Chen, Y.: A data-centric capability-focused approach for system-of-systems architecture modeling and analysis. Systems Engineering 16(3), (2013) 16. Volkert, R., Stracener, J.T., Yu, J.: A framework for performance prediction during development of systems of systems. Intl J. of System of Syst. Eng. 3, (2012) 17. Smith, C.: Introduction to soft. performance engineering: origins and outstanding problems. In: 7th Intl. Conf. on Formal Meth. for Perf. Evaluation, pp (2007) 18. Koziolek, H.: Performance evaluation of component-based software systems: A survey. Performance Evaluation 67(8), (2010)