ECSS E Test Platform Features and Applicability Area

Transcription

1 SpaceOps 2008 Conference (Hosted and organized by ESA and EUMETSAT in association with AIAA) AIAA ECSS E Test Platform Features and Applicability Area F. Croce 1 and A.Simonic 2 Vitrociset PEG, Darmstadt, Germany, Lise-Meitner-Strasse 10, The Space System Model (SSM) is the answer of the ECSS E-70 Ground Systems and Operations working groups to the need to improve the exchange of knowledge between actors in the Space System Development and Operations domains. The SSM is defined within the Monitoring and Control (M&C) Data Definition Standard (ECSS-E-70-31). Its test and operation procedure aspects are defined in the Procedure Definition Language Standard (ECSS-E-70-32). ECSS-E introduces a formal specification of a language to be used for the automation of procedures in the development and exploitation phases of space systems. The language is known as Procedure Language for Users in Test and Operations (PLUTO) and is used or is going to be used in a number of applicability areas, mainly mission control system, automatic operations and check-out systems. What differentiate PLUTO from other adopted test languages is the possibility for the operator to reference the external system through standard and generic services acting on the model representation (SSM) of the unit, the availability of dedicated constructs and execution environment behavior specifically designed for testing and a natural language syntax. Regarding the modeling aspect, it is the SSM with its static definition and dynamic behavior to be in charge to directly interface the external elements, leaving the procedure definition completely decoupled from them. With PLUTO it is natural to approach the test definition and implementation in terms of behaviors (sequential and parallels goals, watchdogs, pre/post conditions, etc..) as the dynamic part of the model. Besides the unit/system under test (UUT/SUT) the model can include the representation of the check-out system itself and the test plan (with associated test scenarios, test cases and procedures) allowing to define the whole test environment in terms of a single unique model to be referenced with the same paradigm and type of services (i.e. the interface between PLUTO execution environment and the SSM). I. Introduction As the complexity of ground data systems increases, the budget available for a complete project life decreases and the re-usage of legacy or generic functional components is a common design choice, the assembly, integration and system testing activities (AIT) face an increase demand of reliability and formalism. In the case of systems made up of heterogeneous elements, the AIT infrastructure must be flexible and scalable to adapt to different test configurations providing the same level of reliability, throughout the whole verification and validation process. The paper addresses an AIT approach based on applicability of ECSS standards, used to model test artifacts and the description of an AIT infrastructure reflecting such approach. This paper includes the following sections: Section I this Introduction Section II ECSS-E and ECSS-E Brief Description Section III Assembly Integration and System Testing Infrastructure Model Section IV AIT Infrastructure for System Software Applications Section V Conclusions 1 Software Programs, francesco.croce@vitrociset.de. 2 Senior Software Engineer, ales.simonic@vitrociset.de. 1 Copyright 2008 by 08 VITROCISET S.p.A. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission.

2 II. ECSS-E And ECSS-E Brief Description ECSS-E-00 (Space engineering - Policy and principles) defines the overall space system as comprising a space segment, a ground segment and a launch service segment. Knowledge relating to the space system is used during all phases of a mission (design, development, testing and operation) and across all domains of expertise (quality, management and engineering). This knowledge has a structure that reflects the structure of the space system itself. For monitoring and control purposes, the ECSS-E standard introduces the concept of a space system model (SSM), which is hierarchically broken down into system elements mirroring the physical and/or functional breakdown of the space system. The ECSS-E addresses the definition of activities, reporting data and events associated to system elements and the ECSS-E has the purpose to encourage the use of a common procedure language across missions to facilitate the transfer of knowledge from space system development (and testing domain) to mission operations domain. A. ECSS-E [1] : Test and Operations Procedure Language The driving philosophy of the standard is: Specification of a language which is near to the needs of the end user of the space system (natural language); Embedded default support for expressing runtime behavior with minimum overhead for the user; Capability to interact with a model of the space system through well defined and generic services. The standard specifies clauses to be applied to a procedure language in terms of capabilities to be used for space system testing and mission operations, as well as the PLUTO language as an implementation of the specifications. The standard does not limit the user to the set of defined specifications, but allowing extending the language as needed, as long as tailoring does not break the fundamental rules of the language syntax and semantics. A PLUTO procedure has the following main structures: Declaration Body Preconditions Body Main Body Subgoal Confirmation Body Subgoal Subgoal Subgoal Subgoal Watchdog Body Watchdog Step Watchdog Step Figure 1 : PLUTO Procedure Structure An optional Declaration Body where events that can be raised at procedure level are declared An optional Pre-conditions Body responsible for ensuring that the procedure only executes if (or when) pre-defined initial conditions are satisfied The Main Body responsible for achieving the objective (goal) of the procedure. It is made of procedure statements and steps. Steps can be executed in sequence or in parallel An optional Confirmation Body that assesses whether the objective(s) of the procedure have been achieved or not (i.e. postcondition ) An optional Watchdog Body that manages contingency situations that can arise during the execution of the procedure. Watchdog body is started at the same time as the main body and is active until the main body ends execution Procedure main body can include simple statements or self-contained entities called steps. A step has the same structure of the procedure (preconditions body, main body, confirmation body, watchdog body) effectively allowing the user to design an overall procedure goal made up of lower level sub-goals. A procedure and a procedure step can include references to system elements object that is activities, reporting data and events. The abstract E definition of activity allows PLUTO to handle the activity regardless its implementation, whether it is a spacecraft telecomand or another procedure. The definition of the Pre-conditions body allows checking and/or waiting the validity of some conditions that must be true before executing the main body of a statements or a step. The definition of the Watchdog body allows the user to model contingencies and any recovery actions which the procedure goal must take into account. The definition of Confirmation Body allows defining procedure/step assertion expressions to determine the set of states under which the procedure/step is passed or failed. The execution model of E includes the following major characteristics: 2

3 Steps can be executed sequentially and/or in parallel. Steps can be nested as well as blocks of parallel steps. Block of parallel steps can exit until one step completes or until all steps complete; More than one watchdog step can be defined both at procedure level and at step level. In terms of language capability the E presents the following major features: Procedural language with the possibility to define independent flows of execution (parallel steps); Data Typed declarative with the possibility to associate engineering units to variables (e.g. Kg, m/s, Newton,); Binary multiples of engineering units (2^10 kibi, 2^20 mebi, 2^30 gibi, 2^40 tebi, 2^50 pebi, 2^60 exbi) ; Support to standard control blocks (e.g. if-then-else, while, for, case-switch,); Built in mathematical, string and time handling functions ; Possibility to raise events at procedure and step level. can be used to trigger execution branches upon conditions raised within other bock of procedure or step. must be declared in the declaration body; Enumerated set reference: defines a reference to a set of enumerated values; Property data type: defines a data type inherited from a property of a given system element, reporting data, activity or event. The following figure provide as example of a PLUTO procedure. procedure start preconditions body main body watchdog body procedure end step confirmation body watchdog step Figure 2 : Example of PLUTO Procedure Space System Model B. ECSS-E [2] : Monitor and Control Data Definition - Space System Model A Space System Model is a means to organize information accompanying the space system. The space system consists of hierarchies of functional and physical elements (e.g. the on-board thermal subsystem, a ground control software application). The SSM mirrors this structure into a hierarchical tree of System Elements (SEs) with each element capturing the knowledge of logical or physical functionality of the corresponding element of the space system. Each actor in the overall space system development and operation lifecycle (e.g. a subsystem manufacturer) has a domain-specific view of the SMM that corresponds to his particular domain of interest (e.g. quality assurance, thermal engineering, monitoring and control). A domain-specific view consists of a sub-set of the SSM information i.e. only those system elements of interest and only the relevant sub-set of system element properties. 3

4 System Element Knowledge Activities Reporting Data Figure 3 : Space System Model and System Element Knowledge The monitoring and control (M&C) view of the SSM is the subject of E and consists of hierarchies of system elements with associated activities, reporting data and events. With the formalization of the space system model (representing real physical or functional elements and resources), a combined SSM and PLUTO procedure can be defined. In such system procedures do not need to know the specific interface details for each system element, but procedures interact with each SE through the SSM implemented itself as functional unit. PLUTO System Procedure Environment // procedure A procedure A procedure // procedure statements B procedure A end procedure procedure // procedure A statements B procedure A end procedure B procedure statements end procedure C ECSS-E defines standard methods to access the monitoring and control characteristics of the SSM objects Space System Model Resources Monitoring Data Activities The SSM provides a logical view of the elements of the overall space system and is in charge to interface the services responsible of low level interfaces with these elements. Figure 4 : Combined ECSS-E and 32 System It is the implementation of the SSM model that will resolve on behalf of the procedure the interaction with the actual SE. C. Example of Systems Applying The Standards As an example of real-world systems making use of the standards we can mention: ESA VEGA Launch Vehicle (LV) Electrical Ground Special Equipment (EGSE), in charge to test the VEGA small launcher in a number of testing configuration (avionics equipments, vehicle stages, etc..); RADARSAT-2 Mission Control System (MCS) operational procedures automation component, in charge to execute mission planning products (satellite pass activities schedules) as set of procedures. Systems limited to ECSS-E are: ESOC / EGOS MATIS (Mission Automation) component; GALILEO Spacecraft Constellation Control Facility (SCCF) Automation and Planning Component. For the scope of issues addressed by this paper, the VEGA LV EGSE development has particular relevance, since the system needed to support the operational concept of units under test through the applicability for the first time of the standards. III. Assembly Integration and System Testing Infrastructure Model PLUTO has been specifically designed to be a test language and in this sense its usage for integration and system testing is a natural choice. An AIT infrastructure should provide the following high level functions: Requirements management; Test plan design and validation; Test execution and tracing; Test results management; Issues tracking management. 4

5 The test infrastructure should support consistency of all test artifacts (requirements, test cases, test procedures, etc..) across the five functions and infrastructure tools. It also must be capable to handle the testing functions and related data with a level of scalability and flexibility to cover different configuration of the elements under test (EUT). D. Model of Element Under Test Interfaces The concept of unit can refer to a hardware or software element which requirements must be validated and verified. In a widely accepted convention, the Unit-Under Test (UUT) is referred to hardware elements, while System-Under Test (SUT) or Application Under Test (AUT) to software systems. In the rest of the paper we use the term Element Under Test (EUT) as abstract concept. A EUT can be considered at different level of abstraction: a complete system, a subsystem or a single component. Subsystems contributing to more complex systems need to undergo formal verification and validation process with the same level of formalism and quality required for the assembled system. A unit is characterized by the set of interfaces through which it can be controlled and observed by the test conductor/executor application. Each interface handles services and data relevant for the domain of functionality provided by the unit. This can be compliant to formal specifications (ruled by Interface Control Document and as such to be formally verified), or to an interface specifically designed to support EUT inspection during formal testing. Interfaces can support the following type of interactions: Figure 5 : EUT Interactions Data Poll Service Request/Response Data Notifications Notifications used to query state or data values of resources managed by the EUT used to issue a command to the EUT aimed to change the status of an EUT managed resources. An (optional) response is sent back as a result of command execution used by the EUT to asynchronously send data values information used by the EUT to asynchronously send events information Table 1: EUT Interaction Types To be noted that by events, we refer here to occurrences (e.g. change of status) formalized with notification message typically carrying event information such as: event id, event source, event category, severity, etc. In terms of model representation, the EUT interface can be represented as a system element with associated activities, reporting data and events modeling the actual services, data and events exposed by the interface. While the first three interaction types and associated SE objects can be easily modeled in terms of supported data types and meta definition of services and responses, in case of complexes EUT integrated with many subelements (possibly from different vendors/suppliers) there is the need to cope with events of different structures. As later described in the paper this is a specific issue the test infrastructure must be able to cope with the introduction of an common structure event.. Figure 6 : EUT Model Used in Test Execution The definition of all EUT interfaces (in terms of SEs) can be stored in the test infrastructure space system model repository referenced and loaded by the tool in charge to execute test procedures and to interact with the EUT. 5

6 Due to the need to interact with heterogeneous elements (part of the EUT or part of the test infrastructure) the test execution tool needs to support all possible interfaces involved in the test of the EUT. An intermediate interface layer provides a means to bridge the specific interface exposed by an EUT with the test execution tool. E. Test Elements and Artifacts Modeling The concept of SSM modeling, used to define physical entities (EUT, simulators, test conductor, etc...) and their interfaces can be extended for the modeling of test artifacts such as test plan, test cases and test procedures. In this respect there are no differences between an activity acting on an EUT interface and a test procedure included in a test plan: they both can be represented as PLUTO procedures, where each one of them references and invokes different branches of the SSM (i.e. different knowledge views of the model). In the case of test plan branch, each test case/procedure can be linked to EUT requirements for which an EUT is under verification allowing an automatic handling of the EUT verification based on the built-in confirmation body logic of the test plan procedure. The whole modeling approach (EUT interfaces and test artifacts) leads to a deployment of common knowledge implemented as an overall SSM defined in terms of a test environment logically including three main branches: test infrastructure (TEST_INF), EUT and the test plan (TEST_PLAN) as depicted in figure 6. TEST_ENV TEST_INF EUT TEST_PLAN Linked to EUT Interfaces Linked to EUT requirements Execution of activities defined in the whole SSM can be formally traced and logged as tests results which, if produced according to a common schema, can ease the test evaluation (post-analysis) process and EUT interfaces and requirements verification Figure 7 : Test Knowledge Each branch includes knowledge and activities specific to a domain of responsibility in the test environment with each domain making references to other SSM branches through references of system elements activities when interdomain interaction is needed. Figure 8 : Test Elements and Artifacts Modeled as System Elements Figure 8 reports a basic example of the contents of each branch. In the case of TEST_PLAN the following modeling guidelines can be put in place: Each test area (functional, performance, reliability, etc) can be modeled as a root SE as well as each test case present in a test area; Each test procedure in a test case is implemented as a PLUTO procedure linked to EUT requirements; Each test procedure logical step can be modeled as PLUTO step mapping the test case procedure logical subgoals. Test assertion (verification logic) of each test procedure can be modeled in the PLUTO procedure confirmation body; 6

7 Test data sets required for a test can be modeled either as actual reporting data defined in the SSM or as references to external data items (e.g. data files available in a external repository). Applicability of modeling has benefits on the consistency of the production of testing elements during a system development process. The two main team roles involved in the testing activities can refer to a common modeling approach: the EUT designer (architect or developer) responsible for design and implement the element and provide a model of the related relevant test interfaces; the tester (AIT manager) in charge to design on one side the test plan (test plan model) according to requirements and EUT model and to model the test bed (test conductor, simulators, etc..); Another fundamental benefit is the possibility to re-use branches of the SSM definition across test environments. For example the definition of a test infrastructure and some sub-elements of the EUT can be stable and reused during different test phases. IV. AIT Infrastructure for System Software Applications In order to verify and tune the approach described in the previous sections a test infrastructure has been deployed capable to support the approach in its main concepts and functionality. The infrastructure does not include a number of functionality like test session management and automatic documentation production, since the major goal of the system is to provide support for the fundamental aspects of AIT activities such as test plan definition, test execution and test results analysis. The overall architecture is reported in figure 9: Management Interface (jmx-rmi) Test Definition Test Infrastructure Middleware I/F Specific (jmx-rmi) TIM I/F TI TI I/F I/F Agent Agent Agent Specific /F Specific Specific EUT EUT I/F I/F Test Execution Test Conductor/Executor Test Procedures Scheduling and Execution Runtime SSM (complete test environment model) Specific EUT I/F Specific Interface Test Infrastructure Middleware I/F Specific (jmx-rmi) TIM I/F TI TI I/F I/F Agent Agent Agent Specific EUT I/F Specific Specific EUT EUT I/F I/F (jms) EUT Requirements Definition Requirements Management Test Analysis and Issues Tracking General EV (jms) Issues Tracking System General EV (jms) Test Environment Editors & Consistency Checker Message Bus (jms) EUT Interfaces Definition Test Results Viewer / Analyzer Recorder (jms) Test Infrastructure Middleware Test Environment & Test Plan Definition (static SSM) E-70-31/32 Repository Man I/F Evt I/F Man Man I/F I/F Evt Evt I/F I/F Equipment Sub-element EUT EUT Sub-element Sub-element Test Bed Resource EUT Specific I/F EUT Sub-element EUT Man I/F Evt I/F Man Man I/F I/F Evt Evt I/F I/F EUT Sub-element EUT EUT Sub-element Sub-element Figure 9 : Test Infrastructure Implementation Design The infrastructure provides: Test Definition layer with editors and consistency checker for test environment definition. It is possible to associate requirements to test cases and procedures; Test Execution layer with test conductor/executor capable to execute test procedures and interacting with the EUT elements through the test infrastructure middleware; Test Infrastructure Middleware layer capable to provide all interaction services among the elements involved in the execution and tracing of a test session. Services include supports for the interactions types with EUT reported in section III-D. Furthermore the middleware provide support for a central repository of test results; 7

8 A Test Analysis and Issues Tracking layer for test results inspection and analysis and issue tracking system for formal observations/problem report tracking as resulting from test results analysis. As the infrastructure must be capable to support the modeling approach with a high degree of flexibility and scalability, a number of design guidelines have been put in place: Manager-agent paradigm where the test executor (manager) interacts with one or more agents in charge to provide management and events interface with the EUT and all relevant test bed elements (e.g. emulators); Agents based on a strong component (plugin) architecture with a generic set of components common to all agents and plugins implementing specific EUT interfaces specifications; Possibility to have a direct interface between the tests executor and the EUT in cases where the reporting data flow from EUT to be managed by the test conductor is demanding and a close loop with the test procedure is needed. In this respect the test executor must include the same plugin approach to be put in place for the agent design; Hierarchical and centralized test data definition including space system model repository (modeling test artifacts, test infrastructure and EUT interfaces) with links to EUT requirements and test data sets; Possibility to decouple the events traffic (potentially resources consuming) from the EUT management interface; Centralized storage repository for all events generated during a test; Common structure event for all test environment generated events, with automatic mapping performed by agent between the specific structure of events generated by the EUT and the common structure format. F. Test Environment Definition The test environment definition tools are required to provide all features for the editing and maintenance of all test items and artifacts with the major requirement to maintain consistency across all items belonging to different data definition domain (system elements, procedures, requirements, etc..). A major requirement is the need to associate test procedures with EUT requirements such as it is possible to determine requirements verification by correlating test procedure, test results and requirements. As in standard projects, requirements are maintained in a dedicated database and since the test environment is defined in the SSM database the association between test procedures and requirements (a 1-to-0,,n relationship) has been achieved through links between the two databases. G. Test Infrastructure Middleware Technology A number of technology frameworks have been adopted as infrastructure middleware and agent technology. These are: JMX [3] (Java Management Extension) and JMS [4] Java Message Service. JMX is a Java framework technology specifically designed for remote resources management with a formal customization approach through the concept of Management Bean (MBean) development. An MBean is a Java specification defining services and data associated to a resource to be managed by a remote application (client), transparently with respect to the resource interface logic and behavior. A remote management application interact with the agent (though a common adapter protocols) while the agent instantiates the interface towards the resource to be managed. The customization of a JMX management infrastructure is performed through the development of the required MBean(s), with each MBean implementing the instrumentation for the management of a resource. In the case of the test infrastructure each MBean is mapped to an EUT (or in general a test infrastructure element) interface and in modeling terms the MBean maps a system element. JMX is used in combination with the Java Message Service (JMS) which provides a standard Java-based interface to message services. Messaging services are classified into different models that determine message producer consumer interactions. The most common messaging models are: Publish-Subscribe messaging, Point-To- Point messaging, Request-Reply messaging. In the test infrastructure the Publish-Subscribe messaging is used, which allows asynchronous interaction between producer and consumer through the availability of a message bus implemented as persistent resource (topic). The availability of such bus (maintained by a dedicated server) allows to deploy a message bus infrastructure. The central concept in such model is that any message produced on the bus can be generated independently whether there are any consumers for that message allowing a decoupling from producer and consumer (the consumer can consume messages in different time frames the messages are produced). 8

9 In our test infrastructure this mechanism is used to support messages (i.e. events used as test results) production and storage without imposing constraints at producer side as it would be in the case of a point-to-point connection with the consumer. H. Agent The agent is implemented as java application using strong components design approach with the possibility to reference the required components upon configuration when required. Management Interface JMX -RMI Common Structure Event JMS Common Structure Event The agent is deployed with an agent configuration defining: - the MBeans to support - the SE definition associated to each Mbean - part of the SE definition which EU events needs to be forwarded to the middleware via JMX-RMI JAVA VM Agent JMX-RMI JMS infrastructure Middleware Interface A sampler for each MBean can be present with the functionality to poll the EUT interface for data retrieval to be forwarded to the test executor EUT Sampler Mbean Server Dynamic MBeans MBean with event Message Adapter Common Structure Event are generated according to the EUT events structure Agent Layer Instrumentation Layer generated by the EUT are transformed into a common structure event Figure 10 : Test Infrastructure Agent As summarized in figure 10, the agent is made up of generic components and specialized components according to EUT (or other test infrastructure element) management interface requirements. The specialized components are JMX MBeans components providing the specialized support for managing a specific EUT interface. Figure 11 presents a possible mapping between agents / MBean with system elements and EUT interface definition reported in the space system model Figure 11 : System Elements Mapping to Agents Configuration of each agent (in terms of which MBean to load hence which SE to support with respect to the SSM definition) is performed through the definition of XML files. Part of the agent functionality is the possibility to have a data sampler which allows to poll EUT according to the interaction type Data Poll as described in section III-D. Each sample can be checked for change with respect to the previous sampling and forwarded to the test executor accordingly (it is also possible to forward all samples regardless whether a change has been detected). The usage of sampling allows avoiding to implement the polling mechanism at test executor side, with benefits of test infrastructure resources usage. 9

10 In the case of events management, the agent (through a dedicated customizable component) is capable to translate the events generated by the EUT (compliant to the EUT events structure) with a common structure event. I. Test Executor The Automatic Schedule Execution (ASE) system is used ECSS-E compliant application. ASE has been used as PLUTO execution kernel component for the implementation of the systems mentioned in section II.C. The system includes: procedure definition, validation and execution capability. It implements a runtime representation of the space system model (RT-SSM) as the components instantiating the SSM and providing the external world interaction capabilities on behalf of the procedures execution environment. The runtime space system model implements a components based design, allowing to customize the RT-SSM for different external element interactions through the concept of plugins. In the case of the test infrastructure as here described, the RT-SSM has been equipped with a dedicated plugin, capable to interact with the test infrastructure middleware both in terms of protocols used (JMX-RMI, JMS) and data structures processing. Figure 12 provides both the system components layered architecture and the MMI snapshot representation SSM Navigators Figure 12 : ASE Architecture and User MMI System Elements Editors J. Test Results processing and Message Management During a test execution a number of event messages are produced by both the agents and the test executor. can carry in their payload indications of occurrences of special conditions, as well as relevant information of reporting data values. are stored during a test execution session in a test results database for later results analysis. The concept of the common structure event is required to for logging messages originated by EUT into test results repository such that test analysis can be performed without deploying specific processing for different events sources (e.g. different EUT). In the simplification and re-usage of the test infrastructure this is considered a fundamental issue. V. Conclusions As the complexity of space systems increases and the suppliers involved in their development are required to be consistent at system level, applicability of standards is inevitable. ECSS-E and 32 are specifically designed to address the testing and operations of the space system with a high degree of modeling approach allowing building knowledge among their users decoupled from the actual implementation of systems. Testing infrastructure deployed for complex projects like VEGA Launcher EGSE has proved the applicability of such standards in real world is effective but a formalization of how they are used and applied to the whole project artifacts and product life cycle is still driven by the interpretation and needs of the specific project. The paper has tried to highlight the major aspects involved in the usage guidelines of the standards in the system AIT domain mapped to a possible real test infrastructure architecture. It is believed that combination of standards applicability and formalization of their usage can improve: best usage and customization of today s available technology for a test infrastructure deployment; exchange of test plans artifact models among the different actors involved in complex systems development according to a common definition and usage knowledge based approach. AND DIsplays Execution Control Execution Control User Interaction 10

11 It is believed that both aspects should lead to a more effective usage of projects resources and budgets without affecting the required level of expected verification and validation quality. References [1] ECSS-E Test and operations procedure language April-24 th 2006 [2] ECSS-E Ground systems and operations Monitoring and control data definition [3] JMX - [4] JMS