DABYS: EGOS Generic Database System

Size: px

Start display at page:

Download "DABYS: EGOS Generic Database System"

Ruth Bethany Quinn
5 years ago
Views:

SpaceOps 2010 Conference<br><b><i>Delivering on the Dream</b></i><br><i>Hosted by NASA Mars 25-30 April 2010, Huntsville, Alabama AIAA 2010-1949 DABYS: EGOS Generic base System Isabel del Rey 1 and

1 SpaceOps 2010 Conference Delivering on the Dream Hosted by NASA Mars April 2010, Huntsville, Alabama AIAA DABYS: EGOS Generic base System Isabel del Rey 1 and Ramiro Peñataro 2 GMV, Madrid 28760, Spain Vicente Navarro 3 ESA/ESOC, Darmstadt 64295, Germany Various ground systems belonging to ESA ground segments rely on configuration data which are prepared offline with a database management application. This is a common practice for different generic systems, which are customized for a particular mission through the population of a configuration database. Different applications have been developed in the past for preparing and managing the configuration data needed by these systems, each targeted to a different system and/or mission. DABYS is a new generic system dedicated to the management of mission and application configuration data, which standardizes and harmonizes the software used for this specific purpose and the associated operations. It is in particular used for the preparation and management of the operational TM/TC database of ESA missions. I. Introduction ABYS is a new system developed by GMV for the European Space Agency, motivated by the need to D standardize the software used to manage configuration data within ESA ground segments. There are several systems within ESA ground systems infrastructure which rely on configuration data prepared offline with a database management application. This is a common practice for different generic systems, including Mission Control Systems, Simulators and Ground Station Monitoring & Control Systems, which are customized for a particular mission through the population of a configuration database. As a result, several applications have been developed for preparing and managing the configuration data needed by these systems, each targeted to a different system and/or mission. These applications provide similar functionality and are used in a similar way in all cases, although they are based on different technologies and technical approaches. In essence they differ mainly in the type of data being managed in each case (e.g. TM/TC data, GS data, etc). In this context, DABYS has been implemented with the following main objectives in mind: 1) Provide a single solution for configuration data management, which can be used for all existing use cases 2) Improve standardization and harmonization across ground data systems 3) Reduce the cost of developing and maintaining new database management applications DABYS provides a framework for building database management applications, called s (DM), which standardizes both the technology and the operations that support the management of configuration data. Specific s are implemented on top of the framework, by configuration and extension of the framework. The DABYS Framework has two elements: 1) The DABYS SDE, a development environment for implementing s 2) The DABYS Generic (GDM), a generic system that implements the functionality needed by most s, in a generic and data model independent way. The Generic provides a standard set of functions that can be applied to the management of any kind of configuration data, as long as the data can be represented with a relational data model. The implementation of a particular using DABYS consists primarily of the definition of the relational data model that represents the data to be managed, and the configuration of the Generic for 1 DABYS Project Manager, Ground Control Systems Business Unit, idelrey@gmv.es. 2 DABYS Project Engineer, Ground Control Systems Business Unit, jrpenataro@gmv.es. 3 DABYS Technical Officer, Infrastructure Systems Division, vnavarro@esa.int. 1 Copyright 2010 by the, Inc. All rights reserved.

2 such data model. In addition, if the has specific requirements which are not fulfilled by the Generic, it can also be extended with new functionality. This process is illustrated in the following figure. Framework Generic Model Conf. configuration Generic Model Conf. extension Generic Model Conf. Development Environment New Functionality The first created with the DABYS framework is called SCOS-2000 and is dedicated to the management of TM/TC data for Mission Control Systems based on SCOS SCOS-2000 is the Spacecraft Control & Operations System Kernel developed by ESA for monitoring and control of ESA missions (although it is also used now for many non-esa missions as well). SCOS-2000 uses a relational database as the source of TM and TC definitions required for spacecraft monitoring and control. This database is populated from the Spacecraft base provided by the satellite manufacturer and from other sources that provide additional operational data. The SCOS-2000 has been implemented to manage this database, supporting all involved operations. II. Problems and limitations of the current approach As introduced above, at the moment several applications are used for preparing and managing the configuration data needed by different systems. The functionality and operational concept that they have in common are implemented differently in each case, which makes it difficult to take advantage of synergies across domains. The main problems and limitations of the current approach are the following: 1) High maintenance costs: One of the reasons for this is that some of the existing tools were never designed to be extended or easily maintained. In addition, as reuse is not possible across domains, the maintenance effort is proportional to the number of existing tools. When a tool is extended with certain functionality, it cannot be incorporated into other tools which would also benefit from it. Similarly, problems that are common to more than one tool, have to be solved separately and repeatedly for each of them. Besides, the knowledge acquired on one tool is not applicable to the others, which also results on a steep learning curve for newcomers. 2) Lack of a standard approach that enforces coherency and good practices: Most of the existing tools support slightly different ways to do the same things, and in some cases rely on good practices and manual procedures that have to be followed by their users. For example, the configuration data are sometimes kept under configuration control by saving copies of the configuration database on specific locations using a naming convention. Furthermore the concepts involved in this operation, like the name of the operations and user inputs involved, are different in each case. 3) Lack of multi-user support: Most of the existing tools provide no multi-user support, although the configuration data is typically accessed by several users for different purposes. It is important to provide a controlled environment where several users can access the data in parallel and have different access rights. 4) Lack of advanced support for change management: Change management is extremely important for managing configuration data, and is generally done in a semi-manual way with existing tools. In particular it is important to be able to support incremental upgrades of the data involving sophisticated merge operations, to be able to keep a log of changes, to keep databases under configuration control, etc. 5) Difficult adaptation to changes in the data model: All existing applications have been designed to work with a certain data model, and need to be modified whenever the data model changes. Since the changes in the data model are actually quite frequent (either due to upgrades of the associated system, or due to mission-specific needs) an easier adaptation to changes in the data model would considerably improve the current situation. 2

3 III. Solutions offered by DABYS The DABYS Generic provides a generic solution and a standardized approach for managing configuration data, which solves all the problems described in the previous section. It supports all the functions which have been identified as typically required for managing any type of configuration data, and it is designed to be easily extended in a number of ways, to support new mission or application specific functionality. The following sections describe the built-in functions supported by the Generic for any data model. A. base lifecycle management Configuration data are stored in databases. There can be just one database, which holds all the data, or more than one, each holding a subset of the whole configuration data, e.g. a separate TM/TC database can be kept for each of the satellites of a multi-satellite mission like Galileo. In any case, the lifecycle of the database or databases needs to be managed consistently. The database lifecycle supported by DABYS involves the following operations: 1) base creation, to start the lifecycle. The database can be created empty or from a certain version of another database. 2) Management of reference versions. The database is kept under configuration control; it is possible to create a reference version at any time, and it is possible to restore any version later if needed. Upon request reference versions can be browsed, compared and merged with the latest version of the database. 3) Management of development branches. base branches can be created to work on a separate copy of the database during a period of time, for different reasons. A development branch will typically be merged with the main branch at some point, or just be deleted. 4) Change logging. A log containing all the changes made to the database is kept during the whole database lifecycle. The log includes the date and time of the change and the user that made it, as well as the task associated to the change, if defined. Tasks represent high level operations which typically involve several changes to the database. DABYS allows users to gain exclusive access to the database for implementing tasks, and to review the changes associated with each task. 5) base deletion, at the end of the lifecycle, when the database is not needed anymore. In fact, the process of populating and maintaining the configuration database during a period of time (e.g. during operations preparation) has a lot of similarities with the process of developing and maintaining software. Therefore, some of the supported functions are based on ideas taken from best practices and tools used for source code configuration control during a software development process. B. User management The population of the configuration database can be a team task, and even if a single user is responsible for editing the data, several other users may need to access the data in read-only mode. Therefore it is important to allow several users to access the database in parallel, and to be able to define the privileges that each user has for reading and writing on each database, and for performing administrative tasks. With DABYS only privileged users can create and manage other users, and perform configuration control tasks such as creating or restoring reference versions of the database. These users can create other users and define their read-only or write rights on the different databases and database tables. C. Import Typically, configuration databases are not populated from scratch. The bulk of the data comes from external data sources, like TM/TC data provided by the satellite manufacturer, or configuration data available from another mission. Therefore, it is necessary to have a means to import data into the database from external data sources. This is a function that depends on the type of data being managed, and the operational context of the Manager. Most s will require their own specific import functions depending on the way the configuration data is provided, e.g. as a set of text files, a single XML file, etc. On the other hand, most database frameworks support the import of data from generic sources such as XML or Comma-Separated Value (CSV) files. The DABYS Generic supports the import from Tab-Separated Value files, in a similar way. In addition, the GDM provides an extension point by design, which facilitates the implementation of new import functions for other types of data sources. 3

4 During import, sometimes it is also useful to be able to perform some operations on the database before and after the import process. The GDM is also prepared to be extended with pre-processing and post-processing functions that can be added to the standard import process. Finally, the import of data normally implies a merge between the existing data and the new data, except for the first import when the database is empty. The GDM provides a lot of flexibility for performing safe incremental imports, reducing the risks of overwriting important data during the process by mistake. D. Export The export is the function needed to extract the database content into external files in a certain format. This is the means by which the data is made available to the system that receives the configuration data. The export requirements of a particular will typically differ depending on what is expected by the system that will receive the data. As with import, the GDM supports the export to Tab-Separated Value files, as a generic format which any DM could use as exchange format, and facilitates the support for new export formats through an extension point. It also supports the addition of pre-processing and post-processing functions that can be used to customize the export process. E. Editing Another functionality which is generally required by any is the capability to edit the database content. The data to be edited must be presented in a user-friendly and structured way, making it possible to edit related data easily and also facilitating the navigation through the different database tables and records. To this end, the GDM provides a generic editing engine which is able to dynamically generate form-based editors from XML files that define the editor layout and content. An editor is a combination of forms and sub-forms which provides a high level view of a set of database tables, and allows their editing and browsing through the following operations: 1) Add, edit and delete records 2) Check the user input and prevent errors at edit time, based on configurable rules 3) Copy and paste records 4) Navigate through the data set 5) Filter the data set 6) Sort the data set 7) Follow a link to another related editor This allows for the creation of a different suite of editors for each, through the definition of the corresponding XML files. F. Reporting The reporting function consists of the capability to generate reports of the data, mainly for visualization, distribution and printing purposes. The GDM provides a generic reporting engine based on the Eclipse Business Intelligence and Reporting Tool (BIRT), which is able to generate reports of the data based on pre-defined report templates. These templates can be graphically defined with an external BIRT tool. The GDM integrates the BIRT functionality for rendering, filtering, printing and exporting the reports. As with editing, the reporting engine allows for the creation of a different suite of reports for each, through the definition of the corresponding report templates. G. Consistency checking The purpose of the consistency checking function is to check the database content against a set of predefined rules, which define one or more checks that shall be verified for the data to be consistent, sometimes involving complex queries. The GDM provides a configurable consistency checker that implements generic check rules which can be applied to the database tables and fields differently depending on the data model and the constraints that need to be verified. The checks to be applied for each data model are defined by configuration. For example, a generic rule defines that the value of a field should be within a certain range. This rule can be applied to a particular data model by specifying the actual field to be checked and the applicable range. The GDM also provides an extension point by design to implement new check rules as needed by each specific, because some rules are very specific to particular data models. 4

5 H. Compare and merge As described above, a will typically manage more than one database instance. There are different scenarios where it is useful to be able to compare and merge two database instances, e.g. to compare two versions of a certain database, or to merge two databases that have been evolving in parallel during a period of time. The GDM can compare any two selected databases, producing a report of the differences at field level. For each compared table, the records existing only on one of the databases, and the records that exist on both but differ in some field value, are clearly highlighted. Another interesting function is the capability to merge the content of two databases into one of them. This operation has the risk of accidentally loosing data if the two databases have data in common, so it is important to be able to control the process. The GDM supports different types of merge algorithms, which can be used for different purposes: 1) Merge of new data only, ensuring that the existing data is not modified. This is useful when the user is only interested in new data, and not in changes to existing data. 2) Merge of new and modified data, overriding the existing data. This is useful when it is known a priori that the new data should take preference over the old data in all cases. 3) Partial merge guided by a configurable merge profile. This is the most flexible approach supported, and it is useful in cases where it is known a priori that only certain parts of the database should be modified by the merge. A merge profile defines how the merge shall be performed: at table level, it is possible to define what to do with new, modified and deleted records, and define filters to ignore records that match some condition; at field level it is possible to define patterns for each individual field so that when the field value matches the pattern, the new value is ignored, either by keeping the original value, or by setting it to a certain value defined in the profile. Once the type of merge is selected, the merge can be performed automatically or interactively. In the latter case the system will provide a summary of the changes to be applied to the user, to be confirmed or discarded individually or in groups. In summary, there is a lot of built in functionality in the Generic that aims at facilitating the merge of databases in a flexible and secure way. IV. Current status and future evolution At the moment DABYS has only been used to implement the SCOS This is currently being used for operations preparation of the Gaia mission, and is expected to be used for other ESA missions in the near future. The feedback from the Gaia project has been very positive, especially in the areas of DB configuration control, DB comparison and multi-user support. Each mission will configure and extend the SCOS-2000 as required in each case, and both the configuration and the new functionality can be easily made available to other missions through the retrofitting of enhancements to the generic SCOS-2000 (through source code merge), or the exchange of configuration files across missions (e.g. editor layouts, report templates, consistency check configurations, etc, can be defined for one mission and then reused by another). In the medium/long term, DABYS is also intended to be used for managing other types of data: 1) The GS will manage Ground Station data used by the Ground Station Monitoring & Control System. 2) The Simulator will manage configuration data used by ESA Spacecraft Simulators. 3) The EGOS will manage the configuration data used by an EGOS system. The implementation of new DABYS s will make it possible to enhance the Generic with new generic functionality. For example the implementation of support for import/export of data in XML format for the EGOS can be a new generic feature to be added to the Generic, which would be readily available to other s by a simple software upgrade. V. Conclusion DABYS has brought a number of benefits to the management of configuration data within ESA ground segments: 1) It provides a standardized and homogeneous approach to the management of configuration data, flexible enough to support different operational scenarios. 5

6 2) It greatly simplifies the development of s, thus reducing their cost. New s can be created by configuration, i.e. defining a relational data model and a set of configuration data which can even be reused from previous missions; Adding functionality is also simple using pre-designed extension points. 3) It improves the maintainability of the s, as they share a common part that can be maintained independently. 4) It provides great adaptability to mission specific needs, as it easily adapts to changes in the data model, it is highly configurable, and it is designed to be extensible. The DABYS SCOS-2000 also provides significant benefits versus previous applications used to manage the SCOS-2000 database, both in terms of functionality and in terms of software quality and maintainability. 6

Operation Preparation Environment (OPEN)

Operation Preparation Environment (OPEN) OPEN, OPEN-CC, an introduction Francois Trifin, ESA/ESOC 21/06/2017 ESA UNCLASSIFIED - For Official Use ESAW 2017 ADM-Aeolus BepiColombo Cluster II Cryosat-2 EarthCare