Pascal Gilles H-EOP-GT Meeting ESA-FFG-Austrian Actors ESRIN, 24 th May 2016
ESA EO GS Operations Concept Evolution: Objectives Mission improvement Improve the availability and facilitate the exploitation of data Better serve a broader user community Improve the overall missions operability, reliability and availability Reduce Costs Sustainably reduce operating cost to meet budget constraints Leverage technological evolution and continuous decrease of HW costs Increase efficiency Standardise, streamline and simplify interfaces to support modular concepts Develop advanced tools supporting functions/processes whenever necessary. Use commercially available (European) products whenever possible Perpetuate and sustain Ensure preservation and of data and associated information in the long term Head of sections Meeting ESRIN 3 rd December 2015 Pag. 2
ESA EO GS Operations Concept Evolution: Core features of the target concept One core EO GS Reference Operations Concept for all ESA and TPM missions Modularity and flexibility: can be tailored as necessary to meet individual mission needs Compatible with historic (ERS/Envisat/TPMs), current (Sentinels, Earth Explorers, TPMs) and future EO missions Definition of the reference operations concept based on the following principles: Limit HW procurement (e.g. rely on IAAS type services) Streamline, simplify and standardize interfaces between functional blocks with a view of minimizing dependencies between those blocks and allow maximum flexibility for their implementation Focus on long term data and associated information preservation, curation and The CORE PDGS shall be one of the elements fuelling the future network of Earth Observation Community Platforms Head of sections Meeting ESRIN 3 rd December 2015 Pag. 3
Data Generation Element 1/2 Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Head of sections Meeting ESRIN 3 rd December 2015 Pag. 4
Data Generation Element 2/2 Data consolidation Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Head of sections Meeting ESRIN 3 rd December 2015 Pag. 5
Data Preservation Element Data consolidation Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Head of sections Meeting ESRIN 3 rd December 2015 Pag. 6
Data Access Element Data As-is dissemination (all products processed systematically) On-the-fly processing (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Data consolidation Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Head of sections Meeting ESRIN 3 rd December 2015 Pag. 7
User Services Element User services User Management (registration, authentication, authorization ) (Guide & Directory, web pages ) Data (Catalogue of data products offering) Data As-is dissemination (all products processed systematically) On-the-fly processing (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Data consolidation Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Head of sections Meeting ESRIN 3 rd December 2015 Pag. 8
Data improvement Element User services User Management (registration, authentication, authorization ) (Guide & Directory, web pages ) Data (Catalogue of data products offering) Data As-is dissemination (all products processed systematically) On-the-fly processing (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Data consolidation Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data improvement CAL/VAL (Calibration files, product validation) Algorithm improvement (new prototypes and IPF) LTPA (QC Datasets and trends Preservation analysis) Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Head of sections Meeting ESRIN 3 rd December 2015 Pag. 9
Coordination Element User services User Management (registration, authentication, authorization ) (Guide & Directory, web pages ) Data (Catalogue of data products offering) Reporting consolidation Mission performance monitoring Configuration management Data consolidation (Changes, incidents, anomalies_) Data improvement CAL/VAL (Calibration files, product validation) Algorithm improvement (new prototypes and IPF) LTPA (QC and trends analysis) Data As-is dissemination (all products processed systematically) On-the-fly processing (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Head of sections Meeting ESRIN 3 rd December 2015 Pag. 10
Main interfaces Reporting consolidation Mission performance monitoring Configuration management User services Data improvement User Management Query/product (registration, list authentication, authorization ) & browse (Guide & Directory, web pages ) Data (Catalogue of data products offering) Data As-is dissemination (all products processed systematically) product & On-the-fly processing retrieve (cache with all required input, e.g lvl0, aux ) Data consolidation K Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) QC CAL/VAL (Calibration files, product validation) Algorithm improvement (new prototypes and IPF) LTPA (QC and trends analysis) Algo-IPF Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings Virtual Research environment (hosted processing) EO Dataset Management All Data related information Knowledge Head of sections Meeting ESRIN 3 rd December 2015 Pag. 11
Operations model ITIL theoretical model EOP Reference model (compatible with QMS) Mapping of each mission on the EOP Reference model Roadmap for Convergence to the EOP Reference model (standard) Homogeneity between missions (efficiency) Quick wins and long term objectives
Procurement Terminology Operations contract: classical contract for an operational facility (e.g. PACs or Stations) where ESA is deploying own HW and SW in the facility and places a contract with the facility owner to operate these systems Service Contracts: contract where the Contractor executes ESA-defined processes for the activities in scope under agreed levels of quality and performance (Service Level Agreement). While ESA is defining and enforcing the appropriate technical requirements, e.g., on equipment compatibility, interfaces, performance, security and reliability, it is the Prime Contractor s responsibility to propose and manage a Consortium and to deploy the necessary means, including adequate infrastructure and competences, to provide the abovementioned services (i.e. ESA will refrain to impose or provide any technological platform or CFIs).
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Master Inventory User Registration Information User management ESA WEB Portal Knowledge Information Package - Knowledge Object - - Documentation - Software - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val User Owned Services Catalogue search Product retrieve Product list & Browse Catalogue Algorithm/IPFs Algorithms Improvement Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) KIP Data Retrieval L1-L2 Data (as-is) Harvesting - Index files - browse Dissemination & QC EO Datasets (PDSC) Management Management - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) - Know ledge Packaging & Publication (7) - Curation and Valorization ( 8 ) Product list Processing Product on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Data Systematic Generation systematic Processing & QC Data Acquisition and Procurement Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
Head of sections Meeting ESRIN 3 rd December 2015 Pag. 15 Spare slides
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User User management Knowledge Information Registration Package ESA - Knowledge Object Information WEB Portal - - Documentation - Software Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val User Owned Services Cold Back-up Archive Catalogue search Functionality: Product retrieve Configuration, Change Algorithms Improvement Management, Problem and - Long Term Cold Archiving (2b) Product list & - Transformation, Browse Catalogue Algorithm/IPFs (3) Non-conformance Management, Actions Management. KIP Past implementation: most of the functionality of the Configuration Harvesting EO Datasets (PDSC) Management - Index files - browse Management Database was delegated to each separate & QC - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & operations contract. Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification Dissemination Effective (6) - Know ledge Packaging & Publication (7) L1-L2 Data (as-is) Data Retrieval end-to-end configuration control was difficult. - Curation and Valorization ( 8 ) Processing on-the-fly Final product Data Systematic Generation Product list processing Product systematic Processing & Data Flow Intended implementation: input product - EO Data Sets Reception/ Delivery ( 1, Storage cache Implementation of an end-to-end 3) L0 & AUX Data (Restricted Access for s or on-demand Data Acquisition and processing) Procurement core ground segment activities. Virtual Research Environment Knowledge Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Configuration Service procured from Industry, and covering all Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User Registration Information Master archive User management ESA WEB Portal Knowledge Information Package - Knowledge Object - - Documentation - Software User Owned Catalogue search Master Services Archive Product retrieve Functionality: Preservation of Algorithms all Improvement ESA managed Product list & Browse Catalogue Algorithm/IPFs data records within a Managed Archive. Cal/Val KIP Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Past implementation: the storage part of the functionality was Harvesting EO Datasets (PDSC) Management - Index files - browse Management provided as part of the Operations Contracts with the PACs & QC - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & (mainly ERS and ENVISAT Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification Dissemination data) and the Stations (mainly Third (6) - Know ledge Packaging & Publication Data Retrieval L1-L2 Data (as-is) (7) - Curation and Valorization ( 8 ) Part Missions). Processing on-the-fly Final product Data Systematic Generation Product list processing Product input product systematic Processing & QC Data Flow - EO Data Sets Reception/ Delivery ( 1, Storage cache Intended implementation: 3) L0 & AUX Data (Restricted Access for functionality intended to be provided s or on-demand Data Acquisition and as part of a service contracts hosting processing) Procurement the ESA managed data records for preservation purposes. Virtual Research Environment Knowledge Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Master Inventory User User management Registration ESA WEB Portal Information Cold Backup archive Knowledge Information Package - Knowledge Object - - Documentation - Software - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) User Owned Catalogue search Cold back-up Services Product retrievearchive Functionality: provide a Algorithms Disaster Improvement Product list & Browse Catalogue Algorithm/IPFs Recovery element, not providing continuity of service,. for all ESA managed data records Cal/Val KIP Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Data Retrieval L1-L2 Data (as-is) Harvesting - Index files - browse Past implementation: this functionality was not & QC provided. Dissemination Processing on-the-fly Data Systematic Generation Product list Intended implementation: functionality systematic Processing intended Final product & to be Product processing input product QC Storage cache provided as part L0 & AUX of Data (Restricted service Access for contracts, relying on a s or on-demand Data Acquisition and processing) Procurement specific dedicated ESA infrastructure. EO Datasets (PDSC) Management Management - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) - Know ledge Packaging & Publication (7) - Curation and Valorization ( 8 ) Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User User management Knowledge Information Registration Package ESA - Knowledge Object Information WEB Portal - EO Datasets - Documentation (PDSC) Management - Software User Owned Catalogue search EO Datasets Services Product retrieve Management Functionality: Product list & Browse Catalogue Algorithm/IPFs Preservation of all Information and knowledge related to ESA owned data records ensuring future data curation capability. Harvesting - Index files - browse & QC Cal/Val Algorithms Improvement Past implementation: Disseminationthis functionality was not L1-L2 Data (as-is) Data Retrieval provided. Processing on-the-fly Final product Data Systematic Generation Product list processing Product input product systematic Processing & QC Storage cache Intended implementation: L0 & AUX Data (Restricted Access for functionality intended s or on-demand Data Acquisition and to be provided as part of Service Contracts, processing) Procurement KIP EO Datasets (PDSC) Management Management - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) - Know ledge Packaging & Publication (7) - Curation and Valorization ( 8 ) Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Master Inventory User Registration Information Master Inventory User management ESA WEB Portal Knowledge Information Package - Knowledge Object - - Documentation - Software - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) User Owned Catalogue search Master Services Inventory Product retrieve Functionality: Provide a single Algorithms Improvement view of Product list & Browse Catalogue Algorithm/IPFs all ESA managed data sets, wherever find themselves within the core ground segments. Cal/Val KIP Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Harvesting EO Datasets (PDSC) Management - Index files - browse Management Past implementation: partially provided for the data & QC - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & residing in the multi-mission Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification Dissemination infrastructure, through the (6) - Know ledge Packaging & Publication Data Retrieval L1-L2 Data (as-is) (7) - Curation and Valorization ( 8 ) Multi Mission Master Catalogue functionality. Processing on-the-fly Final product Data Systematic Generation Product list processing Product input product systematic Processing & QC Data Flow - EO Data Sets Reception/ Delivery ( 1, Storage cache Intended implementation: 3) L0 & AUX Data (Restricted Access for Functionality intended to be s or on-demand Data Acquisition and provided though a specific SW development, processing) Procurement operated as part of Service Contracts. Virtual Research Environment Knowledge Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User Data Generation Registration Information User management ESA WEB Portal Systematic Processing Functionality: User Owned Services Catalogue search Generation, after acquisition, of all Product retrieve Product list & systematic processed Catalogue Browse data Knowledge Information Package - Knowledge Object - - Documentation - Software Algorithm/IPFs Cal/Val Algorithms Improvement KIP Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Past implementation: most of the Harvesting functionality was provided as part of - Index files - browse the Operations Contracts with the Dissemination stations (systematic products. L1-L2 Data (as-is) Data Retrieval & QC EO Datasets (PDSC) Management Management - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) - Know ledge Packaging & Publication (7) - Curation and Valorization ( 8 ) Processing on-the-fly Final product Product list Product Intended implementation: processing input product Storage cache functionality intended to be provided L0 & AUX Data (Restricted Access for s or on-demand processing) as part of the Operations Contracts with the stations, as all the new missions are based Virtual on Research Environment systematic production of all products Federated Data / products (operational QC is also part of this functionality). EO Collaborative Element & thematic exploitation platforms Data / products Access Data Systematic Generation systematic Processing & QC Data Acquisition and Procurement Processing hosting & Support to Scientific Exploitation Disasster Recovery Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) Knowledge Knowledge information Sources Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3)
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User User management Registration Information Product list & Browse ESA WEB Portal Reprocessing Functionality: User Owned Services Catalogue search Generation of all reprocessed Product retrieve datasets Catalogue Knowledge Information Package - Knowledge Object - - Documentation - Software Algorithm/IPFs Cal/Val Algorithms Improvement Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) KIP Past implementation: most of the Harvesting functionality was provided as part - Index files - browse of the Operations Contracts with Dissemination the stations and the PACs L1-L2 Data (as-is) Data Retrieval (reprocessed data or on- Processing on-the-fly Final product Product list processing Product processed data). input product The functionality has been Storage gradually cache L0 & AUX Data (Restricted Access for s or on-demand processing) moved to a service contract for heritage and Earth Explorers data. Virtual Research Environment & QC Data Systematic Generation systematic Processing & QC Data Acquisition and Procurement EO Datasets (PDSC) Management Management - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) - Know ledge Packaging & Publication (7) - Curation and Valorization ( 8 ) Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) Knowledge Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Federated Data / products Intended implementation: functionality EO Collaborative intended to be Element & thematic exploitation platforms provided as part of service contracts.. Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
PDGS Operations concept Functional Elements USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User User management Data Improvement Registration Information ESA WEB Portal Knowledge Information Package - Knowledge Object - - Documentation - Software Functionality: Instrument performance User Owned Services Catalogue search assessment, Mission quality control and Product retrieve Product list & calibration/validation, Catalogue Browse algorithm improvements, Algorithm/IPFs development of new processors. Past implementation: most of the functionality for Harvesting - Index files - browse heritage data (ERS/Envisat, TPMs) and Earth & QC Dissemination Explorer data was provided as part of the service L1-L2 Data (as-is) Data Retrieval contracts and through the ESLs, often based on Processing on-the-fly Final product Data Systematic Generation Product list Product infrastructure provided processing input product by ESA. Functionality systematic Processing & QC implemented through Service contracts for Storage cache L0 & AUX Data (Restricted Access for s on-demand Data Acquisition and processing) Sentinel (Mission Performance Centres MPC). Procurement Intended implementation: Virtual Research Environmentfunctionality mainly to Disasster Recovery Federated Data / products (DISCs, potentially based on tools like the Collaborative Data / products Access Mission EO Platforms - s). No foreseen Processing hosting & Element & thematic exploitation platforms Support to Scientific Exploitation be provided by enlarged service contracts changes for Sentinels. Cal/Val Algorithms Improvement KIP EO Datasets (PDSC) Management Management - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) - Know ledge Packaging & Publication (7) - Curation and Valorization ( 8 ) Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) Knowledge Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Knowledge information Sources Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3)
PDGS Operations concept Functional Elements User Owned Services USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User helpdesk, ESA Web Portals and Data Master Inventory User Registration Information Catalogue search Product retrieve Product list & Browse User management ESA WEB Portal Catalogue Knowledge Information Package - Knowledge Object - - Documentation - Software Documentation KIP - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val Functionality: EO helpdesk, EO Order desk, EO Cold Back-up Archive on-line portal, Algorithms Improvement EO-portal, GEO-Portal, Disaster - Long Term Cold Archiving (2b) - Transformation, Algorithm/IPFs (3) Charter Portal Data Retrieval L1-L2 Data (as-is) Harvesting - Index files - browse Dissemination Past implementation: functionality partially EO Datasets (PDSC) Management implemented on ESA infrastructure and Management & QC - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & operated as part of a Cooperation series ( 5 ) of different service - Appraisal, MGM & E2 E Certification (6) - Know ledge Packaging & Publication contracts. (7) - Curation and Valorization ( 8 ) Product list Processing Product on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Data Systematic Generation systematic Processing & Master Archive QC Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) regrouped and provided as two service Data Acquisition and Procurement Intended implementation: functionality contracts. - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
PDGS Operations concept Functional Elements User Owned Services Product list User Registration Information Catalogue search Product retrieve Data Retrieval Product list & Browse L1-L2 Data (as-is) Processing Product User management ESA WEB Portal Catalogue Harvesting - Index files - browse Dissemination on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Knowledge Information Package - Knowledge Object - - Documentation - Software Data USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Algorithms Improvement Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val catalogue of data products available to the Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Algorithm/IPFs KIP Past implementation: functionality implemented on ESA infrastructure and operated as part of EO Datasets (PDSC) Management service contracts. Many separated catalogues Management & QC - Docum ents/ SW Repository ( 2 b) - ESA EO Preservation Scenario & exist, sometimes providing overlapping services Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) - Know ledge Packaging & Publication to the users. (7) - Curation and Valorization ( 8 ) Functionality: provide to the user to the users Data Systematic Generation systematic Processing Intended & implementation: functionality Master to Archivebe QC Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) - Long Term Archiving (2a) implemented on ESA infrastructure and - Transformation, Extraction, Data Acquisition and Delivery (3) Procurement operated as part of Service Contracts. Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
PDGS Operations concept Functional Elements User Owned Services USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Product list User Registration Information Catalogue search Product retrieve Data Retrieval Product list & Browse L1-L2 Data (as-is) Processing Product User management ESA WEB Portal Catalogue Harvesting - Index files - browse Dissemination on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Knowledge Information Package - Knowledge Object - - Documentation - Software Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val direct retrieval of products that are already generated, or seamless on-the-fly production and subsequent Cold Back-up Archive retrieval. Algorithms Improvement - Long Term Cold Archiving (2b) - Transformation, Algorithm/IPFs (3) Past implementation: direct retrieval of products KIP on the mission providing many different (and sometimes EO Datasets (PDSC) Management confusing) opportunities for the user to retrieve the data. Management & QC - Docum ents/ SW Repository ( 2 b) On- data processing - ESA EO and Preservation Scenario retrieval & was provided Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) through a cumbersome data - Know ledge ordering Packaging & Publication process involving (7) - Curation and Valorization ( 8 ) Data Access Functionality: allow of the data to the users, for functionality implemented on ESA and non-esa depending several elements of the ground segment (user services, Data Systematic Generation archive, processing systems, dissemination, etc ) and systematic Processing & Master Archive QC shall be replaced in the future Data Flow by an automatic process - EO Data Sets Reception/ Delivery ( 1, 3) - Long Term Archiving (2a) - Transformation, Extraction, based on On-the-fly production. Data Acquisition and Delivery (3) Procurement Federated Data / products EO Collaborative Element & thematic exploitation platforms Virtual Research Environment Data / products Access Intended implementation: functionality to implemented on virtual infrastructure procured by Knowledge ESA through service contracts and providing product as-is retrieval capability, Disasster Recovery and on-the-fly processing and retrieval capability in an Knowledge information identical manner for the users. Sources Processing hosting & Support to Scientific Exploitation
PDGS Operations concept Functional Elements User Owned Services User Registration Information Catalogue search Product retrieve Product list & Browse User management ESA WEB Portal Catalogue Virtual Research Environment USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Knowledge Information Package - Knowledge Object - - Documentation - Software Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Functionality: Provide hosted processing capability to specific Cal/Val users (e.g. implementing a for a DISC). Cold Back-up Archive Algorithms Improvement Past implementation: functionality partially implemented as - Long Term Cold Archiving (2b) - Transformation, Algorithm/IPFs (3) part of the G-POD platform. KIP Data Retrieval Product list L1-L2 Data (as-is) Processing Product Harvesting - Index files - browse Dissemination on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Intended implementation: functionality to be implemented on EO Datasets (PDSC) Management virtual infrastructure procured by ESA through service Management & QC - Docum ents/ SW Repository ( 2 b) contracts and providing hosted - ESA EO processing Preservation Scenario & capability, in Cooperation ( 5 ) - Appraisal, MGM & E2 E Certification (6) particular benefitting from the - Knowpresence ledge Packaging & Publication on the same platform (7) - Curation and Valorization ( 8 ) of the data hosted for the data functionality. Data Systematic Generation systematic Processing & QC Data Acquisition and Procurement Data Flow - EO Data Sets Reception/ Delivery ( 1, 3) Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources
Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 28