Future Core Ground Segment Scenarios

Transcription

1 Future Core Ground Segment Scenarios Pascal Gilles EOP-G Ground Segment Coordination Body Workshop 2015 ESRIN, 24 September2015 Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 1

2 Reasons for Change: Programmatic Programmatic End of in-orbit life of major ESA EO satellite missions (Envisat, ERS ) Budget composition and scales are changing: overall increase over years to come but decrease in ESA member states budgets and increase in EU budgets linked to Copernicus where focus is on provision of data to service providers designated by EC. End of EOEP funded mission operations and transition to LTDP. Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 2

3 Reasons for Change: Industry & Technology Trends Industry & Technology Trends Maturity of industry and technologies; increased appearance of service offerings. Mature IT and communications sector looking for opportunities also in EO. Aggressive stance of Google and appearance of initiatives from other, non-european Google type Cloud Service Providers mainly focussed on data/products. New media and channels for communication with user communities: Cloud, Social media, Link EO data. Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 3

4 Reasons for Change: Operations concept Missions profile From Envisat type missions to Earth Explorers type missions Smaller Focus on science Maturity of instruments requires technical and schedule flexibility Data Management User driven mission planning to systematic acquisition and limited mission planning (e.g. calibration, housekeeping) Systematic processing vs delayed on- processing Cyber Security protect data integrity and avoid data loss. Growing & Evolving customer community Science communities not in focus of currently funded GMES ground segment but called upon nationally to help deliver benefits of GMES. Application and service industry looking for improved to data/products. Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 4

5 ESA UNCLASSIFIED For Official Use AWG - 12th March - Page5 ESA EO GS Operations Concept Evolution: Objectives Improve the availability and facilitate the exploitation of data Better serve a broader user community Improve the overall missions operability, reliability and availability Sustainably reduce operating cost to meet budget constraints Ensure preservation and of data and associated information in the long term Develop advanced tools supporting such functions/processes. Leverage technological evolution and continuous decrease of HW costs Use commercially available (European) products whenever possible Standardise, streamline and simplify interfaces to support modular concepts

6 ESA UNCLASSIFIED For Official Use AWG - 12th March - Page6 ESA EO GS Operations Concept Evolution: Core features of the target concept One core EO GS Reference Operations Concept for all ESA and TPM missions Modularity and flexibility: can be tailored as necessary to meet individual mission needs Compatible with historic (ERS/Envisat/TPMs), current (Sentinels, Earth Explorers, TPMs) and future EO missions Definition of the reference operations concept based on the following principles: Limit HW procurement (e.g. rely on IAAS type services) and focus on Streamline, simplify and standardize interfaces between functional blocks with a view of minimizing dependencies between those blocks and allow maximum flexibility for theit implementation Focus on long term data and associated information preservation, curation and The CORE PDGS shall be one of the elements fuelling the future network of Earth Observation Community Platforms

7 Data Generation Element 1/2 Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data Quality Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 7

8 Data Generation Element 2/2 Data consolidation Data Quality Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data Quality Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 8

9 Data Preservation Element Data consolidation Data Quality Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data Quality Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 9

10 Data Access Element Data As-is dissemination (all products processed systematically) On-the-fly processing (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Data consolidation Data Quality Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data Quality Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 10

11 User Services Element User services User Management (registration, authentication, authorization ) (Guide & Directory, web pages ) Data (Catalogue of data products offering) Data As-is dissemination (all products processed systematically) On-the-fly processing (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Data consolidation Data Quality Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data Quality Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 11

12 Data improvement Element User services User Management (registration, authentication, authorization ) (Guide & Directory, web pages ) Data (Catalogue of data products offering) Data As-is dissemination (all products processed systematically) On-the-fly processing (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Data consolidation Data Quality Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data Quality Data improvement CAL/VAL (Calibration files, product validation) Algorithm improvement (new prototypes and IPF) LTPA (QC Datasets and trends Preservation analysis) Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 12

13 Coordination Element Coordination, Monitoring and User services User Management (registration, authentication, authorization ) (Guide & Directory, web pages ) Data (Catalogue of data products offering) Reporting consolidation Mission performance monitoring Configuration management Data consolidation Data Quality (Changes, incidents, anomalies_) Data improvement CAL/VAL (Calibration files, product validation) Algorithm improvement (new prototypes and IPF) LTPA (QC and trends analysis) Data As-is dissemination (all products processed systematically) On-the-fly processing (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data Quality Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 13

14 Main interfaces Coordination, Monitoring and Reporting consolidation Mission performance monitoring Configuration management User services User Management Query/product (registration, list authentication, authorization ) & browse (Guide & Directory, web pages ) Data (Catalogue of data products offering) Data As-is dissemination (all products processed systematically) product & On-the-fly processing retrieve (cache with all required input, e.g lvl0, aux ) Virtual Research environment (hosted processing) Data consolidation Data Quality Systematic Generation Data Acquisition (or procurement, and including mission planning) Data processing (including partial reprocessing) Data Quality Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 14 K QC Data improvement CAL/VAL (Calibration files, product validation) Algorithm improvement (new prototypes and IPF) LTPA (QC and trends analysis) Algo-IPF Datasets Preservation Data Master Archive (all preserved datasets) Cold back-up (including partial reprocessing) Master inventory All Data Holdings EO Dataset Management All Data related information Knowledge

15 Procurement Terminology Operations contract: classical contract for an operational facility (e.g. PACs or Stations) where ESA is deploying own HW and SW in the facility and places a contract with the facility owner to operate these systems Service Contracts: contract where the Contractor executes ESA-defined processes for the activities in scope under agreed levels of quality and performance (Service Level Agreement). While ESA is defining and enforcing the appropriate technical requirements, e.g., on equipment compatibility, interfaces, performance, security and reliability, it is the Prime Contractor s responsibility to propose and manage a Consortium and to deploy the necessary means, including adequate infrastructure and competences, to provide the abovementioned services (i.e. ESA will refrain to impose or provide any technological platform or CFIs).

16 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Master Inventory User Registration Information User management ESA WEB Portal Knowledge Information Package - Knowledge Object - - Documentation - Software - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val User Owned Services Catalogue search Product retrieve Product list & Browse Catalogue Algorithm/IPFs Algorithms Improvement Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) KIP Data Retrieval L1-L2 Data (as-is) Harvesting - Index files - browse Dissemination & QC EO Datasets (PDSC) Management Management - Documents/SW Repository (2b) - ESA EO Preservation Scenario & Cooperation (5) - Appraisal, MGM & E2E Certification (6) - Knowledge Packaging & Publication (7) - Curation and Valorization (8) Product list Processing Product on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Data Systematic Generation systematic Processing & QC Data Acquisition and Procurement Data Flow - EO Data Sets Reception/Delivery (1, 3) Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

17 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User User management Knowledge Information Registration Package ESA - Knowledge Object Information WEB Portal - Coordination, Monitoring and - Documentation - Software Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val User Owned Services Cold Back-up Archive Catalogue search Functionality: Product retrieve Configuration, Change Algorithms Improvement Management, Problem and - Long Term Cold Archiving (2b) Product list & - Transformation, Browse Catalogue Algorithm/IPFs (3) Non-conformance Management, Actions Management. KIP Past implementation: most of the functionality of the Configuration Harvesting EO Datasets (PDSC) Management - Index files - browse Management & QC Database was delegated to each separate - Documents/SW Repository (2b) - ESA EO Preservation Scenario & operations contract. Cooperation (5) - Appraisal, MGM & E2E Certification Dissemination Effective (6) - Knowledge Packaging & Publication (7) L1-L2 Data (as-is) Data Retrieval end-to-end configuration control was difficult. - Curation and Valorization (8) Processing on-the-fly Final product Data Systematic Generation Product list processing Product systematic Processing & Data Flow Intended implementation: input product - EO Data Sets Reception/Delivery (1, Storage cache Implementation of an end-to-end 3) L0 & AUX Data (Restricted Access for s or on-demand Data Acquisition and processing) Procurement core ground segment activities. Virtual Research Environment Knowledge Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Configuration Service procured from Industry, and covering all Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

18 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User Registration Information Master archive User management ESA WEB Portal Knowledge Information Package - Knowledge Object - - Documentation - Software User Owned Catalogue search Master Services Archive Product retrieve Functionality: Preservation of Algorithms all Improvement ESA managed Product list & Browse Catalogue Algorithm/IPFs data records within a Managed Archive. Cal/Val KIP Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Past implementation: the storage part of the functionality was Harvesting EO Datasets (PDSC) Management - Index files - browse Management provided as part of the Operations Contracts with the PACs & QC - Documents/SW Repository (2b) - ESA EO Preservation Scenario & (mainly ERS and ENVISAT Cooperation (5) - Appraisal, MGM & E2E Certification Dissemination data) and the Stations (mainly Third (6) - Knowledge Packaging & Publication Data Retrieval L1-L2 Data (as-is) (7) - Curation and Valorization (8) Part Missions). Processing on-the-fly Final product Data Systematic Generation Product list processing Product input product systematic Processing & QC Data Flow - EO Data Sets Reception/Delivery (1, Storage cache Intended implementation: 3) L0 & AUX Data (Restricted Access for functionality intended to be provided s or on-demand Data Acquisition and as part of a service contracts hosting processing) Procurement the ESA managed data records for preservation purposes. Virtual Research Environment Knowledge Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

19 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Master Inventory User User management Registration ESA WEB Portal Information Cold Backup archive Knowledge Information Package - Knowledge Object - - Documentation - Software - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) User Owned Catalogue search Cold back-up Services Product retrievearchive Functionality: provide a Algorithms Disaster Improvement Product list & Browse Catalogue Algorithm/IPFs Recovery element, not providing continuity of service,. for all ESA managed data records Cal/Val KIP Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Data Retrieval L1-L2 Data (as-is) Harvesting - Index files - browse Past implementation: this functionality was not & QC provided. Dissemination Processing on-the-fly Data Systematic Generation Product list Intended implementation: functionality systematic Processing intended Final product & to be Product processing input product QC Storage cache provided as part L0 & AUX of Data (Restricted service Access for contracts, relying on a s or on-demand Data Acquisition and processing) Procurement specific dedicated ESA infrastructure. EO Datasets (PDSC) Management Management - Documents/SW Repository (2b) - ESA EO Preservation Scenario & Cooperation (5) - Appraisal, MGM & E2E Certification (6) - Knowledge Packaging & Publication (7) - Curation and Valorization (8) Data Flow - EO Data Sets Reception/Delivery (1, 3) Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

20 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User User management Knowledge Information Registration Package ESA - Knowledge Object Information WEB Portal - EO Datasets - Documentation (PDSC) Management - Software User Owned Catalogue search EO Datasets Services Product retrieve Management Functionality: Product list & Browse Catalogue Algorithm/IPFs Preservation of all Information and knowledge related to ESA owned data records ensuring future data curation capability. Harvesting - Index files - browse & QC Cal/Val Algorithms Improvement Past implementation: Disseminationthis functionality was not L1-L2 Data (as-is) Data Retrieval provided. Processing on-the-fly Final product Data Systematic Generation Product list processing Product input product systematic Processing & QC Storage cache Intended implementation: L0 & AUX Data (Restricted Access for functionality intended s or on-demand Data Acquisition and to be provided as part of Service Contracts, processing) Procurement KIP EO Datasets (PDSC) Management Management - Documents/SW Repository (2b) - ESA EO Preservation Scenario & Cooperation (5) - Appraisal, MGM & E2E Certification (6) - Knowledge Packaging & Publication (7) - Curation and Valorization (8) Data Flow - EO Data Sets Reception/Delivery (1, 3) Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

21 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Master Inventory User Registration Information Master Inventory User management ESA WEB Portal Knowledge Information Package - Knowledge Object - - Documentation - Software - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) User Owned Catalogue search Master Services Inventory Product retrieve Functionality: Provide a single Algorithms Improvement view of Product list & Browse Catalogue Algorithm/IPFs all ESA managed data sets, wherever find themselves within the core ground segments. Cal/Val KIP Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Harvesting EO Datasets (PDSC) Management - Index files - browse Management Past implementation: partially provided for the data & QC - Documents/SW Repository (2b) - ESA EO Preservation Scenario & residing in the multi-mission Cooperation (5) - Appraisal, MGM & E2E Certification Dissemination infrastructure, through the (6) - Knowledge Packaging & Publication Data Retrieval L1-L2 Data (as-is) (7) - Curation and Valorization (8) Multi Mission Master Catalogue functionality. Processing on-the-fly Final product Data Systematic Generation Product list processing Product input product systematic Processing & QC Data Flow - EO Data Sets Reception/Delivery (1, Storage cache Intended implementation: 3) L0 & AUX Data (Restricted Access for Functionality intended to be s or on-demand Data Acquisition and provided though a specific SW development, processing) Procurement operated as part of Service Contracts. Virtual Research Environment Knowledge Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

22 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User Data Generation Registration Information User management ESA WEB Portal Systematic Processing Functionality: User Owned Services Catalogue search Generation, after acquisition, of all Product retrieve Product list & systematic processed Catalogue Browse data Knowledge Information Package - Knowledge Object - - Documentation - Software Algorithm/IPFs Cal/Val Algorithms Improvement KIP Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Past implementation: most of the Harvesting functionality was provided as part of - Index files - browse the Operations Contracts with the Dissemination stations (systematic products. L1-L2 Data (as-is) Data Retrieval & QC EO Datasets (PDSC) Management Management - Documents/SW Repository (2b) - ESA EO Preservation Scenario & Cooperation (5) - Appraisal, MGM & E2E Certification (6) - Knowledge Packaging & Publication (7) - Curation and Valorization (8) Processing on-the-fly Final product Product list Product Intended implementation: processing input product Storage cache functionality intended to be provided L0 & AUX Data (Restricted Access for s or on-demand processing) as part of the Operations Contracts with the stations, as all the new missions are based Virtual on Research Environment systematic production of all products Federated Data / products (operational QC is also part of this functionality). EO Collaborative Element & thematic exploitation platforms Data / products Access Data Systematic Generation systematic Processing & QC Data Acquisition and Procurement Processing hosting & Support to Scientific Exploitation Disasster Recovery Data Flow - EO Data Sets Reception/Delivery (1, 3) Knowledge Knowledge information Sources Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3)

23 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User User management Registration Information Product list & Browse ESA WEB Portal Reprocessing Functionality: User Owned Services Catalogue search Generation of all reprocessed Product retrieve datasets Catalogue Knowledge Information Package - Knowledge Object - - Documentation - Software Algorithm/IPFs Cal/Val Algorithms Improvement Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) KIP Past implementation: most of the Harvesting functionality was provided as part - Index files - browse of the Operations Contracts with Dissemination the stations and the PACs L1-L2 Data (as-is) Data Retrieval (reprocessed data or on- Processing on-the-fly Final product Product list processing Product processed data). input product The functionality has been Storage gradually cache L0 & AUX Data (Restricted Access for s or on-demand processing) moved to a service contract for heritage and Earth Explorers data. Virtual Research Environment & QC Data Systematic Generation systematic Processing & QC Data Acquisition and Procurement EO Datasets (PDSC) Management Management - Documents/SW Repository (2b) - ESA EO Preservation Scenario & Cooperation (5) - Appraisal, MGM & E2E Certification (6) - Knowledge Packaging & Publication (7) - Curation and Valorization (8) Data Flow - EO Data Sets Reception/Delivery (1, 3) Knowledge Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Federated Data / products Intended implementation: functionality EO Collaborative intended to be Element & thematic exploitation platforms provided as part of service contracts.. Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

24 PDGS Operations concept Functional Elements Coordination, Monitoring and USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User User management Data Improvement Registration Information ESA WEB Portal Knowledge Information Package - Knowledge Object - - Documentation - Software Functionality: Instrument performance User Owned Services Catalogue search assessment, Mission quality control and Product retrieve Product list & calibration/validation, Catalogue Browse algorithm improvements, Algorithm/IPFs development of new processors. Past implementation: most of the functionality for Harvesting - Index files - browse heritage data (ERS/Envisat, TPMs) and Earth & QC Dissemination Explorer data was provided as part of the service L1-L2 Data (as-is) Data Retrieval contracts and through the ESLs, often based on Processing on-the-fly Final product Data Systematic Generation Product list Product infrastructure provided processing input product by ESA. Functionality systematic Processing & QC implemented through Service contracts for Storage cache L0 & AUX Data (Restricted Access for s on-demand Data Acquisition and processing) Sentinel (Mission Performance Centres MPC). Procurement Intended implementation: Virtual Research Environmentfunctionality mainly to Disasster Recovery Federated Data / products (DISCs, potentially based on tools like the Collaborative Data / products Access Mission EO Platforms - s). No foreseen Processing hosting & Element & thematic exploitation platforms Support to Scientific Exploitation be provided by enlarged service contracts changes for Sentinels. Cal/Val Algorithms Improvement KIP EO Datasets (PDSC) Management Management - Documents/SW Repository (2b) - ESA EO Preservation Scenario & Cooperation (5) - Appraisal, MGM & E2E Certification (6) - Knowledge Packaging & Publication (7) - Curation and Valorization (8) Data Flow - EO Data Sets Reception/Delivery (1, 3) Knowledge Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Knowledge information Sources Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3)

25 PDGS Operations concept Functional Elements Coordination, Monitoring and User Owned Services USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION User helpdesk, ESA Web Portals and Data Master Inventory User Registration Information Catalogue search Product retrieve Product list & Browse User management ESA WEB Portal Catalogue Knowledge Information Package - Knowledge Object - - Documentation - Software Documentation KIP - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val Functionality: EO helpdesk, EO Order desk, EO Cold Back-up Archive on-line portal, Algorithms ImprovementEO-portal, GEO-Portal, Disaster - Long Term Cold Archiving (2b) - Transformation, Algorithm/IPFs (3) Charter Portal Data Retrieval L1-L2 Data (as-is) Harvesting - Index files - browse Dissemination Past implementation: functionality partially EO Datasets (PDSC) Management implemented on ESA infrastructure and Management & QC - Documents/SW Repository (2b) - ESA EO Preservation Scenario & operated as part of a Cooperation series (5) of different service - Appraisal, MGM & E2E Certification (6) - Knowledge Packaging & Publication contracts. (7) - Curation and Valorization (8) Product list Processing Product on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Data Systematic Generation systematic Processing & Master Archive QC Data Flow - EO Data Sets Reception/Delivery (1, 3) regrouped and provided as two service Data Acquisition and Procurement Intended implementation: functionality contracts. - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

26 PDGS Operations concept Functional Elements Coordination, Monitoring and User Owned Services Product list User Registration Information Catalogue search Product retrieve Data Retrieval Product list & Browse L1-L2 Data (as-is) Processing Product User management ESA WEB Portal Catalogue Harvesting - Index files - browse Dissemination on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Knowledge Information Package - Knowledge Object - - Documentation - Software Data USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Algorithms Improvement Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val catalogue of data products available to the Cold Back-up Archive - Long Term Cold Archiving (2b) - Transformation, (3) Algorithm/IPFs KIP Past implementation: functionality implemented on ESA infrastructure and operated as part of EO Datasets (PDSC) Management service contracts. Many separated catalogues Management & QC - Documents/SW Repository (2b) - ESA EO Preservation Scenario & exist, sometimes providing overlapping services Cooperation (5) - Appraisal, MGM & E2E Certification (6) - Knowledge Packaging & Publication to the users. (7) - Curation and Valorization (8) Functionality: provide to the user to the users Data Systematic Generation systematic Processing Intended & implementation: functionality Master to Archivebe QC Data Flow - EO Data Sets Reception/Delivery (1, 3) - Long Term Archiving (2a) implemented on ESA infrastructure and - Transformation, Extraction, Data Acquisition and Delivery (3) Procurement operated as part of Service Contracts. Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

27 PDGS Operations concept Functional Elements Coordination, Monitoring and User Owned Services USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Product list User Registration Information Catalogue search Product retrieve Data Retrieval Product list & Browse L1-L2 Data (as-is) Processing Product User management ESA WEB Portal Catalogue Harvesting - Index files - browse Dissemination on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Knowledge Information Package - Knowledge Object - - Documentation - Software Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Cal/Val direct retrieval of products that are already generated, or seamless on-the-fly production and subsequent Cold Back-up Archive retrieval. Algorithms Improvement - Long Term Cold Archiving (2b) - Transformation, Algorithm/IPFs (3) Past implementation: direct retrieval of products KIP on the mission providing many different (and sometimes EO Datasets (PDSC) Management confusing) opportunities for the user to retrieve the data. Management & QC - Documents/SW Repository (2b) On- data processing - ESA EO and Preservation Scenario retrieval & was provided Cooperation (5) - Appraisal, MGM & E2E Certification (6) through a cumbersome data - Knowledge ordering Packaging & Publication process involving (7) - Curation and Valorization (8) Data Access Functionality: allow of the data to the users, for functionality implemented on ESA and non-esa depending several elements of the ground segment (user services, Data Systematic Generation archive, processing systems, dissemination, etc ) and systematic Processing & Master Archive QC shall be replaced in the future Data Flow by an automatic process - EO Data Sets Reception/Delivery (1, 3) - Long Term Archiving (2a) - Transformation, Extraction, based on On-the-fly production. Data Acquisition and Delivery (3) Procurement Federated Data / products EO Collaborative Element & thematic exploitation platforms Virtual Research Environment Data / products Access Intended implementation: functionality to implemented on virtual infrastructure procured by Knowledge ESA through service contracts and providing product as-is retrieval capability, Disasster Recovery and on-the-fly processing and retrieval capability in an Knowledge information identical manner for the users. Sources Processing hosting & Support to Scientific Exploitation

28 PDGS Operations concept Functional Elements Coordination, Monitoring and User Owned Services User Registration Information Catalogue search Product retrieve Product list & Browse User management ESA WEB Portal Catalogue Virtual Research Environment USERS DATA ACCESS DATA GENERATION DATA IMPROVEMENT DATA Sets PRESERVATION Knowledge Information Package - Knowledge Object - - Documentation - Software Master Inventory - Inventory of all EO Data Records & Alignment of Master/Cold BUP Archives (4) Functionality: Provide hosted processing capability to specific Cal/Val users (e.g. implementing a for a DISC). Cold Back-up Archive Algorithms Improvement Past implementation: functionality partially implemented as - Long Term Cold Archiving (2b) - Transformation, Algorithm/IPFs (3) part of the G-POD platform. KIP Data Retrieval Product list L1-L2 Data (as-is) Processing Product Harvesting - Index files - browse Dissemination on-the-fly processing Storage cache L0 & AUX Data (Restricted Access for s or on-demand processing) Final product input product Intended implementation: functionality to be implemented on EO Datasets (PDSC) Management virtual infrastructure procured by ESA through service Management & QC - Documents/SW Repository (2b) contracts and providing hosted - ESA EO processing Preservation Scenario & capability, in Cooperation (5) - Appraisal, MGM & E2E Certification (6) particular benefitting from the - Knowledge presence Packaging & Publication on the same platform (7) - Curation and Valorization (8) of the data hosted for the data functionality. Data Systematic Generation systematic Processing & QC Data Acquisition and Procurement Data Flow - EO Data Sets Reception/Delivery (1, 3) Master Archive - Long Term Archiving (2a) - Transformation, Extraction, Delivery (3) Virtual Research Environment Knowledge Federated Data / products EO Collaborative Element & thematic exploitation platforms Data / products Access Processing hosting & Support to Scientific Exploitation Disasster Recovery Knowledge information Sources

29 Ground Segment Coordination Body Workshop 2015 ESRIN 24 September2015 Pag. 29