DLR Online Data Access Lessons Learnt. Torsten Heinen, Stephan Kiemle

Transcription

1 DLR Online Data Access Lessons Learnt Torsten Heinen, Stephan Kiemle

2 DFD Merging technical and scientific expertise Payload Data Ground Segment Applications and Services Acquisition Processing Data- Management Ecosystems, Environment Climate, Atmosphere Security, Disaster Mapping Ensuring access and log-term preservation to remote sensing data and geoinformation Innovation thru product-oriented research, development, and value adding Research, technology, products, and services for the benefit of society serving scientific, industrial, governmental, and public needs DLR Online Data Access Lessons Learnt, ODA Workshop Slide 2 Folie 2

3 DLR in GMES FP7 Core Services co-lead Key partner partner partner Key partner Direct action MACC key partner DLR Online Data Access Lessons Learnt, ODA Workshop Slide 3 Folie 3

4 5) 4) German Contributions to GMES Space Segment Sentinel 1: SAR Imaging: SRTM TerraSAR-X TanDEM-X (Oct 2009) Sentinel 2: Superspectral Imaging: BIRD RapidEye 1) IRS-P6; ResourceSAT EnMAP ( 2012) Sentinel 2b: Very High resolution optical mapping: CartoSAT-1 IKONOS, GeoEye-1 2) WorldView-1, WorldView-2 (Sept 2009) 3) Sentinel 3: Ocean Monitoring: EnMAP ( 2013) DLR partner/ commercial distributors Sentinel 4 & 5 atmospheric monitoring: GOME Sciamachy Earth System understanding: Grace, Champ 1) RapidEye science data distribution by DLR 2) Europ. Licence by Telespazio; IKONOS operations by EUSI/DLR in OP GeoEye-1 processing by EuroMap/DLR in NZ 3) Europ. Licence by EUSI; Acquisition and processing by EUSI/DLR in OP (est. Q1/10) 4) By EOX, Austria DLR Online Data Access Lessons 5) tbd; Learnt, for DLR ODA Science Workshop data Slide 4 Folie 4

5 German Remote Sensing Data Center Missions Public & Science EO Missions 1986ff NOAA / AVHRR NOAA 1984 ff Landsat-5 ESA/USGS Aktivny AIP, IZMIRAN 1991 ff Meteosat / MSG EUMETSAT ERS-1 ESA APEX / MAGIOS AIP, IZMIRAN OKEAN NPO Planeta JERS JAXA / ESA 1995 ff ERS-2 ESA IRS-P3 /MOS DLR, ISRO MOMS / Priroda DLR, Russia Landsat-7 ESA CORONAS-I IZMIRAN, AIP CORONAS-I IZMIRAN, AIP TUB-,MarocSAT TU Berlin 2000 ff Champ national, GFZ 2001 ff BIRD DLR 2001 ff Terra / Aqua NASA/ESA 2002 ff Grace 1/2 national, GFZ / JPL 2002 ff Envisat ESA / national 2006 ff ALOS JAXA / ESA 2007 ff METOP-A EUMETSAT 2011 ff EnMAP DLR 2011 ff Sentinel-1 ESA 2012 ff Sentinel-2 ESA National & PPP Missions 1994 SIR-C/X-SAR NASA & DLR 2000 SRTM DLR/companies 2007 ff TerraSAR-X DLR / InfoTerra 2008 ff Rapid Eye 1..5 Rapid Eye AG** 2009 ff TanDEM-X DLR / InfoTerra 2011 ff EnMAP DLR Support for commercial partners IRS-IC ISRO / EuroMap IRS-1D ISRO / EuroMap 2004 ff IRS-P6 ISRO / EuroMap 2003 ff Ikonos GeoEye / EUSI OrbView-2 GeoEye 2006 ff IRS-P5 Cartosat ISRO / EuroMap 2008ff GeoEye-1 GeoEye/Telespazio 2009ff CartoSat-2 ISRO / EuroMap ** only science support * launch dates black: mission completed or no more G/S activities red: in operation blue: in preparation DLR Online Data Access Lessons Learnt, ODA Workshop Slide 5 Folie 5

6 Various Characteristics of Data Sets for Online Access Land/sea surface temperature NOAA AVHRR low res., thematic, time series, daily/weekly/monthly products Digital elevation models Atmospheric trace gas concentration SAR image SRTM X- SAR, TanDEM-X MetOp GOME-2 TerraSAR-X high res., thematic, 1 mio scenes low res., nrt (one scene every 3 min), daily composites, multiple layers, time series high res.-very high res., loose scenes, time series (Water) masks Derived various res., on-demand generation Multi-level, multi-domain and multi-application datasets DLR Online Data Access Lessons Learnt, ODA Workshop Slide 6 Folie 6

7 Various Characteristics of Primary Data Access Browse and evaluate datasets with dynamic visualisation supporting multichannel RGB mapping, histogram functions, custom colortables and identification of data values Access and retrieve geophysical values of dataset with Conditional filter (only if pixel value in (other) dataset has certain property, e.g. quality, error, cloud) Temporal filter (time instant, time span) with specific aggregation type (means of all values vs. retrieval of all datasets vs. newest dataset) Spatial filter (spatial subset of dataset) Band filter (access specific channel of multispectral dataset) Retrieve multiple datasets and optionally combine them with function (add, subtract, ) Retrieve single (full) datasets without transformation Access related properties of datasets, e.g. footprint geometry and metadata documents Asynchronous access (pull or push) for complex datasets / filters DLR Online Data Access Lessons Learnt, ODA Workshop Slide 7 Folie 7

8 Geospatial Data Access Service (GDAS) Service-Oriented dissemination of remote sensing products based on OGC web services in addition to traditional FTP delivery Standardized access to geospatial core datasets such as boundaries, landmask, infrastructure, elevation for analysis, processing and visualisation Bridge the gap between EO and GEO domain to allow exploitation of datasets through GIS DLR Online Data Access Lessons Learnt, ODA Workshop Slide 8 Folie 8

9 GDAS Interoperable Service Infrastructure Provides standardized web services (OGC) for accessing geospatial datasets and remote sensing products Enables interoperable data access with different software tools, web portals and national & international Spatial Data Infrastructure Programs Provides geospatial core datasets (boundaries, land / seamask, infrastructure, elevation) for value-adding geoinformation chain Extendible platform supporting rapid service instanciation for new missions, projects and portals Missions, Core Data, Service Provider, Projects, DLR Online Data Access Lessons Learnt, ODA Workshop Slide 9 Folie 9

10 Use of Service Infrastructure through DLR portals C-AF Geoportal hyperlinks to portal web pages C-AF Portal WDC-RSAT GeoVIS ZKI EOWEB Web Portals Web Services service access Level 3 and above Services (partitioned according to GMES domains + core datasets) Atmosphere Marine Land Emergency Core Level 2 Mission Services (partitioned according to sensor types, types and granularity tbd.) SRTM ENMAP TDM DLR Online Data Access Lessons Learnt, ODA Workshop Slide 10 Folie 10

11 Lessons learnt (1) Infrastructure Data specific policies, property rights and access restrictions increase infrastructural security needs IT infrastructure challenged through VHR / hyperspectral datasets Virtualization helps to separate development and IT teams and enhance scalability Development Different service implementation (OSS and COTS) behave differently Enterprise Service Integration: Avoid or at least abstract from proprietary (service) APIs Non-functional requirements should be taken seriously ( a slow service is a bad service! ) Multiple ways of accessing multidimensional coverages (x,y,z vs. band vs. elevation) Layer / coverage hierarchies not support by some server and clients (EO use cases require differentiation between collections and products) DLR Online Data Access Lessons Learnt, ODA Workshop Slide 11 Folie 11

12 Lessons learnt (2) Operation Outreach Complex licensing and security constraints (national laws and data policies) Service infrastructure roles and organizational policies need to be defined and allocated (IT maintainance over SW development) Service configuration and maintainance need to be minimised, Trained personel required with GI skills, SQL, Web Services in general OGC concepts in particular Find right granularity of services and group services according to discipline, user communities or (license) constraints Educate and train users and operator of GI concepts and service-oriented access paradigm (differences to file-based access) Demonstrate and show capabilities Discuss different community needs (multi-domain and multi-application requirements) Metadata is important, provide simple tools for creation and allocate and train personnel DLR Online Data Access Lessons Learnt, ODA Workshop Slide 12 Folie 12

13 DLR Online Data Access Lessons Learnt, ODA Workshop Slide 13 Folie 13

14 DLR Online Data Access Lessons Learnt, ODA Workshop Slide 14 Folie 14