Archiving Low-Cost Mission Data in NASA s Planetary Data System

Transcription

1 Archiving Low-Cost Mission Data in NASA s Planetary Data System Nancy Chanover New Mexico State University PDS Atmospheres Node 8/15/17 LCPM-12 1

2 Why Archive Data? Many of the data are irreplaceable and are national treasures (funded by the tax payers!). There is valuable insight to be gained by analyzing data acquired over long time intervals (e.g. climate changes). Broader perspectives from a wide range of scientists leads to enhanced creativity, innovation, and science return. It s the law!! (Space Act of 1958: NASA must provide for the widest practicable and appropriate dissemination of information concerning its activities and the results thereof ). 8/15/17 LCPM-12 2

3 PDS Background Established in 1989 as an outgrowth of 3 reports following National Academy of Sciences Committee on Data Management and Computation (CODMAC), charted by NASA Guiding principles: Archives should be managed by scientists who USE the data distributed archive with central archive managed for long-term storage Data should be provided in standard and welldocumented formats Archive should be usable by scientists and strive to balance flexibility vs. not changing too often 8/15/17 LCPM-12 3

4 PDS Organization Though the PDS does not fund the production of archive data from active missions, it works closely with project teams to help them design well-engineered products that can be released quickly. Federation of 8 nodes organized based on scientific expertise, target body type, or sensing modality Discipline nodes selected through peerreviewed proposal process Current as of Aug /15/17 LCPM-12 4

5 What Does PDS *DO*? Major PDS Functions Core PDS Data Providers Trans form Ingest PDS Data Management Distribution Trans form Users Improve efficiency and support to deliver high quality science products to PDS Preserve and ensure the stability and integrity of PDS data Improve user support and usability of the data in the archive 8/15/17 LCPM

6 Growth of Planetary Data Archived PDS Archive Growth from U.S. Solar System Research Yes, size matters, but so does variety 6/29/16 4 8/15/17 LCPM

7 PDS Holdings Total volume ~ 1 PB Over 40M data products (from > 500 instruments and > 70 spacecraft) Some missions have few instruments but many data products Key challenge is the variety of data PDS4 enables transparent cross-node accessibility NASA Science Division Planetary Science Heliophysics Astrophysics Earth Science Total Data Holdings (approx.) 1 PB 10 PB 3 PB 12 PB 8/15/17 LCPM-12 7

8 PDS4 Current data standards as of 2012 LADEE and MAVEN were first missions to archive data using PDS4 system 4 fundamental data structures are recognized in PDS4: array (e.g. images, spectral cubes) table (repeating scalar records, binary and/or characters) parseable byte stream (e.g. text files, XML files) encoded byte stream (e.g. PDF files, JPEG images) All data are described using XML labels easier management/searching of metadata 8/15/17 LCPM-12 8

9 8/15/17 LCPM-12 9

10 8/15/17 LCPM-12 10

11 Archiving Aids There is more than one way to generate an archive! Each node has its own favorite set of tools and approaches to get you there: Label generation tools Data manipulation tools Data visualization tools Contact the appropriate node for your science early to discuss how to optimize your archive generation 8/15/17 LCPM-12 11

12 Recent Events Affecting PDS Ongoing development of PDS4 Roadmap activity ( _20jun17.pdf) Increased internationalization of planetary data standards - the PDS is a key driver in the International Planetary Data Alliance New requirements from above in areas of: increased demand/desire for archiving derived data products IT security issues Increased opportunities for the development of SmallSats for solar system exploration! 8/15/17 LCPM-12 12

13 What is the Archiving Process? Data providers PDS 1. Open communication channel with appropriate Node Establish POC, provide letter of support (if needed) Describe types/volume of data to be archived 2. Define personnel and schedule Who will be in charge of archiving activities? Establish data delivery schedule Request representative data samples Begin developing tailored PDS4 XML label templates 8/15/17 LCPM-12 13

14 Data providers PDS 3. Iteratively define metadata and bundle structure Define bundle structure Iterate on tailored XML label templates 4. Documentation & Review Preparation Finalize Software Interface Specification Provide all necessary documentation Provide calibration information Set up peer review of sample archive 8/15/17 LCPM-12 14

15 Data providers PDS 5. Generate archive Resolve liens from peer review Generate complete archive Design/host web site for archive distribution 6. Data acceptance! Provide all data and documentation to PDS Work with PDS on XML Schema and Context collections Review all data for completeness and integrity Certify archive 8/15/17 LCPM-12 15

16 Archiving Timeline 8/15/17 LCPM-12 16

17 Examples of Small Mission Data in the PDS BOPPS (Balloon Observation Platform for Planetary Science) - SBN BRRISON (Balloon Rapid Response for ISON) - SBN VeSpR (Venus Spectral Rocket) - ATM Future opportunities (some under discussion): Lunar IceCube, other lunar CubeSat missions? 8/15/17 LCPM-12 17

18 The PDS is here to help you! Final Thoughts SmallSat data is relevant to the PDS and should be preserved for posterity Contact us if you have archiving questions! 8/15/17 LCPM-12 18