Town Hall Meeting on the National and International Data Infrastructure. Jim Warren, NIST Lisa Friedersdorf, NNCO

Transcription

1 Town Hall Meeting on the National and International Data Infrastructure Jim Warren, NIST Lisa Friedersdorf, NNCO 1

2 Data Models Today s Approach to Computa.onal Materials Design Experiment Quantum Nano Micro Macro Simula.on Materials w/ Targeted Proper.es

3 What s the purpose of this Town Hall? Get the word out What s happening across government, and a few government funded activities A few extra items on NIST 3

4 MS&T Town Hall: The Materials Data Infrastructure Name Affiliation Topic [Draft] Dr. James A Warren National Institute of Standards and Materials Data Infrastructure: Dr. Lisa E. Friedersdorf Technology National Nanotechnology Current Status and Development Nanotechnology Knowledge Mr. Scott D. Henry Coordination Office ASM International, Computational Infrastructure and the Concept of Lessons Learned from Pilot and Laura Bartolo Materials Data Network Demonstration Projects for Data Kent State University Laying MGI Infrastructure: Dr. David L. McDowell Georgia Institute of Technology Materials Data Format Schemas Materials Innovation Ecosytem Dr. Carelyn E. Campbell National Institute of Standards and and Accelerator Network Data Initiatives of the Center for Mr. William Joost Technology U.S. Department of Energy Hierarchical Materials Data Materials Data for Lightweight Dr. John Allison U. Michigan Materials PRISM and the Materials Dr. Charles Ward AFRL Commons Thoughts on Publishing and other Dr. Stefano Curtarolo Duke University cool stuff AFLOWLIB and Other Quantum Materials Databases We are not well representing the DFT Database efforts 4

5 What s NIST, the USG and International Community doing about Data Carrie Campbell will talk about specific examples in the context of the CHiMaD This is really a helicopter view of the broad issues You can hopefully not understand much of this talk and it shouldn t matter Two issues getting the data in getting the data out Who are we working with, who else is a player?

6 The Issues: What do we Need? 1. Some sort of interface/api that the user interacts with to deposit their information such that (2) is enabled. This entails a. The establishment of repositories to store the data b. The reposed data should then be marked up with sufficient metadata to inform someone else how the data was created, including attribution or provenance information relevant to citation c. Assignment of a persistent digital identifier (like the DOI for journals) so the data can be cited and discovered by others d. Tools must be developed to i.enable the ingest of data from computation or experiment for deposition in a repository ii. simplify metadata collection, e.g. by 1. Automatically assigning some basic metadata 2. Implementing automatic extraction of metadata to the extent possible iii. Assign persistent digital identifiers 2. Some sort of interface/api that the user interacts with to find needed information. This requires (at least) a. the registration of the availability of the data into some sort of registry to enable discovery without prior knowledge of the existence of the repository/specific data features. b. Various types of policy enforcement c. Tools to evaluate the quality of the data i.enabling determinations of the data s uncertainty, validity, sensitivity, as well as other metadata qualities ii.enabling assessments of the relevance of the data to the question at hand, such as quick-look plotting or imaging capabilities] d. Tools for data manipulation e. Terminologies and ontologies to enhance search

7 Notional architecture for NIST CAP NIST Common Access Platform Data Consumers (Humans and Machine Agents) Portals / Federated Search / Discovery (e.g. Data.gov, agency portals) Services? Metadata Registries/Catalogs PID Resolvers, Harvesters? PIT Data Type Servers Data Services Agency Repos Federal Repos Publisher Repos Domain Repos Other Repos bare data

8

9 NIST Data Efforts Collaborations ASM International: Structural Data Demonstration Project DOE/EERE Kinetics of Cast Mg Alloys Journals collaboration IMMI Others under discussion DATA CAPTURE TRC: Guided Data Capture Materials Data Curator Workflow Tools DSpace Ontologies Schemas (XML based) NLP TRC: ThermoML Archive Data Dissemination WebFF- Repository Interatomic Potential Repository Semantic Media Wiki Meta data standards DATA PROCESSING Uncertainty Analysis Data Analytics Data Mining Tools Bench marking activities (DFT)

10 Repositories

11

12 Metadata Curation Markup

13 Desired Use Case Normalized Data Complexity Normalized Data Size Human data entry of published data Add new tasks to do API Data Repository API Query for unfinished tasks, compute, update data record HPC High throughput computation Photo credit: energy.gov Photo credit: nist.gov API Data Repository File Repository Automated capture of measurement data API Consume data

14 PIDS

15 Persistent Identifiers (PID) Persistent Identifiers (PID) are labels in the form of globally unique numbers or codes allocated to an object or resource (whether online or offline), that are guaranteed to be managed and kept up to date over a defined time period. Generally they are long- lasting references to a digital object a single file or set of files. The types of persistent identifiers we have mentioned are: Handles (HDL) Digital Object Identifiers (DOI), a type of HDL PID Archival Resource Keys (ARK) ARK HDL DOI

16 Persistent Identifiers (e.g. Handle System) hdl: /456a9.c4f protocol prefix identifier via PKI optionally In this example the Handle System maps a persistent identifier to a set of metadata /456 1 PK publickey 2 IP rights data 3 Publisher NIST 4 GUID a8-0c- 22-7f- c URL 6 HDL /9934

17 Data Policy A NIST Approach (in progress)

18 Data Taxonomy: STRAWMAN- DRAFT SRD Reference Data Resource Data Published Results Publishable Results Derived Data Working Data Working Data The digital equivalent of entering data in a laboratory notebook. Working data may be raw observational data that is acquired directly from an instrument or a measurement system, or digital values acquired or generated during experiments or simulations. In some cases the researcher responsible for generating the working data may determine that this data has immediate value and is worth preserving, or the researcher expects the data will have value after it has been manipulated or further evaluated, and the data has the potential to develop into a publication or will be used to draw conclusions. In other cases working data may be recognized as not appropriate for broader use in its present form. It may have value to the data producers and their collaborators, but it should be recognized that the data could be easily misinterpreted by people not closely involved in its production because some metadata and important facts about its status or acquisition are not readily available beyond the immediate research team (i.e. adequate metadata for re- purposing is not attached to the data itself, expending resources to codify needed metadata is not justified, etc

19 Data Taxonomy: STRAWMAN- DRAFT Preservation Levels 1. No additional requirements 2. Individual user responsibility 3. Data backed up using a tested/automated process Discoverability Levels 1. No additional requirements 2. Persistent Identifier (PID) assigned 3. Entered in NIST Enterprise Data Inventory 1 4. Inventory record flagged for public access 1 PID assigned + NIST minimum metadata present

20 Data policy under discussion mapping levels onto categories STRAW MAN- DRAFT Preserv = Preservation Discov = Discoverability SRD Reference Data Resource Data Published Results Publishable Results Derived Data Working Data Preserv Discov

21 Who else is up to stuff Not counting many of the following speakers

22 Research Data Alliance Interest Group RDA/CODATA Materials Data, Infrastructure & Interoperability IG James A Warren and Laura Bartolo, Co-Chairs REGISTRIES

23 EUDAT

24 22

25 NIST Center of Excellence for Advanced Materials

26 STOP

27 PANEL What do you think the biggest gaps in the data/model infrastructure are? How much does it depend on material discipline or applicafon What's "low hanging fruit"? What's really hard? Do you need people to test your infrastructures? Are you looking for volunteers? What doesn't/hasn't worked? Which "minimal metadata" exemplars exist? What should we develop? What should we avoid? How will you sustain your infrastructure over the long term? What new government data and resource policies would be most helpful? What would be most huriul? What about the culture of sharing? (Who is submijng data and why?) What tools are needed? What about code disseminafon? What role do the publishers/journals have in this? 27