Data Symposium 2012 SeWHIP & CTSI John W. Cobb, Ph.D. Milwaukee, WI March 1, 2012

Transcription

1 : Some Lessons Learned Data Symposium 2012 SeWHIP & CTSI John W. Cobb, Ph.D. Milwaukee, WI March 1, 2012

2 Acknowledgement and collaborators DataONE Cal Dig. Lib. Cornell Lab of Ornithology TeraGrid (now XSEDE) National Science Foundation 2 Managed by UT-Battelle

3 Outline Distinctions between cyberinfrastructure, information technology, computer science, computational science a personal view Requirements analysis for interoperable long-term data archive and curation service (a datanet) with focus on ecological, biological, and environmental sciences Description of DataONE datanet services and related services as an exemplar (bulk of talk) 3 Managed by UT-Battelle

4 Outline Distinctions between cyberinfrastructure, information technology, computer science, computational science a personal view Requirements analysis for interoperable long-term data archive and curation service (a datanet) with focus on ecological, biological, and environmental sciences Description of DataONE datanet services and related services as an exemplar (bulk of talk) 4 Managed by UT-Battelle

5 Data driven science A supercomputer is just one more source of petabytes of data Rorshach test: What is This? A computing center or a data center? Images courtesy of the NCCS, ORNL 5 Managed by UT-Battelle

6 A (personal) taxonomy of computing research activities Computer Science (CS) Algorithm development Compu6ng architecture development Language research Cyberinfrastructure (CI) Developing methods for accessing compu6ng services Virtual organiza6on research Computa6onal Science (Computa6onal Sci) PDE solver algorithms Finite math representa6ons of con6nous PDE s Numerical linear algebra Informa6on Technology (IT) Research about IT opera6ons Provisioning Produc6on networking (internal and external connec6ons) Cybersecurity In addition to research efforts IT, CI, and Computational science often have coupled operational tasks. 6 Managed by UT-Battelle

7 Contrasts: CI and IT CI Enables research initiatives Expands topline Seeks new services Externally focused Project partner IT Enables enterprise integration Optimizes bottom line Optimizes current services Internally focused Project component However, pragmatically, university CIO s often are charged with CI and IT responsibilities as well as a partner in computational science provisioning. Usually computer science is not under CIO function. Library science is also separate although occasionally academic library operations sometimes also fall under the CIO function. Some CI and IT services can look quite similar (system administration, network security, ) 7 Managed by UT-Battelle

8 My focus today: CI Like the physical infrastructure of roads, bridges, power grids, telephone lines, and water systems that support modern society, cyberinfrastructure refers to the distributed computer, information and communication technologies combined with the personnel and integrating components that provide a long-term platform to empower the modern scientific research endeavor. (Atkins Report, NSF Blue-Ribbon Advisory Panel on Cyberinfrastructure 2003) 8 Managed by UT-Battelle

9 Outline Distinctions between cyberinfrastructure, information technology, computer science, computational science a personal view Requirements analysis for interoperable long-term data archive and curation service (a datanet) with focus on ecological, biological, and environmental sciences Description of DataONE datanet services and related services as an exemplar (bulk of talk) 9 Managed by UT-Battelle

10 What is needed to build a robust set of data services? Archiving Management Metadata regularization Curation Retention Practioner education Socio-cultural barriers 10 Managed by UT-Battelle

11 Information Content Poor data practice data entropy Accident Time of publication Specific details General details In what sense is modern science reproducible? cf. Brian Athey and Clifford Lynch this morning Retirement or career change Death Time (Michener et al. 1997) 11 Managed by UT-Battelle

12 Data management plans now required NSF Data Management Plan Requirements Beginning January 18, 2011, proposals submitted to NSF must include a supplementary document of no more than two pages labeled "Data Management Plan" (DMP). This supplementary document should describe how the proposal will conform to NSF policy on the dissemination and sharing of research results. Proposals that do not include a DMP will not be able to be submitted. Other agencies have or are instituting requirements 12 Managed by UT-Battelle

13 Data deluge and interoperability the flood of increasingly heterogeneous data Data are heterogeneous Syntax (format) Schema (model) Semantics (meaning) By hand is timeconsuming and brittle Jones et al Managed by UT-Battelle

14 Baseline assessment: scientists (2010) Demographics Discipline Work Sector Tenopir, C, Allard S, Douglass K, Aydinoglu AU, Wu L, Read E, Manoff M, Frame M Data Sharing by Scientists: Practices and Perceptions. PLoS ONE. 6(6) medicine 2% ecology 18% other 7% social sciences 16% computer science/ engineering 9% commercial 2% government 13% non-profit 3% other 2% biology 14% physical sciences 12% academic 80% atmospheric science 4% 14 Managed by UT-Battelle n=1317 environmental sciences 18% n=1315

15 What standard do you currently use? DIF DwC DC EML FGDC Open GIS Metadata language ISO My Lab none 15 Managed by UT-Battelle

16 Many are interested in sharing data Willing to share data across a broad group of researchers Willing to place at least some of my data into a central data repository with no restric6ons Appropriate to create new datasets from shared data 81% 78% 76% Willing to place all of my data into a central data repository with no restric6ons 41% 16 Managed by UT-Battelle 0% 20% 40% 60% 80% 100% Percent agree

17 Needs of scientists: the data lifecycle Will I get credit for my work? Collect What is a data management plan? Analyze Assure What tools do I use? What is metadata? Integrate Describe Who can help me? Discover Deposit How much will it cost? 17 Managed by UT-Battelle Where do I preserve my data? Preserve How do I preserve my data?

18 ebird pilot project exploration and visualization Diverse bird observa6ons and environmental data from 300,00 loca6ons in the US integrated and analyzed using High Performance Compu6ng Resources Model results Occurrence of Indigo Bun=ng (2008) Land Cover Meteorology MODIS Remote sensing data Spa6o- Temporal Exploratory Model iden6fies factors affec6ng pa^erns of migra6on Jan Apr Jun Sep Dec Examine pa^erns of migra6on Infer how climate change may affect bird migra6on 18 Managed by UT-Battelle

19 Secretary Salazar on Birds (May 3, 2011): The State of the Birds report is a measurable indicator of how well we are fulfilling our shared role as stewards of our nation s public lands and waters. Acadian Flycatcher Distribution ebird.org 19 Managed by UT-Battelle 19

20 Multiple Scales Building the Knowledge Pyramid 90:10 à 10:90 Decreasing Spatial Coverage Increasing Process Knowledge Intensive science sites and experiments Extensive science sites Volunteer & education networks Remote sensing 20 Managed by UT-Battelle Adapted from CENR-OSTP

21 Tracing requirements Multiple scales Interoperable across repositories Cross organizational (VO s) Multiple identities Data heterogeneity Manage disparate rights policies Support all phases of the data life cycle Include education and outreach to change community practies 21 Managed by UT-Battelle

22 Outline Distinctions between cyberinfrastructure, information technology, computer science, computational science a personal view Requirements analysis for interoperable long-term data archive and curation service (a datanet) with focus on ecological, biological, and environmental sciences Description of DataONE datanet services and related services as an exemplar (bulk of talk) 22 Managed by UT-Battelle

23 DataONE Movie (with Sound) 23 Managed by UT-Battelle

24 DataONE is Cyberinfrastructure Three major components form a flexible, scalable, sustainable network Member Nodes diverse institutions Coordinating Nodes serve local community retain complete Investigator provide resources Toolkit for metadata catalog managing their data indexing for search retain copies of data network-wide services ensure content availability (preservation) replication services Source: DataONE/Michener 24 Managed by UT-Battelle

25 Examples of data holdings Metadata Interoperability Across Data Holdings Data Archive Types of Data Managed Biodiversity, taxonomic, ecological Biogeochemical dynamics, terrestrial ecological Earth observation imagery Ecological, biodiversity, biophysical, social, genomics, and taxonomic Avian populations and molecular biology Metadata Standard(s) BDP, DwC, DC, OGIS DIF, BDP, ECHO EML DwC Biological and taxonomic Biophysical, biodiversity, disturbance, and Earth observation imagery Biodiversity, biotic structure, function/ process, biogeochemical, climate, and hydrologic DC subset EML EML BDP=Biological Data Profile DC subset=dublin Core subset DwC=Darwin Core DC=Dublin Core 25 Managed by UT-Battelle EML=Ecological Metadata Language DIF=Directory Interchange Format ECHO=EOS ClearingHOuse OGIS=OpenGIS 25" 25

26 Initial member nodes ORNL- DAAC Dryad KNB Community Agency repository Journal consor6um Research network Data Size Services Ecology and biogeochemical dynamics 900 data products, ~ 1 TB Tools for data preservation, replication, discovery, access, subsetting, and visualization Biosciences ~ 1,000 data products, ~ 3 GB Tools for data preservation, replication, discovery and access Metadata stds. FGDC subset Dublin Core applica6on profile Degree of cura6on High Medium Low Data submission Agency-approved, staffassisted submission and curation of final data product Web-based data submission at time of journal article submission Sponsor NASA NSF/JISC, socie6es, publishers Biodiversity, ecology, environment 20,000 data products, 100s GBs Tools for data preservation, replication, discovery, access, management, and visualization EML, FGDC Self-submission via desktop tool at any time NSF 26 Managed by UT-Battelle

27 Preserve Data and Metadata Metadata copied to Coordinating Nodes Mirrored between CNs Data replicated between Member Nodes CNs manage copies Checksums recorded Promote quality metadata * 27 Managed by UT-Battelle