Key cyberinfrastructure elements implemented as RESTful webservices

Transcription

1 Key cyberinfrastructure elements implemented as RESTful webservices Investigator Toolkit Web Interface Analysis, Visualization Data Management Client Libraries Java Python Command Line Member Nodes Service Interfaces Bridge to non-dataone Member Node services Data Repository Coordinating Nodes Service Interfaces Resolution Discovery Replication Coordination Layer Identifiers Catalog Preservation Object Store Registration Monitor Index 28 Managed by UT-Battelle SW repo at

2 Identify objects Goal: Uniquely identify data or metadata objects Support the several identifier types widely used Identifiers assigned by Member Nodes Uniqueness ensured by Coordinating Nodes Resolution through Coordinating Nodes GUID! {3F2504E Managed by UT-Battelle LSID PURL

3 Identify people: federated identity Identity provider selected by the user Member nodes define access rules Rules propagated by Coordinating Nodes Identity and access control consistent across entire infrastructure (note similarity with Globus Online approach) 30 Managed by UT-Battelle

4 Discover Content: ONEMercury? 31 Managed by UT-Battelle

5 Tools of interest: DataONE API s RESTful interfaces give flexibility Investigator toolkit consumes the REST API s Reference implementations: Investigator Toolkit (ITK) Extensible Investigator Toolkit 32 Managed by UT-Battelle 32

6 The Investigator Toolkit Inves=gator Toolkit Web Interface Analysis, Visualiza6on Data Management Client Libraries Java Python Command Line Member Nodes Developer, end-user tools Creation, search, retrieval, management Plugins, extensions for analysis tools Coordina6ng Nodes Kepler 33 Managed by UT-Battelle

7 ONEDrive Use DataONE discovery REST service Overlay with a Fuse interface (implemented in Python) Result: a Posix file system interface to the entire DataONE set of collections (Note similarity to XSEDE GFFS) Caveat: beware the metadata issues: query 34 Managed by UT-Battelle latency Nice coupling with ONEMercury faceted search to pare down metadata universe with selective mount commands Nice contrast to schlepping files for selected problems Useful for COTS SW that assumes a file-open dialogue

8 DataONE File System Structure data_provider project title keywords decade abscission abundance accretion knb-lter-gce knb-lter-vcr.58.7 metadata system metadata.xml data abstract.txt <value> <iden6fier> <iden6fier> <facet> <value> <iden6fier> describes 35 Managed by UT-Battelle 35

9 One Drive Demo 36 Managed by UT-Battelle

10 Other project tools (beyond SW stack) 37 Managed by UT-Battelle

11 DCXL: Excel extensions for data management Collaboration between CDL, Gordon and Betty Moore foundation, and Microsoft Managed by Carly Strasser at CDL Examining ways to better capture metadata associated with Excel data sets As an Excel add-in, or/and As a web-service Because that s where the data is Recent slide deck: Why Excel? Because it is widely used 38 Managed by UT-Battelle Refer to Cliff Lynch this morning Willie Sutton

12 Data Management Planning Tool Create ready-to-use data management plans for specific funding agencies Meet funder requirements for data management plans Get step-by-step instructions and guidance for your data management plan as you build it Learn about resources and services available at your institution to help fulfill the data management requirements of your grant Released: Oct Support for NIH requirements added 2/22/2012 Other similar efforts now also underway at institutional levels or with other entities. Note: Invitation to Data Management Boot Camp with Dorothea Salo in your symposium materials packet 39 Managed by UT-Battelle

13 Support for Entire Data Lifecycle Plan Analyze Collect Integrate Assure Kepler Discover Describe 40 Managed by UT-Battelle Preserve

14 User Matrix Data Service Investigator ToolKit Data Management Planning Best Practices Tools Database Training Curricula Scientist Data Librarians Ecological Modeler Resource Manager 41 Managed by UT-Battelle

15 How can this CI change the way we do science? A new cloud layer: DaaS Data-as-a-Service Use REST interfaces to deploy DataONE services Document and open Rest services so they can be used collaboratively by new-partners and 3 rd parties Enable workflow mediated wide-area large scale analysis (Caveat: you may crash them!) Enable a bottoms-up standardization of services from experienced based discovery of design patterns Example: curation micro-services (tapas instead of hoagies 42 Managed by UT-Battelle

16 Provide Credit for Data Publication Data citation standards and courtesy customs Needs to metrics how often cited Socio-cultural change: include data citations in promotion and tenure DataONE needs to nurture Member Node needs not work against them 43 Managed by UT-Battelle

17 Engaging citizens in science" 44 Managed by UT-Battelle 44

18 DataONE education and socio-cultural efforts ½ of DataONE project team is CEO oriented Best practices web area Tenopir paper (Tenopir C, Allard S, Douglass K, Aydinoglu AU, Wu L, et al Data Sharing by Scientists: Practices and Perceptions. PLoS ONE 6(6): e doi: /journal.pone ) Many workshops (SC 11 tutorial, ESA2011, ESA 2010, ) 45 Managed by UT-Battelle

19 Interoperablity Some of the best science results from re-purposing data in unanticipated ways. Similar projects under different management need to interoperate Dissimilar projects need to interoperate Multiple institutions need to interact for the best science (SeWHIP and CTSI are good examples) 46 Managed by UT-Battelle

20 Interoperability Shibboleths The future is NOT an accretion event Data locality canard Bandwidth limits are real But data will not all be generated in one location Methods must be able to cope with non-local data retrieval The trouble with centers: Big-Data hostage crisis Cloud services, in general, are not a means for interoperability. Clouds are not necessarily VO friendly 47 Managed by UT-Battelle

21 Data rights issue: hard problems Not just a technical issue Currently in flux (c.f. RWA kerfuffle) Open is nice but not always possible Researcher and discipline cultural behavior External requirements (e.g. HIPAA) Commercial interests, IP-related data Science is international but rights vary across political borders 48 Managed by UT-Battelle Technology may uncover new anti-social cultural norms Approach: evolve along with community of practice norms Change may only be on a generational scale Openness can be dictated by funder (some NIH successful examples) Rights on Metadata Cf. Clifford Lynch on PII data future reuse this morning

22 Building global communities of practice: creating long-lived CI enterprises, Broad, active community engagement Involvement of library and science educators engaging new generations of students in best practices Existing outreach and education programs Transparent, participatory governance cf. Clifford Lynch this morning and forward link to Serge Goldstein this afternoon Adoption/creation of innovative and sustainable business and organizational models 49 Managed by UT-Battelle

23 DataONE ultimate goals A one-stop shop for all aspects of data lifecycle A known, reliable data management brand A resource enabling connections and interoperability among many (often disparate) data repositories An advocate and educational resource for improving data management practices An recognized enabler for improved research data practices A productive partner for data repositories (Member Nodes) 50 Managed by UT-Battelle

24 Closing plug: DataONE summer internships available URL: internships Available for Summer of 2012 Time Period: May 23 July 29, 2012 Applications Due: March 12, 2012 Up to 8 slots available 51 Managed by UT-Battelle Solicited topics: 1. Publish (data) or Perish: Best Practices for Creating, Reviewing, and Publishing Data Products 2. Enriching the Content of the DMPTool for the DataONE Community 3. A Portable Web Application for Data and Metadata Submission 4. Querying Scientific Workflow Provenance 5. Data Usage and Citation Visualization 6. Evaluating the Feasibility of Using Bottom-Up Text Mining Approaches to Complement Thesaurus and Ontology-based Approaches for Supporting Data Discovery 7. Enhancing Semantic Search in ONEMercury 8. An Information Model for Observational Data within DataONE 9. Components of Successful Metadata Registry Frameworks 10. Developing a DaaS (Data as a Service) view of DataONE

25 DataONE Community 52 Managed by UT-Battelle

26 Question & Discussion John W. Cobb Managed by UT-Battelle