NEES Community and Communication (NEEScomm) Data Sharing and Archiving Policies and Guidelines, Version 1. December

Transcription

1 Introduction Policies Definitions References This document promulgates policies and guidelines from the George E. Brown, Jr. Network for Earthquake Engineering Simulation (NEES) Consortium, Inc. for archiving and sharing data resulting from all research using the NEES Equipment Sites and for data obtained from other sources within the scope of NEES. 1. Introduction 1.1 Vision for Data Archiving and Sharing NEEScomm is committed to promoting the sharing of high-quality data generated in the course of research conducted using the NEES Equipment Sites and data from other sources within the scope of the NEES mission. NEEScomm is committed to establishing and maintaining well-documented, properly safeguarded data archives and to developing uniform standards for data access by members of the scientific and engineering research communities. The overall purpose of this document is to ensure and facilitate full and open access to quality data archived in the NEES Data Repository. 1.2 Data Goals of NEEScomm NEEScomm facilitates the use of the NEES experimental facilities, also known as NEES Equipment Sites, and establishes data archiving and sharing policies for all data submitted to and archived in the NEES Data Repository. It is the goal of NEEScomm that no data will be lost. Moreover, to ensure that the archived data are of high quality, the data documentation (metadata) must contain sufficient information about the data and its creation to ensure that any qualified user could reproduce the experiment or simulation, through the life of the repository. NEEScomm promotes the creation of unique and diverse datasets, rich sources of information that cannot be readily replicated. Data sharing allows engineers and scientists to expedite the translation of research results into knowledge, products, and procedures that will advance the state-of-the-art in earthquake engineering and help reduce earthquake risk. It also reduces unnecessary duplication of data collection. Building preservation systematically into routine data management is good research practice; it strengthens quality, enables replication and audit, and provides a sound basis for data sharing. There are many reasons to share data. Sharing data reinforces open scientific inquiry, encourages diversity of analysis and opinion, promotes new research and collaborations, makes possible the testing of new or alternative hypotheses and methods of analysis, supports studies on data collection methods and measurement, avoids duplication of effort, facilitates the education of new researchers, enables the exploration of topics not envisioned by the original investigators, and permits the creation of new datasets when data from multiple sources are combined. NEEScomm recognizes that the investigators who create and collect the data have a legitimate interest in benefiting from their investment of time and effort. It expects that the original investigators and data producers will benefit first from the data for a period of time, but that period should be limited to facilitate access to the data by the broader earthquake engineering research community. Ultimately, the entire public community will be able to benefit from access to the data. Page 1 of 15

2 1.3 Data Goals of the National Science Foundation These policies and guidelines conform to National Science Foundation (NSF) Grant General Conditions (GC-1, 01/05/2009), Section "Sharing of Findings, Data and Other Research Products, which states: NSF expects significant findings from research and education activities it supports to be promptly submitted for publication, with authorship that accurately reflects the contributions of those involved. It expects investigators to share with other researchers, at no more than incremental cost and within a reasonable time, the data, samples, physical collections and other supporting materials created or gathered in the course of the work. It also encourages grantees to share software and inventions or otherwise act to make the innovations they embody widely useful and usable. Adjustments and, where essential, exceptions may be allowed to safeguard the rights of individuals and subjects, the validity of results, or the integrity of collections or to accommodate legitimate interests of investigators.. Page 2 of 15

3 Introduction Policies Definitions References 2. Policies for Data Archiving and Sharing This section describes the policies for data archiving and sharing resulting from all research developed using NEES Equipment Sites and for data obtained from other sources within the scope of NEES. 2.1 Applicability These policies apply to all information curated and archived in the NEES Data Repository. These include experimental and simulated data generated from research performed using the NEES Equipment Sites and computational facilities. These also include data generated from other sources, related to the scope of NEES, and approved by NEEScomm. Prior to the start of a research project, project-specific data information, which indicates the type and expected amount shall be submitted to NEEScomm as a part of the Equipment Utilization Form. 2.2 Archiving This section describes the data flow from production through packaging to curation and to the permanent repository. The timeline for accessibility to the data is described in the next section. Together with the data, investigators must provide sufficient information about the data, the experiment, or computational simulation, to enable prospective users to reproduce the experiment or simulation. All data resulting from experiments and the information required to reproduce the experiments and the computational simulations should be stored. Data from measurements and associated simulations must be stored as preliminary data in an initial repository as they are recorded. NEES Equipment Sites are to provide and maintain offsite backup and storage of all raw data developed at their Site. Raw data must be stored offsite within 24 hours after each shared-use experiment, unless otherwise justified on an approved Equipment Site Utilization Form. Each Equipment Site shall keep a current plan on file with NEEScomm detailing how the Site will achieve the offsite data storage requirement. The investigators must submit structured data to the permanent repository within six months from the end of experiment or simulation, after proper quality evaluation, integration with documentation and other necessary preparation for curation. The data submitted to the permanent repository will be curated and archived in the permanent repository from which these data will be made accessible to the public. Researchers should periodically update the curated structured data following further data processing and quality evaluation as soon as reasonably possible. Project investigators must provide all additional interpreted information as soon as reasonably possible, but not later than two years after the end of the experiment. Investigators are strongly encouraged to include additional interpretations and analyses that will help users interact with the measured data. Data inventories will be recorded into the public databases and published periodically by NEEScomm indicating accessibility status and any changes made in the archived information. 2.3 Accessibility and sharing NEEScomm expects investigators using the NEES Equipment Sites to make their research data available in a timely and responsible manner to the scientific community for subsequent research with as few restrictions as possible in keeping with the following: 1. Data will be made accessible to project participants as it is recorded. With the PI's Page 3 of 15

4 permission, data can be released to particular users before it is publicly accessible. Certain personnel from NEEScomm Headquarters, as approved by the NEEScomm Director, will be given read-only access to all NEES Data Repository shared-use project files for tracking purposes only unless a compelling justification denying such access is provided in an approved Equipment Utilization Form. NEEScomm Headquarters personnel accessing these files will observe strict standards of confidentiality, and access accounts only for the purpose of monitoring progress along the data flow process. 2. Information placed in the archive will be made automatically accessible to the public no later than twelve months after the end of the experiment or simulation, unless there are exceptional circumstances or other arrangements as identified on the approved Equipment Utilization Form. 3. The automatic release period may be extended under exceptional circumstances. The request for exceptions should be submitted as part of the Equipment Site Utilization Form, which is subject to the approval of the NEES Equipment Site and NEEScomm. 4. In the event that proprietary information must be used for the conduct of a project, researcher shall provide justification for the use of said information on the Equipment Site Utilization Form and request its use. However the Equipment Site and NEEScomm shall have the right to deny the request. 2.4 Quality Assurance All researchers and NEES Equipment Site personnel are responsible for providing high quality data. The data producers are responsible for the accurate recording and reporting of data. The experiment or simulation must be documented to enable full reproducibility; it should be possible for another investigator to verify and reproduce these data throughout the life of the permanent repository. Implementation of quality assurance (QA) will be accomplished through Training and Review using current best practices. NEES Equipment Sites may undergo a data management review. In cases where data is produced by a non-nees facility and is provided to the NEES Data Repository, the same QA policies will apply. Support information for such compliance shall be supplied to NEEScomm by the data producer. The data in the NEES Data Repository are provided as is. All data archived should have a standardized disclaimer of warranty. 2.5 Attribution Data providers are responsible for ensuring that all data files are internally documented and include data producer and sponsoring agency information in the headers, or in associated logs, or other documentation. In exceptional circumstances, in agreement with NEEScomm and the Sponsoring Agencies, the identity of subjects and/or sponsors and/or researchers could be protected from public disclosure. 2.6 Acknowledgements Users shall give appropriate acknowledgement of the data producer, the NEES Equipment Site, the research project, NEEScomm, and the Sponsoring Agency as contained in the data files, or in the accompanying documentation. 2.7 Implementation Prior to the start of research that will make use of NEES Equipment Sites for their projects, all Principal Investigators must submit, on an Equipment Utilization Form, information about project-specific data use, which in most cases would have been approved by a Sponsoring Agency as part of an award. The plan should include justified deviations from the Equipment Site s plan for offsite backup and storage of all raw data, and required resources. The plan should follow the accessibility and sharing policies and guidelines specified above. In Page 4 of 15

5 exceptional circumstances requiring extension of the release period (see items (3) and (4) under Accessibility and Sharing), the Principal Investigaors must include an accessibility scheme and a detailed schedule for data sharing prior to the release to the public. 2.8 Compliance The release of the data to the public will be monitored by NEEScomm. Failure to comply with the described policies and guidelines may disqualify a researcher from future use of NEES Equipment Sites. Page 5 of 15

6 ` Introduction Policies Definitions References Definitions: This section provides definitions and explanations of terms. All the definitions and explanations are an integral part of these policies and guidelines, and provide for some terms more detailed requirements. The definitions are listed in alphabetical order. Access to data can be either public or restricted (by the data producer within the conditions and time frame described elsewhere in these policies). Access control is provided by the Equipment Sites for local repositories and by NEEScomm for the permanent NEES Data Repository according to the rules established by these policies. Acknowledgement: All data users must include in their contracts, reports or other publications acknowledgement of the data producer, the research project, the NEES Equipment Site, NEEScomm, and the Sponsoring Agency supporting the data production. It is expected that authors of reports, papers, or other publications based on data they did not generate themselves will acknowledge the source of data upon which their manuscript is based. The users shall sign an agreement with NEEScomm indicating their commitment to provide proper acknowledgement. See the following specific acknowledgement definitions. Acknowledgement of Data Producer: The following statement should be used: These data are part of the results obtained by <names of PI s or researchers indicated in the header or the associated metadata who produced the data> from <name of associated institution> using <name of NEES Equipment Site (if applicable)>. Acknowledgement of Sponsoring Agency: For data from research supported through NSF/NEESR programs, the following statement should be used: This data is based upon work supported by the National Science Foundation under Grant No. (grantee must enter NSF grant number). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This data was produced using the experimental facilities and repository operated and maintained by NEEScomm, under National Science Foundation cooperative agreement CMMI For data supported by other Sponsoring Agencies but making use of NEES equipment or data from the permanent repository, the following statement should be used in addition to any sponsor requirements: This data is based upon work supported by the (grantee must enter Sponsor s name) under Grant No. (grantee must enter grant number). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the (grantee must enter Sponsor s name). This data was produced using the experimental facilities and repository operated and maintained by NEEScomm, under National Science Foundation cooperative agreement CMMI Acknowledgement of NEEScomm: The following statement should be used: These data were obtained from the data repositories of George E. Brown, Jr. Network for Earthquake Engineering Simulation (NEES) operated and maintained by NEEScomm under National Science Foundation cooperative agreement CMMI Page 6 of 15

7 Automatic Access or removal of restrictions in accessing the data will be made at the end of the term specified in these policies. Automatic access to data will also be made upon publication of the data in peer-reviewed publications. NEEScomm will provide the infrastructure to expedite the publication of data in peer-reviewed publications such as an Electronic Journal curated by NEEScomm, or a report series. Curation: is the process of integrating the packaged data (see Data Flow Diagram) in the permanent repository with information on copies, versions, revisions, editions, adaptations, derivations and information on the software and operating systems which allow retrieval of digital data. The process of cataloging and referencing will be done by the NEEScomm Data Librarian. The packaged data shall be reviewed by the NEEScomm Data Librarian for compliance with the metadata models (templates). In case of non-compliance the NEEScomm Data Librarian will request repackaging of data before completing the curation process. Data Management Review: Data management practices at NEES Equipment Sites may be internally reviewed by NEEScomm as part of their annual review. The objective of the review and assessment is to establish whether a NEES Equipment Site can competently generate the data for which accreditation is sought. Compliance of a NEES Equipment Site with accreditation requirements is determined primarily by an on-site assessment of its resources, procedures, and documentation. Guidelines for the accreditation process are shown in Attachment A. Data Producer is the researcher or the staff person who generates the data in the laboratory experiments or computational simulations. Data Producer Information: Each data file will include in its header, or in the metadata associated with the data file, at a minimum the investigator s name, her/his affiliation, equipment site, and time of data collection. Data Recording refers to the production of data in a series of trials linked in a research experiment. Each data component will be recorded in the initial repository as it is created. The completion of data recording refers to the end of the experiment. The investigators must define the experiments in the Equipment Site Utilization Form. Data Sharing Agreement will be recognized as a "license agreement," or "data distribution agreement," between the Recipients of the Data and the Data Producer and will include the requirements to protect the confidentiality of restricted data. Note that all data developed under NEES shared-use projects are intended for public release, and the term restricted data pertains to the limited time interval prior to public release that is identified in the Plan. A Data Sharing Agreement may prohibit the recipient from transferring the restricted data to other users or require that the data be used for research purposes only and require that the user consult the PI before any data can be used for publication, among other provisions, and it may stipulate penalties for violations. The procedures adopted to share data while protecting privacy should be individually tailored to the specific dataset. Page 7 of 15

8 Documentation of Data (Metadata): Regardless of the mechanism used to share data, each dataset will require documentation. Some other science and engineering fields refer to data documentation by other terms, such as metadata, or codebooks. The term data documentation is used in this document synonymously with metadata. Proper documentation is needed to ensure that others can use the dataset and to prevent misuse, misinterpretation, and confusion. Data documentation provides information about the methodology, or procedures used to collect the data, details about software code, definitions of variables, variable field locations, frequencies, and the like. The precise content will vary by scientific area of study, type of data collected, and characteristics of the dataset. All documentation will be provided in special metadata files and notebooks in formats which will facilitate searching and retrieving the information archived in repositories. Data and metadata formats are specified by NEEScomm IT which operates the NEES Data Repository. Disclaimer of Warranty: When data from the NEEScomm permanent repository is used, all users must indicate in their publications that: The views expressed herein are those of the authors and do not necessarily reflect the views of Data Producers, the NEES Equipment Sites, the Sponsoring Agencies, or NEEScomm. NEEScomm provides the data at the highest quality possible according to the best practice at the time of its generation. However, the data are provided as recorded or calculated by the data producer. It is the responsibility of the user to validate and interpret the data. NEEScomm, Data Producers, NEES Equipment Sites, and Sponsoring Agencies are not liable for any consequences resulting from use of the data in the permanent NEES Data Repository or the NEES Equipment Sites initial repositories. Equipment Sites (or Facilities) described in this document consists of NEES Equipment Sites and NEES Computing Sites networked and linked with the NEES Data Repository operated by NEEScomm. Encapsulated Data refers to the package including all data, metadata and associated tools required to interpret the data. End of Experiment specified in this document refers to the completion of a series of trials linked in a research experiment which form a complete set of information. Engineering Units specified in this document refer to the relevant units, characteristic to the data measured. For example, force will be measured in either Newtons (in the SI system), or pounds (in US customary Units). NEEScomm recommends use of SI system at all times. Whenever the US customary system is used, additional information should be included for conversion to the SI system. All electrical units in the raw data should be converted to the appropriate engineering units as illustrated in the example above. All transformations shall be documented accordingly. Equipment Site Utilization Form: All investigators, including investigators internal to the institution hosting the NEES Equipment Site, must submit an Equipment Utilization Form, which details the research project to be conducted. The Form shall be reviewed and approved by the NEES Equipment Site and NEEScomm. The Equipment Utilization Form shall include, at a minimum, the following: Page 8 of 15

9 (a) Description of Research Project, including timeline (b) Key personnel (c) Equipment needed (d) Identification of any unusual costs that will need to be incurred and covered by the researcher. (e) Project-specific data use information and requested deviations from the above policies and guidelines. Exceptional Circumstances. NEEScomm understands the need for exclusive use of information in developments related to work in the academic environment or in industry requiring confidentiality to assure competitiveness. Exceptional Circumstances refer to unusual requirements such as, but not exclusively, those arising from patent applications, or from protection of innovative ideas leading to and improving industrial competitiveness. Those circumstances must be documented in the Equipment Site Utilization Form. NEEScomm with assistance from the Sponsoring Agency will evaluate those circumstances before approval. Experiment is defined here as a series of trials made on one or more specimens prepared for one purpose of testing. In general an experiment should produce a stand-alone set of information. Experimental and Simulated Data in this document relates to information obtained from measurements and associated simulations produced during a research project which include the following: a) Experimental Data refers to all measurements, calibrations, observations, analyses, images, commentary, reports, logs, notes, and/or electronic notebook entries which relate directly to the experiments. Any data (as just described), which are recorded in hardcopy of any form, must be transcribed / converted into an appropriate searchable format onto electronic media. Data samples are obtained from measuring instrumentation and other supporting materials created or gathered during the course of the research work. Measuring instruments in this case are defined in the broad sense as measuring or observing sensors and the associated conditioning devices (such as powering, amplifying, filtering, and scaling), as well as analog-to-digital converters, where applicable. Moreover, the measuring instruments may include conversions of data from individual channels of information automatically computed during the recording. Such data may include numerical quantities in engineering units relevant to the measured quantities, video and audio data captured from the measuring instruments directly. b) Simulation Data refers to computational simulations developed for the experimental research, such as preliminary and post test evaluations and the simulation tools required to produce such simulations. Data must be accompanied by documentation (metadata) of all the procedures and transformations of the original mechanical, physical, or electrical observations to the numeric data stored in resulting electronic files. This must include calibration and scaling factors and transformation procedures performed automatically by the electronic instrumentation. Data should be stored in an appropriate format, usually defined by the instrumentation system. Investigators must document the data format in the associated documentation / metadata. This information must be properly characterized with appropriate documentation in the form of metadata descriptors and then subsequently sorted into one of the NEEScomm IT accepted digital formats and data models to facilitate archiving in accordance with standards established by the earthquake engineering community through NEEScomm. All data resulting from experiments and the information required to reproduce the experiments and the computational simulations should be stored. These policies encourage Page 9 of 15

10 archiving key results of simulations including all the results expected to be published. Final Data consists of the structured data with the optional addition of data derived during the course of the investigation, which may better characterize the information measured. The updated structured data in the final data are the result of progressive corrections, cleaning, and processing accompanied by appropriate metadata explaining all of the procedures and transformations from the original mechanical, physical, or electrical observations to the numerical data stored in resulting files obtained during the research process. A versioning system is provided by the services of NEEScomm IT to keep track of and control any changes made after the package of structured data is originally submitted to the permanent repository. Initial Repository refers to the electronic storage in which the recorded data are captured from the instruments and sensors. The initial repository may be located at a NEES Equipment Site, the institution collecting the data, or in a central location for those institutions that do not have a local repository. The information in the initial repository may be removed after transfer to a permanent repository following proper data cleaning, processing, documenting, packaging and curating. The NEES permanent repository is located at and operated by NEEScomm. Other Data Sources: Data generated by the earthquake engineering community other than the NEES Equipment Sites may be stored in the NEES permanent repository if it fits the mission of NEES and is approved by NEEScomm Other Programs and Agencies: While NEEScomm was to manage NEES resources, other programs within NSF or other Sponsoring Agencies, federal, local or private may utilize the NEES resources. The terms and conditions of obtaining such services related to data archival and sharing will be subject to written agreements between NEEScomm and such other programs or other Sponsoring Agencies. Permanent Repository, or NEES Data Repository, is the electronic permanent storage for NEES data located at the NEEScomm facility. The same storage may also serve as an initial repository for some NEES Equipment Sites if large storage capacity is not available on site. Placing Data in the Initial Repository: The capture and transfer of data in the initial repository will typically be accomplished by moving files from the recording components to the initial storage repository (see also Data Flow Diagram). At this stage all the information indicating the sources of data, the procedures for transformations from mechanical or physical quantities to recorded information, including conditioning, calibration, and aggregation done automatically or deliberately during the recording process, also should be stored in the initial repository. Placing Data in the Permanent Repository The structured data (encapsulated) will be transferred / uploaded to the NEES permanent repository from the initial repository by the Data Producer with assistance of the NEEScomm Data Librarian (see also Data Flow Diagram). The upload will include the curation performed by the NEEScomm Data Librarian Page 10 of 15

11 at NEEScomm IT with assistance of the Data Producer. Proprietary Information: Issues related to proprietary information can arise with corresponding limitations on public disclosure. NEEScomm recognizes the need to protect temporarily patentable and other proprietary information. Any restrictions on information sharing required to protect intellectual property should be presented on the Equipment Utilization Form and will be considered for approval by NEEScomm. While NEEScomm understands that an institution's desire to exercise its intellectual property rights may justify a need to delay disclosure of research findings, the information disclosure should not be delayed more than twenty four (24) months from the time of placement in the NEEScomm permanent repository. An extended delay of disclosure of proprietary information will have to be approved by NEEScomm. Raw Data are the data samples obtained from measuring instrumentation and other supporting materials created or gathered during the course of the research work. Measuring instruments in this case are defined in the broad sense as measuring or observing sensors and the associated conditioning devices such as powering, amplifying, filtering, and scaling, as well as analog-to-digital converters. Moreover, the measuring instruments may include conversions of data from individual channels of information automatically computed during the recording. Such data may include numerical quantities in engineering units relevant to the measured quantities, video and audio data captured from the measuring instruments directly. Review for Quality Control: NEEScomm will rely upon an annual review to ensure data archived in the repository is of high quality (see Accreditation Process). Any data to be curated in the NEES permanent repository will also be reviewed by the NEEScomm Data Librarian to ensure it satisfies all technical requirements for curation. Site User Agreement: All institutions with investigators interested in using NEES facilities must enter into an agreement with the relevant NEES Equipment Site(s). The agreement is defined as the Site User Agreement. Resources for Data Production: NEEScomm recognizes that it takes time and financial resources to prepare quality data for sharing. Investigators are strongly encouraged to prepare a detailed resources request to the sponsor dedicated to data production. The resources must include at a minimum human and financial resources required to train for, prepare and transfer the data to a NEES Equipment Site initial repository or the permanent NEES Data Repository. Sponsoring Agency: Sponsoring Agency refers here to the National Science Foundation Programs (NEESR and other programs) and other Sponsoring Agencies supporting research or investigations using the NEES Equipment Sites or data. Sponsoring Agency Information: Each data file will include in its header, or in the metadata associated with the file, at a minimum the name of the Sponsoring Agency, and the information about the sponsoring grant. See Acknowledgement definition. Structured Data refers to the integrated package of data from all sensors, electronic or Page 11 of 15

12 physical notebooks and other sources, and the documentation/metadata which explain how the data was created and manipulated before it was recorded and stored. The integrated structured data will follow the data models developed by NEEScomm. The integrated package is then appended with additional information required to identify and access the data over the life of the NEEScomm permanent repository. This information is obtained during curation performed before archiving in the NEEScomm permanent repository with assistance of the NEEScomm Data Librarian. The structured data must comply with the Quality Assurance Plan. It is the responsibility of the investigator to verify compliance. Training for Quality Control: NEEScomm will organize training programs to educate data producers as to best data practices available. It is the responsibility of the data producer to ensure that best practices are followed during data collection. Training will consist of community-led workshops as well as online and printed material for self-guided training. NEEScomm will support staff to organize training of personnel at NEES Equipment Sites on electronic data practices. The training program will focus on: Experiment design, e.g., proposal preparation and detailed experiment data design. Model construction and testing, e.g., specimen construction and materials selection. Physical accuracy of data, e.g., instrumentation and data acquisition systems. Documentation and archiving for reproducibility, e.g., best practices on metadata. The NEES Equipment Sites and NEEScomm must maintain (i) qualified, trained and experienced staff; (ii) equipment appropriately calibrated and maintained in an appropriate testing environment; (iii) appropriate sampling practices; (iv) sound testing/inspection procedures; and (v) adequate documentation and quality control procedures. Introduction Policies Definitions References References and Links 1. NATA Quality Assurance and Certification Procedures 2. Jim Gray, Microsoft Research; Alexander S. Szalay, Johns Hopkins University; Ani R. Thakar, Johns Hopkins University; Christopher Stoughton, Fermi National Accelerator Laboratory; Jan VandenBerg, Johns Hopkins University (2002), Online Scientific Data Curation, Publication, and Archiving, Technical Report MSR-TR , July 2002, Microsoft Research / Microsoft Corporation, One Microsoft Plaza, Seattle WA 3. e/policy/index.htm Medical Research Council UK MRC MRC policy on data sharing and preservation 4. NIH Statement of Sharing Research Data 5. TOP Page 12 of 15

13 Page 13 of 15