NEUROIMAGING RESEARCH DATA LIFE- CYCLE MANAGEMENT

Transcription

1 NEUROIMAGING RESEARCH DATA LIFE- CYCLE MANAGEMENT Hurng-Chun (Hong) Lee, Robert Oostenveld, Erik van den Boogert, Eric Maris

2 Outlines Lifecycle RDM: objectives and challenges The method the RDM protocol Donders Research Data Repository (DRDR) - usage of irods Strength and weakness of DRDR Future focuses 2

3 Lifecycle RDM RDM spans the entire research lifecycle Objectives: conception of research long-term data preservation scientific-process documentation data publication publication data analysis 3

4 Challenges large institute with heterogeneous scientific-administrative workflows 600 researchers in PI groups more than 150 projects per year 3 centres with 4 administrative domains data complexity: text, audio/video, imaging or signal data, etc. sensitive data size ranges from a few large (>2GB) files to a huge amount of small files (<1MB) user expectation 4

5 The RDM protocol conception of research data analysis data publication 5 * Box icons are made by Roundicons and Pixel Buddha from are licensed by CC 3.0 BY.

6 The RDM protocol conception of research : a container of data (files/folders) metadata has a state attribute data analysis data publication data sharing research documentation 5 * Box icons are made by Roundicons and Pixel Buddha from are licensed by CC 3.0 BY.

7 The RDM protocol conception of research : a container of data (files/folders) metadata has a state attribute PID data analysis data publication data sharing research documentation 5 * Box icons are made by Roundicons and Pixel Buddha from are licensed by CC 3.0 BY.

8 The RDM protocol conception of research : a container of data (files/folders) metadata research administrator (RA) manager contributor has a state attribute PID data analysis data publication data sharing viewer research documentation 5 * Box icons are made by Roundicons and Pixel Buddha from are licensed by CC 3.0 BY.

9 The RDM protocol conception of research workflow responsibility eligibility PID data publication data sharing research administrator (RA) manager contributor viewer research documentation : a container of data (files/folders) metadata has a state attribute data analysis 5 * Box icons are made by Roundicons and Pixel Buddha from are licensed by CC 3.0 BY.

10 The RDM protocol Organisational Unit (OU) conception of research workflow responsibility eligibility PID data publication data sharing research administrator (RA) manager contributor viewer research documentation : a container of data (files/folders) metadata has a state attribute data analysis 5 * Box icons are made by Roundicons and Pixel Buddha from are licensed by CC 3.0 BY.

11 The data repository a irods-based ICT system implementing the workflow defined by the protocol a single data-management system enabling internal collaboration, and external data sharing management portal irods interfaces WebDAV data management middleware management rules storage system Stager ELK stack file-based system single and uniform namespace access-controlled metadata management authentication via trusted identity providers role-based authorisation data replication for disaster recovery workflow automation and policy enforcement 6

12 Storage resources data of s of OU2 asynchronous data replication vault_ou1_1 vault_ou1_2 vault_ou2_1 vault_ou2_2 quota quota resc_ou2 quota resc_ou1 quota filesystem irods resources data of s of OU1 resc_nl vault_nl_1 dataflow controle load-balancing/ou-level quota NFS export storage NFS export two identical copies of data (disaster recovery) Location A (first replica) Location B (second replica) 7

13 Collection namespace namespace reflects administrative hierarchy metadata in KVU triplets role-based authorisation with irods groups irods zone organisation organisational unit DRDR /rdm/di/dccn/dac_ _123/ 8 admin viewer manager contributor viewer

14 Management rules a RPC-like interface for management Client inputs: *collname, *kvp Server (core.re) rdmupdatecollectionmetadata Filter out attributes the client doesn t have right to set Verify attribute value Update attributes output: *errorcode, *Attrs return up-to-date attributes 9

15 User provisioning and authentication authentication via a national federated IdP user is provisioned upon sign-up to the management portal IdP attributes are stored as user KVU-triplets in irods setup PAM authentication on OTP (one-time password) for data access 10

16 Event logging essential user actions are logged as events non-blocking way streaming events to Elastic stack via filebeat reporting icat PEP rodslog filebeat auditing 11

17 User interfaces separating data-access from management web portal for management WebDAV (Davrods) for easy data transfer file stager: a service intelligently managing bulk file transfer between a local storage and the repository file stager web interface transfer job manager/queue transfer agents local storage actual transfer irsync DRDR 12

18 Strength and weakness It fits to a combined scientific-administrative workflow of a large and heterogeneous institute It provides sufficient functionality for 1. sharing data for publication 2. implementing Data Management Plan (DMP) It has weak integration with data analysis facility It doesn t implement standard way of organising content 13

19 Future focuses seamless integration with computing (dataanalysis) facility FAIR-ness of published data s adoption beyond neuroimaging 14

20 Summary We structured a RDM workflow which covers the entire research lifecycle in which both researcher and administrator take part of responsibility which is specified by protocol; implemented by a irodsbased digital repository 15