Open Data and its enemies

Transcription

1 Open Data and its enemies Dries van Oosten Debye Institute for NanoMaterials Science Center for Extreme Matter and Emergent Phenomena 1

2 Content 2

3 FAIR Findable Accessible Interoperable Reusable 3

4 4

5 Funding agencies (will) demand it 4

6 Funding agencies (will) demand it Less room for monkey business 4

7 Funding agencies (will) demand it Less room for monkey business Contributions can be made by groups that cannot obtain the data 4

8 Why not Why do we fear opening our data? 5

9 Why not Why do we fear opening our data? What if someone does better with our data? 5

10 Why not Why do we fear opening our data? What if someone does better with our data? What if someone does something horrible with our data? 5

11 Why not Why do we fear opening our data? What if someone does better with our data? What if someone does something horrible with our data? Preparing data to be FAIR is a lot of work 5

12 Why not Why do we fear opening our data? What if someone does better with our data? What if someone does something horrible with our data? Preparing data to be FAIR is a lot of work What does interoperability even mean? 5

13 Why not What if someone does better with our data? 6

14 Why not What if someone does better with our data? Good for science, right? 6

15 Why not What if someone does better with our data? Good for science, right? Bad for us, no glossy no grant! 6

16 Why not What if someone does better with our data? Good for science, right? Bad for us, no glossy no grant! This is why publishing of data should count! 6

17 Why not What if someone does something horrible with our data? 7

18 Why not What if someone does something horrible with our data? Monkey business: people (sadly) photoshop data 7

19 Why not What if someone does something horrible with our data? Monkey business: people (sadly) photoshop data What if they do this with our data? 7

20 Why not What if someone does something horrible with our data? Monkey business: people (sadly) photoshop data What if they do this with our data? Will this reflect badly on us? 7

21 Why not What if someone does something horrible with our data? Monkey business: people (sadly) photoshop data What if they do this with our data? Will this reflect badly on us? We should be able to show this was their doing! 7

22 Why not Preparing data to be FAIR is a lot of work 8

23 Why not Preparing data to be FAIR is a lot of work I Our data storage system might look like this 8

24 Why not Preparing data to be FAIR is a lot of work I Our data storage system might look like this 8

25 Why not Preparing data to be FAIR is a lot of work I Our data storage system might look like this I The metadata are the handwritten notes of our students 8

26 Why not Preparing data to be FAIR is a lot of work I Our data storage system might look like this I The metadata are the handwritten notes of our students I Or in an electronic labbook that is abandonware 8

27 Why not Preparing data to be FAIR is a lot of work I Our data storage system might look like this I The metadata are the handwritten notes of our students I Or in an electronic labbook that is abandonware All true, but we already know this is bad! 8

28 Why not Interoperability, what does it even mean? 9

29 Why not Interoperability, what does it even mean? Ready to be combined with other datasets by humans as well as computer systems 9

30 Why not Interoperability, what does it even mean? Ready to be combined with other datasets by humans as well as computer systems No vendor lock-in! 9

31 Why not Interoperability, what does it even mean? Ready to be combined with other datasets by humans as well as computer systems No vendor lock-in! No old pc in the corner to open old files 9

32 Why not Interoperability, what does it even mean? Ready to be combined with other datasets by humans as well as computer systems No vendor lock-in! No old pc in the corner to open old files Again, we already know we hate that old pc! 9

33 Why not Interoperability, what does that mean in 10 years? 10

34 Why not Interoperability, what does that mean in 10 years? 10

35 11

36 Easy to use (so we use it) 11

37 Easy to use (so we use it) In our own influence sphere (so it will continue to exist) 11

38 Easy to use (so we use it) In our own influence sphere (so it will continue to exist) Using open standards (preferably an existing one) 11

39 Easy to use (so we use it) In our own influence sphere (so it will continue to exist) Using open standards (preferably an existing one) 11

40 Easy to use (so we use it) In our own influence sphere (so it will continue to exist) Using open standards (preferably an existing one) With data protection, auditing, etc. 11

41 Our workflow 12

42 Our workflow Data publication right before submission of manuscript 12

43 Our workflow Data publication right before submission of manuscript Contains all data and all Python scripts to generate graphs 12

44 Our workflow Data publication right before submission of manuscript Contains all data and all Python scripts to generate graphs Python scripts are used as roadmap 12

45 Our workflow Data publication right before submission of manuscript Contains all data and all Python scripts to generate graphs Python scripts are used as roadmap Scripts require no user input 12

46 Our workflow Data publication right before submission of manuscript Contains all data and all Python scripts to generate graphs Python scripts are used as roadmap Scripts require no user input All scripts and data under Creative Commons license 12

47 Our workflow Data publication right before submission of manuscript Contains all data and all Python scripts to generate graphs Python scripts are used as roadmap Scripts require no user input All scripts and data under Creative Commons license No proprietary software used 12

48 Our workflow Data publication right before submission of manuscript Contains all data and all Python scripts to generate graphs Python scripts are used as roadmap Scripts require no user input All scripts and data under Creative Commons license No proprietary software used I ask most of this of my students anyway! 12

49 YODA Some nerding out 13

50 YODA Some nerding out Use irods (integrated Rule-Oriented Data management System) 13

51 YODA Some nerding out Use irods (integrated Rule-Oriented Data management System) data grid software 13

52 YODA Some nerding out Use irods (integrated Rule-Oriented Data management System) data grid software order and use data spread across multiple locations 13

53 YODA Some nerding out Use irods (integrated Rule-Oriented Data management System) data grid software order and use data spread across multiple locations with their metadata 13

54 YODA Some nerding out Use irods (integrated Rule-Oriented Data management System) data grid software order and use data spread across multiple locations with their metadata Rules to determine who can do what etc. 13

55 YODA Some nerding out Use irods (integrated Rule-Oriented Data management System) data grid software order and use data spread across multiple locations with their metadata Rules to determine who can do what etc. Interfaces that hide that complexity 13

56 YODA portal *.yoda.uu.nl network-disk *.data.uu.nl Apache Web Server administra on i-commands irods API UU Data Policies and -services irods 14

57 YODA portal *.yoda.uu.nl network-disk *.data.uu.nl Apache Web Server administra on i-commands irods API UU Data Policies and -services irods Blue: open source projects 14

58 YODA portal *.yoda.uu.nl network-disk *.data.uu.nl Apache Web Server administra on i-commands irods API UU Data Policies and -services irods Blue: open source projects Yellow: UU developed (also open sourced) 14

59 Our YODA workflow 15

60 Our YODA workflow Store data (looks like a shared volume) 15

61 Our YODA workflow Store data (looks like a shared volume) Fill out metadata (through command line or web portal) 15

62 Our YODA workflow Store data (looks like a shared volume) Fill out metadata (through command line or web portal) Lock data 15

63 Our YODA workflow Store data (looks like a shared volume) Fill out metadata (through command line or web portal) Lock data Submit to vault (approved by groupleader) 15

64 Our YODA workflow Store data (looks like a shared volume) Fill out metadata (through command line or web portal) Lock data Submit to vault (approved by groupleader) Submit Publish (approved by data manager) 15

65 Our YODA workflow Store data (looks like a shared volume) Fill out metadata (through command line or web portal) Lock data Submit to vault (approved by groupleader) Submit Publish (approved by data manager) DOI assigned, leads to landing page with metadata 15

66 Our YODA workflow Store data (looks like a shared volume) Fill out metadata (through command line or web portal) Lock data Submit to vault (approved by groupleader) Submit Publish (approved by data manager) DOI assigned, leads to landing page with metadata And data itself, if you wish 15

67 Our YODA workflow Store data (looks like a shared volume) Fill out metadata (through command line or web portal) Lock data Submit to vault (approved by groupleader) Submit Publish (approved by data manager) DOI assigned, leads to landing page with metadata And data itself, if you wish Cite data in your manuscript 15

68 YODA 16

69 YODA You can download that data 16

70 YODA You can download that data Look for the figure that caught your eye 16

71 YODA You can download that data Look for the figure that caught your eye Find the accompanying Python script 16

72 YODA You can download that data Look for the figure that caught your eye Find the accompanying Python script And run it (or modify it first) 16

73 YODA You can download that data Look for the figure that caught your eye Find the accompanying Python script And run it (or modify it first) Use it in your work (just cite the data package) 16

74 YODA 17

75 YODA Under our own control, immediate support (ITS-UU) 17

76 YODA Under our own control, immediate support (ITS-UU) Low cost ( 2 e/ TB/ Month) 17

77 YODA Under our own control, immediate support (ITS-UU) Low cost ( 2 e/ TB/ Month) Redundant (across at least two locations) 17

78 YODA Under our own control, immediate support (ITS-UU) Low cost ( 2 e/ TB/ Month) Redundant (across at least two locations) So easy to use there is no reason not to! 17

79 YODA Under our own control, immediate support (ITS-UU) Low cost ( 2 e/ TB/ Month) Redundant (across at least two locations) So easy to use there is no reason not to! Does this solve everything? 17

80 What are the enemies of open data? 18

81 What are the enemies of open data? Our own lack of discipline 18

82 What are the enemies of open data? Our own lack of discipline Assessment policies, these have to be modernized 18

83 What are the enemies of open data? Our own lack of discipline Assessment policies, these have to be modernized See related discussion on Open Access publishing 18

84 What are the enemies of open data? Our own lack of discipline Assessment policies, these have to be modernized See related discussion on Open Access publishing Propriety software/formats 18

85 What are the enemies of open data? Our own lack of discipline Assessment policies, these have to be modernized See related discussion on Open Access publishing Propriety software/formats These are by definition not interoperable 18

86 What are the enemies of open data? Our own lack of discipline Assessment policies, these have to be modernized See related discussion on Open Access publishing Propriety software/formats These are by definition not interoperable But some hard- or software might force you 18

87 What are the enemies of open data? Our own lack of discipline Assessment policies, these have to be modernized See related discussion on Open Access publishing Propriety software/formats These are by definition not interoperable But some hard- or software might force you In the long term we should not accept this 18

88 What are the enemies of open data? Our own lack of discipline Assessment policies, these have to be modernized See related discussion on Open Access publishing... Propriety software/formats These are by definition not interoperable But some hard- or software might force you In the long term we should not accept this 18