Developing Online Databases and Serving Biological Research Data

Transcription

1 Developing Online Databases and Serving Biological Research Data 1

2 Last Time PHP scripts can be used to create better web interfaces for specific database schemas If you want something more user friendly than PHPMyAdmin's scaffolding, you have to learn a web programming language such as PHP 2

3 Last Time If you choose PHP as your web programming language to build a better web interface, you will have to have an understanding of PHP, SQL, and HTML Fortunately in learning these languages you have a number of resources at your disposal 3

4 Last Time Using PHP Scaffolding that has been automatically generated from your schema isn't a bad start Even though these interfaces aren't as useful, You can get a sense of how PHP can query MySQL You can 'piece together' the code from multiple files into a single, more usable interface 4

5 Last Time Approach to building better interfaces: Decide what you want your user to see (view) e.g.: all species view, specific species view Make static HTML files that correspond to each (view), format them such that they can be filled out easily using PHP (indentation) Figure out which queries each of these views will need in order to do their job (Scaffolding can help) Turn each view into a PHP script (.php file), that executes the necessary SQL commands, and prints (echos) the HTML along side the SQL results 5

6 Database Tier Day 1&2 Server Tier Day 3 &4 Database Server Web Server Network Intermediary Tier Day 5 Client Tier Client Browser GBIF Server Network Network Network 6

7 Darwin Core Darwin Core (DwC) is a standard set of terms that biological data can be 'mapped' to Developed and maintained by the Biodiversity Information Standards non-profit group Formerly TDWG (The International Working Group on Taxonomic Databases) 7

8 Darwin Core A set of terms to which database columns can be mapped These terms essentially get everyone on 'the same page' and allow for generically built database schemas to store other researcher's data as long as they can be mapped onto the same set of terms 8

9 Darwin Core Terms 9

10 Darwin Core In addition to being a unifier of research database fields, it is an interchange format Once database fields are mapped to Darwin core terms, they can be shared in an XML (extensible Markup Language) format Darwin core XML looks quite similar to HTML, but it's tags are defined by the Biodiversity Information Standards (TDWG) 10

11 11

12 GBIF GBIF - Global Biodiversity Information Facility, an organization that facilitates sharing of biological data through it's web portal Records are published to GBIF through publishing software - typically these are web applications GBIF IPT- Integrated Publishing Toolkit (described) TapirLink (described) Digr BioCASE 12

13 GBIF 13

14 GBIF Integrated Publishing Toolkit GBIF IPT (Integrated Publishing Toolkit) Easy registration Used for publishing mapped subsets of records in a database (resource) Real time record holdings as well as static archives Can map records using darwin core terms GBIF IPT Wiki: 14

15 15

16 IPT 16

17 IPT 17

18 Types of Records 18

19 GBIF In order to get your data indexed and searchable through gbif.org, you must first register Within ITP automatically Or for other publishing methods, contact them via

20 GBIF In order to share records to GBIF, a database needs to be transformed into a format that GBIF knows how to index Darwin Core standard 20

21 GBIF Integrated Publishing Toolkit GBIF IPT (Integrated Publishing Toolkit) Easy registration Used for publishing mapped subsets of records in a database (resource) Real time record holdings as well as static archives Can map records using darwin core terms GBIF IPT Wiki: 21

22 GBIF 22

23 TapirLink A PHP web application for mapping arbitrary database schemas to Darwin Core Terms, and making them available in DwC XML format over the web 23

24 24

25 TapirLink Instillation of TapirLink consist of: Downloading the software from: Unzipping the files and copying them to a PHP web server 25

26 TapirLink Configuring a resource to be shared requires mapping the schema using the configure.php interface 26

27 Hands on The following directions assume that Uconn's central web server, directions for different hosting companies will differ If you have your own hosting, you can upload this file to your own web server, via any means they provide, such as FTP or SSH 27

28 TapirLink Go to: Click the add button 28

29 TapirLink Step 1 29

30 TapirLink - Step1 Type your initials into the Location ID field 30

31 TapirLink Step 1 Click inside the 'Accesspoint' field Change: To Add: www/ Verify that LOCAL_ID is identical to the 'Local ID' you entered (i.e., your initials) 31

32 TapirLink Step1 Fill in all the other required fields that are marked with an (*) You don't have to fill in real information, as we are not sharing with GBIF Normally we would, but UITS's web server is not configured to use TapirLink correctly When all the required fields have been filled in, click the 'next step' button 32

33 TapirLink Step2 Here we configure the database connection Fill in the information from the next slide 33

34 TapirLink Step2 Driver: MySQL PHP driver with transaction support Database Encoding: UTF-8 Datasource String: mysql Password: (up on board) Database name: cyberinfra14 34

35 TapirLink - Step2 Click the 'next step' button 35

36 TapirLink - Step3 Select CestodeSpecies.idCestodeSpecies from the 'Root table and key field' 36

37 TapirLink Step 3 37

38 TapirLink Step3 New Join: --between--: CestodeSpecies.idCestodeSpecies CestodeOrder.FK_idCestodeSpecies --and-- : Then click the add button 38

39 TapirLink Step3 New Join: --between--: CestodeSpecies.idCestodeSpecies CestodeFamily.FK_idCestodeSpecies --and-- : Then click the add button 39

40 TapirLink Step3 New Join: --between--: CestodeSpecies.idCestodeSpecies CestodeGenus.FK_idCestodeSpecies --and-- : Then click the add button 40

41 TapirLink Step3 We just selected the portions of each record we wish to share Now click the 'next step' button 41

42 TapirLink - Step4 Click 'next step' 42

43 TapirLink Step 5 Check 'DarwinCore terms ( )' Click 'load the specified schema above' (We will be mapping our selected portions of the records to DarwinCore terms next) 43

44 TapirLink - Step5 A large number of selectable fields will appear The left hand column lists Darwin Core Terms The right hand column allows you to map values from your database to the corresponding Darwin Core Terms Darwin Core Term Mapping 44

45 TapirLink Step5 For the scientificname term configure the mapping as follows: Change unmapped-- to --single column-- Change table-- to CestodeSpecies Change column-- to CestodeSpeciesName --type-- should automatically fill to text 45

46 TapirLink Step 5 For the Order term configure the mapping as follows: Change unmapped-- to --single column-- Change table-- to CestodeOrder Change column-- to CestodeOrder --type-- should automatically fill to text 46

47 TapirLink Step 5 For the Family term configure the mapping as follows: Change unmapped-- to --single column-- Change table-- to CestodeFamily Change column-- to CestodeFamily --type-- should automatically fill to text 47

48 TapirLink Step 5 For the Genus term configure the mapping as follows: Change unmapped-- to --single column-- Change table-- to CestodeGenus Change column-- to CestodeGenus --type-- should automatically fill to text 48

49 TapirLink - Step5 Click the 'next step' button at the bottom 49

50 TapirLink Step 6 Click the 'set to now' button in the Date last modified row Click the 'save new resource' button 50

51 TapirLink You can click the link with your initials on the left to run a test to make sure the resource would be available to GBIF Click the link that is your initials 51

52 TapirLink Now you are in the resource specific view Next we will test the 'Accesspoint' link that GBIF will use to access your data Next, we will see what our mapped data looks like to GBIF 52

53 TapirLink Unfortunately the web server we're using isn't configured to work correctly with TapirLink, hence the 'Accesspoint' error 53

54 TapirLink The Accesspoint link won't work as is, so copy the link and open it in a new browser tab, and change the URL as follows: Add 's' to the end of 'http' to make it 'https' 54

55 TapirLink What do you see? You might see something boring like this: What happened to our mapping? This URL doesn't ask for data it just defaults to giving some general information about the provider 55

56 TapirLink In order to get the browser to display the mapped records, add the following to the end of the Accesspoint URL:?op=inventory&concept= This modified link should look familiar, in fact TapirLink uses $_GET to make requests for data Once you have modified the Accesspoint URL, press enter in order to have the browser send a new request to the Accesspoint 56

57 TapirLink Now we (sort of) see the family, order, and species name of each record, but this might not be clear depending on what browser you are using It is XML interchange format being displayed Chrome 57

58 TapirLink Viewing the source, or 'saving the page as' and opening it in a syntax highlighting text editor will show you what it really looks like: Chrome: FireFox View Source 58

59 Workflow (Day 1) (Day 2) (Day 3&4) (Day 5) (Day 5) 59

60 (Day 1) (Day 2) (Day 5) (Day 3&4) (Day 5) 60

61 Database Tier Day 1&2 Server Tier Day 3 &4 Database Server Web Server Network Intermediary Tier Day 5 Client Tier Client Browser GBIF Server Network Network Network 61