Introduction to RDF and the Semantic Web for the life sciences

Transcription

1 Introduction to RDF and the Semantic Web for the life sciences Simon Jupp Sample Phenotypes and Ontologies Team European Bioinformatics Institute

2 Practical sessions Converting data to RDF Three questions 1. What types of things are in my data? 2. Can I identify these things? 3. How are these things related to other things?

3 Gene expression data example Experiment Gene name Ensembl id organism organism_part expression t- stat p- value E- TABM- 865 Ms4a1 ENSMUSG mus musculus liver DOWN E- 34 E- TABM- 865 Ms4a1 ENSMUSG mus musculus spleen UP E- 34 E- TABM- 865 MMp ENSMUSG mus musculus liver UP E- 34 E- TABM- 865 MMp ENSMUSG mus musculus spleen DOWN E- 34 E- TABM- 865 Akr1c14 ENSMUSG mus musculus liver UP E- 33 E- TABM- 865 Akr1c14 ENSMUSG mus musculus spleen DOWN E- 33 E- TABM- 865 Gulo ENSMUSG mus musculus liver UP E- 33 E- TABM- 865 Gulo ENSMUSG mus musculus spleen DOWN E- 33 E- TABM- 865 Marc1 ENSMUSG mus musculus liver UP E- 33 E- TABM- 865 Marc1 ENSMUSG mus musculus spleen DOWN E- 33 E- GEOD Gulo ENSRNOG ramus norvegicus kidney DOWN E- 42 E- GEOD Gulo ENSRNOG ramus norvegicus liver UP E- 42 E- GEOD Akr1c14 ENSRNOG ramus norvegicus kidney DOWN E- 39 E- GEOD Akr1c14 ENSRNOG ramus norvegicus liver UP E- 39 E- GEOD Akr1c14 ENSRNOG ramus norvegicus kidney DOWN E- 25 E- GEOD Akr1c14 ENSRNOG ramus norvegicus liver UP E- 25 E- GEOD Amacr ENSRNOG ramus norvegicus kidney DOWN E- 08 E- GEOD Amacr ENSRNOG ramus norvegicus liver UP E- 08

4 What is it? What concepts do we have in this dataset? Some hints are already in the column names Experiment Gene name Ensembl id organism organism_part expression t- stat p- value E- TABM- 865 Ms4a1 ENSMUSG mus musculus liver DOWN E- 34 E- TABM- 865 Ms4a1 ENSMUSG mus musculus spleen UP E- 34 E- TABM- 865 MMp ENSMUSG mus musculus liver UP E- 34 E- TABM- 865 MMp ENSMUSG mus musculus spleen DOWN E- 34 E- TABM- 865 Akr1c14 ENSMUSG mus musculus liver UP E- 33 E- TABM- 865 Akr1c14 ENSMUSG mus musculus spleen DOWN E- 33 E- TABM- 865 Gulo ENSMUSG mus musculus liver UP E- 33 E- TABM- 865 Gulo ENSMUSG mus musculus spleen DOWN E- 33 E- TABM- 865 Marc1 ENSMUSG mus musculus liver UP E- 33 E- TABM- 865 Marc1 ENSMUSG mus musculus spleen DOWN E- 33 E- GEOD Gulo ENSRNOG ramus norvegicus kidney DOWN E- 42 E- GEOD Gulo ENSRNOG ramus norvegicus liver UP E- 42 E- GEOD Akr1c14 ENSRNOG ramus norvegicus kidney DOWN E- 39 E- GEOD Akr1c14 ENSRNOG ramus norvegicus liver UP E- 39 E- GEOD Akr1c14 ENSRNOG ramus norvegicus kidney DOWN E- 25 E- GEOD Akr1c14 ENSRNOG ramus norvegicus liver UP E- 25 E- GEOD Amacr ENSRNOG ramus norvegicus kidney DOWN E- 08 E- GEOD Amacr ENSRNOG ramus norvegicus liver UP E- 08

5 Exercise 1 Concept maps Write down the concepts represented in this data (e.g. Experiment) Organise the concepts into a graph and write down some relationships between the concepts

6 Exercise 1 solution Gene name Experiment Has result Expression Value Ensembl gene Gene name Ensembl id Organism Factor value Factor value Experimental factor T-statistic T- stasssc P-value P- value Congratulations on building your first Ontology!

7 Instance vs Types The world (of information) is made up of things and lots of them Instances, individuals, objects, tokens, particulars. The Earth is a kind of Planet Simon Jupp (NE A) is a Person E-MTAB-62 is a type of Experiment our liver is a type of Organ

8 Exercise 2 Identify Types vs Instance data Experiment E- TABM- 865 Ms4a1 Gene name ensembl id Instance Type ENSMUSG organism mus musculus organism_part liver expression DOWN t- stat p- value 8.40E- 34

9 Exercise 2 solution Experiment E- TABM- 865 Ms4a1 Gene name ensembl id Instance Type ENSMUSG organism mus musculus organism_part liver expression DOWN t- stat p- value 8.40E- 34

10 Giving things identity Choose a URI scheme for resources. Re-use URIs for types of things where possible Shared URIs for the same things make integration happen General rule 1. If it s your data, give it a URI in your namespace. 2. If it s someone else's data (e.g. UniProt) use a URI from them (if they have one)

11 Exercise 3 your data vs shared data Experiment E- TABM- 865 Ms4a1 Gene name ensembl id Instance Type Mine ENSMUSG N organism mus musculus organism_part liver expression DOWN t- stat p- value E- 34

12 Exercise 3 solution Instance Type Mine Experiment N E- TABM- 865 Ms4a1 N Gene name N ensembl id N ENSMUSG N organism N mus musculus N organism_part N liver N expression N DOWN N t- stat N p- value N 8.40E- 34 our data is usually the instance data (the experiment or the results) Types of things usually belong in external reference ontologies. Good practice try and connect your data to these ontologies

13 URI for a instance Ensembl Gene ENSMUSG g=ensmusg Is this a good URI? Is it stable? What does it represent? This is a URL for the web page, it may change It doesn t return RDF

14 Identifiers.org Identifiers.org is a system providing resolvable persistent URIs used to identify data for the scientific community, with a current focus on the Life Sciences domain. The provision of a resolvable identifiers (URLs) fits well with the Semantic Web vision, and the Linked Data initiative.

15 Exercise 4 Use the identifiers.org website to find the URI for ENSMUSG

16 Exercise 4 solution Search identifiers.org for ensembl Got to Find root URL See what it resolves to

17 URI for types Experimental factor liver liver is an organ. We would expect to find an ontology term that describes what a liver is BioPortal is a repository or bio-medical ontologies

18 Exercise 5 Go to and find ontologies that contain terms for liver, spleen and kidney Get the URIs for liver, spleen and kidney from the Experimental Factor Ontology (EFO)

19 Exercise 5 solution liver spleen kidney

20 Exercise 5 Find URIs using BioPortal for types and identifiers.org for instances Restrict types search to EFO, UBERON, SIO, OBI and EDAM Ontology Instance Type Mine URI Experiment N E- TABM- 865 Ms4a1 N Gene name N ensembl id N ENSMUSG N organism N mus musculus N organism_part N liver N expression N DOWN N t- stat N p- value N 8.40E- 34

21 Exercise 5 solution- find some more URIs Instance Type Mine URI Experiment N E- TABM- 865 N/A Ms4a1 N N/A this is just a label for the ensembl gene Gene name N ensembl id N ENSMUSG N organism N mus musculus N organism_part N liver N expression N DOWN N N/A t- stat N p- value N E- 34 N/A

22 Building the RDF graph We have identified our types with URIs We know what data is ours Now we need to translate each row in the file to an RDF representation using N-triples <Subject> <Predicate> <Object> Remember the Object can be a URI or a value For predicates create URIs in our own namespace

23 Example row conversion to RDF E- TABM- 865 Ms4a1 ENSMUSG mus musculus liver DOWN E- 34 Experiment type E- TABM- 865 RDF Triples SUBJECT PREDICATE OBJECT mydata:e- TABM- 865 rdf:type efo:efo_

24 Example row conversion to RDF E- TABM- 865 Ms4a1 ENSMUSG mus musculus liver DOWN E- 34 Experiment type E- TABM- 865 has result Down Expression Value mydata:result1 type RDF Triples SUBJECT PREDICATE OBJECT mydata:e- TABM- 865 rdf:type efo:efo_ mydata:e- TABM- 865 mydata:hasresult mydata:result1 mydata:result1 rdf:type sio:sio_001078

25 Exercise 6 Using the following schema write out some RDF in N-triples to represent this single row of data E- TABM- 865 Ms4a1 ENSMUSG mus musculus liver DOWN E- 34 Gene name Experiment has result Expression Value dbxref label Ensembl id Organism Factor value Factor value Experimental factor T-stat T- stasssc P-value P- value

26 Exercise 6 solution E- TABM- 865 Ms4a1 ENSMUSG mus musculus liver DOWN E- 34 RDF Triples SUBJECT PREDICATE OBJECT mydata:e- TABM- 865 rdf:type efo:efo_ mydata:e- TABM- 865 mydata:hasresult mydata:result1 mydata:result1 rdf:type sio:sio_ mydata:result1 mydata:factorvalue obo:ncbitaxon_10090 mydata:result1 mydata:factorvalue obo:uberon_ mydata:result1 mydata:t- stat mydata:result1 mydata:p- value 8.40E- 34 mydata:result1 mydata:dbxref idensfiers:ensmusg idensfiers:ensmusg rdfs:label Ms4a1

27 Generating RDF CSV2RDF OpenRefine Scripts Output serialised RDF Simple to print out N3 to files Use an RDF API Most programming language will have RDF libraries Other options RDB2RDF: Work directly off your relational database

28 A simple CSV 2 RDF in Perl Example script data2rdf.pl Read input file (raw-data.csv) Convert rows into triple statements according to my schema Generate appropriate URIs for things Print out triple statement in simple N3 format

29 Exercise 7 Look at the N-triple file generated (raw-data.rdf) See if you understand how that translates to the Schema Convert this file to RDF/XML using online converter

30 Blank nodes (bnode) ou can use an anonymous resource in RDF They can be the subject or object of any triple Denote the existence of a thing but you don t have to explicitly give it a URI In our scenario we created a URI for the Gene expression value, we didn t have to Using turtle syntax we could have said mydata:e- TABM- 865 rdf:type efo:efo_ mydata:e- TABM- 865 mydata:hasresult [ rdf:type sio:sio_ ; mydata:factorvalue obo:uberon_ ; mydata:t- stat ]

31 Querying RDF Specialised databases for indexing RDF graphs Apache Jena OWLIM Allegrograph Stardog Virtuoso Sesame

32 OpenRDF sesame OpenRDF Sesame is a de-facto standard framework for processing RDF data. This includes parsers, storage solutions (RDF databases a.ka. triplestores), reasoning and querying, using the SPARQL query language. It offers a flexible and easy to use Java API that can be connected to all leading RDF storage solutions. Easy to deploy (Java servlet) Provides SPARQL endpoint and workbench for administration tasks Scalable to millions of triples Other more scalable implementations of the storage and inference layer available OWLIM Virtuoso Bigdata

33 The Sesame workbench We have a workbench online for you to play with ( Use this to create a repository Upload data Test queries

34 Exercise 8 Create a new in memory store repository for your data

35 Exercise 9 Load RDF Data file (use raw-data.rdf form the dropbox folder) Set Data format to N-Triples Set base URI to

36 SPARQL endpoint

37 Exploring a SPARQL endpoint Show me some triples SELECT * WHERE {?subject?predicate?object } Select all data = not a very friendly query! Find the types of things PREFIX rdf:< SELECT DISTINCT?type WHERE {?subject rdf:type?type } LIMIT 10

38 Describing a resource What is known about DESCRIBE <

39 Exercise 11 SPARQL endpoint Try some of the previous queries on the SPARQL endpoint Explore clicking around URIs to follow links through the data Explore download formats SPARQL query results XML, JSON, CSV

40 Binding variables Get all things that are types of experiment Experiment URI PREFIX rdf:< PREFIX rdfs:< PREFIX efo:< SELECT DISTINCT?thing WHERE {?thing rdf:type efo:efo_ } LIMIT 10

41 Exercise 12 Write a SPARQL query to get the labels for all experiments (hint: Use the rdfs:label relation) Tip: Store SPARQL queries that work in a text file, easier to edit and re-use previous queries

42 Exercise 12 solution Select labels for all classes PREFIX rdf:< PREFIX rdfs:< PREFIX efo:< SELECT DISTINCT?label WHERE {?thing rdf:type efo:efo_ ?thing rdfs:label?label }

43 Exercise 13 Explore the raw-data.rdf files and try and write a SPARQL query that would show you all the genes UP in liver samples Hint: UP = liver =

44 Exercise 13 solution Get genes up regulated in liver samples PREFIX rdf:< PREFIX rdfs:< PREFIX mydata:< PREFIX sio:< PREFIX obo:< SELECT DISTINCT?geneid?label WHERE {?result mydata:dbxref?geneid.?geneid rdfs:label?label.?result rdf:type sio:sio_ ?result mydata:hasfactorvalue obo:uberon_ }

45 Filtering SPARQL queries Restrict values in results from matches in the graph patterns String matching FILTER regex(?x, "pattern" [, "flags"]) E.g. FILTER regex (?label, E-TABM-865 ) Testing values FILTER (?tstat >0 24)

46 Exercise 14 Get all experiments where label contain GEOD Get all genes up regulated with a t-statistic < 0

47 Exercise 14 solutions PREFIX rdf:< PREFIX rdfs:< PREFIX efo:< SELECT DISTINCT?label WHERE {?thing rdf:type efo:efo_ ?thing rdfs:label?label. FILTER regex(?label, "geod", "i") } PREFIX rdfs:< PREFIX mydata:< SELECT DISTINCT?geneid?label?tstat WHERE {?result mydata:dbxref?geneid.?geneid rdfs:label?label.?result mydata:haststatistic?tstat. FILTER (?tstat < 0) }

48 Enriching data Our dataset is still a bit sparse e.g. no labels or descriptions for sample information We used URIs form external ontologies to define some concepts Let s integrate our dataset with those ontologies and do some querying

49 Exercise 15 Find the Experimental Factor Ontology ontology file Can get from Web or efo.owl in the course material Load the ontology file into the same repository as your raw data RDF Now describe the liver URI Create a SPARQL query to pull out labels for all of the factor values

50 Exercise 15 solution DESCRIBE < PREFIX rdfs:< PREFIX mydata:< SELECT DISTINCT?factor?label WHERE {?result mydata:hasfactorvalue?factor.?factor rdfs:label?label }

51 Exploiting knowledge As an ontology, EFO contains lots of biological domain knowledge E.g. classification of diseases, organism parts etc.. We can exploit this knowledge to enhance queries over our datasets E.g. What are all the parent types (or categories) for liver in EFO PREFIX rdfs:< PREFIX obo:< SELECT DISTINCT?parent?label WHERE { obo:uberon_ rdfs:subclassof?parent.?parent rdfs:label?label }

52 Property paths We can query along paths of relations using SPARQL This is useful for exploiting transitive relationships Special SPARQL 1.1 syntax for property paths * PREFIX rdfs:< PREFIX obo:< SELECT DISTINCT?parent?label WHERE { obo:uberon_ rdfs:subclassof*?parent.?parent rdfs:label?label }

53 Exercise 16 Ontology query Get all genes expressed in your data where the factor values is a child of organism part (efo:efo_ )

54 Exercise 16 solution Get all genes expressed in your data where the factor values is a child of organism part (efo:efo_ ) PREFIX rdfs:< PREFIX mydata:< PREFIX efo:< SELECT DISTINCT?geneid?label?factor WHERE {?result mydata:dbxref?geneid.?geneid rdfs:label?label.?result mydata:hasfactorvalue?factor.?factor rdfs:subclassof* efo:efo_ }

55 End of 1 st practical session Introduced modeling data in RDF Three questions I always ask of data What is it (types)? What is it (id)? What is it related to? Generating RDF statements in N-Triples Loading RDF into a triple store Basic querying with SPARQL