Linked Life Data. Vassil Momtchev 19/04/2011

Transcription

1 Linked Life Data Vassil Momtchev 19/04/2011

2 Outline Seman2c Data Integra2on Linked Life Data concept Integrated datasets Behind the scene #2

3 Interlinking Text and Data #3

4 Seman6c Technologies vs. AI If It Works, It's Not AI: A Commercial Look at Ar7ficial Intelligence Startups Eve M. Phillips, M.Sc. Thesis, 1999 MIT One can think of Seman2c Technologies like as AI, made less abstract and more robust, predictable and manageable #4

5 Seman6c Technologies Seman6c technologies (ST) is a general term for any sooware that involves some kind and level of understanding the meaning of the informa2on it deals with Examples: A search engine that can match a query for bird with a document men2oning eagle A database that will return Ivan as a result of a query for?x rela.veof Maria, when the fact asserted was Maria motherof Ivan A naviga2on system that is more intelligent than what we are already used to #5

6 Ontotext Posi6oning Leading seman6c technology provider Top- 5 core seman2c technology developer Supplying engines and components to vendors and solu2on developers Unique technology porvolio: Seman6c Databases: high- performance RDF DBMS, scalable reasoning Seman6c Search: text- mining (IE), Informa2on Retrieval (IR) Web Mining: focused crawling, screen scraping, data fusion Good recogni6on in the SemTech community Ontotext pages are ranked #1 for seman2c annota2on and seman2c repository at GYM #6

7 Time to Guess It? #7

8 Massive Data Integra6on Problem Extreme amount of data with inconsistent syntax, structure and seman2cs Data is supported by different organiza2ons Informa2on is highly distributed and redundant Knowledge is locked in vast data silos Isolated communi2es which could not reach cross- domain understanding Increase the abstrac6on level of the data! #8

9 Data representa6on: RDBMS vs. RDF Person ID Name Gender 1 Maria P. F 2 Ivan Jr. M 3 Parent Spouse ParID ChiID S1ID S2ID From To Rela6onal Tables Statement Subject Predicate Object myo:person rdf:type rdfs:class myo:gender rdfs:type rdfs:property myo:parent rdfs:range myo:person myo:spouse rdfs:range myo:person myd:maria rdf:type myo:person myd:maria rdf:label Maria P. myd:maria myo:gender F myd:maria rdf:label Ivan Jr. myd:ivan myo:gender M myd:maria myo:parent Myd:Ivan myd:maria myo:spouse myd:john RDF Representa6on #9

10 Data representa6on: XML vs. RDF <document> <person> <name>maria</name> <gender>f</gender> <rellist> <rel type= child >Ivan</rel> <relliist> </person> ptop:male ptop:person ptop:woman No agreement over the structure and the vocabulary Could not be seman2cally compared by machine rdf:type mydata:ivan ptop:childof rdf:type mydata: Maria XML Documents RDF Representa6on #10

11 RDF Graph inferred ptop:agent owl:symmetricproperty owl:inverseof rdf:type owl:rela6veof rdf:type owl:inverseof ptop:parentof rdfs:subpropertyof rdf:type mydata:ivan owl:inverseof ptop:person rdfs:range owl:inverseof ptop:childof rdf:type ptop:woman mydata: Maria #11

12 Linked Data Design Principles Unambiguous iden2fiers for objects (resources) Use URIs as names for things Use the structure of the web Use HTTP URIs so that people can look up the names Make is easy to discover informa2on about an object (resource) When someone lookups a URI, provide useful informa2on Link the object (resource) to related objects Include links to other URIs

13 PWC on Seman6c Technologies Spring of the data Web Technology forecast, A quarterly journal, Spring 2009, hqp:// techforecast/ #13

14 There is Nothing You Can Do There is nothing you can do with ontologies that cannot be done without them The same holds for language technology: given unlimited resources, all methods will deliver comparable results for any text analysis task (Y. Willks) BTW, there is also nothing you can on Java than cannot be done on Assembler #14

15 Conceptual idea LINKED LIFE DATA #15

16 Seman6c Data Integra6on Current A lot of biomedical data available on the web and internally Very hard to locate the informa2on and put it into context Scien2sts unable to u2lize exis2ng informa2on well Difficult to automa2cally combine public domain knowledge with private company exper2se Desired Single integra2on model based on linked data technology and open standards Computerized support to interpret the informa2on Assists scien2sts to combine internal data from experiments with external knowledge 16

17 The Original Idea Molecular Interac6on Disease Gene Target Protein Drugs Pa6ent #17

18 Data Integra6on Levels Semantics Structure Syntax Generalization/ specialization (Nexium vs. Esomeprazole) Homonyms, synonyms Different metric units Aggregation (full name with initials vs full name) Schema mismatch and internal path discrepancy File format (CSV, XML, flat file) Character encoding (ASCII, UTF-8, UTF-16) #18

19 System Levels in the Knowledge Driven Process Scientific Intelligence Linked Life Data Knowledge Operational Transactional Advanced visualiza2on and sta2s2cal analyzes Informa2on extrac2on Schema alignment Shared iden2fiers Data silos applica2ons Databases File system 19

20 Syntax and Structure Ambiguity RDF data model resolves all syntax level ambigui2es It helps you express all data in a common data model ID GRAA_HUMAN STANDARD; PRT; 262 AA. AC P12544; DT 01-OCT-1989 (Rel. 12, Created) DT 01-OCT-1989 (Rel. 12, Last sequence update) DT 15-JUN-2002 <PubmedArticle> (Rel. 41, <MedlineCitation Last annotation update) Owner="NLM" DE Granzyme Status="In-Process"> A precursor (EC <PMID ) (Cytotoxic T- lymphocyte Version="1"> </PMID> proteinase <DateCreated> DE 1) (Hanukkah <Year>2011</Year> factor) (H factor) <Month>04</Month> (HF) (Granzyme 1) (CTL tryptase) <Day>15</Day> </DateCreated> <Article DE (Fragmentin PubModel="Print"> 1). GN <Journal> <ISSN GZMA OR IssnType="Electronic"> </ISSN> CTLA3 OR HFSP. OS Homo sapiens (Human). <JournalIssue CitedMedium="Internet"> <Volume>82</Volume> <Issue>20</Issue> <PubDate> <Year>2010</Year> <Month>Oct</ Month> <Day>15</Day> </PubDate> </ JournalIssue> #20

21 Linked Data Mapping How well interlinked is the linked data cloud? Many interes2ng queries are difficult to be expressed in SPARQL String func2ons could not be index OOen there are misplaced iden2fiers biopax- 2:SHORT- NAME CD40L_HUMAN cpath:cpath CD40L_HUMAN biopax- 2:DB UNIPROT uniprot:mnemonic TNF5_HUMAN P29965 uniprot:mnemonic uniprot:p29965 CD4L_HUMAN cpath:cpath- LOCAL cpath:cpath- LOCAL #21

22 Linked Data Mappings Iden2fied 6 linked data integra2on paqerns Define meta- rules to connect resources with various predicates Manually controlled process Namespace mapping X ns- x: id Mismatched identifiers Transitive link Y ns- y: id X Y accession db: accession X db: id Y Reference node Value dereference Semantic Annotations The blue lines and the blue text of the cap2ons (used either as part of the URI or literals) designate the criteria for linking the informa2on Y db: id X X Y X name Y term db id text with name #22

23 Instance Level Iden6fy Alignment Relationship Semantics Example Exact match Transitive equivalence Close match Equivalent only for search purposes Broader match Generalization of a concept Narrower match Specialization of a concept Inverse of broader match Related Unspecified relation (no real semantics) #23

24 Quick Facts! Public and free RDF warehouse service Integrates more than 25 popular data sources Apply text mining technology to link the text with en22es Computer friendly API to access the informa2on #24

25 Type of possible ques2ons, analysis and interpreta2on INTEGRATED DATASET #25

26 Linked Life Data Datasets #26

27 Rest API SPARQL endpoint Co- Occurrence Relation Finder #27

28 New Type of Possible Query #1 Select drugs related to asthma that are linked to a curated molecular interac2on in the literature where the protein is known to cause inflammatory response PREFIX rdf: <hqp:// rdf- syntax- ns#> PREFIX skos: <hqp:// PREFIX biopax2: <hqp:// level2.owl#> PREFIX uniprot: <hqp://purl.uniprot.org/core/> PREFIX drugbank: <hqp://www4.wiwiss.fu- berlin.de/drugbank/resource/ drugbank/> SELECT DISTINCT?fullname?drugname WHERE {?interac2on rdf:type biopax2:physicalinterac2on.?interac2on biopax2:participants?par2cipant.?par2cipant biopax2:physical- ENTITY?physicalEn2ty.?physicalEn2ty skos:exactmatch?protein.?protein uniprot:classifiedwith <hqp://purl.uniprot.org/go/ >.?protein uniprot:recommendedname?name.?name uniprot:fullname?fullname.?target skos:exactmatch?protein.?drug drugbank:target?target.?drug drugbank:genericname?drugname.?drug drugbank:indica2on?indica2on. } The red graph paqerns indicate the usage of mapping rules. #28

29 men2ons Seman6c Annota6ons umls:c Respiration Disorders Bronchial Diseases Chronic Obstructive Airway Diseases umls:c COPD broader Asthma umls:c men2ons journal Clinical and experimental pharmacology pmid: Ian A Yang #29

30 New Type of Possible Query #2 Select all located in Y- chromosome, human genes with known molecular interac2ons, which are analysed with 'Transfec2on' PREFIX rdf: <hqp:// rdf- syntax- ns#> PREFIX skos: <hqp:// PREFIX gene: <hqp://linkedlifedata.com/resource/entrezgene/> PREFIX core: <hqp://purl.uniprot.org/core/> PREFIX biopax2: <hqp:// level2.owl#> PREFIX lifeskim: <hqp://linkedlifedata.com/resource/lifeskim/> PREFIX umls: <hqp://linkedlifedata.com/resource/umls/> PREFIX pubmed: <hqp://linkedlifedata.com/resource/pubmed/> SELECT dis2nct?genedescrip2on?preflabel?pmid WHERE {?interac2on rdf:type biopax2:interac2on.?interac2on biopax2:participants?p.?p biopax2:physical- ENTITY?protein.?protein skos:exactmatch?uniprotaccession.?uniprotaccession core:organism <hqp://purl.uniprot.org/ taxonomy/9606>.?geneid gene:uniprotaccession?uniprotaccession.?geneid gene:descrip2on?genedescrip2on.?geneid gene:pubmed?pmid.?geneid gene:chromosome 'Y'.?pmid lifeskim:men2ons?umlsid.?umlsid skos:preflabel 'Transfec2on'.?umlsid skos:preflabel?preflabel. } #30

31 Query Results #31

32 New Type of Possible Query #3 Select all par2cipa2ng in interac2ons human genes which are a drug target and are analysed with 'Transfec2on' PREFIX rdf: <hqp:// rdf- syntax- ns#> PREFIX rdfs: <hqp:// schema#> PREFIX skos: <hqp:// PREFIX gene: <hqp://linkedlifedata.com/resource/entrezgene/> PREFIX core: <hqp://purl.uniprot.org/core/> PREFIX biopax2: <hqp:// level2.owl#> PREFIX lifeskim: <hqp://linkedlifedata.com/resource/lifeskim/> PREFIX umls: <hqp://linkedlifedata.com/resource/umls/> PREFIX pubmed: <hqp://linkedlifedata.com/resource/pubmed/> PREFIX drugbank: <hqp://www4.wiwiss.fu- berlin.de/drugbank/ resource/drugbank/> SELECT dis2nct?genedescrip2on?preflabel?drugname?pmid WHERE {?interac2on rdf:type biopax2:interac2on.?interac2on biopax2:participants?p.?p biopax2:physical- ENTITY?protein.?protein skos:exactmatch?uniprotaccession.?uniprotaccession core:organism <hqp://purl.uniprot.org/ taxonomy/9606>.?geneid gene:uniprotaccession?uniprotaccession.?geneid gene:descrip2on?genedescrip2on.?geneid gene:pubmed?pmid.?pmid lifeskim:men2ons?umlsid.?umlsid skos:preflabel 'Transfec2on'.?umlsid skos:preflabel?preflabel.?target skos:closematch?geneid.?drug drugbank:target?target.?drug rdfs:label?drugname. } #32

33 Query Results #33

34 Classical Informa6on Retrieval Queries Lucene based index Special predicate to execute full- text queries Mul2ple retrieval modes Literal RDF molecules select * where {?ar2cle <hqp:// "+lung COPD^5 asthma^3".?ar2cle <hqp:// rdf- syntax- ns#type> <hqp://linkedlifedata.com/resource/ pubmed/cita2on>.?ar2cle <hqp://linkedlifedata.com/resource/ pubmed/ar2cletitle>?2tle. } limit 1000 #34

35 Loading procedure, tes2ng environment, used environment BEHIND THE SCENE #35

36 Linked Life Data Architecture Web services Data Queries Exports Data Access Layer (store, retrieve, manage) Data Indexing and Storage (database, text index, data cubes) Federation Extract Transform Load (ETL) Warehousing Data Sources Excel DMS PLM LIMS Products www flat files #36

37 LLD Current Sta6s6cs #37

38 Maintain Two Parallel and Independent Processes Data source updates and transforma2on Download the data source Apply ETL script that generates RDF data Load and index the RDF data with a global En2ty Pool Do a local inference for the data source graph LLD release builds Merge previously loaded repositories Execute post processing instance mappings Do a global inference across all graphs RDF data S P O C URI1 URI2 Literal1 URI3 URI1 URI2 Literal2 URI3 Node URI1 1 URI2 2 Literal1 3 Id S P O C URI3 4 Entity Pool RDF index #38

39 Life Cycle of a LLD Data Source Node Id umls:asthma umls:copd Data Source files release info ETL process for each source OWLIM Image Global Entity Pool RDF Index umls:p S P O C Ts #39

40 Combining Arbitrary Data Sources Fast network storage OWLIM Image 1 OWLIM Image 2 RDF Index RDF Index merge RDF Index Repository merging OWLIM Image Global Entity Pool RDF Index OWLIM Image 3 RDF Index #40

41 Advantages of the LLD Approach Each data source has a consistent query- able repository All repositories are compa2ble and could be efficiently combined You can maintain mul2ple versions of the repository Fixes in the RDF schema are very quick The data source updates are absolutely independent from the produc2on releases We can maintain mul2ple LLD versions op2mized for different needs The extension with new data sources is trivial You have the capability to support global reasoning #41

42 Wrap- up Free and ac2vely developed public service available: hqp://linkedlifedata.com OWLIM engine which is experimentally proven to scale up to: 20 billion RDF statements (15 billions explicit) On a computer that costs less than $ Warehouse methodology that scales for tens of data sources Integrate informa2on- extrac2on algorithms #42