Semantic Web. Oracle Semantic Web im Einsatz. Author : Marc Lieber Date : 11/03/2013

Transcription

1 Semantic Web Oracle Semantic Web im Einsatz Author : Marc Lieber Date : 11/03/2013 BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 1 Title of Presentation Insert Header & Footer Date

2 AGENDA 1. Introduction 2. Semantic Web concepts 3. Oracle RDF Triple Store 4. Our Project : Pharma Ontology searching tool 5. Questions and answers 2 Title of Presentation Insert Header & Footer Date

3 Introduction 1. Oracle Semantic Web is a way to store and maintain Ontology oriented data in the Oracle relational database 2. Our case study is a semantic data integration platform for the biomedical domain using Oracle Semantic Web 3 Title of Presentation Insert Header & Footer Date

4 Definition of an ontology 1. An ontology is a shared conceptualization of knowledge in a particular domain 2. It consists of a collection of classes, properties, relations and instances 3. It is a systematic arrangement of all of the important categories of objects or concepts which exist in some field of discourse, showing the relations between them. For example: The Gene Ontology project provides a controlled vocabulary to describe gene and gene product attributes in any organism Friend of a Friend (FOAF) is an ontology for describing persons, their activities and their relations to other people and objects 4

5 Why should you choose Semantic Web? 1. You want a flexible, adaptable, transparant information architecture 2. project requires integration of data from different sources 3. heterogeneous sets of metadata and vocabulary concepts, originating from multiple sources 4. Need for semantic annotations using controlled vocabularies and thesauri such as FOAF, OWL, SKOS, etc 5. There is a need for making logical deductions based on rules defined by these controlled vocabularies. The ability to draw inferences from existing data using the precision and rigor of mathematical logic is the most important property that distinguishes semantic data from others 5

6 Implementing an Ontology in a relational database 1. Not always a good idea! 2. The ever growing complexity of the ontolgy model generates : Growing number of table attributes and many relation to other tables Some attributes are actually interconcept relations lack of lisibility Growing number of new tables 3. Bad performances because of the complexity of the SQL queries 4. Lack of flexibility: a relational datamodel is not meant to be modified frequently. Changes in database objects, especially relational tables, might impact or invalidate other objects. Needs often the assistance of the database administrator. 6

7 Relational Datamodel versus RDF Model A Semantic web graph representation of an ontology is easier to visualize and maintain. The same data stored in a relational table are a mix of values and relations 7

8 Migration Relational to Triple Store 1. Any relational datamodel can be translated into triples Each cell of table or a view becomes a Triple Example : SQL> INSERT INTO emp_tpl VALUES (SDO_RDF_TRIPLE_S('emp1', '< '< "Jones"^^< Add new rule : Employee s rdf:type is owl:class ); SQL> INSERT INTO emp_tpl VALUES (SDO_RDF_TRIPLE_S('emp1', '< '< '< 8

9 Complexity of the queries A SPARQL query of this type would be quite difficult to translate into SQL queries : 9

10 Relational, Big Data or RDF Graph Storage What should we use? RDF Triple store : Complex Meta Data, Flexibility, adaptable, integration of other database Big Data : huge amounts of unstructured data, low cost per Terabyte Generated logs s Videos images documents Regular Enterprise Applications Unstructured and semistructured data Complex Meta Data applications Relational Database Haadoop, Hbase, etc Graph Databases and Triples Stores 10 Title of Presentation Insert Header & Footer Date

11 AGENDA 1. Introduction 2. Semantic Web concepts 3. Oracle RDF Triple Store 4. Our Project : Pharma Ontology searching tool 5. Questions and answers 11 Title of Presentation Insert Header & Footer Date

12 RDF Triples RDF as core data format Uniform structure to represent data (triples) [subject] [predicate] [object] JFK president of the United States [resource] [property] [value] JFK PresidentOf The United States quad = triple + named graph, quint = quad + technical ID (rowid) RDF Triple representations : n3 triple, RDF/XML, Turtle or RDFa (XHTML) use of namespaces to differentiate terms Some are predefined, but you can create your own namespaces < "John Fitzgerald Kennedy"^^< < New York City. 12 Presentation Title Presenter Name Date Subject Business Use Only

13 RDF Graphs Triples together form a graph Graphical tools : Cytoscape or Gruff in Allegrograph (Franz Corp.) With Gruff you can visualize Graph pattern but also use it as a graphical SPARQL Query Builder. 13 Title of Presentation Insert Header & Footer Date

14 RDF Model 1. Each model contains a set of subject object relationship triples are organized as an RDF/OWL graph of directed labeled edges. 2. Example of a family model: Person is a class that has two subslasses: Male and Female. parentof is a property that has two subproperties: fatherof and motherof siblingof is a property that has two subproperties: brotherof and sisterof. The domain of the fatherof and brotherof properties is Male. The domain of the motherof and sisterof properties is Female. < < < < < < 14

15 What is Inferencing? Inferencing is the ability to make logical deductions based on Ontology rules. The reasoning tools use the rules defined in the RDF Model (RDFS, OWL, SKOS, ) to detect new properties and new relationships. Previous Example : One triple states that John is the father of Steve One triple states that Donna is the sister of Steve the inferencing generates new triples John is the father of Donna Donna is female Steve is male,. 15 Kurs- bzw. TC-Namen eingeben

16 SPARQL end points Link your data to external RDF data sources SPARQL syntax supports joins with external data sources Overview of the most known SPARQL End points Title of Presentation Insert Header & Footer Date

17 Triples stores architectures Native in-memory : In-Memory Triple Stores, also called Fast Data store the RDF Triples in Memory. They are fast but limited by the size of the memory Native : The triples are stored in a persistent storage system. Provides support for transactions, Queries and has its own development environment Non-native triple store : persistent storage in an environment not specific to RDF Graph databases : Graph databases do not physically store the data as RDF Triples. Each provider has its own way to store the data Mapping between relational database schemata and RDF : The mapping tool translates relational data into RDF triples 17 Title of Presentation Insert Header & Footer Date

18 Graph databases big picture In memory Jena Sesame native Allegrograph Virtuoso Owlim Sesame Jena Anzo Cambridge bigdata Kowari Mapping tools D2RQ graph Neo4j HyperGraphDB InfiniteGraph Non native Oracle Semantic Web 18

19 Triple stores Tripe Store Scalability (Billion Triples) Query Reasoning support Full text Search support Programming Jena (TDB) up to 1.7 BT SPARQL 1.1 OWL, RDFS Yes (lucene integration) Java Sesame Millions Triples SPARQL 1.1 RDFS Yes (through Lucene SAIL) Java OpenLink Viruoso 15.4 BT SPARQL 1.1 RDFS, subsets of OWL yes Java Oracle Billons Triples SPARQL 1.0 (11g) Sparql 1.1 (12c), SEM_MATCH, SEM_RELATED RDFS, OWL, OWLIM, SKOS, SNOMED Yes (Oracle Text) Java, SQL, PL/SQL OWLIM 20 BT SPARQL 1.1 RDFS, OWL, OWLIM yes Java Allegrograph 300 BT SPARQL 1.1, Prolog RDFS, Prolog rules yes Java, LISP, Python, Ruby, C# 4 Store 15 BT SPARQL 1.1 RDFS yes Java BigData over 10 BT SPARQL 1.1 RDFS, OWL Lite Internal, external through Lucene Java Mulgara 500 MT SPARQL 1.1 RDFS Yes (lucene integration) Java Anzo Cambridge unknown SPARQL 1.1 RDFS, OWL Yes (Information Mining) Java, SOAP 19 Title of Presentation Insert Header & Footer Date

20 AGENDA 1. Introduction 2. Semantic Web concepts 3. Oracle RDF Triple Store 4. Our Project : Pharma Ontology searching tool 5. Questions and answers 20 Title of Presentation Insert Header & Footer Date

21 Oracle RDF Triple Store Oracle Semantic Web is an add-on to Oracle Spatial. Spatial allows geotemporal search and inferencing on semantic data Supports most of the W3C rules Use of named graphs (quad) since Scales up to 100 s billions of triples Oracle specific adapters available for JENA, SESAME, TopBraid, Cytoscape and Protégé. 21

22 ORACLE Database RDF Query engine 22 Kurs- bzw. TC-Namen eingeben

23 SPARQL Query SQL query on a relational table SQL> SELECT ename from EMP where JOB= CLERK ; SPARQL in the SEM_MATCH function {?s?p?o FILTER(sameTerm(?p,:ENAME)).?s :hasjob :Clerk} 23 Gigantisch Semantisch

24 AGENDA 1. Introduction 2. Oracle RDF Triple Store 3. Semantic Web concepts 4. Our Project : Pharma Ontology searching tool 5. Questions and answers 24 Title of Presentation Insert Header & Footer Date

25 Project Overview Metastore Fundamentals 1. Consists of a semantic data federation layer based on controlled terminologies extracted from scientific data repositories 2. Organized around scientific concepts: Genes, Proteins, Indications, Anatomy etc ; some hierarchically organized and classified 3. Complemented by referential knowledge (cross references to internal and external knowledge repositories) 4. Ontological relations between concepts materializing semantic network of scientific concepts 5. Content up to monthly updated (concept type centric updates) during dedicated loading exercises 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

26 Why migrating to Oracle Semantic Web? Extend the current model Inter-concept relation Lexical Tagging Associate more meaning to enterprise data to enable usage across applications Continuously evolving 26 Presentation Title Presenter Name Date Subject Business Use Only

27 Life cycle management Project Overview Building blocks Service Layer AUTHENTICATION XML/SOAP XML/RDF XML/JSON Views SOAP WS III Extractor Service SPARQL Endpoint REST API Hierarchical JSON Service Data Services METASTORE TERMINOLOGY SEMANTIC CONTENT ( inter concept relations ) REFERENTIAL DATA ( pointers ) Back-end Parsing, Entity Extraction, Typing, Normalization, Classification, Rules 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

28 Project Overview Workflow Loading Exercise (1) Proteins Genes Loading Exercise Contains one to many RDF/XML files to be uploaded taxonomy There is one file per concept type Each file is checked first against an XML Schema Import with Jena into the staging Table Pre-validate Bulk Load Post- Validate Materialize in tables Swap model Live and Work 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

29 Project Overview Workflow Loading Exercise (2) Jena Relational Tables Pointers History Versions Logs Reference tables Staging Table RDF Triple store LIVE SPARQL end Point Joseki RDF Triple store WORK Materialized Views Query SQL and PL/SQL APIs D A T A - S e r v i c e s 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

30 Technical Implementation Staging table and Model implementation 1. One partitioned Staging table keeps all the triples uploaded by Jena The RDF/XML files read by Jena can be up to 1.2 Gigabytes in size - This is not a problem for the load into the staging table The staging table is not truncated after each loading exercise - This allows us to rebuild the complete triple store from scratch if required 2. The Metastore RDF Model has been duplicated (MS3_LIVE + MS3_WORK) to separate the productive from work-inprogress Data Reason : Our experience with Oracle Workspace management was not successful 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

31 Technical Implementation Data volume 1. Each model contains 1,001,544 concepts / 22 concept types 2. Stored 74,000,000 triples + 35,000,000 inferred triples for each Model RDF_LINK$ table size 183,000,000 rows; 19,237 Mb RDF_VALUE$ table size 137,000,000 rows; 23,314Mb - Only 34,000,000 rows are actually used in our Models will be fixed We use Keep Pool to cache the partition Model LIVE on RDF_LINK$ + Keep Pool on RDF_VALUE$ - Required an alter table storage (Buffer pool keep) 3. A new concept type will be added this year that will double the size of today s number of concepts.. 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

32 Physical implementation of Oracle Semantics It is essential to understand how Oracle has physically implemented the triple store in the 11g Database Staging Table Partitioned VIRTUAL MODEL Models owned by Metastore Object tables Model M1 Table Model M2 Table index index MDSYS.RDF_LINK$ Partitions of other owners Partition M1 Partition M2 Partition M1 inferenced Partition M2 inferenced MDSYS.RDF_VALUE$ Indexes GPSCM, used Unused Metamodel Table Partition Meta- Model MDSYS Views M1, M2 M1_IDX, M2_IDX 32 Title of Presentation Insert Header & Footer Date

33 Technical Implementation Bulk Load using SEM_APIS.bulk_load_from_staging_table 1. The Bulk load from the staging table into the Semantic table can be incremental for small changes - Advantage : faster - Problem : risk of dangling triples, also the Blank node IDs are getting larger and larger. They keep track of the old model ID + the new model ID.. complete rebuild of the semantic model - The model is dropped and then recreated - The staging Table contains all the triples loaded in metastore since the beginning. - The bulk operation uploads the latest version of each concept stored in the staging table - Advantage : we have a clean rebuild of the model - Problem : takes currently about 10 hours but this can be improved with hints The Oracle development team in Cambridge is helping us solve this problem 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

34 Technical Implementation Use of Named Graphs 1. For better performance we switched to named graph one for the semantic model, one for each concept Every triple is now associated to a named graph 2. Issues with blank nodes getting larger and larger.. 3. SEM_MATCH Query using name graphs : SELECT * FROM TABLE(SEM_MATCH( 'select?rep?obj { GRAPH :gnvmtax9606 { :NVMTAX9606 :CONCEPT_isRepresentedBy?rep. OPTIONAL{?rep :REPRESENTATION_hasSource?obj. }}}', sem_models('ms3_live'),sem_rulebases(''), SEM_ALIASES( SEM_ALIAS(null,' ' GRAPH_MATCH_UNNAMED=T PLUS_RDFT=T ',null,sem_graphs(':gnvmtax9606'))); 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

35 Client SPARQL End Point Use of Named Graphs; Joseki SPARQL queries 1. We decided to rewrite the SPARQL end point to have a better control on what end users can do and to support Session Kills Timeouts Oracle Hints Named Graphs Predefined queries 2. Note : The function Describe is extremely slow 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

36 Technical Implementation PL/SQL implementation 1. The REST Webservice calls PL/SQL functions to retrieve the triples in a nt triple format Example using our function nt_describe SELECT ms3_util.nt_describe('nvmtax9606',0,0,1) FROM DUAL; Returns triples in a CLOB This way, we can optimize the SEM_MATCH queries but we still have sometimes performance issues (more than 5 sec. waiting time) 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

37 Next Step : Virtual Model Splitting the current triple store into multiple triple stores One Model per concept type A copy of each model for the working environment - If 22 concept types, then Metastore owns 44 models + 2 models for the metamodel Remove the swap model process and replace it by a drop and recreate Virtual Model Instead of a Drop use Truncate for the WORK models Advantage : The model ID stays the same Replace all blank nodes by a URI to solve the problem of Blank Nodes getting larger and larger This will reduce also the size of RDF_VALUE$ Performance should stay the same, maybe better because of the partitioning of RDF_LINK$ 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

38 Next Step : Security 1. What about Security on triple level? The default control of access to the Oracle Database semantic data store is at the model level. Virtual Private Database allows security administrators to define policies that conditionally restrict a user's access to triples that involve instances of a specific RDF class or property - Oracle does not recommend to use VPD in the triple store - Instead, use new 11gR2 feature Oracle Label Security 2 TMS Service layer Katia Vella & Marc Lieber 06/03/2013 Business Use Only

39 Core messages Oracle 11gR2 implementation of RDF Web semantics is a powerful new way of storing data in a database The advantage of using the Oracle Triple store are multiple SPARQL and SQL interaction with relationally stored data Standard DB Administration : Backup/recovery/replication, etc PL/SQL or Java programming Supports large volumes of data Good integration with standard RDF client tools such as Jena and Sesame Newcomers to RDF Web Semantics will need some time to get used to the various modeling concepts and to the SPARQL syntax Newcomers to Oracle Web semantics will need some time to fine tune the Oracle specific features but the effort is worthwhile! 39 Kurs- bzw. TC-Namen eingeben

40 AGENDA 1. Introduction 2. Oracle RDF Triple Store 3. Our Project : Pharma Ontology searching tool 4. Questions and answers 40 Title of Presentation Insert Header & Footer Date

41 THANK YOU. Marc Lieber BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 41 Title of Presentation Insert Header & Footer Date