Linked Data and Geosparql

Transcription

1 Linked Data and Geosparql OGh NL-Oracle User Group Martien Vos (Redforce/DPA) June 22, 2017

2

3 Program Agenda Theory What is Linked Data and what standards are behind it? Product Oracle Spatial and Graph Use Cases and some customer stories Markets Demo 1 Break What is GeoSPARQL? Demo 2

4 Mindset: which Business problems? Silos To much copies: unreliable data No realtime total view Changes in dataset are time consuming and expensive (DWH, SOA) Structured and unstructured data Use of external data

5 What is Linked Data? Connect Distributed Data across the Web Linked Data is about using the Web to connect related data that wasn't previously linked, or using the Web to lower the barriers to linking data currently linked using other methods. More specifically, Wikipedia defines Linked Data as "a term used to describe a recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF."

6 What is Linked Data? Datavirtualization Internet of data Paradigm shift: open standards, W3c Open World assumption; sharing data instead of owning them Flexibility: IT is not slowing down changes the business needs Combining many datasources (more then in relational world)

7 Linked Data uses internet technology The world has a network that connects people and businesses: the internet. Worldwide coupling. Even future clienst and partners have internet. Cheap and avalable nearly everywhere With security, internet banking Use this technology for extern and intern to connect (http adress, links etc.).

8 Loosely coupled Less IT work Source: Joep Creusen, NS DB-links, batchprocesses ESB, webservices Linked Data, API s system centric proces centric data centric yesterday today tomorrow

9 Relational Model vs. Graph Model Relational Model Based on the assumption that the relationship is vast and applicable to all the records Graph Model Suitable to model when the relationship is variable and changing Courtesy: Tom Sawyer 2016

10 Graph as data modeling foundation What is a graph? A set of vertexes and edges (and optionally attributes) A graph is simply linked data Why do we care? Graphs are everywhere Road networks, power grids, biological networks Social networks/social Web (Facebook, Linkedin, Twitter, Baidu, Google+, ) Knowledge graphs (RDF, OWL) Graphs are intuitive and flexible Easy to navigate, easy to form a path, natural to visualize

11 Various Kinds of Graphs (alias triple store, the database) Different application scenarios RDF Semantic Graphs modeling data as triples for social network, linked data and other semantic applications (Oracle Database Spatial and Graph option ) Property Graphs allowing the association of K/V pairs (attributes) with vertexes/edges for social network analytics (Oracle Big Data Spatial and Graph + Database Spatial and Graph option 12.2 ) Link-node graphs representing physical/logical networks used in transportation, utilities and telco (Oracle Spatial and Graph Network Data Model (NDM))

12 Why Graph Databases? Increasing recognition of Graph Databases potentials New data collections & new analytics requirements Big data analytic, Social Network Analysis, data federation, Linked Open Data, meta-modelling,. Increasing cases that are hard to model using the traditional relational methods Enables new type of queries that are either impossible or too costly to run on the traditional platforms Property Graphs or RDF Triple Stores? RDF is appropriate for Data Federation and Linked Open Data technologies Property Graphs are best for traversal and Graph analytics Oracle provides support for both Property Graph and RDF Graph Oracle can translate Property Graph into RDF Graphs

13 RDF (Semantic) Graph Resource Description Framework (RDF) URIs are used to identify Resources, entities, relationships, concepts Creates Subject-Property-Object triples Data identification is a must for integration URIs are globally unique Properties of subjects are triples RDF Graph defines semantics Standards defined by W3C & OGC RDF, RDFS, OWL, SKOS SPARQL, RDFa, RDB2RDF, GeoSPARQL Implementations Oracle, IBM, Cray, Bigdata, Franz, Ontotext, Openlink, Jena, Sesame,.

14 What is RDF ( Resource Description Framework )? A graph data model for web resources and their relationships The graph can be serialized into RDF/XML, N3, N-TRIPLE, Construction unit: Triple subject + action + object < subject <:produces> action/property <:mp3> object Quads (named graphs) add context, provenance, identification, etc. to assertions < <:produces> <:mp3 > <:ProductGraph> /locatedin CA /produce /customerof /uses /produce

15 SPARQL Query language RDF Data :res1 rdf:type :House. :res1 :baths "2.5"^^xsd:decimal. :res1 :bedrooms "3"^^xsd:decimal. :res2 rdf:type :Condo. :res2 :baths "2"^^xsd:decimal. :res2 :bedrooms "2"^^xsd:decimal. :res3 rdf:type :House :res3 :baths "1.5"^^xsd:decimal. :res3 :bedrooms "3"^^xsd:decimal. SPARQL Query SELECT?r?ba?br WHERE {?r rdf:type :House.?r :baths?ba.?r :bedrooms?br } Result Bindings?r?ba?br =================== :res1 "2.5" "3" :res3 "1.5" "3"

16 Inferencing: deriving new facts from existing data Enabling the software to perform reasoning normally done in the human brain Inference is discovering new relationships Facts + Rules = Inferencing Rules : ontology & user own rules OWL rules native to Oracle Spatial - RDFS, OWL 2 RL, OWL 2 EL, SKOS Example : :California 1. California is part of USA + USA is part of North America California is part of North America 2. Mr. A has a son Mr. B + Mr. B has a son Mr. C Mr. A is a grandfather :USA :partof :partof :NorthAmerica :partof

17 Program Agenda Theory What is Linked Data and what standards are behind it? Product Oracle Spatial and Graph Use Cases and some customer stories Markets Demo 1 Break What is GeoSPARQL? Demo 2

18 Oracle database Spatial and Graph 12.2 Polyglot: seamless from Linked Data to Relational, Sparql statement in a SQL query Geo relational combined with attributes from a Graph Things you are used to from an Oracle database: availability, scalability, compression, security, performance etc. Use of open standards

19 Integrating Oracle SQL space with the RDF Graph world SPARQL Queries using SPARQL language RDF Graph triples OWL : ontology's ( triples ) RDF Graph triples RDF view over relational data Conversion to RDF Triple store RDB2RDF SQL + SPARQL ( SEM_MATCH ) SQL User own rules

20 Oracle DB s SEM_MATCH operator : Adding SPARQL to SQL Extends SQL with full SPARQL 1.1 query constructs Benefits: Integrates graph data with existing enterprise data JOINs with other relational (and object-relational) data Allows SQL constructs/functions DDL Statements: create tables/views Allows use of enterprise SQL development tools

21 Example of a SQL statement using SEM_MATCH SELECT n1, n2 FROM TABLE_X( SEM_MATCH( 'PREFIX foaf: < SELECT?n1?n2 FROM < WHERE {?p foaf:name?n1 OPTIONAL {?p foaf:knows?f.?f foaf:name?n2 } FILTER (REGEX(?n1, "^A")) } ORDER BY?n1?n2', SEM_MODELS('M1'), ));

22 Connectivity and integration with many tools Full choice for the users due to complying to the W3C and other standards Extraction Tools Lymba, Expert Systems Data Ninja, Linguamatics, Thompson Reuters' Open Calais, GATE and many others ( some NLP or industry specific ) Graph management Tools Oracle Development or SQL access tool like SQL Developer,. Analytic Tools Oracle BI EE, Oracle R, 3 RD party tools. Data exchange Tools Oracle SQL Loader, and many other 3 rd party tools, web services, Oracle RDBMS with RDF Store Ontology management Tools Protégé, TopQuadrant TopBraid Composer Visualization Tools Tom Sawyer, Cytoscape, IO Informatics,..

23 Reporting RDF Data with Oracle BI EE Powerful BI dashboards Visually appealing 100% thin client Across all styles of analsis R-OLAP, M-OLAP, Scorecards, Reporting, Collaboration, Actions Across all data sources Federated data access Share, collaborate, & publish

24 Program Agenda Theory What is Linked Data and what standards are behind it? Product Oracle Spatial and Graph Use Cases and some customer stories Markets Demo 1 Break What is GeoSPARQL? Demo 2

25 Linked Data Use Cases: Industries Life Sciences Finance Media Networks & Communications Defense & Intelligence Police Hutchinson 3G Austria

26 Ordnance Survey Ireland Linked Data project at the Irish National Land Registry (combination with Esri software) Formulation of two use case scenarios from which requirements were distilled: 1. Accessing the same features with different geometric representations, i.e., different generalizations or resolutions. 2. Capturing the provenance and evolution of features and their geometric representations. E.g., Statutory Instruments to change boundaries.

27 EU Publication Office ( Cellar System ) Linked Metadata Platform for European Union Archive Long term preservation EUR-Lex Common Content Repository (CCR) Official publications Validation Tendering documents TED Common Portal General publications EU Bookshop Common search service Common indexes Common Metadata Repository (CMR) CORDIS CORDIS Metadata production External sources Portal Index and Search Content and Metadata = CELLAR Postproduction Production Objectives Common metadata model supports: Search and discovery of EU Publications Multiple domains and languages Solution Validate and tag EU law, tenders, and publicity to standardized vocabularies Unified RDF graph metadata model Supports discovery of content through user s terminology and language Provides variety of dissemination modes Environment ( September 2016 ) 8 million requests / 100 K SPARQL query per day 1500 milion trippels in Oracle RDF Store 4.2 TB Oracle DB ( compressed )

28 Markets All of our and your clients have data in silos

29 Program Agenda Theory What is Linked Data and what standards are behind it? Product Oracle Spatial and Graph Use Cases and some customer stories Markets Break 20 minutes What is GeoSPARQL? Some of the implementation examples

30 Program Agenda Theory What is Linked Data and what standards are behind it? Product Oracle Spatial and Graph Use Cases and some customer stories Markets Break 20 minutes What is GeoSPARQL? Some of the implementation examples

31 Issues with Geo Data Many Linked Open Data (LOD) datasets have geospatial components Barriers to integration Vendor-specific geometry support Different vocabularies W3C Basic Geo, GML XMLLiteral, Vendor-specific Different spatial reference systems WGS84 Lat-Long, British National Grid

32 Semantic GIS GIS applications with semantically complex thematic aspects Logical reasoning to classify features Land cover type, suitable farm land, etc. Complex Geometries Polygons and Multi-Polygons with 1000 s of points Complex Spatial Operations Union, Intersection, Buffers, etc. Find parcels with an area of at least 3 sq. miles that touch a local feeder road and are inside an area of suitable farm land.

33 As shown: a SPARQL Query RDF Data :res1 rdf:type :House. :res1 :baths "2.5"^^xsd:decimal. :res1 :bedrooms "3"^^xsd:decimal. :res2 rdf:type :Condo. :res2 :baths "2"^^xsd:decimal. :res2 :bedrooms "2"^^xsd:decimal. :res3 rdf:type :House :res3 :baths "1.5"^^xsd:decimal. :res3 :bedrooms "3"^^xsd:decimal. SPARQL Query SELECT?r?ba?br WHERE {?r rdf:type :House.?r :baths?ba.?r :bedrooms?br FILTER (?ba > 2) } Result Bindings?r?ba?br =================== :res1 "2.5" "3"

34 Spatial SPARQL QUERY Spatial RDF Data :res1 rdf:type :House. This is what :res1 :baths "2.5"^^xsd:decimal. GeoSPARQL :res1 :bedrooms "3"^^xsd:decimal. standardizes :res1 ogc:hasgeometry :geom1. :geom1 ogc:aswkt "POINT( )"^^ogc:wktLiteral. :res3 rdf:type :House Vocabulary & :res3 :baths "1.5"^^xsd:decimal. Datatypes :res3 :bedrooms "3"^^xsd:decimal. :res3 ogc:hasgeometry :geom3. :geom3 ogc:aswkt "POINT( )"^^ogc:wktLiteral. Find houses within a search polygon GeoSPARQL Query Extension SELECT?r?ba?br Functions WHERE {?r rdf:type :House.?r :baths?ba.?r :bedrooms?br.?r ogc:hasgeometry?g.?g ogc:aswkt?wkt FILTER(ogcf:sfWithin(?wkt, "POLYGON( )"^^ogc:wktliteral)) }

35 GeoSPARQL Vocabulary: Basic Classes and Relations ogc:spatialobject Same as ISO GM_Object ogc:feature ogc:hasgeometry 0.. * ogc:geometry ogc:hasdefaultgeometry metadata ogc:dimension : xsd:int ogc:coordinatedimension : xsd:int Same as ISO GFI_Feature ogc:spatialdimension : xsd:int ogc:isempty : xsd:boolean ogc:issimple : xsd:boolean Geometry encoded as a Literal serializations ogc:aswkt : ogc:wktliteral ogc:asgml : ogc:gmlliteral

36 Details of ogc:wktliteral All RDF Literals of type ogc:wktliteral shall consist of an optional IRI identifying the spatial reference system followed by Simple Features Well Known Text (WKT) describing a geometric value [ISO ]. WGS84 longitude latitude is the default CRS "< POINT( )"^^ogc:wktLiteral "POINT( )"^^ogc:wktLiteral European Petroleum Survey Group (EPSG) maintains a set of CRS identifiers.

37 Topological Relations between ogc:spatialobject A/B A B A B A B ogc:sfequals ogc:sftouches ogc:sfoverlaps ogc:sfcontains A A B A B A B B ogc:sfwithin ogc:sfdisjoint ogc:sfintersects ogc:sfcrosses

38 GeoSPARQL Query Functions ogcf:distance(geom1: ogc:wktliteral, geom2: ogc:wktliteral, units: xsd:anyuri): xsd:double geom1 geom2 ogcf:buffer(geom: ogc:wktliteral, radius: xsd:double, units: xsd:anyuri): ogc:wktliteral geom ogcf:convexhull(geom: ogc:wktliteral): ogc:wktliteral geom

39 GeoSPARQL Query Functions ogcf:intersection(geom1: ogc:wktliteral, geom2: ogc:wktliteral): ogc:wktliteral geom2 geom1 ogcf:union(geom1: ogc:wktliteral, geom2: ogc:wktliteral): ogc:wktliteral geom2 geom1

40 GeoSPARQL Query Functions ogcf:difference(geom1: ogc:wktliteral, geom2: ogc:wktliteral): ogc:wktliteral geom2 geom1 ogcf:symdifference(geom1: ogc:wktliteral, geom2: ogc:wktliteral): ogc:wktliteral geom2 geom1

41 GeoSPARQL Query Functions ogcf:envelope(geom: ogc:wktliteral): ogc:wktliteral geom ogcf:boundary(geom1: ogc:wktliteral): ogc:wktliteral geom ogcf:getsrid(geom: ogc:wktliteral): xsd:anyuri

42 Spatial Function Library Standard OGC functions Topological Relations ogcf:relate, ogcf:sfcontains, ogcf:sfcrosses, ogcf:sfdisjoint, ogcf:sfequals, ogcf:sfintersects, ogcf:sfoverlaps, ogcf:sftouches, ogcf:sfwithin Distance-based Operations ogcf:distance, ogcf:buffer Geometry Operations ogcf:boundary, ogcf:convexhull, ogcf:envelope, ogcf:getsrid, Geometry-Geometry Operations ogcf:difference, ogcf:intersection, ogcf:symdifference, ogcf:union

43 Spatial Function Library Oracle Extensions Topological Relations orageo:relate Distance-based Operations orageo:distance, orageo:withindistance, orageo:buffer, orageo:nearestneighbor Geometry Operations orageo:area, orageo:length orageo:centroid, orageo:mbr, orageo:convexhull Geometry-Geometry Operations orageo:intersection, orageo:union, orageo:difference, orageo:xor

44 Example Query PREFIX : < PREFIX ogc: < PREFIX ogcf: < PREFIX epsg: < SELECT?parcel WHERE {?parcel rdf:type :Residential.?parcel :exactgeometry?pgeo.?pgeo ogc:aswkt?pwkt. :District1 :exactgeometry?dgeo.?dgeo ogc:aswkt?dwkt. :City1 :extent?cgeo.?cgeo ogc:aswkt?cwkt. FILTER(ogcf:sfWithin(?pWKT, ogcf:intersection(?dwkt,?cwkt)))}

45 Summary GeoSPARQL Defines: Basic vocabulary, Query functions, Entailment component Based on existing OGC/ISO standards WKT, GML, Simple Features, ISO Uses SPARQL s built-in extensibility framework

46 GeoSPARQL GeoSPARQL Support in RDF Semantic Graph

47 GeoSPARQL Support with Oracle Spatial and Graph Support geometries encoded as ogc:wktliterals :semtech2011 ogc:aswkt "POINT( )"^^ogc:wktLiteral. Provide a library of spatial functions SELECT?s WHERE {?s ogc:aswkt?geom FILTER(ogc:distance(?geom, "POINT( )"^^ogc:WKTLiteral, uom:km) <= 10)

48 OGC wktliteral Datatype Optional leading Spatial Reference System URI followed by OGC WKT geometry string. < WGS 84 Longitude, Latitude is the default SRS (assumed if SRS URI is absent) SRS: WGS84 Longitude, Latitude "POINT( )"^^ogc:wktLiteral SRS: NAD27 Longitude, Latitude "< POINT( )"^^ogc:wktLiteral

49 OGC wktliteral Datatype Prepare for spatial querying by creating a spatial index for the ogc:wktliteral datatype SQL> exec sem_apis.add_datatype_index( ' options=>'tolerance=1.0 SRID=8307 DIMENSIONS=((LONGITUDE,-180,180)(LATITUDE,-90,90))');

50 What Types of Spatial Data are Supported? Spatial Reference Systems Built-in support for 1000 s of SRS Plus you can define your own Coordinate system transformations applied transparently during indexing and query Geometry Types Support OGC Simple Features geometry types Point, Line, Polygon Multi-Point, Multi-Line, Multi-Polyon Geometry Collection Up to 500,000 vertices per Geometry

51 RDF Semantic Graph Technologies Partners Ontology Engineering & Visualization External Reasoners NLP Entity Extractors Open Source Frameworks Standards Joseki Sesame Applications & Tools SI / Consulting NL

52 The End. Martien Vos