How can Open Data and Linked Data help your organization and its Big Data? Mark Harrison

Transcription

1 How can Open Data and Linked Data help your organization and its Big Data? Mark Harrison

2 Outline Big Data, Open Data and Linked Data Open Data sources Linked Data technology Using Open Data and Linked Data within your organization

3 Big Data, Open Data and Linked Data Big Data High volume, High velocity, High variety Goal is to extract useful business intelligence and make meaningful predictions and forecasts from the large volumes of data that are being collected from various sources: Website traffic and online sales, B2B e-commerce Web search keywords, trending topics on social networks Sales transactions Sensor data and RFID Open Data Open licence for re-use, Open (non-proprietary) data formats Large sets of publicly-funded data now being released in open formats, e.g. from governments and government agencies Transparent government (e.g. how tax revenue is spent) Enabling data mash-ups, Data re-use, Innovation New digital public services, Support growth of the digital economy

4 Big Data, Open Data and Linked Data Linked Data Transform the web (a web of documents) into a web of facts Fact-level hyperlinking Modelling of domain knowledge / knowledge representations Inference of additional facts from the data Learn new relationships in the data Tools to easily query and combine data from multiple sources Integrate data from heterogeneous sources (relational, XML, etc.) Open World Model Virtually extend fixed schema with additional attributes/relationships Many open data sets are being published as Linked Data (RDF data or SPARQL query endpoints)

5 Open Data sources Government spending Environmental data Public Transport Road traffic Roadworks, Congestion Accidents Population / demographics Local socio-economic data Economy / Industry Health Geographic / Postcode

6 Linked Open Data data.gov data.gov.uk publicdata.eu geonames.org LinkedGeoData.org linkedlifedata.com Linking Open Data cloud diagram, von Richard Cyganiak und Anja Jentzsch.

7 Open Data relevant for logistics Live Traffic Information: Current Roadworks 10 mins 1Mb Current Planned Events 10 mins 0.01Mb Future Roadworks 6 hours 2Mb Future Planned Events 6 hours 0.2Mb Unplanned Events 10 mins 0.05Mb All traffic data 10 mins 30Mb Journey Time Data 10 mins 0.5Mb Variable Message Signs 10 mins 3Mb Matrix Signals 10 mins 9Mb Data is provided in XML Datex2 format ( - not yet as linked data

8 Open Data for weather Weather forecast for London (as web page) Weather forecast for London (as XML data) Provided by the Norwegian Meteorological Institute and the Norwegian Broadcasting Corporation (NRK) Very good fine-grained coverage worldwide. Javascript and PHP script examples are also provided. Free yr.no mobile apps also available for ios and Android.

9 Open Data for UK postcodes Code-Point Open ( postcode data downloads freely available upon registration (reply provides links to text file (CSV) data with the following PostCode, OS Easting, OS Northing, (6 figures, 1m resolution) (Online conversion to latitude, longitude available at: ) Ordnance Survey data and Talis SPARQL endpoint RDF data per postcode, e.g. postcodeunit/cb30fs.ttl

10 5 stars of Open Data Sir Tim Berners-Lee Data available on the web in any format, using an open licence (PDDL, ODC-by, CC0) Availability as structured data(e.g. spreadsheet vs bitmap image of chart) Use non-proprietary formats (e.g. CSV vs.xls) Use URIs to denote things, so other people can point to your data Link your data to other data to provide context

11 Linked Data essentials Identify all kinds of things (resources) - not only web pages but also places, people, products, companies, also relationshipsusing Uniform Resource Identifiers (URIs) Transform a web of documents to a web of 'facts' Publish data 'facts' about things identified by URIs Fact-level hyperlinking using Resource Description Framework (RDF) Subject Predicate Object Combine data 'facts' from multiple sources to enrich our understanding of the data we have - and find what we're really looking for! - using a query language for RDF data, (SPARQL)

12 Examples of Linked Data Identify things with Uniform Resource Identifiers (URIs) Express Facts using Resource Description Framework (RDF) Subject Predicate Object dbpedia:brussels rdfs:label dbpedia:brussels wgs84:lat dbpedia:brussels dbpprop:populationmetro Query Language for RDF-Data: (SPARQL) PREFIX geo: < PREFIX dbpedia: < SELECT?latitude WHERE { dbpedia:brussels geo:lat?latitude. } Test it yourself at: Answer: latitude 50.85

13 User-Defined Rules via SPARQL CONSTRUCT ex:charles CONSTRUCT {?s ex:hasancestor?o} WHERE {?s ex:hasparent+?o.} ex:hasancestor ex:hasparent ex:anne ex:hassister ex:margaret ex:hasparent ex:hasaunt CONSTRUCT {?s ex:hasaunt?o} WHERE {?s ex:hasparent/ex:hassister?o.} ex:mark

14 Following chains of trust using SPARQL CONSTRUCT publishes Report R orga orgb orgc orgd has Root Policy policy A policy B protects Should Organization D be granted read access? policy C chainsforward chainsforward? Organization A published a report R about ID X ID:X Organization D asks for information about ID X

15 Conversion of existing Data? Subject Property Object Direct Mapping R2RML GRDDL & XSLT <subject> Property <property> </subject> object </property> Subject Object See alsod2rq.org

16 Companies and Linked Data Open Data Your company Greater insights and better decision-making Better visibility of your products and services for interested customers + Private Company Data Publish as Open Data Data for consumers (products and services) Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch.

17 Advantages of publishing as Linked Open Data Source:

18 Some existing activities: GPC UNSPSC CPV

19 How BestBuy use Schema.org & GoodRelations BestBuy's store pagese.g. Uses RDFa 1.0 (RDF annotations) and ontologies:goodrelations, vcard, FOAF, Geo (WGS84), to embed:latitude, Longitude, Street Address, City, State, Postcode, Phone No, Opening Hours, link to each store's Facebook page BestBuy's product-specific pagesuse HTML5 microdata (itemscope, itemtype, itemprop etc.) andschema.org ontology to embed: Link to product image, Availability, Price, Manufacturer, Mfr Model Number, Retailer's SKU, Description, Aggregate Rating (1-5), Review Count Best Buy saw a 30% increase in traffic to individual store websites

20 Further reading... W3C Semantic Web Activity (presentations, links to specifications - RDF, RDFS, OWL, SPARQL) Linked Data: Evolving the Web into a Global Data Space (1st edition). Tom Heath and Christian Bizer (2011) Synthesis Lectures on the Semantic Web: Theory and Technology, 1:1, Morgan & Claypool. Linking Enterprise Data David Wood (Editor) 1st Edition., 2010 ISBN:

21 Further reading... Linked Data Structured Data on the Web David Wood (Editor) 1st Edition., 2014 ISBN: Semantic Web Concepts - presentation by Sir Tim Berners-Lee