COMP9321 Web Application Engineering

Transcription

1 COMP9321 Web Application Engineering Semester 2, 2017 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 5 1

2 Semantic Web 2

3 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. 3

4 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. Web of Data beyond the Web of Documents! 4

5 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. Web of Data beyond the Web of Documents! Human Readable un-typed Links 5

6 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. Web of Data beyond the Web of Documents! Human Readable Sydney Are two documents talking about the same thing? un-typed Links 6

7 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. Web of Data beyond the Web of Documents! Human Readable Sydney Are two documents talking about the same thing? un-typed Links Sydney: A city in Australia? A city in Canada? A person? 7

8 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. Web of Data beyond the Web of Documents! Machine Readable (typed Links among THINGS) 8

9 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. Web of Data beyond the Web of Documents! Machine Readable (typed Links among THINGS) was-born-in M. Turnbull Prime-Minister-of Sydney city-in Australia 9

10 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. Web of Data beyond the Web of Documents! Machine Readable (typed Links among THINGS) Degree of Structure: High (based on RDF data model) M. Turnbull was-born-in Prime-Minister-of city-in Sydney Australia e.g. Google KG 10

11 W3C's "Semantic Web Vision 1. Web information has exact meaning! e.g. automatic airline reservation Knows about your preferences: builds up a knowledge-base using your past activities 11

12 W3C's "Semantic Web Vision 1. Web information has exact meaning! e.g. automatic airline reservation Knows about your preferences: builds up a knowledge-base using your past activities Combines the local knowledge with remote services: social networks airline preferences Hospital Information Systems dietary requirements Linkedin Jobs and Calendaring information 12

13 W3C's "Semantic Web Vision 1. Web information has exact meaning 2. Web information can be understood and processed by computers Provide a common way to describe information: so it can be read and understood by computer applications e.g. Describing : properties for publications, such as Author and Title information about web pages such as content, author, and creation date content and rating for web pictures content for search engines electronic libraries 13

14 W3C's "Semantic Web Vision 1. Web information has exact meaning 2. Web information can be understood and processed by computers Provide a common way to describe information: so it can be read and understood by computer applications e.g. Describing : properties for publications, such as Author and Title information about web pages such as content, author, and creation date content and rating for web pictures content for search engines electronic libraries 14

15 W3C's "Semantic Web Vision 1. Web information has exact meaning 2. Web information can be understood and processed by computers 3. Computers can integrate information from the web A. Map the various data onto an abstract data representation B. Merge the resulting representations C. Start making queries on the whole! queries that could not have been done on the individual data sets 15

16 W3C's "Semantic Web Vision 1. Web information has exact meaning 2. Web information can be understood and processed by computers 3. Computers can integrate information from the web A. Map the various data onto an abstract data representation B. Merge the resulting representations C. Start making queries on the whole! queries that could not have been done on the individual data sets 16

17 3A. Map the various data onto an abstract data representation Example: A simplified bookstore data (dataset A ) ID Author-id Title Publisher-id Year ISBN-13: id-bho The Audacity of Hope id ID Author-Name Home Page id-bho Barack Obama ID Pub-Name Headquarter Location id-1954 Vintage Books New York 17

18 3A. Map the various data onto an abstract data representation Example: A simplified bookstore data (dataset A ) The Audacity of Hope 2008 A:Title ISBN-13: Vintage Books A:Pub-Name New York Barack Obama A: Author-Name 18

19 3A. Map the various data onto an abstract data representation Example: A simplified Knowledge Base (dataset B ) A knowledge base (KB) typically consists of a set of concepts organized into a taxonomy, instances for each concept, and relationships among the concepts. Examples: Freebase, DBpedia, YAGO, Wikidata ~ Chai, Xiaoyong, et al. "Social Media Analytics: The Kosmix Story." IEEE Data Eng. Bull (2013):

20 3A. Map the various data onto an abstract data representation Example: A simplified Knowledge Base (dataset B ) Barak Obama Hawaii Michelle Obama Lawyer Harvard Law School 20

21 3A. Map the various data onto an abstract data representation Example: A simplified Knowledge Base (dataset B ) 21

22 W3C's "Semantic Web Vision 1. Web information has exact meaning 2. Web information can be understood and processed by computers 3. Computers can integrate information from the web A. Map the various data onto an abstract data representation B. Merge the resulting representations C. Start making queries on the whole! queries that could not have been done on the individual data sets 22

23 3B. Merge the resulting representations Example: A simplified Knowledge Base (dataset B ) 23

24 W3C's "Semantic Web Vision 1. Web information has exact meaning 2. Web information can be understood and processed by computers 3. Computers can integrate information from the web A. Map the various data onto an abstract data representation B. Merge the resulting representations C. Start making queries on the whole! queries that could not have been done on the individual data sets 24

25 3C. Start making queries on the whole! Queries that could not have been done on the individual data sets : Give me list of the books whose : author was born in Hawaii. author was born in author is married. author s spouse was graduated from Harvard More? We could add extra knowledge to the merged datasets, e.g.: Full classification of various types of library data; Geographical information, maps; Images; Social Media Accounts; 25

26 3C. Start making queries on the whole! Queries that could not have been done on the individual data sets : Give me list of the books whose : author was born in Hawaii. author was born in author is married. author s spouse was graduated from Harvard More? We could add extra knowledge to the merged datasets, e.g.: Full classification of various types of library data; Geographical information, maps; Images; Social Media Accounts; 26

27 Semantic Web It is a collection of standard technologies to realize a Web of Data! RDF, OWL, SPARQL, etc. 27

28 Resource Description Framework (RDF) 28

29 Resource Description Framework (RDF) RDF is a framework for describing resources on the web. e.g. Web documents concepts from the real world (e.g. people, organisations, topics, things) 29

30 Resource Description Framework (RDF) RDF is a framework for describing resources on the web. Web documents concepts from the real world (e.g. people, organisations, topics, things) e.g. RDF uses Web identifiers (URIs) to identify resources. URIs (Uniform Resource Identifiers) are very important, providing both the core of the framework itself and the link between RDF and the Web. 30

31 Resource Description Framework (RDF) On the Semantic Web, URIs identify not just Web documents, but also real-world objects like people and cars. Example (Web Documents): The company Example Inc. has a Web site at Part of the site is a white-pages service listing the names and contact details of the employees. Alice and Bob both work at Example Inc. The structure of the Web site might thus be: each of the pages mentioned above are Web documents. 31

32 Resource Description Framework (RDF) On the Semantic Web, URIs identify not just Web documents, but also real-world objects like people and cars. Example (objects): The company Example Inc. wants to publish contact data of their employees on the Semantic Web. Business partners can import this data into their address books. Hash URIs 303 URIs FOAF (an acronym of Friend of a friend) is a machine-readable ontology describing persons, their activities and their relations to other people and objects. 32

33 RDF Resource, Property, and Property Value Resource: anything that can have a URI e.g. Property: a Resource that has a name, such as author Property Value: the value of a Property, such as Barack Obama" a property value can be another resource Statement: The combination of a Resource, a Property, and a Property value forms a Statement. 33

34 RDF Resource, Property, and Property Value Statement known as the subject, predicate and object Example: Subject Predicate Object Statement: The author of is Barack Obama. Subject: Predicate: author Object: Barack Obama 34

35 <?xml version="1.0" encoding="us-ascii"?> RDF (DBLP Example) <rdf:rdf xmlns:dblp=" xmlns:rdf=" <dblp: Description rdf:about=" <dblp:title>web Services - Concepts, Architectures and Applications</dblp:title> <dblp:bibtextype rdf:resource=" /> <dblp:publicationtype rdf:resource=" /> <dblp:authoredby rdf:resource=" /> <dblp:authoredby rdf:resource=" /> <dblp:authoredby rdf:resource=" /> <dblp:authoredby rdf:resource=" /> <dblp:publishedinseries>data-centric Systems and Applications</dblp:publishedInSeries> <dblp:yearofpublication>2004</dblp:yearofpublication> <dblp:publishedby>springer</dblp:publishedby> <dblp:isbn13> </dblp:isbn13> </dblp: Description > </rdf:rdf> 35

36 <?xml version="1.0" encoding="us-ascii"?> RDF (DBLP Example) <rdf:rdf xmlns:dblp=" xmlns:rdf=" Properties as Resources <dblp: Description rdf:about=" <dblp:title>web Services - Concepts, Architectures and Applications</dblp:title> <dblp:bibtextype rdf:resource=" /> <dblp:publicationtype rdf:resource=" /> <dblp:authoredby rdf:resource=" /> <dblp:authoredby rdf:resource=" /> <dblp:authoredby rdf:resource=" /> <dblp:authoredby rdf:resource=" /> <dblp:publishedinseries>data-centric Systems and Applications</dblp:publishedInSeries> <dblp:yearofpublication>2004</dblp:yearofpublication> <dblp:publishedby>springer</dblp:publishedby> <dblp:isbn13> </dblp:isbn13> </dblp: Description > </rdf:rdf> Properties as Attributes 36

37 RDF (DBLP Example) Beheshti et. al.: Extending SPARQL to Support Entity Grouping and Path Queries. (2012) 37

38 RDF Schema (RDFS) 38

39 RDF Schema (RDFS) RDF Schema provides a data-modelling vocabulary for RDF data. RDF Schema is an extension of the basic RDF vocabulary. RDF Schema provides the framework to describe application-specific classes and properties. Classes in RDF Schema are much like classes in object oriented programming languages. classes are also resources, but they are also a collection of possible resources <?xml version="1.0"?> <rdf:rdf xmlns:rdf=" xmlns:rdfs=" xml:base=" <rdfs:class rdf:id= human" /> <rdfs:class rdf:id= student"> <rdfs:subclassof rdf:resource="#human"/> </rdfs:class> </rdf:rdf> 39

40 RDF Schema (RDFS) RDF Schema provides a data-modelling vocabulary for RDF data. RDF Schema is an extension of the basic RDF vocabulary. RDF Schema provides the framework to describe application-specific classes and properties. Classes in RDF Schema are much like classes in object oriented programming languages. classes are also resources, but they are also a collection of possible resources <?xml version="1.0"?> <rdf:rdf xmlns:rdf=" xmlns:rdfs=" xml:base=" <rdfs:class rdf:id= human" /> <rdfs:class rdf:id= student"> <rdfs:subclassof rdf:resource="#human"/> </rdfs:class> </rdf:rdf> 40

41 Web Ontology Language (OWL) 41

42 Web Ontology Language (OWL) Ontology is about the exact description of things (web information) and their relationships. OWL was designed to provide a common way to process the content of web information OWL and RDF are much of the same thing, but OWL is a stronger language with greater machine interpretability than RDF. OWL provides additional vocabulary along with a formal semantics OWL making it easier for machines to automatically process and integrate information available on the Web. OWL has three sublanguages OWL Lite OWL DL OWL Full OWL is complex: OWL is a large set of additional terms 42

43 OWL: Term equivalences For classes: owl:equivalentclass: two classes have the same individuals owl:disjointwith: no individuals in common <owl:class rdf:about="#us_president"> <owl:equivalentclass rdf:resource="#principalresidentofwhitehouse"/> </owl:class> <owl:class rdf:about="#man"> <owl:disjointwith rdf:resource="#woman"/> </owl:class> 43

44 OWL: Term equivalences For classes: owl:equivalentclass: two classes have the same individuals owl:disjointwith: no individuals in common For properties: owl:equivalentproperty owl:propertydisjointwith For individuals: owl:sameas: two URIs refer to the same concept ( individual ) owl:differentfrom: negation of owl:sameas 44

45 OWL: Term equivalences For classes: owl:equivalentclass: two classes have the same individuals owl:disjointwith: no individuals in common Example: Connecting to French For properties: owl:equivalentproperty owl:propertydisjointwith For individuals: owl:sameas: two URIs refer to the same concept ( individual ) owl:differentfrom: negation of owl:sameas 45

46 OWL: Ontology development The hard work is to create the ontologies requires a good knowledge of the area to be described OWL is just a tool to formalize ontologies large scale ontologies are often developed in a community process Ontologies should be shared and reused can be via the simple namespace mechanisms or via explicit import 46

47 Ontologies Examples eclassowl: ebusiness ontology for products and services, 75,000 classes and 5,500 properties National Cancer Institute s ontology: about 58,000 classes Open Biomedical Ontologies Foundry: a collection of ontologies, including the Gene Ontology to describe gene and gene product attributes in any organism or protein sequence and annotation terminology and data (UniProt) BioPAX: for biological pathway data 47

48 Rules Rule Interchange Format (RIF) 48

49 Rules There is a long history of rule languages and rule-based systems eg: logic programming (Prolog), production rules Lots of small and large rule systems E.g. from mail filters to expert systems Why rules on the Semantic Web? There are conditions that ontologies (ie, OWL) cannot express a well known example is Horn rules: (P1 P2 ) C 49

50 Rules Example a novel with over 500 pages and costing less than $5 is a cheap book. If {?x rdf:type p:novel; p:page_number?p; p:price rdf:value?z?p > "500"^^xsd:integer.?z < "5.0"^^xsd:double. } then {?x rdf:type p:cheapbook } 50

51 A new requirement: exchange of rules Applications may want to exchange their rules: e.g. negotiate ebusiness contracts across platforms e.g. describe privacy requirements and policies Rule Interchange Format (RIF): Goals: expresses the rules a bit like a rule language can be used to exchange rules among engines Challenges: Rule based systems can be very different different rule semantics A universal exchange format is not feasible 51

52 A new requirement: exchange of rules Rule Interchange Format (RIF): The idea is to define cores for a family of languages with variants Introduction to the Semantic Web (tutorial), Ivan Herman, W3C 52

53 A new requirement: exchange of rules Rule Interchange Format (RIF): The idea is to define cores for a family of languages with variants Possible dialects: F-logic, production rules, fuzzy or probabilistic logic, Introduction to the Semantic Web (tutorial), Ivan Herman, W3C 53

54 A new requirement: exchange of rules Rule Interchange Format (RIF): The idea is to define cores for a family of languages with variants Introduction to the Semantic Web (tutorial), Ivan Herman, W3C 54

55 Rules vs OWL? In a SW application, should I use RIF, OWL, or both? The two approaches are complimentary there are things that rules cannot really express or infer eg, inferencing complex relationships among classes there are things that ontologies cannot really express or in only a very complicated manner eg, complex Horn rules Often, applications require both Interesting paper: Paul Suganthan, G. C., et al. "Why Big Data Industrial Systems Need Rules and What We Can Do About It." (SIGMOD 2015). Introduction to the Semantic Web (tutorial), Ivan Herman, W3C 55

56 Linking Open Data (LOD) 56

57 Linking Open Data (LOD) - Goal: expose open datasets in RDF - Set RDF links among the data items from different datasets - Set up query endpoints - Result: billions of triples and millions of links - The important point here is that : the data becomes available to the World via a unified format (ie, RDF). the various datasets are interlinked together e.g. DBpedia project 57

58 Linking Open Data (LOD) 58

59 Linking Open Data (LOD) Linked Data: exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF 59

60 More? The Google Knowledge Graph is a system that Google launched in May 2012 that understands facts about people, places and things and how these entities are all connected

61 Query RDF Data (SPARQL) 61

62 Query RDF Data (SPARQL) RDF is a directed, labelled graph data format for representing information in the Web. 62

63 Query RDF Data (SPARQL) SPARQL Protocol And RDF Query Language, is an RDF query language. This specification defines the syntax and semantics of the SPARQL query language for RDF. SPARQL can be used to express queries across diverse data sources. SPARQL contains capabilities for querying required and optional graph patterns The results of SPARQL queries can be results sets or RDF graphs. 63

64 Query RDF Data (SPARQL) The (subject,?p,?o) is a pattern for what we are looking for with?p and?o as unknowns SPARQL query: SELECT?p?o WHERE { subject?p?o } The triples in WHERE define the graph pattern The query returns all p,o pairs 64

65 SPARQL: Simple Query Data: Query: Result: 65

66 SPARQL: Simple Query Data: Query: SPARQL is not based on XML Names beginning with a? or a $ are variables. Graph patterns are given as a list of triple patterns enclosed within braces {} The variables named after the SELECT keyword are the variables that will be returned as results. Result: 66

67 SPARQL: Multiple Matches Data: Query: Result: 67

68 SPARQL: Constraints SPARQL FILTER functions like regex can test RDF literals. Data: Query: Result: 68

69 SPARQL: Constraints SPARQL FILTERs can restrict on arithmetic expressions. Data: Query: Result: 69

70 SPARQL usage in practice SPARQL is usually used over the network separate documents define the protocol and the result format SPARQL Protocol for RDF with HTTP and SOAP bindings SPARQL results in XML or JSON formats Big datasets usually offer SPARQL endpoints using this protocol typical example: SPARQL endpoint to DBpedia 70

71 SPARQL usage in practice SPARQL is usually used over the network separate documents define the protocol and the result format SPARQL Protocol for RDF with HTTP and SOAP bindings SPARQL results in XML or JSON formats Big datasets usually offer SPARQL endpoints using this protocol typical example: SPARQL endpoint to DBpedia 71

72 Apache Jena

73 Apache Jena A free and open source Java framework for building Semantic Web and Linked Data applications. The framework is composed of different APIs interacting together to process RDF data

74 Apache Jena Main Java packages: oaj -> org.apache.jena 74

75 Apache Jena ARQ - A SPARQL Processor for Jena ARQ is a query engine for Jena that supports the SPARQL RDF Query language. ARQ - Application API The application API is in the package org.apache.jena.query Key Classes Query - a class that represents the application query. QueryExecution - represents one execution of a query. QueryExecutionFactory - a place to get QueryExecution instances. DatasetFactory - a place to make datasets. 75

76 SELECT queries: Apache Jena 1. A query is created from a string using the QueryFactory. 2. The query and model or RDF dataset to be queried are then passed to QueryExecutionFactory to produce an instance of a query execution. 3. QueryExecution objects are java.lang.autocloseable and can be used in tryresource. 4. Result are handled in a loop and finally the query execution is closed. 76

77 Graphs 77

78 Graphs Graph is a fundamental data structure that captures relationships between different data entities. Graphs are widely used for modelling complicated data in different application domains such as: social networks, protein networks, transportation networks, bibliographical networks, knowledge bases, many more. 78

79 Graphs Graphs with millions and billions of nodes and edges have become very common For example, there are over 1.86 billion monthly active Facebook users (Facebook MAUs) which is a 17 percent increase year over year. (Source: Facebook as of 02/01/17) and billions and billions of friendship relationships (edges). The enormous growth in graph sizes requires huge amounts of computational power to analyze. Graph analytics is an important and effective discovery tool. For example, it enables identifying influential persons in a social network and inspecting fraud operations in a complex interaction network. 79

80 Graphs Graphs with millions and billions of nodes and edges have become very common For example, there are over 1.86 billion monthly active Facebook users (Facebook MAUs) which is a 17 percent increase year over year. (Source: Facebook as of 02/01/17) and billions and billions of friendship relationships (edges). The enormous growth in graph sizes requires huge amounts of computational power to analyze. Graph analytics is an important and effective discovery tool. For example, it enables identifying influential persons in a social network and inspecting fraud operations in a complex interaction network. 80

81 Twitter Example 81

82 Twitter Example 82

83 References Introduction to the Semantic Web (tutorial), Ivan Herman, W3C

84 84