NoSQL (Not only SQL)

Transcription

1 NoSQL (Not only SQL) - Many distributed application in the Web, don t need much of the guarantees and features of relation databases, and of SQL. - These guarantees and features usually come at a high price! - Take Amazon e-commerce platform Hundreds of services; millions of simultaneous customers at peek time; tens of thousands servers in geographically distributed server Most services only require read/write by primary key; complex queries are mostly unnecessary; availability is usually more crucial than consistency - NoSQL movement - abandon some of the guarantees and features, for having performance boosts Big Web 2.0 companies developed and use NoSQL systems (e.g. Google, Facebook, Amazon) José Alferes - Adaptado de Database System Concepts - 6 th Edition 364

2 CAP conjecture - Conjecture [Eric Brewer, PODC 2000]: It is impossible for a webservice to provide the 3 guarantees of: Consistency: all nodes should see the same data at the same time; Availability: node failures do not prevent others from continuing to operate (read and write); Partition-tolerance: the system continues to operate despite network partition - Devise systems that have: highly availability; high performance; scalability. José Alferes - Adaptado de Database System Concepts - 6 th Edition 365

3 CAP - Pick two José Alferes - Adaptado de Database System Concepts - 6 th Edition 366

4 The relational model vs key-value - The relational model is sometimes too rigid the attributes of objects may vary it is hard (or impossible) to anticipate which are the needed attributes for objects of a given type - Adopted a much simpler model - Key-Value (e.g. Amazon s Dinamo) (persistent) objects are accessible by a key, and have a value (which may be complex) API with: get(key) - return the value of the object(s) with the key put(key,context,value) - creates/updates an object with the key to have the value; context contains metadata about the object Objects can further be ordered and linked José Alferes - Adaptado de Database System Concepts - 6 th Edition 367

5 Keyspace based key-value - Group objects in keyspaces e.g. Cassandra - Keyspace are similar to schemas - API: insert(table,key,rowmutation) get(table,key,columnname) delete(table,key,columnname) José Alferes - Adaptado de Database System Concepts - 6 th Edition 368

6 Document based key-value - View each object as a document, and group objects in collections (similar to keyspaces) e.g. MongoDB - Insert objects in a collection: E.g. db.insert.customer(id: 1234, name: John, ) - More elaborate constructs to find objects (not only by key) db.customer.find() - returns all costumers db.customer.find( {name= A } ) - returns all customer whose name is John Also patterns for conjunctions, disjunctions, distinct values, etc José Alferes - Adaptado de Database System Concepts - 6 th Edition 369

7 More complex queries - For joins, use (more low-level) cursors, to iterate over different collections - For aggregations use map-reduce map function determines a list for each group reduce function determines what to do with each group - Example (sum of balances of accounts per branch) db.accounts.mapreduce( function () { emit(this.branch, this.balance) } function (key, values) { return Array.sum(value) }. {out: balances_sum } ) José Alferes - Adaptado de Database System Concepts - 6 th Edition 370

8 NoSQL evolution José Alferes - Adaptado de Database System Concepts - 6 th Edition 371

9 NoSQL evolution José Alferes - Adaptado de Database System Concepts - 6 th Edition 371

10 NoSQL evolution José Alferes - Adaptado de Database System Concepts - 6 th Edition 371

11 NoSQL evolution José Alferes - Adaptado de Database System Concepts - 6 th Edition 371

12 NoSQL evolution José Alferes - Adaptado de Database System Concepts - 6 th Edition 371

13 NoSQL evolution :-) José Alferes - Adaptado de Database System Concepts - 6 th Edition 371

14 Web of Data José Júlio Alferes ISCTEM - janeiro de 2017

15 Linked Data José Júlio Alferes ISCTEM - janeiro de 2017

16 Semantic Web José Júlio Alferes ISCTEM - janeiro de 2017

17 Web of Data José Júlio Alferes ISCTEM - janeiro de 2017

18 A Web of Documents Most of today s Web content is organised in a big graph of Documents Even Web content that is generated automatically is usually then processed and presented as a document, without the original structural information (e.g. from databases) Typical usage of this graph of documents requires people - for seeking and making use of information, - searching for and getting in touch with other people, - reviewing catalogues of online stores and ordering products by filling out forms, ISCTEM - janeiro 2017

19 A Web of Documents Most of today s Web content is organised in a big graph of Documents Even Web content that is generated automatically is usually then processed and presented as a document, without the original structural information (e.g. from databases) Typical usage of this graph of documents requires people - for seeking and making use of information, - searching for and getting in touch with other people, - reviewing catalogues of online stores and ordering products by filling out forms, A Web in machines, for humans usage ISCTEM - janeiro 2017

20 Planning a trip ISCTEM - janeiro 2017

21 Planning a trip One has to consult several sites, with different styles, purposes, languages,... Mentally integrate the data and understand it Apply personal preferences ISCTEM - janeiro 2017

22 Planning a trip One has to consult several sites, with different styles, purposes, languages,... Mentally integrate the data and understand it Apply personal preferences How could we do a program to perform such a task? Note that those pages most likely are already based in structured data, but one can t access it (easily) ISCTEM - janeiro 2017

23 Querying the Web Google is great! - for keyword search ISCTEM - janeiro 2017

24 Querying the Web Google is great! - for keyword search ISCTEM - janeiro 2017

25 Querying the Web Google is great! - for keyword search ISCTEM - janeiro 2017

26 Querying the Web Google is great! - for keyword search What if we want to make more elaborate queries? - Similar to what you may do in databases. ISCTEM - janeiro 2017

27 Querying the Web Google is great! - for keyword search What if we want to make more elaborate queries? - Similar to what you may do in databases. ISCTEM - janeiro 2017

28 Querying the Web Google is great! - for keyword search What if we want to make more elaborate queries? - Similar to what you may do in databases. What about very specialised queries? - E.g. in specific domains - What are the genes and phenotypes associated with OMIM (Online Mendelian Inheritance in Man) diseases? - Again similar to those to databases. ISCTEM - janeiro 2017

29 A Web of Linked Data For these queries, what we need is a Web of (raw) Data ISCTEM - janeiro 2017

30 A Web of Linked Data For these queries, what we need is a Web of (raw) Data Use the Linked Data in the same way we do with linked documents - be able to link data independently of presentation - use the data to query it, present it, mine it - have data in a machine processable format ISCTEM - janeiro 2017

31 A Web of Linked Data For these queries, what we need is a Web of (raw) Data Use the Linked Data in the same way we do with linked documents - be able to link data independently of presentation - use the data to query it, present it, mine it - have data in a machine processable format And we need this in a way that makes it easy to write applications using the linked combined data - Opens the way for a whole new range of applications! ISCTEM - janeiro 2017

32 A Semantic Web of Data ISCTEM - janeiro 2017

33 A Semantic Web of Data But if we have all that data, highly distribute and built in a highly collaborative way, how can we know its meaning? ISCTEM - janeiro 2017

34 A Semantic Web of Data But if we have all that data, highly distribute and built in a highly collaborative way, how can we know its meaning? In the Web, the data must be accompanied by its meaning! - The meaning of Web documents is not readily machineaccessible: lack of semantics - Natural language processing cannot be the answer in such a huge scale ISCTEM - janeiro 2017

35 A Semantic Web of Data But if we have all that data, highly distribute and built in a highly collaborative way, how can we know its meaning? In the Web, the data must be accompanied by its meaning! - The meaning of Web documents is not readily machineaccessible: lack of semantics - Natural language processing cannot be the answer in such a huge scale The Web of Data must come with a machine processable model of what the data means. - It must be a Semantic Web ISCTEM - janeiro 2017

36 The Semantic Web Vision I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web the content, links, and transactions between people and computers. A Semantic Web, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The intelligent agents people have touted for ages will finally materialize. Tim Berners-Lee, 1999 ISCTEM - janeiro 2017

37 The Semantic Web Vision Represent Web content in a form that is more easily machine-processable. Use intelligent techniques to take advantage of these representations. Have a Web of Data, together with a Web of Documents ISCTEM - janeiro 2017

38 The Semantic Web Vision Represent Web content in a form that is more easily machine-processable. Use intelligent techniques to take advantage of these representations. Have a Web of Data, together with a Web of Documents A Web in machines, for humans and machines usage Evolving into a Web of Data ISCTEM - janeiro 2017

39 An example of integration Consider the following data(base) on a bookstore, to be published in a Web page ISCTEM - janeiro 2017

40 Data Integration Example Besides publishing the data on a nice Web page, also export data as relations - Relations form a graph, where nodes refer to either data or identifiers (URI) ISCTEM - janeiro 2017

41 Data Integration Example Data in another bookstore: ISCTEM - janeiro 2017

42 Data Integration Example ISCTEM - janeiro 2017

43 Data Integration Example ISCTEM - janeiro 2017

44 Data Integration Example ISCTEM - janeiro 2017

45 Data Integration Example What if we have additional knowledge? E.g. - we could know that a:author is the same as f:auteur - we could know that in both cases they represent persons In other words, we could have - ontologies describing the concepts and relations - mappings of our relations into the ontologies ISCTEM - janeiro 2017

46 Data Integration Example ISCTEM - janeiro 2017

47 Data Integration Example With this merged information, richer queries are possible: - The second bookstore can query what is the web page of the author of the book - Possibly can ask about other information on the author, or translator via foaf - The first bookstore can query information about the translator - ISCTEM - janeiro 2017

48 A Web of Data Example ISCTEM - janeiro 2017

49 Automatic data integration This process of merging knowledge is done easily by users in the Web For doing it automatically, some more rigour is needed - Have data structured in the web - Name relations in a standard manner - Have ontologies for describing general concepts - Refer to the ontologies when exporting relations The Semantic Web provides technologies to make all this possible - For structuring data; defining relations; defining ontologies; querying data; reasoning over data; ISCTEM - janeiro 2017

50 The Semantic Web Cake ISCTEM - janeiro 2017

51 A new Model for Data Having tables (relations) to store the data is not adequate in this setting - parts of similar information are in different places - different places may have different ways to organise local data Data must be organised as a (huge) graph - a labeled digraph, linking small pieces of data at different places A standard language for expressing the labeled digraphs - RDF (Resource Description Framework) - Statements are triples of the form - <objectnode> <propertyedge> <subjectnode> - Sometimes also viewed as property(object,subject) ISCTEM - janeiro 2017

52 RDF Challenges How to access data organised in huge digraphs? ISCTEM - janeiro 2017

53 RDF Challenges How to access data organised in huge digraphs? - How to detect patterns in graphs that span across different sites? ISCTEM - janeiro 2017

54 RDF Challenges How to access data organised in huge digraphs? - How to detect patterns in graphs that span across different sites? - How to grab the data in this distributed framework? ISCTEM - janeiro 2017

55 RDF Challenges How to access data organised in huge digraphs? - How to detect patterns in graphs that span across different sites? - How to grab the data in this distributed framework? - How can this be done efficiently? - With what algorithms? - With what data structures to ease the (local and global) access? ISCTEM - janeiro 2017

56 RDF Challenges How to access data organised in huge digraphs? - How to detect patterns in graphs that span across different sites? - How to grab the data in this distributed framework? - How can this be done efficiently? - With what algorithms? - With what data structures to ease the (local and global) access? Studying properties of data is like studying properties of graphs - And, as we shall see, there are huge graphs to play with ISCTEM - janeiro 2017

57 A vocabulary for RDF How do we know the vocabulary used by data, to associate to it, or to query it? - We can have global vocabularies defined somewhere, so that people can adhere to it ISCTEM - janeiro 2017

58 A vocabulary for RDF How do we know the vocabulary used by data, to associate to it, or to query it? - We can have global vocabularies defined somewhere, so that people can adhere to it We need a language for defining vocabularies - describe properties and classes of RDF data - allow typing objects and properties - provide semantics for generalisation hierarchies of properties and classes ISCTEM - janeiro 2017

59 A vocabulary for RDF How do we know the vocabulary used by data, to associate to it, or to query it? - We can have global vocabularies defined somewhere, so that people can adhere to it We need a language for defining vocabularies - describe properties and classes of RDF data - allow typing objects and properties - provide semantics for generalisation hierarchies of properties and classes RDFS is a standard language for defining vocabularies - RDFS is itself encoded as RDF - So, the definition of the vocabulary is itself a graph ISCTEM - janeiro 2017

60 Beyond vocabularies In general, a vocabulary (in RDFS) is not enough to characterise the objects and properties For really connecting data, and make joint use of it, one may need to know, e.g. - that one concept is a subconcept of another (e.g. authors are persons) - that two concepts are equivalent, or disjoint, or complementary (e.g. author and auter are the same concept) - that some logical relations exist between objects (e.g. a country must have a capital) - Value and cardinality restriction (e.g. min and max value, etc) A richer ontology language is needed ISCTEM - janeiro 2017

61 Ontology language We need a language that - is formal - to be machine processable - is able to express properties about object - comes with procedures for knowing what follows from the ontology and the data - comes with procedures to check whether the data complies with the ontology (and to test whether the ontology alone is correct) ISCTEM - janeiro 2017

62 Ontology language We need a language that - is formal - to be machine processable - is able to express properties about object - comes with procedures for knowing what follows from the ontology and the data - comes with procedures to check whether the data complies with the ontology (and to test whether the ontology alone is correct) What about First Order Logic? ISCTEM - janeiro 2017

63 Ontology language We need a language that - is formal - to be machine processable - is able to express properties about object - comes with procedures for knowing what follows from the ontology and the data - comes with procedures to check whether the data complies with the ontology (and to test whether the ontology alone is correct) What about First Order Logic? Good for every requirement, as expressed above ISCTEM - janeiro 2017

64 Ontology language We need a language that - is formal - to be machine processable - is able to express properties about object - comes with procedures for knowing what follows from the ontology and the data - comes with procedures to check whether the data complies with the ontology (and to test whether the ontology alone is correct) What about First Order Logic? Good for every requirement, as expressed above But not good enough, if I had added efficient before procedures - They aren t even decidable! ISCTEM - janeiro 2017

65 Fragments of FOL But maybe we don t need the full power of FOL ISCTEM - janeiro 2017

66 Fragments of FOL But maybe we don t need the full power of FOL Challenge: Use fragments of FOL: - that are decidable - are still expressive enough for the applications - when possible, allow for efficient proof procedures ISCTEM - janeiro 2017

67 Fragments of FOL But maybe we don t need the full power of FOL Challenge: Use fragments of FOL: - that are decidable - are still expressive enough for the applications - when possible, allow for efficient proof procedures Example: - If we only use binary and unary predicates - and all formulas are either facts or of the form x a(x) F[x], where a is a unary predicate and F[x] is a formula such that: - it doesn't use disjunction nor negation - and quantifiers are only used in formulas of the form - y r(y,x) b(x) - y r(y,x) b(x) - then the language is decidable and tractable ISCTEM - janeiro 2017

68 Description Logics Study of decidable fragments of FOL, their computational complexity/expressivity, and of corresponding proof procedures The language that I described is usually not good enough for ontologies in the Web - We have to add some more things - But if we add too much, procedures become intractable - And if we add even more, they may become undecidable What is needed depends on the application domain - E.g. if we have languages for which the complexity of the procedures depend more on the ontology than on the data, use them for domains with small ontologies and lots of data ISCTEM - janeiro 2017

69 Web Ontology Language OWL is a family of languages for defining ontologies for linked data in the Web - It is a standard set of languages, all of them based on Description Logics (basically it is DL written with another syntax!) - Different languages with different complexity, for different uses - All of them with proof procedures There are plenty of tools for OWL - Theorem provers (for knowing what follows from the ontology and the data) - Consistency checker (for testing the correctness of the ontology, and compliance of the data with the ontology) - Ontology editors - ISCTEM - janeiro 2017

70 Query Languages For querying the data and ontology, we could just use the ontology language itself - build a concept with the conditions that specify the desired result - use the theorem provers to get the object in that concept ISCTEM - janeiro 2017

71 Query Languages For querying the data and ontology, we could just use the ontology language itself - build a concept with the conditions that specify the desired result - use the theorem provers to get the object in that concept But this is not good enough! ISCTEM - janeiro 2017

72 Query Languages For querying the data and ontology, we could just use the ontology language itself - build a concept with the conditions that specify the desired result - use the theorem provers to get the object in that concept But this is not good enough! Things we want to use in querying were thrown way - E.g. join conditions (quite typical in database queries) - a,b,n author(b,a) name(a,n) result(b,n) - This, together with basic things for defining ontologies, immediately makes the language undecidable ISCTEM - janeiro 2017

73 Query Languages For querying the data and ontology, we could just use the ontology language itself - build a concept with the conditions that specify the desired result - use the theorem provers to get the object in that concept But this is not good enough! Things we want to use in querying were thrown way - E.g. join conditions (quite typical in database queries) - a,b,n author(b,a) name(a,n) result(b,n) - This, together with basic things for defining ontologies, immediately makes the language undecidable We need a more expressive query language, on top of the ontology language ISCTEM - janeiro 2017

74 SPARQL (Sparql Protocol And Rdf Query Language) For queries, use an algebra on graphs - Define operators that given graphs (the data, and consequences from the ontology) return graphs (the result) - A query is an algebraic expression it that algebra - This is similar to databases, where the query language is relational algebra Then have a nice, CS-like, language - In the same way that in databases we have SQL - In fact SPARQL looks a lot like SQL ISCTEM - janeiro 2017

75 SPARQL (Sparql Protocol And Rdf Query Language) For queries, use an algebra on graphs - Define operators that given graphs (the data, and consequences from the ontology) return graphs (the result) - A query is an algebraic expression it that algebra - This is similar to databases, where the query language is relational algebra Then have a nice, CS-like, language - In the same way that in databases we have SQL - In fact SPARQL looks a lot like SQL PREFIX dc: < PREFIX ns: < SELECT?title?price WHERE {?x ns:price?price. FILTER (?price < 30).?x dc:title?title. } ISCTEM - janeiro 2017

76 The Semantic Web at work Until recently, all of this was mainly a vision for a future, better, Web Today it is a reality! The Semantic Web can only work if there is enough linked web data out there, and enough ontologies to go with it - The same could have been said about the Web, in its beginning Today there is plenty of Linked Data in the Web ISCTEM - janeiro 2017

77 Linked Open Data Project An open initiative project Exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF. Large number of datasets with connections between them Billions of triples, millions of links! See at ISCTEM - janeiro 2017

78 Linked Open Data Project An open initiative project Exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF. Large number of datasets with connections between them Billions of triples, millions of links! See at ISCTEM - janeiro 2017

79 Linked Open Data Project An open initiative project Exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF. Large number of datasets with connections between them Billions of triples, millions of links! See at ISCTEM - janeiro 2017

80 Linked Open Data Project An open initiative project Exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF. Large number of datasets with connections between them Billions of triples, millions of links! See at ISCTEM - janeiro 2017

81 Linked Open Data Project And there are quite large datasets! - E.g. bio2rdf has 5 billion triples, dbpedia has 3 billion, An open has initiative project geonames 100 million, dblp has 88 millions Exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF. Large number of datasets with connections between them Billions of triples, millions of links! See at ISCTEM - janeiro 2017

82 Linked Open Data Project And there are quite large datasets! - E.g. bio2rdf has 5 billion triples, dbpedia has 3 billion, An open has initiative project geonames 100 million, dblp has 88 millions Exposing, sharing, and connecting pieces of data, information, knowledge on the Number ofand axioms in 2011 Semantic Web using URIs and RDF. Large number of datasets with connections between them Billions of triples, millions of links! See at ISCTEM - janeiro 2017

83 DBpedia It is one of the nodes in that Linked Data graph Includes (up-to-date) raw data extracted from wikipedia (infobox) Interlinks with other datasets on the linked open data ISCTEM - janeiro 2017

84 DBpedia It is one of the nodes in that Linked Data graph Includes (up-to-date) raw data extracted from wikipedia (infobox) Interlinks with other datasets on the linked open data ISCTEM - janeiro 2017

85 DBpedia It is one of the nodes in that Linked Data graph Includes (up-to-date) raw data extracted from wikipedia (infobox) Interlinks with other datasets on the linked open data ISCTEM - janeiro 2017

86 Brief intro to RDF Resource Description Framework

87 Brief intro to RDF Resource Description Framework

88 A bit of history RDF was created by the W3C in 1999 as a format for structuring metadata about web pages (title, author, modification date, ) It has been designed to allow information to be exchanged without loss of meaning It became a general data format/model for data in general (rather than only metadata) - Proposed in 2001 as the model for data exchange in the Semantic Web - Nowadays there are quite big data stores in RDF - The today s data model of Semantic Technologies!

89 Basic Ideas of RDF Represent the meaning independently of the syntax Basic building block: object-attribute-value triple - It is called a statement - In the example, object is José Alferes, attribute is lectures, value is Semantic Web Fundamental concepts in RDF are: - Resources (like the object and value above) - Properties (like the attribute above) - Statements (the triple above)

90 RDF syntax With these basic ideas, an RDF dataset is a labeled directed graph - Modelling data as a graph, rather than modelling as a set of relations (in databases), or as a tree (in XML) There are several syntaxes for writing such a graph: - N3 (Notation 3) comprehensive formalism - N-Triples: fraction of N3 - Turtle (Terse RDF Triple Language) A standard encoding of RDF into XML has also been defined - Facilitates usage of XML tools - RDF should not be confused with its XML syntactical representation!

91 First RDF example (in Turtle)

92 Resources A resource is just any thing, an object we want to refer to - E.g. an author, a book, a place, a person, a hotel, etc In RDF, every resource has a unique identifier - Unique identifiers are crucial to disambiguate resources! - On the Web we already have unique identifiers, and RDF just uses them URIs will be used as identifiers

93 Properties Properties describe relations between resources - E.g. written by, has age, has title, etc. Each property is itself also a resource - So, properties are also identified by URIs Advantages of using identifying URIs: - Α global, worldwide, unique naming scheme - Reduces the homonym problem of distributed data representation - (Basically, with URIs everything is guaranteed to be uniquely identified, by a key)

94 Statements Statements assert properties of resources - Relate resources via properties In RDF, a statement is an object-attribute-value triple - It consists of a resource, a property, and a value They can be seen as binary predicates: attribute(object,value) Values can be resources or literals - Literals are just atomic values (e.g. strings), that don t need to have a URI.

95 Representation of statements A statement can be viewed as: - A triple (Object, Property, Value) - An arc connecting two nodes in a graph - A piece of XML code, representing the triple Accordingly, an RDF document can be viewed as: - A set of triples - A graph (semantic net) - An XML document with the triples represented according to a given predefined syntax

96 Representing triples in a graph centria.di.fct.unl.pt/~jja site-owner José Alferes This piece of a graph is representing the triple: - (centria.di.fct.unl.pt/~jja, site-owner, José Alferes ) - Or the predicate site-owner(centria.di.fct.unl.pt/~jja, José Alferes ) (these things are not proper URIs, but for the sake of the example ) An RDF document can be seen as a directed graph with labeled nodes and arcs - from the resource (the subject of the statement) - to the value (the object of the statement) Known in AI as a semantic net

97 An example of RDF graph centria.di.fct.unl.pt/~jja site-owner José Alferes has-phone is-friend-of uses João Leite site-owner ssdi.di.fct.unl.pt/bd The owner of the site ~jja is José Alferes who has phone and is a friend of João Leite, who owns /bd web page; José Alferes uses this web page

98 RDF as a data model A schema-less data model - Based on a graph - With unambiguous identifiers - With named relations between pair of resources The graph structure trivialises merging data with shared identifiers Triples act as least common identifier for expressing data Eases navigation through data in different locations

99 A bit of (Turtle) syntax Turtle was defined to be a simple syntax for RDF, and is standardised by W3C since February 2014 (see - Triples are directed lists: Subject Property Object - URI are in <angle brackets> - End with. - White spaces are ignored - Prefixes (simple string concatenation) - Grouping of triples with the same subject with ; - Grouping of triples with the same subject and property with,

100 Turtle example

101 Literals Represent data values - Encoded as strings but can be interpreted by means of datatypes - Literals without any type are treated as strings - A literal without a type is called a plain literal, and may have a language tag Datatypes are borrowed from XML Schema (XSD) - RDF does not require an implementation of datatypes, though systems usually implement most of XSD datatypes Some examples - Typed literals - Bay State ^^xsd:string or ^^xsd:integer - Plain literals - Germany or

102 Blank nodes bnodes are used for resources that do not need to be universally identified Just like existentially quantified ex:< ex:john ex:hascourse _:id1; ex:hascourse _:id2. _:id1 ex:course course1 ; ex:withgrade 12. _:id2 ex:course course1 ; ex:withgrade 12.

103 Blank nodes bnodes are used for resources that do not need to be universally identified Just like existentially quantified ex:< ex:john ex:hascourse [ex:course course1 ;ex:withgrade 12 ], [ex:course course2 ;ex:withgrade 15 ].

104 Blank ex:< ex:john ex:hascourse [ex:course course1 ;ex:withgrade 12 ], [ex:course course2 ;ex:withgrade 15 ].

105 Blank ex:< ex:john ex:hascourse [ex:course course1 ;ex:withgrade 12 ], [ex:course course2 ;ex:withgrade 15 ] course course2 15

106 RDF Schema Defining schemas for RDF data

107 RDF Schema Defining schemas for RDF data

108 Why a schema language Like in databases, to understand the data one needs some formalisation of what it is about - what classes, with each types of attributes, with what relationship between classes, etc - this is especially important when the data is highly distributed, and provided in a collaborative way RDF provides a data model to state propositions about individual resources In a schema language we need to state propositions about generic sets of individuals - and also logical interdependencies between them

109 RDF Schema (RDFS) - RDF is a universal language that lets users describe resources - It does not assume any meaning for the vocabulary used - It does not assume, nor does it define semantics of any particular application domain RDFS allows for specifying terminological knowledge, that RDF data can refer to - It is a language for describing (simple) semantics of arbitrary RDF - Uses RDF itself (with dedicated vocabulary) RDFS is part of RDF s W3C recommendation - xmlns:rdfs = - Notice: The relation between RDF Schema and RDF is not the same as that between XML Schema and XML

110 RDFS Basic Ideas Schemas are specified with: - Classes and Properties - Class (and Property) hierarchies and inheritance - Property Restriction (e.g. stating that authors of a book must be persons) Classes and Instances - Instances (defined in RDF) refer to concrete individual objects (e.g. me, this book) - Classes denote sets of individuals sharing some properties (e.g. persons, books) - In RDFS classes are also seen as objects (with URIs) - The relationship between instances and classes is made via special attribute rdf:type of the instance - amalgamating data and meta-data

111 First Schema example ex:jja ex:name Jose Alferes ; ex:teaches ex:sw; rdf:type uni:professor. The last statement characterises me as an instance of class professor. - An individual can belong to more than one class. E.g. uni:professor rdf:type ex:sw ex:teaches ex:jja ex:name Jose Alferes

112 First Schema example ex:jja ex:name Jose Alferes ; ex:teaches ex:sw; rdf:type uni:professor. The last statement characterises me as an instance of class professor. - An individual can belong to more than one class. E.g. ex:jja rdf:type rel:father. rel:father uni:professor rdf:type rdf:type ex:sw ex:teaches ex:jja ex:name Jose Alferes

113 The class of all classes In the example ex:professor and ex:father are objects whose type is rdfs:class rdfs:class is also of type rdfs:class (the class of all classes) - The triple rdfs:class rdf:type rdfs:class virtually belongs to all dataset (already some kind of semantics) rel:father rdf:type uni:professor rdf:type rdfs:class rdf:type rdf:type rdf:type ex:sw ex:teaches ex:jja ex:name Jose Alferes

114 Class Hierarchies Suppose we are searching for all Academic Staff - jja should be considered This can be done by having a general statement, saying that all professors are academic staff (with refs:subclassof) rdfs:subclassof is - a property - it is reflexive and transitive - can be used to enforce that two URIs refer to the same class - with A rdfs:subclassof B and B rdfs:subclassof A

115 Simple taxonomy rdf:type is like rdfs:subclassof is like uni:members rdfs:subclassof uni:student uni:academicstaff uni:administrative rdfs:subclassof rel:father uni:professor uni:associate uni:assistant rdf:type rdf:type ex:sw ex:teaches ex:jja ex:name Jose Alferes

116 Properties Properties specify in which ways two resources are related - usually appear in the predicate position of triples - mathematically represented as binary relations (sets of pairs) - their rdf:type is rdf:property Hierarchical relationships may also be defined for properties - E.g., teaches is a sub-property of isinvolvedin - If a professor P teaches course C, then P is involved in C P rdf:subpropertyof Q, if Q(x,y) is true whenever P(x,y) is true

117 Property restriction Defined by rdfs:domain and rdfs:range Restrict the kind of resources that can be related via the property - E.g. a property teaches only makes sense if it is relating an academic staff to a course - It can also be used to declare datatypes for literals ex:teaches rdfs:domain uni:academicstaff; rdfs:range uni:course. ex:name rdf:range xsd:string.

118 Property restriction Property restriction are interpreted globally and conjunctively - E.g. ex:teaches rdfs:domain uni:professor; rdfs:domain uni:associate. ex:cd rdf:teaches ex:sbd. - entails that ex:cd is both a uni:professor and a uni:associate. In this case (and in general) this is not what is wanted! - When designing the schema, choose the most general class for domain and range!

119 An example schema

120 An example schema ex:teaches ex:name ex:sw ex:jja Jose Alferes

121 An example schema rdf:type uni:professor uni:course ex:teaches ex:name ex:sw ex:jja Jose Alferes

122 An example schema rdf:type rdfs:subclassof uni:members uni:student uni:academicstaff uni:administrative uni:professor uni:associate uni:assistant uni:course ex:teaches ex:name ex:sw ex:jja Jose Alferes

123 An example schema rdf:type rdfs:subclassof uni:members uni:student uni:academicstaff uni:administrative rdfs:domain uni:professor uni:associate uni:assistant uni:course xsd:string rdfs:range rdfs:range ex:teaches ex:name ex:sw ex:jja Jose Alferes

124 An example schema rdf:type rdfs:subclassof uni:members rdfs:subpropertyof rdfs:domain uni:student uni:academicstaff uni:administrative ex:isinvolvedin rdfs:range rdfs:domain uni:professor uni:associate uni:assistant uni:course xsd:string rdfs:range rdfs:range ex:teaches ex:name ex:sw ex:jja Jose Alferes

125 An example schema rdf:type rdfs:subclassof uni:members rdfs:subpropertyof rdfs:domain uni:student uni:academicstaff uni:administrative ex:isinvolvedin rdfs:range rdfs:domain uni:professor uni:associate uni:assistant uni:course xsd:string rdfs:range ex:teaches ex:name rdfs:range Schema ex:sw ex:jja Jose Alferes Instance

126 An example schema ex:jja ex:teaches ex:sw. ex:jja rdf:type uni:professor. ex:sw rdf:type uni:course. uni:professor rdfs:subclassof uni:academicstaff. uni:associate rdfs:subclassof uni:academicstaff. uni:assistant rdfs:subclassof uni:academicstaff uni:academicstaff rdfs:subclassof uni:member. uni:student rdfs:subclassof uni:member. uni:administrative rdfs:subclassof uni:member. ex:teaches rdf:domain uni:academicstaff; rdf:range uni:course; rdf:subpropertyof ex:isinvolvedin. ex:isinvolvedin rdf:domain uni:member; rdf:range uni:course; ex:name rdf:range xsd:string.

127 And also ex:jja ex:teaches ex:sw. ex:jja rdf:type uni:professor. ex:sw rdf:type uni:course. uni:professor rdfs:subclassof uni:academicstaff. uni:associate rdfs:subclassof uni:academicstaff. uni:assistant rdfs:subclassof uni:academicstaff uni:academicstaff rdfs:subclassof uni:member. uni:student rdfs:subclassof uni:member. uni:administrative rdfs:subclassof uni:member. ex:teaches rdf:domain uni:academicstaff; rdf:range uni:course; rdf:subpropertyof ex:isinvolvedin. ex:isinvolvedin rdf:domain uni:member; rdf:range uni:course; ex:name rdf:range xsd:string.

128 And also ex:jja ex:teaches ex:sw. ex:jja rdf:type uni:professor. ex:sw rdf:type uni:course. uni:professor rdfs:subclassof uni:academicstaff. uni:associate rdfs:subclassof uni:academicstaff. uni:assistant rdfs:subclassof uni:academicstaff uni:academicstaff rdfs:subclassof uni:member. uni:student rdfs:subclassof uni:member. uni:administrative rdfs:subclassof uni:member. ex:teaches rdf:domain uni:academicstaff; rdf:range uni:course; rdf:subpropertyof ex:isinvolvedin. ex:isinvolvedin rdf:domain uni:member; rdf:range uni:course; ex:name rdf:range xsd:string. uni:professor rdf:type rdfs:class. uni:associate rdf:type rdfs:class. uni:assistant rdf:type rdfs:class. uni:academicstaff rdf:type rdfs:class. uni:administrative rdf:type rdfs:class. uni:administrative rdf:type rdfs:class. uni:member rdf:type rdfs:class. ex:teaches rdf:type rdfs:property. ex:isinvolvedin rdf:type rdfs:property.

129 And also ex:jja ex:teaches ex:sw. ex:jja rdf:type uni:professor. ex:sw rdf:type uni:course. uni:professor rdfs:subclassof uni:academicstaff. uni:associate rdfs:subclassof uni:academicstaff. uni:assistant rdfs:subclassof uni:academicstaff uni:academicstaff rdfs:subclassof uni:member. uni:student rdfs:subclassof uni:member. uni:administrative rdfs:subclassof uni:member. ex:teaches rdf:domain uni:academicstaff; rdf:range uni:course; rdf:subpropertyof ex:isinvolvedin. ex:isinvolvedin rdf:domain uni:member; rdf:range uni:course; ex:name rdf:range xsd:string. uni:professor rdf:type rdfs:class. uni:associate rdf:type rdfs:class. uni:assistant rdf:type rdfs:class. uni:academicstaff rdf:type rdfs:class. uni:administrative rdf:type rdfs:class. uni:administrative rdf:type rdfs:class. uni:member rdf:type rdfs:class. ex:teaches rdf:type rdfs:property. ex:isinvolvedin rdf:type rdfs:property. rdfs:class rdf:type rdfs:class. rdfs:property rdf:type rdfs:class. rdf:type rdf:type rdfs:property. rdf:domain rdf:type rdfs:property. rdfs:subclassof rdf:type rdfs:property. rdfs:subpropertyof rdf:type rdfs:property. rdfs:class rdfs:subclassof rdfs:resource. rdfs:property rdfs:subclassof rdfs:resource. Not up to the user to define. Meaning assigned by RDFS semantics.

130 Core Classes of RDF(S) rdfs:resource, the class of all resources rdfs:class, the class of all classes rdfs:literal, the class of all literals rdf:property, the class of all properties. rdf:statement, the class of all reified statements

131 Core Properties rdf:type, which relates a resource to its class - The resource is declared to be an instance of that class rdfs:subclassof, which relates a class to one of its superclasses - All instances of a class are instances of its superclass rdfs:subpropertyof, relates a property to one of its super-properties

132 SPARQL Querying the Semantic Web

133 SPARQL Querying the Semantic Web

134

135 SPARQL Query Language SPARQL - Sparql Protocol And Rdf Query Language (read as sparkle) provides facilities to: - extract information in the form of URIs, blank nodes, plain and typed literals. - extract RDF sub-graphs. - construct new RDF graphs based on information in the queried graphs. RDF graphs can be obtained from several sources, including middleware capable of generating RDF data. SPARQL is based on matching of graph patterns, using variables in the patterns - Results can come as binding of variables to RDF terms (URIs, literals or blank nodes) in a tabular form

136 Query forms SPARQL has four query forms: - SELECT, returns variable bindings - ASK, returns a boolean indicating whether a query pattern matches or not - DESCRIBE, returns a RDF graph that describes the resources found - CONSTRUCT, returns RDF graphs by substituting variables in a set of triple templates

137 Simple SELECT query The query consists of two parts, the SELECT clause and the WHERE clause. - The SELECT clause identifies the variables to appear in the query results. - The WHERE specifies the graph pattern to match against the RDF data graph. The simplest form of graph patterns are triple patterns - a triple pattern is an RDF triple with optional query variables in any place of the triple. Basic Graph Patterns (BGPs) are sets of triple patterns, in Turtle syntax

138 Our first SPARQL query PREFIX uni: < PREFIX ex: < PREFIX rdf: < SELECT?x?name WHERE {?x ex:teaches ex:sw.?x rdf:type uni:professor.?x ex:name?name } Professor who teach ex:sw, and their names Syntax: - Abbreviated URIs with PREFIX - Variable names signalled by? (or by $) - Literals are as in turtle - ; and, can be used as in Turtle

139 Our first SPARQL query SELECT?x?name WHERE {?x ex:teaches ex:sw ; rdf:type uni:professor ; ex:name?name } - Professors who teach ex:sw, and their names Determines the parts of the graph that match the BGP in the WHERE clause Returns the bindings of variables in the SELECT clause?x?name < Jose Alferes

140 Variables everywhere! Variables are allowed in subject, object and also in predicate positions of triples - E.g. what are the relations between a guy called José Alferes and the object ex:sw? SELECT?relation WHERE {?x ex:name Jose Alferes.?x?relation ex:sw }?relation <

141 Blank nodes Can be used in query patterns - As in RDF, cannot be used in predicates - Act like variables, but cannot be selected - Remember the semantics of bnodes in RDF - Have arbitrary ID, and cannot be reused in different BGP Can be returned in results (if they are bnodes in the original graph) - Placeholders for unknown elements - With arbitrary IDs, with the scope limited to the result, that can be different from those in the input RDF - But repeated occurrences in the result denote the same element SELECT?name WHERE {_:a ex:teaches ex:sw ; rdf:type uni:professor ; ex:name?name }

142 Datatypes Exact match for the datatypes is required - E.g. {?x ex:p test.} does not match with ex:ex1 ex:p test ^^xsd:string. But abbreviation for numbers are allowed - E.g. {?x ex:p 123.} matches with ex:ex1 ex:p 123^^xsd:integer. - The datatype is determined from the syntax - xsd:integer (e.g. 123) - xsd:decimal (e.g ) - xsd:double (e.g. 1.23e2)

143 Some syntactic simplifications Collections - RDF collections can be written using the syntax ( element1 element2 ) - rdf:nil can be replaced by () - E.g. {(1?x) ex:p w } is the same as rdf:first 1 ; rdf:rest _:b. {_:a _:b rdf:fist?x; rdf:rest rdf:nil. ex:p w } _:a rdf:type - Can be replaced by a. E.g. {?x a uni:professor} - Reads better

144 RDF Datasets When compared with SQL, what is missing?

145 RDF Datasets When compared with SQL, what is missing? - The FROM clause

146 RDF Datasets When compared with SQL, what is missing? - The FROM clause No FROM clause is required in SPARQL - A SPARQL query is executed against an RDF Dataset which represents a collection of graphs - RDF data stores hold multiple RDF graphs, and record information about each graph, allowing an application to make queries that involve information from more than one graph (a collection of graphs) But more (named) graphs can be specified

147 Graph provenance SPARQL can use the keywords FROM and FROM NAMED, to specify the default graph and named graphs, respectively. The FROM and FROM NAMED are followed by an URI. The graphs retrieved by FROM (NAMED) clauses are merged for constructing the default graph The GRAPH construct can be used to obtain the provenance of results, or to query a particular named graph

148 FROM NAMED example SELECT?name?mbox FROM NAMED < FROM NAMED < WHERE { [ foaf:name?name; foaf:mbox?mbox ] } Returns all names and mailboxes in the result of merging the default graph with the graphs in < a> and in <

149 Provenance example SELECT?graph?name?mbox FROM NAMED < FROM NAMED < WHERE { GRAPH?graph { [ foaf:name?name; foaf:mbox?mbox ] } }?graph?name?mbox < Jose <mailto:j@example.pt> < Maria <mailto:m@example.pt> < Ana <mailto:a@example.pt> < Jose <mailto:j@example.pt>

150 Provenance example SELECT?name?mbox FROM NAMED < FROM NAMED < WHERE { GRAPH < { [ foaf:name?name; foaf:mbox?mbox ] } GRAPH < { [ foaf:name?name; foaf:mbox?mbox ] } }?name Jose?mbox <mailto:j@example.pt>

151 Multiple BGP and UNION BGP can be grouped, and several can be used. {} stand for the empty BGP, and matches any graph UNION between BGPs stand for the union (disjunction) of the BGPs. - E.g. Who are the associate or full professors SELECT?x WHERE { {?x a uni:professor } UNION {?x a uni:associate } }

152 Unbound results UNION may give rise to unbound results. - E.g. What are the names or nomes of courses? SELECT?x?name?nome WHERE {?x a uni:course. {?x ex:name?name } UNION {?x ex:nome?nome} }

153 Unbound results UNION may give rise to unbound results. - E.g. What are the names or nomes of courses? SELECT?x?name?nome WHERE {?x a uni:course. {?x ex:name?name } UNION {?x ex:nome?nome} }?x?name?nome ex:sw Semantic Web ex:sw ex:sbd Web Sistemas de

154 OPTIONAL patterns Another keyword that may return unbound results is OPTIONAL It allows for the specification of optional parts of a graph - E.g. What are the nomes of courses, and names if they exist?

155 OPTIONAL patterns Another keyword that may return unbound results is OPTIONAL It allows for the specification of optional parts of a graph - E.g. What are the nomes of courses, and names if they exist? SELECT?x?name?nome WHERE {?x a uni:course.?x ex:nome?nome OPTIONAL {?x ex:name?name} }

156 OPTIONAL patterns Another keyword that may return unbound results is OPTIONAL It allows for the specification of optional parts of a graph - E.g. What are the nomes of courses, and names if they exist? SELECT?x?name?nome WHERE {?x a uni:course.?x ex:nome?nome OPTIONAL {?x ex:name?name} }?x?name?nome ex:sw Semantic Web Web ex:sbd Sistemas de

157 Combination of UNION and OPTIONAL When used in combination UNION has higher precedence - I.e. X UNION Y OPTIONAL Z is to be interpreted as {X UNION Y} OPTIONAL Z - E.g. The names of either authors or editors of ex:swbook, and also the when available?

158 Combination of UNION and OPTIONAL When used in combination UNION has higher precedence - I.e. X UNION Y OPTIONAL Z is to be interpreted as {X UNION Y} OPTIONAL Z - E.g. The names of either authors or editors of ex:swbook, and also the when available? SELECT?name? WHERE {?x ex:name?name. {ex:swbook ex:author?x } UNION {ex:swbook ex:edit?x } OPTIONAL {?x ex: ? } }

159 Combination of UNION and OPTIONAL When used in combination UNION has higher precedence - I.e. X UNION Y OPTIONAL Z is to be interpreted as {X UNION Y} OPTIONAL Z - E.g. The names of either authors or editors of ex:swbook, and also the when available? SELECT?name? WHERE {?x ex:name?name. { {ex:swbook ex:author?x } UNION {ex:swbook ex:edit?x } } OPTIONAL {?x ex: ? } }

160 Filters Even with complex graph patterns, some quite reasonable queries are not expressible: - Who are the professors aged more than 50? - Professors with a name started by A? FILTER allows us to specify conditions over variables, restricting the solutions to those that satisfy the condition SELECT?prof WHERE {?prof a uni:professor; ex:age?age. FILTER{?age > 50 } }

161 Filter conditions Conditions evaluate to truth values (booleans) Usual comparison operator are allowed - = and!= for all RDF types - >, <, <= and >= for numerical datatypes, dates, string and booleans Function can be used - Arithmetic function (+, -, *, /) for numerical datatypes - e.g. FILTER {?x /?y >?z } - XQuery and XPath functions - Specific SPARQL functions

162 Specific functions BOUND(A) ISURI(A) ISBLANK(A) ISLITERAL(A) SAMETERM(A,B) REGEX(A,B) DATATYPE(A) LANG(A) returns true if A is a bound variable returns true if A is (bound to) a URI returns true if A a bnode returns true if A a literal returns true if A and B are the same RDF term returns true if the string A matches the regular expression B returns the datatype of A returns the language tag of A (or if none exists) Several more functions, especially in SPARQL Check the manual/specification

163 Combining conditions Conditions can be combined - Conjunctively, by adding multiple filters - Disjunctively, with UNION But boolean operators &&, and! are also available - E.g. Professors between 20 and 30, or not younger than 50? SELECT?prof WHERE {?prof a uni:professor; ex:age?age. FILTER{ (?age > 20 &&?age < 30 )!(?age < 50) } }

164 Solution modifiers The result of a SELECT query is a bag of tuples (a table) - But one can view it as a sequence of tuples, and consider their order - Or as a set, without repeated elements Similarly to SQL, there can be an ORDER BY clause, and select only DISTINCT values - By default the order is ascending - For descending, use DESC - String and numbers are sorted as usual - URIs are sorted as strings - unbound variables < bnodes < URIs < Literals One can also LIMIT the number of results, and define an OFFSET for the first result shown

165 Modifiers examples 10 oldest professors sorted by age, from oldest to younger:

166 Modifiers examples 10 oldest professors sorted by age, from oldest to younger: SELECT?prof?age WHERE {?prof a uni:professor; ex:age?age} ORDER DESC(?age) LIMIT 10

167 Modifiers examples 10 oldest professors sorted by age, from oldest to younger: SELECT?prof?age WHERE {?prof a uni:professor; ex:age?age} ORDER DESC(?age) LIMIT 10 Second oldest professor:

168 Modifiers examples 10 oldest professors sorted by age, from oldest to younger: SELECT?prof?age WHERE {?prof a uni:professor; ex:age?age} ORDER DESC(?age) LIMIT 10 Second oldest professor: SELECT?prof?age WHERE {?prof a uni:professor; ex:age?age} ORDER DESC(?age) LIMIT 1 OFFSET 2

169 Modifiers examples 10 oldest professors sorted by age, from oldest to younger: SELECT?prof?age WHERE {?prof a uni:professor; ex:age?age} ORDER DESC(?age) LIMIT 10 Second oldest professor: SELECT?prof?age WHERE {?prof a uni:professor; ex:age?age} ORDER DESC(?age) LIMIT 1 OFFSET 2 Ages of professors

170 Modifiers examples 10 oldest professors sorted by age, from oldest to younger: SELECT?prof?age WHERE {?prof a uni:professor; ex:age?age} ORDER DESC(?age) LIMIT 10 Second oldest professor: SELECT?prof?age WHERE {?prof a uni:professor; ex:age?age} ORDER DESC(?age) LIMIT 1 OFFSET 2 Ages of professors SELECT DISTINCT?age WHERE { [] a uni:professor; ex:age?age}

171 Aggregates SPARQL 1.1 includes aggregates and aggregate functions - As in SQL, aggregates group solutions and compute values over the groups - Groups are defined by a GROUP BY clause - There can be filters over groups, with HAVING Available aggregate function: - COUNT, MIN, MAX, SUM, AVG are as usual in SQL - SAMPLE, picks one random value from the group - GROUP_CONCAT, concatenate strings in the group

172 Aggregates examples How many courses does each lecturer who teach more than 2, teach

173 Aggregates examples How many courses does each lecturer who teach more than 2, teach SELECT?prof COUNT(?course) AS?ncourses WHERE {?prof ex:teaches?course } GROUP BY?prof HAVING?ncourses > 2 - Note the renaming of results with AS

174 Aggregates examples How many courses does each lecturer who teach more than 2, teach SELECT?prof COUNT(?course) AS?ncourses WHERE {?prof ex:teaches?course } GROUP BY?prof HAVING?ncourses > 2 - Note the renaming of results with AS String with list of professors names by age

175 Aggregates examples How many courses does each lecturer who teach more than 2, teach SELECT?prof COUNT(?course) AS?ncourses WHERE {?prof ex:teaches?course } GROUP BY?prof HAVING?ncourses > 2 - Note the renaming of results with AS String with list of professors names by age SELECT?age GROUP_CONCAT(?name, separator=, ) WHERE { _:x a uni:professor; ex:age?age; ex:name?name } GROUP BY?age

176 Subqueries In SPARQL 1.1 one can have subqueries, whose results are conjunctively joined with the remainder BGP - E.g. name of courses that are taught by exactly two lecturers

177 Subqueries In SPARQL 1.1 one can have subqueries, whose results are conjunctively joined with the remainder BGP - E.g. name of courses that are taught by exactly two lecturers SELECT?cname WHERE {?course ex:name?cname. { SELECT?course COUNT(?lect) AS?nl WHERE {?lect ex:teaches?course } GROUP BY?course HAVING?nl = 2 } }

178 Variable BINDing In SPARQL 1.1 variables can be bound to expressions with BIND and AS SELECT?name?yearBirth WHERE { [] a uni:professor; ex:age?age BIND ((2015 -?age) AS?yearBirth) }

179 Variable BINDing In SPARQL 1.1 variables can be bound to expressions with BIND and AS SELECT?name?yearBirth WHERE { [] a uni:professor; ex:age?age BIND ((2015 -?age) AS?yearBirth) } Or SELECT?name (2015 -?age) AS?yearBirth WHERE { [] a uni:professor; ex:age?age }

180 Property paths Another feature of SPARQL 1.1 is property paths - Extend patterns from simple tripes to paths, eventually with arbitrary length Property paths are built with regular expressions over predicates - Alternative paths:?s (exp1 expn)?o - Inverse paths:?s ^exp?o (the same as?o exp?s) - Negation of paths:?s!exp?o - Sequence of paths:?s exp1/ /expn?o - Paths with arbitrary length:?s exp+?o,?s exp*?o, and?s exp??o - Paths with fixed size:?s exp{n}?o,?s exp{n,}?o, and?s exp{,n}?o,?s exp{n,m}?o

181 Property paths example The ancestors of ex:jja:

182 Property paths example The ancestors of ex:jja: SELECT?ans WHERE { ex:jja (ex:mother ex:father)+?ans }

183 Property paths example The ancestors of ex:jja: SELECT?ans WHERE { ex:jja (ex:mother ex:father)+?ans } The descendants of ex:jja:

184 Property paths example The ancestors of ex:jja: SELECT?ans WHERE { ex:jja (ex:mother ex:father)+?ans } The descendants of ex:jja: SELECT?desc WHERE { ex:jja (^ex:mother ^ex:father)+?desc }

185 Property paths example The ancestors of ex:jja: SELECT?ans WHERE { ex:jja (ex:mother ex:father)+?ans } The descendants of ex:jja: SELECT?desc WHERE { ex:jja (^ex:mother ^ex:father)+?desc } The classes to which ex:jja belongs:

186 Property paths example The ancestors of ex:jja: SELECT?ans WHERE { ex:jja (ex:mother ex:father)+?ans } The descendants of ex:jja: SELECT?desc WHERE { ex:jja (^ex:mother ^ex:father)+?desc } The classes to which ex:jja belongs: SELECT?class WHERE { ex:jja rdf:type/rdfs:subclassof*?class}

187 Property paths example The ancestors of ex:jja: SELECT?ans WHERE { ex:jja (ex:mother ex:father)+?ans } The descendants of ex:jja: SELECT?desc WHERE { ex:jja (^ex:mother ^ex:father)+?desc } The classes to which ex:jja belongs: SELECT?class WHERE { ex:jja rdf:type/rdfs:subclassof*?class} The object to which ex:jja is related to, except for its type:

188 Property paths example The ancestors of ex:jja: SELECT?ans WHERE { ex:jja (ex:mother ex:father)+?ans } The descendants of ex:jja: SELECT?desc WHERE { ex:jja (^ex:mother ^ex:father)+?desc } The classes to which ex:jja belongs: SELECT?class WHERE { ex:jja rdf:type/rdfs:subclassof*?class} The object to which ex:jja is related to, except for its type: SELECT?o WHERE { ex:jja!(rdf:type)?o }

189 Negation SPARQL 1.1 includes two forms of negation, besides the negation in filter conditions - Test for patterns that do not exist - Set difference between patterns The first with FILTER NOT EXISTS {BGP} The second with MINUS between BGPs

190 Negation examples Professors for which there is no age

191 Negation examples Professors for which there is no age SELECT?prof WHERE {?prof a uni:professor. FILTER NOT EXISTS {?prof ex:age _:a} }

192 Negation examples Professors for which there is no age SELECT?prof WHERE {?prof a uni:professor. FILTER NOT EXISTS {?prof ex:age _:a} } Can also be done with MINUS

193 Negation examples Professors for which there is no age SELECT?prof WHERE {?prof a uni:professor. FILTER NOT EXISTS {?prof ex:age _:a} } Can also be done with MINUS SELECT?prof WHERE {?prof a uni:professor. MINUS {?prof ex:age?age} }

194 Testing for existence If instead of wanting to know the results (variable bindings) one is only interested in knowing whether there are results, we can simply use ASK Results of ASK are yes or no E.g. Is any lecturer of ex:sw called Jose Alferes?

195 Testing for existence If instead of wanting to know the results (variable bindings) one is only interested in knowing whether there are results, we can simply use ASK Results of ASK are yes or no E.g. Is any lecturer of ex:sw called Jose Alferes? ASK WHERE {?prof ex:name Jose Alferes ; ex:teaches ex:sw }

196 Returning graphs SELECT takes a (set of) RDF graph(s), and returns a table - In this sense, it is not algebraic SPARQL includes two commands that are algebraic - DESCRIBE: given a set of URIs returns the graph with all edge departing from that URI - CONSTRUCT: Similar to SELECT, but returning the result in the form of an RDF graph

197 Describe If given just a URI, returns all the direct data about the resource. E.g. - DESCRIBE < It can also be used to describe the result of a query that returns just one variable, which is always bound to URIs - E.g. The data about those that teach ex:sw DESCRIBE?prof WHERE {?prof ex:teaches ex:sw }

198 Construct CONSTRUCT constructs an RDF graph from a given graph template, using the variables of a query E.g. graph with name (as nome) and year of birth of lecturers of ex:sw CONSTRUCT {?prof my_ex:nome?name; my_ex:birth?year } WHERE {?prof ex:teaches ex:sw; ex:age?age; ex:name?name BIND ((2015-?age) AS?year)}

199 Construct CONSTRUCT constructs an RDF graph from a given graph template, using the variables of a query E.g. graph with name (as nome) and year of birth of lecturers of ex:sw CONSTRUCT {?prof my_ex:nome?name; my_ex:birth?year } WHERE {?prof ex:teaches ex:sw; ex:age?age; ex:name?name BIND ((2015-?age) AS?year)} E.g. all data from lecturers of ex:sw

200 Construct CONSTRUCT constructs an RDF graph from a given graph template, using the variables of a query E.g. graph with name (as nome) and year of birth of lecturers of ex:sw CONSTRUCT {?prof my_ex:nome?name; my_ex:birth?year } WHERE {?prof ex:teaches ex:sw; ex:age?age; ex:name?name BIND ((2015-?age) AS?year)} E.g. all data from lecturers of ex:sw CONSTRUCT {?prof?prop?obj } WHERE {?prof ex:teaches ex:sw;?prop?obj }

201 SPARQL Updates SPARQL update make it possible to - insert and delete contents to graph stores - manage the lifecycle of graph stores - Somehow, similar to DML of SQL INSERT and DELETE insert or delete a given set of triple into a store - Like in CONSTRUCT - Delete cannot be applied to bnodes INSERT {?person ex:birth?year } DELETE {?person ex:age?age} WHERE {?person a ex:person; ex:age?age. BIND ((2015-?age) AS?year)}

202 Managing graph stores There are extra construct to manage RDF stores - LOAD <URI1> [INTO GRAPH <URI2>] loads all the triples in URI1 to the store [in URI2] - CLEAR [GRAPH <URI>] deletes all the triples in the store [in URI] - COPY GRAPH <URI1> TO <URI2> copies and overwrites - MOVE GRAPH <URI1> TO <URI2> moves and overwrites - CREATE GRAPH <URI> - DROP GRAPH <URI>

203 Revisiting Queries We can query, e.g. inside a program, the Web of Data Let s try it (in a SPARQL query web interface) - The countries in Europe with less population than Portugal ISCTEM - janeiro 2017

204 Revisiting Queries We can query, e.g. inside a program, the Web of Data Let s try it (in a SPARQL query web interface) - The countries in Europe with less population than Portugal SELECT?Pais?Populacao WHERE {?Pais rdf:type db-owl:country ; dbpedia2:populationestimate?populacao ; purl:subject dbcat:countries_in_europe. :Portugal dbpedia2:populationestimate?populacaopt. FILTER (?Populacao <?PopulacaoPT). } ISCTEM - janeiro 2017

205 Revisiting Queries We can query, e.g. inside a program, the Web of Data Let s try it (in a SPARQL query web interface) - The countries in Europe with less population than Portugal SELECT?Pais?Populacao WHERE {?Pais rdf:type db-owl:country ; dbpedia2:populationestimate?populacao ; purl:subject dbcat:countries_in_europe. :Portugal dbpedia2:populationestimate?populacaopt. FILTER (?Populacao <?PopulacaoPT). } ISCTEM - janeiro 2017

206 Revisiting Queries We can query, e.g. inside a program, the Web of Data Let s try it (in a SPARQL query web interface) - The countries in Europe with less population than Portugal SELECT?Pais?Populacao WHERE {?Pais rdf:type db-owl:country ; dbpedia2:populationestimate?populacao ; purl:subject dbcat:countries_in_europe. :Portugal dbpedia2:populationestimate?populacaopt. FILTER (?Populacao <?PopulacaoPT). } - What are the genes and phenotypes associated with OMIM diseases? ISCTEM - janeiro 2017

207 Revisiting Queries We can query, e.g. inside a program, the Web of Data Let s try it (in a SPARQL query web interface) - The countries in Europe with less population than Portugal SELECT?Pais?Populacao WHERE {?Pais rdf:type db-owl:country ; dbpedia2:populationestimate?populacao ; purl:subject dbcat:countries_in_europe. :Portugal dbpedia2:populationestimate?populacaopt. FILTER (?Populacao <?PopulacaoPT). } - What are the genes and phenotypes associated with OMIM diseases? ISCTEM - janeiro 2017

208 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

209 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

210 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

211 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

212 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

213 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

214 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

215 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

216 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017

217 Applications are popping up Development and querying application - Yagoo - isparql - OpenLink data Explorer;... Application making use of data - Relation Finder - DBpedia mobile - GoPubMed - TrialX; A lot of prototypes at: - ISCTEM - janeiro 2017