Database Optimization Techniques for Semantic Queries

Transcription

1 Database Optimization Techniques for Semantic Queries Ioana Manolescu INRIA, France SEBD 2015, Gaeta I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 1 / 73

2 Part I Motivation and outline I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 2 / 73

3 Motivation and context Querying Semantic data Most popular data model: RDF (W3C s Resource Description Framework) Famous application: the Linked Open Data cloud I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 3 / 73

4 Motivation and context Querying Semantic data Most popular data model: RDF (W3C s Resource Description Framework) Famous application: the Linked Open Data cloud (2007) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 3 / 73

5 Motivation and context Querying Semantic data Most popular data model: RDF (W3C s Resource Description Framework) Famous application: the Linked Open Data cloud (2008) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 4 / 73

6 Motivation and context Querying Semantic data Most popular data model: RDF (W3C s Resource Description Framework) Famous application: the Linked Open Data cloud (2009) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 5 / 73

7 Motivation and context Querying Semantic data Most popular data model: RDF (W3C s Resource Description Framework) Famous application: the Linked Open Data cloud (2010) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 6 / 73

8 Motivation and context Linked Open Data cloud, ,129 documents describing 8,038,396 resources (Schmachtenberg, Bizer, Paulheim, ISWC 2014) There is more (Billion Triple Challenge etc.) Topic Datasets % Government Publications Life sciences User-generated Cross-domain Media Geographic Social web Total I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 7 / 73

9 Motivation and context Querying Semantic Data 1 RDF: three-attribute relation (subject, property, object) The subject is the resource being described The resource has the property property whose value is object Resource type is a property, specified just like any other I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 8 / 73

10 Motivation and context Querying Semantic Data 1 RDF: three-attribute relation (subject, property, object) The subject is the resource being described The resource has the property property whose value is object Resource type is a property, specified just like any other 2 RDFS semantics providing information about the properties and classes (types) of resources: Any undergraduatestudent is a Student Anyone having a graduationdate is a Student (but may also be of other types)... I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 8 / 73

11 Motivation and context Querying Semantic Data 1 RDF: three-attribute relation (subject, property, object) The subject is the resource being described The resource has the property property whose value is object Resource type is a property, specified just like any other 2 RDFS semantics providing information about the properties and classes (types) of resources: Any undergraduatestudent is a Student Anyone having a graduationdate is a Student (but may also be of other types)... Semantics leads to implicit data I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 8 / 73

12 Motivation and context Querying Semantic Data Main RDFS constraints: rdfs:subclassof, rdfs:subpropertyof, rdfs:domain, rdfs:range Beyond RDFS Many (richer) constraint (or ontology) languages W3C standards: OWL 2 profile family DL Lite family Calvanese, De Giacomo, Lembo, Lenzerini, Rosati, JAR 2007 Datalog ± Cali, Gottlob Lukasiewick, Marnette, Pierris, LICS 2010 I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 9 / 73

13 Motivation and context Do we really need the semantics? Yes. All the time. Application knowledge / constraints: Every Senator is an ElectedOfficial which is a Person (On Wikipedia) being BornInAPlace means being a Person The source and destination of a tripfromto are either a streetaddress, or a cityaddress or a countryaddress I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 10 / 73

14 Motivation and context Do we really need the semantics? Yes. All the time. Application knowledge / constraints: Every Senator is an ElectedOfficial which is a Person (On Wikipedia) being BornInAPlace means being a Person The source and destination of a tripfromto are either a streetaddress, or a cityaddress or a countryaddress 1 Without the semantics, we may miss query answers 2 Semantic contraints are a compact way of encoding information ( every ElectedOfficial is a Person stated only once) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 10 / 73

15 Motivation and context Outline 1 Motivation 2 RDF data model and query language 3 Query answering techniques 4 Cover-based query reformulation for the database fragment of RDF 5 Cover-based query reformulation framework for FOL-reducible settings 6 Performance and concluding remarks I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 11 / 73

16 Part II RDF data and queries I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 12 / 73

17 RDF data model and query language RDF The Resource Description Framework (RDF) RDF graph set of triples Assertion Triple Relational notation Class s rdf:type o o(s) Property s p o p(s, o) resource (URI) blank node literal (string) property 1949 Book publishedin rdf:type writtenby doi 1 hastitle El Aleph :b 1 hasname J. L. Borges I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 13 / 73

18 RDF data model and query language RDFS RDF Schema (RDFS) Declare deductive constraints between classes and properties Constraint Triple OWA interpretation Subclass s rdfs:subclassof o s o Subproperty s rdfs:subpropertyof o s o Domain typing s rdfs:domain o Π domain (s) o Range typing s rdfs:range o Π range (s) o Publication rdfs:subclassof Book hasauthor rdfs:subpropertyof rdfs:domain writtenby rdfs:range Person I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 14 / 73

19 RDF data model and query language RDF entailment Open-world assumption and RDF entailment RDF data model based on the open-world assumption. deductive constraints implicitly propagate triples Implicit triples: part of the graph not explicitly present Entailment reasoning mechanism I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 15 / 73

20 RDF data model and query language RDF entailment Open-world assumption and RDF entailment RDF data model based on the open-world assumption. deductive constraints implicitly propagate triples Implicit triples: part of the graph not explicitly present Entailment reasoning mechanism set of explicit triples + derive implicit triples some entailment rules I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 15 / 73

21 RDF data model and query language RDF entailment Open-world assumption and RDF entailment RDF data model based on the open-world assumption. deductive constraints implicitly propagate triples Implicit triples: part of the graph not explicitly present Entailment reasoning mechanism set of explicit triples + derive implicit triples some entailment rules Exhaustive application of entailment saturation (closure) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 15 / 73

22 RDF data model and query language RDF entailment RDF entailment The semantics of an RDF graph G is its saturation G. Sample RDFS Instance entailment from combining schema and instance entailment rules triples rdfs9 c 1 rdfs:subclassof c 2 s rdf:type c 1 rdfs9 RDF s rdf:type c 2 rdfs7 p 1 rdfs:subpropertyof p 2 s p 1 o rdfs7 RDF s p 2 o rdfs2 rdfs3 p rdfs:domain c s p o rdfs2 RDF p rdfs:range c s p o rdfs3 RDF s rdf:type c o rdf:type c Publication rdfs:domain hasauthor rdfs:subclassof rdfs:subpropertyof 1949 rdf:type Book rdfs:domain writtenby publishedin rdf:type rdfs:range doi 1 writtenby hasauthor :b 1 rdf:type Person hastitle hasname El Aleph J. L. Borges I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 16 / 73

23 RDF data model and query language Query answering SPARQL query language and SPARQL conjunctive queries SPARQL is the W3C query language for RDF. SPARQL conjunctive queries = Basic Graph Pattern (BGP) queries Sample BGP query: q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 17 / 73

24 RDF data model and query language Query answering Query semantics / Answer set query evaluation query answering The evaluation of a query only uses the graph s explicit triples For the (complete) answer set, evaluate q against the graph s saturation I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 18 / 73

25 RDF data model and query language Query answering Query answering example Given the query q(x, y):- x rdf:type y: Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin rdf:type writtenby doi 1 hasauthor hastitle El Aleph rdfs:range :b 1 rdf:type Person hasname J. L. Borges I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 19 / 73

26 RDF data model and query language Query answering Query answering example Given the query q(x, y):- x rdf:type y: Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin rdf:type writtenby doi 1 hasauthor hastitle El Aleph rdfs:range :b 1 rdf:type Person hasname J. L. Borges q(g) = {(doi 1, Book)} I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 19 / 73

27 RDF data model and query language Query answering Query answering example Given the query q(x, y):- x rdf:type y: Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin rdf:type writtenby doi 1 hasauthor hastitle El Aleph rdfs:range :b 1 rdf:type Person hasname J. L. Borges q(g) = {(doi 1, Book)} q(g ) = {(doi 1, Book), (doi 1, Publication), ( :b 1, Person)} I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 19 / 73

28 Part III Query answering techniques I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 20 / 73

29 Query answering techniques Reasoning The need for reasoning Query answering needs explicit and implicit data! Saturation-based query answering (when the saturation result is finite) Reformulation-based query answering Hybrids of the above J. Urbani, F. van Harmelen, S. Schlobach, and H. Bal, QueryPIE: Backward reasoning for OWL Horst over very large knowledge bases, ISWC 2011 I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 21 / 73

30 Query answering techniques Graph saturation Saturation-based query answering schema G answer G query q I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 22 / 73

31 Query answering techniques Graph saturation Saturation-based query answering schema G answer G query q q(g ) can be computed using an RDBMS G needs time to be computed and space to be stored Not suitable for high update rate (data and/or schema triples) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 22 / 73

32 Query answering techniques Graph saturation Saturation maintenance schema G answer G query q Compute for an update of an RDF graph G s.t. (µ(g)) = G when µ an insertion (µ(g)) = G \ when µ a deletion I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 23 / 73

33 Query answering techniques Query reformulation Reformulation-based query answering schema G answer query q query q ref I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 24 / 73

34 Query answering techniques Query reformulation Reformulation-based query answering schema G answer query q query q ref q ref (G) can be evaluated using an RDBMS Robust to updates Reformulated queries are complex, thus costly to evaluate I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 24 / 73

35 Query answering techniques Query reformulation Reformulation-based query answering schema G answer query q query q ref I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 25 / 73

36 Query answering techniques Query reformulation Reformulation-based query answering schema G answer query q query q ref q ref (G) can be evaluated using an RDBMS Robust to updates Reformulated queries are complex, thus costly to evaluate I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 25 / 73

37 Query answering techniques Query reformulation Semantic query answering in the presence of mappings Di Pinto, Lembo, Lenzerini, Mancini, Poggi, Rosatti, Ruzzi, Savo, EDBT 2013 The logical storage of the data is related to the ontology by means of mappings. A mapping states that a query on the database is included into a query on the ontology. Reformulation-based query answering in the presence of (GAV) mappings: Lanti, Rezk, Xiao and Calvanese, EDBT Start-up phase (may perform some materialization) 2 Query rewriting phase (based on the schema) 3 Query translation phase (e.g., into SQL, based on the mappings) 4 Query execution phase (e.g., through an RDBMS) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 26 / 73

38 Query answering techniques Query reformulation Semantic query answering in the presence of mappings Reformulation-based query answering in the presence of (GAV) mappings: Lanti, Rezk, Xiao and Calvanese, EDBT Start-up phase (may perform some materialization) 2 Query rewriting phase (based on the schema) 3 Query translation phase (e.g., into SQL, based on the mappings) 4 Query execution phase (e.g., through an RDBMS) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 27 / 73

39 Query answering techniques Query reformulation Semantic query answering in the presence of mappings Reformulation-based query answering in the presence of (GAV) mappings: Lanti, Rezk, Xiao and Calvanese, EDBT Start-up phase (may perform some materialization) 2 Query rewriting phase (based on the schema) 3 Query translation phase (e.g., into SQL, based on the mappings) 4 Query execution phase (e.g., through an RDBMS) This talk: Optimized reformulation technique based on considering translation and rewriting together Can be composed with mappings (and further mapping-based optimizations) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 27 / 73

40 Query answering techniques Query reformulation Semantic query answering vs. Ontology-Based Data Access (OBDA) Fundamentally the same Input data (stored somehow), ontology, query Output query answer In general, mappings may be GAV, LAV, GLAV etc. LAV can be very practical (see: RDBMS indexes!) We optimize query rewriting and translation independently of the global local (source) data mappings Integration/interactions are intriguing and current/future work I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 28 / 73

41 Query answering techniques Query reformulation Reformulation-based query answering landscape schema G query q query q ref answer Target reformulation languages for conjunctive queries (CQs): mainly unions of CQs (UCQs) F. Goasdoué, I. Manolescu, A. Roatiş: Efficient query answering against dynamic RDF databases, EDBT 2013 joins of single-atom CQs (SCQs) M. Thomazo: Compact Rewritings for Existential Rules, IJCAI 2013 joins of UCQs (JUCQs) D. Bursztyn, F. Goasdoué, I. Manolescu: Optimizing Reformulation-based Query Answering in RDF, EDBT 2015 I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 29 / 73

42 Query answering techniques Query reformulation Reformulation-based query answering landscape schema G query q query q ref answer Target reformulation languages for conjunctive queries (CQs): mainly unions of CQs (UCQs) F. Goasdoué, I. Manolescu, A. Roatiş: Efficient query answering against dynamic RDF databases, EDBT 2013 Wait: is this about SQL syntax?!... joins of single-atom CQs (SCQs) M. Thomazo: Compact Rewritings for Existential Rules, IJCAI 2013 joins of UCQs (JUCQs) D. Bursztyn, F. Goasdoué, I. Manolescu: Optimizing Reformulation-based Query Answering in RDF, EDBT 2015 I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 29 / 73

43 Query answering techniques Query reformulation Reformulation-based query answering landscape schema G query q query q ref answer Target reformulation languages for conjunctive queries (CQs): Wait: is this about SQL syntax?!... mainly unions of CQs (UCQs) F. Goasdoué, I. Manolescu, A. Roatiş: Efficient query answering against dynamic RDF databases, EDBT 2013 Yes. And it makes a big difference. joins of single-atom CQs (SCQs) M. Thomazo: Compact Rewritings for Existential Rules, IJCAI 2013 From failing to feasible, or 4 orders of magnitude speed-up on the 8 M triples DBLP dataset. joins of UCQs (JUCQs) D. Bursztyn, F. Goasdoué, I. Manolescu: Optimizing Reformulation-based Query Answering in RDF, EDBT 2015 I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 30 / 73

44 Part IV Cover-based reformulation for the database fragment of RDF I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 31 / 73

45 Cover-based reformulation for the DB fragment of RDF The database fragment of RDF Largest RDF fragment for which an FOL reformulation-based query answering technique is known Restrict RDF entailment to the rules dedicated to RDF Schema only (a.k.a. RDFS entailment). No restriction to graphs: any triple allowed by the RDF specification is also allowed in the DB fragment (e.g., blank nodes in data and schema) Algorithm Reformulate(q, db) for BGP queries over schema and data. Output size bound O((6 #db 2 ) #q ) Goasdoué, Manolescu, Roatiş, EDBT 2013 I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 32 / 73

46 Cover-based reformulation for the DB fragment of RDF CQ-to-UCQ query reformulation example Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin doi 1 hastitle El Aleph rdf:type writtenby hasauthor :b 1 hasname rdf:type J. L. Borges rdfs:range Person q(a, t):- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) leads to q ref : (0) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 33 / 73

47 Cover-based reformulation for the DB fragment of RDF CQ-to-UCQ query reformulation example Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin doi 1 hastitle El Aleph rdf:type writtenby hasauthor :b 1 hasname rdf:type J. L. Borges rdfs:range Person q(a, t):- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) leads to q ref : (0) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (1) q(a, t) :- (b, writtenby, x), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 34 / 73

48 Cover-based reformulation for the DB fragment of RDF CQ-to-UCQ query reformulation example Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin doi 1 hastitle El Aleph rdf:type writtenby hasauthor :b 1 hasname rdf:type J. L. Borges rdfs:range Person q(a, t):- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) leads to q ref : (0) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (1) q(a, t) :- (b, writtenby, x), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (2) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, writtenby, a), (b, publishedin, 1949 ) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 35 / 73

49 Cover-based reformulation for the DB fragment of RDF CQ-to-UCQ query reformulation example Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin doi 1 hastitle El Aleph rdf:type writtenby hasauthor :b 1 hasname rdf:type J. L. Borges rdfs:range Person q(a, t):- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) leads to q ref : (0) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (1) q(a, t) :- (b, writtenby, x), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (2) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, writtenby, a), (b, publishedin, 1949 ) (3) q(a, t) :- (b, writtenby, x), (b, hastitle, t), (b, writtenby, a), (b, publishedin, 1949 ) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 36 / 73

50 Cover-based reformulation for the DB fragment of RDF CQ-to-UCQ query reformulation example Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin doi 1 hastitle El Aleph rdf:type writtenby hasauthor :b 1 hasname rdf:type J. L. Borges rdfs:range Person q(a, t):- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) leads to q ref : (0) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (1) q(a, t) :- (b, writtenby, x), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (2) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, writtenby, a), (b, publishedin, 1949 ) (3) q(a, t) :- (b, writtenby, x), (b, hastitle, t), (b, writtenby, a), (b, publishedin, 1949 ) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 37 / 73

51 Cover-based reformulation for the DB fragment of RDF CQ-to-UCQ query reformulation example Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin doi 1 hastitle El Aleph rdf:type writtenby hasauthor :b 1 hasname rdf:type J. L. Borges rdfs:range Person q(a, t):- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) leads to q ref : (0) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (1) q(a, t) :- (b, writtenby, x), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) (2) q(a, t) :- (b, rdf:type, Book), (b, hastitle, t), (b, writtenby, a), (b, publishedin, 1949 ) (3) q(a, t) :- (b, writtenby, x), (b, hastitle, t), (b, writtenby, a), (b, publishedin, 1949 ) q(g ) = q ref (G) = {( :b 1, El Aleph )}. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 37 / 73

52 Cover-based reformulation for the DB fragment of RDF CQ-to-SCQ query reformulation example Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin doi 1 hastitle El Aleph rdf:type writtenby hasauthor :b 1 hasname rdf:type J. L. Borges rdfs:range Person q(a, t):- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) produces the q ref : (0) q(b) :- (b, rdf:type, Book) (b, writtenby, x) (1) q(b, a) :- (b, hasauthor, a) (b, writtenby, a) (2) q(b) :- (b, publishedin, 1949 ) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 38 / 73

53 Cover-based reformulation for the DB fragment of RDF CQ-to-SCQ query reformulation example Publication hasauthor rdfs:domain rdfs:subclassof rdfs:subpropertyof rdfs:domain 1949 rdf:type Book writtenby publishedin doi 1 hastitle El Aleph rdf:type writtenby hasauthor :b 1 hasname rdf:type J. L. Borges rdfs:range Person q(a, t):- (b, rdf:type, Book), (b, hastitle, t), (b, hasauthor, a), (b, publishedin, 1949 ) produces the q ref : (0) q(b) :- (b, rdf:type, Book) (b, writtenby, x) (1) q(b, a) :- (b, hasauthor, a) (b, writtenby, a) (2) q(b) :- (b, publishedin, 1949 ) q(g ) = q ref (G) = {( :b 1, El Aleph )}. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 38 / 73

54 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation RDF Entailment Rules G answer query q Optimiser query q Reformulation algorithm 1. Enlarges the query reformulation language w.r.t. UCQ/SCQ to have more than one reformulation alternative I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 39 / 73

55 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation RDF Entailment Rules G answer query q Optimiser query q Reformulation algorithm 1. Enlarges the query reformulation language w.r.t. UCQ/SCQ to have more than one reformulation alternative 2. Uses a cost model for estimating the cost of evaluating q ref through an RDBMS I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 39 / 73

56 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation RDF Entailment Rules G answer query q Optimiser query q Reformulation algorithm 1. Enlarges the query reformulation language w.r.t. UCQ/SCQ to have more than one reformulation alternative 2. Uses a cost model for estimating the cost of evaluating q ref through an RDBMS 3. Chooses the cheapest alternative from the search space. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 39 / 73

57 Cover-based reformulation for the DB fragment of RDF Optimized reformulation into JUCQs Query q CQ-to-UCQ ref. algo. Graph G Results q 1 c(q 1 ) c(q ref ) q ref q best c(q best ) q n c(q n ) UCQ ref JUCQ ref q ref q best RDBMS I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 40 / 73

58 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation RDF Entailment Rules G answer query q Optimiser query q Reformulation algorithm Given the query q 1 (x, y) :- x rdf:type y, (t 1 ) x ub:degreefrom http : // (t 2 ) x ub:memberof http : // (t 3 ) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 41 / 73

59 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation RDF Entailment Rules G answer query q Optimiser query q Reformulation algorithm Given the query q 1 (x, y) :- x rdf:type y, (t 1 ) x ub:degreefrom http : // (t 2 ) x ub:memberof http : // (t 3 ) and a state-of-the-art CQ-to-UCQ reformulation algorithm. ref : the UCQ reformulation is: (t 1, t 2, t 3 ) ref the SCQ reformulation is: (t 1 ) ref (t 2 ) ref (t 3 ) ref I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 41 / 73

60 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation Given the query q 1 (x, y) :- x rdf:type y, (t 1 ) x ub:degreefrom http : // (t 2 ) x ub:memberof http : // (t 3 ) and a state-of-the-art CQ-to-UCQ reformulation algorithm. ref, the space of JUCQs is: 1. (t 1, t 2, t 3 ) ref 2. (t 1 ) ref (t 2 ) ref (t 3 ) ref 3. (t 1, t 2 ) ref (t 3 ) ref 4. (t 1 ) ref (t 2, t 3 ) ref 5. (t 1, t 3 ) ref (t 2 ) ref 6. (t 1, t 2 ) ref (t 1, t 3 ) ref 7. (t 1, t 2 ) ref (t 2, t 3 ) ref 8. (t 1, t 3 ) ref (t 2, t 3 ) ref I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 42 / 73

61 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation algorithms Exhaustive algorithm Impractical since search space size is the number of minimal covers of a set of n elements Greedy algorithm Driven by cost; starting from 1-atom fragments and growing them Given the query q 1 (x, y) :- x rdf:type y, (t 1 ) x ub:degreefrom http : // (t 2 ) x ub:memberof http : // (t 3 ) a cost-based greedy exploration is: (t 1 ) ref (t 2 ) ref (t 3 ) ref I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 43 / 73

62 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation: greedy algorithm example Given the query q 1 (x, y) :- x rdf:type y, (t 1 ) x ub:degreefrom http : // (t 2 ) x ub:memberof http : // (t 3 ) a cost-based greedy exploration is: (t 1 ) ref (t 2 ) ref (t 3 ) ref (t 1, t 2 ) ref (t 3 ) ref (t 1, t 3 ) ref (t 2 ) ref (t 1 ) ref (t 2, t 3 ) ref I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 44 / 73

63 Cover-based reformulation for the DB fragment of RDF CQ-to-JUCQ query reformulation algorithm Given the query q 1 (x, y) :- x rdf:type y, (t 1 ) x ub:degreefrom http : // (t 2 ) x ub:memberof http : // (t 3 ) a cost-based greedy exploration is: (t 1 ) ref (t 2 ) ref (t 3 ) ref (t 1, t 2 ) ref (t 3 ) ref (t 1, t 3 ) ref (t 2 ) ref (t 1 ) ref (t 2, t 3 ) ref (t 1, t 2, t 3 ) ref (t 1, t 3 ) ref (t 1, t 2 ) ref (t 1, t 3 ) ref (t 2, t 3 ) ref I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 45 / 73

64 Part V Cover-based reformulation in FOL-reducible settings I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 46 / 73

65 Optimised reformulation-based query answering For any setting where query answering is reducible to FOL query evaluation The same overall approach: Query q FOL ref. algo. Deductive constraints Answers q 1 c(q 1 ) c(q ref ) q ref q best c(q best ) q n c(q n ) Fixed FOL reform. Best FOL reformulation q ref q best RDBMS I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 47 / 73

66 DL-Lite R facts (a.k.a. ABox) Given: N C : set of concept names (unary predicates), N R : set of role names (binary predicates), N I : set of individuals (constants). I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 48 / 73

67 DL-Lite R facts (a.k.a. ABox) Given: N C : set of concept names (unary predicates), N R : set of role names (binary predicates), N I : set of individuals (constants). Finite number of: Concept assertions: A(a) with A N C and a N I Role assertions: R(a, b) with R N R and a, b N I I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 48 / 73

68 DL-Lite R constraints (a.k.a. TBox) Given: N C : set of concept names (unary predicates), N R : set of role names (binary predicates), I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 49 / 73

69 DL-Lite R constraints (a.k.a. TBox) Given: N C : set of concept names (unary predicates), N R : set of role names (binary predicates), Set of: Concept inclusions: B C Role inclusions: Q S using the following grammar (where A N C and R N R ): B := A Q, C := B B, Q := R R, S := Q Q I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 49 / 73

70 FOL Reformulation Consider the query q(x):- PhDStudent(x) workswith(y, x) and KB K = T, A with the TBox T and ABox A below: (T 1) PhDStudent Researcher (T 2) workswith Researcher (T 3) workswith Researcher (T 4) workswith workswith (T 5) supervisedby workswith (T 6) supervisedby PhDStudent (T 7) PhDStudent supervisedby (A1) (A2) (A3) workswith(ioana, Francois) supervisedby(damian, Ioana) supervisedby(damian, Francois) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 50 / 73

71 FOL Reformulation The UCQ reformulation of q is: q UCQ (x):- q 1 (x):- PhDStudent(x) workswith(y, x) q 2 (x):- PhDStudent(x) workswith(x, y) q 3 (x):- PhDStudent(x) supervisedby(y, x) q 4 (x):- PhDStudent(x) supervisedby(x, y) q 5 (x):- supervisedby(x, z) workswith(y, x) q 6 (x):- supervisedby(x, z) workswith(x, y) q 7 (x):- supervisedby(x, z) supervisedby(y, x) q 8 (x):- supervisedby(x, z) supervisedby(x, y) q 9 (x):- supervisedby(x, x) q 10 (x):- supervisedby(x, y) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 51 / 73

72 FOL Reformulation The UCQ reformulation of q is: q UCQ (x):- q 1 (x):- PhDStudent(x) workswith(y, x) q 2 (x):- PhDStudent(x) workswith(x, y) q 3 (x):- PhDStudent(x) supervisedby(y, x) q 4 (x):- PhDStudent(x) supervisedby(x, y) q 5 (x):- supervisedby(x, z) workswith(y, x) q 6 (x):- supervisedby(x, z) workswith(x, y) q 7 (x):- supervisedby(x, z) supervisedby(y, x) q 8 (x):- supervisedby(x, z) supervisedby(x, y) q 9 (x):- supervisedby(x, x) q 10 (x):- supervisedby(x, y) Naive exhaustive application of specialization steps leads, in general, to highly redundant reformulations. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 51 / 73

73 FOL Reformulation minimization Minimization of q UCQ by eliminating disjuncts contained in another leads to: q UCQ min (x):- q1 (x):- PhDStudent(x) workswith(y, x) q 2 (x):- PhDStudent(x) workswith(x, y) q 3 (x):- PhDStudent(x) supervisedby(y, x) q 9 (x):- supervisedby(x, y) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 52 / 73

74 Cover-based FOL reformulation Bursztyn, Goasdoué, Manolescu (DL 2015) Consider the query q(x):- A(x) R(x, y) R (z, y) and KB K = T, A with the TBox T and ABox A below: (T 1) B R (T 2) R R (A1) (A2) A(a) B(a) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 53 / 73

75 Cover-based FOL reformulation Bursztyn, Goasdoué, Manolescu (DL 2015) Consider the query q(x):- A(x) R(x, y) R (z, y) and KB K = T, A with the TBox T and ABox A below: (T 1) B R (T 2) R R (A1) (A2) A(a) B(a) The only answer to q(x) is a. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 53 / 73

76 Cover-based FOL reformulation q(x):- A(x) R(x, y) R (z, y) K = T = {B R, R R}, A = {A(a), B(a)} Consider the cover C 1 = {{A(x), R(x, y)}, {R (z, y)}}. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 54 / 73

77 Cover-based FOL reformulation q(x):- A(x) R(x, y) R (z, y) K = T = {B R, R R}, A = {A(a), B(a)} Consider the cover C 1 = {{A(x), R(x, y)}, {R (z, y)}}. The cover-based FOL reformulation of q w.r.t. C 1 is: q JUCQ (x):- q UCQ 1 (x, y):- (A(x) R(x, y)) (A(x) R (x, y)) q UCQ 2 (y):- R (z, y) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 54 / 73

78 Cover-based FOL reformulation q(x):- A(x) R(x, y) R (z, y) K = T = {B R, R R}, A = {A(a), B(a)} Consider the cover C 1 = {{A(x), R(x, y)}, {R (z, y)}}. The cover-based FOL reformulation of q w.r.t. C 1 is: q JUCQ (x):- q UCQ 1 (x, y):- (A(x) R(x, y)) (A(x) R (x, y)) q UCQ 2 (y):- R (z, y) Cover C 1 prevents unification q JUCQ (x) is not a FOL reformulation of q w.r.t. T. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 54 / 73

79 Safe cover q(x):- A(x) R(x, y) R (z, y) K = T = {B R, R R}, A = {A(a), B(a)} Consider the cover C 2 = {{A(x)}, {R(x, y), R (z, y)}} I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 55 / 73

80 Safe cover q(x):- A(x) R(x, y) R (z, y) K = T = {B R, R R}, A = {A(a), B(a)} Consider the cover C 2 = {{A(x)}, {R(x, y), R (z, y)}} The cover-based FOL reformulation of q w.r.t. C 1 is: q JUCQ (x):- q UCQ 1 (x):- (A(x) q UCQ 2 (x):- (R(x, y) R (z, y)) (R (x, y) R (z, y)) (R (x, y)) (B(x)) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 55 / 73

81 Safe cover q(x):- A(x) R(x, y) R (z, y) K = T = {B R, R R}, A = {A(a), B(a)} Consider the cover C 2 = {{A(x)}, {R(x, y), R (z, y)}} The cover-based FOL reformulation of q w.r.t. C 1 is: q JUCQ (x):- q UCQ 1 (x):- (A(x) q UCQ 2 (x):- (R(x, y) R (z, y)) (R (x, y) R (z, y)) (R (x, y)) (B(x)) Cover C 2 preserves unification q JUCQ (x) is a FOL reformulation of q w.r.t. T and C 2 a safe cover. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 55 / 73

82 Root cover We call root cover the minimal safe cover. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 56 / 73

83 Root cover We call root cover the minimal safe cover. C 2 = {{A(x)}, w.r.t. T. {R(x, y), R (z, y)}} is a root cover for q I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 56 / 73

84 Extended cover and extended fragment queries q(x):- A(x) R(x, y) R (z, y) K = T = {B R, R R}, A = {A(a), B(a)} Consider the cover C 3 = {f 1 f 1, f 0 = {A(x)} f 1 = {R(x, y), R (z, y)} f 2 = {A(x), R(x, y)} f 2 f 0 }, with: I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 57 / 73

85 Extended cover and extended fragment queries q(x):- A(x) R(x, y) R (z, y) K = T = {B R, R R}, A = {A(a), B(a)} Consider the cover C 3 = {f 1 f 1, f 0 = {A(x)} f 1 = {R(x, y), R (z, y)} f 2 = {A(x), R(x, y)} f 2 f 0 }, with: The extended cover-based FOL reformulation of q w.r.t. C 3 is: q e (x):- q UCQ f 1 f 1 (x):- ((R(x, y) R (z, y)) R (x, y) B(x)) q UCQ f 2 f 0 (x):- ((A(x) R(x, y)) (A(x) R (x, y)) (A(x) B(x))) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 57 / 73

86 Search space Covers E q (extended covers) L q (safe covers) C root I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 58 / 73

87 Part VI Performance I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 59 / 73

88 Performance Performance potential of optimized reformulation CQ-to-JUCQ query reformulation performance Given the query q 1 (x, y) :- x rdf:type y, (t 1 ) x ub:degreefrom http : // (t 2 ) x ub:memberof http : // (t 3 ) and the LUBM 100M benchmark: JUCQ #reformulations exec. time (ms) (t 1, t 2, t 3 ) ref 2, 256 6,387 (t 1 ) ref (t 2 ) ref (t 3 ) ref 195 1,074,026 (t 1, t 2 ) ref (t 3 ) ref 755 1, 968 (t 1 ) ref (t 2, t 3 ) ref , 710 (t 1, t 3 ) ref (t 2 ) ref (t 1, t 2 ) ref (t 1, t 3 ) ref 1, 316 2, 734 (t 1, t 2 ) ref (t 2, t 3 ) ref 764 2, 289 (t 1, t 3 ) ref (t 2, t 3 ) ref I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 60 / 73

89 Performance Optimized reformulation experiments on three RDBMSs Datasets and RDBMS engines DBLP (8 M) and LUBM (1 M and 100 M) millions triples PostgreSQL System A System B I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 61 / 73

90 Performance Optimized reformulation experiments on three RDBMSs Reformulation algorithms Basic CQ-to-UCQ algorithm Picked that of [Goasdoué et al., EDBT 13] since it handles the largest known fragment of RDF. Comparison: 1 UCQ reformulation 2 SCQ reformulation 3 Greedy JUCQ reformulation 4 Exhaustive JUCQ reformulation I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 62 / 73

91 Performance Optimized reformulation experiments on three RDBMSs Query answering on LUBM 100 M using PostgreSQL 28 queries; 2 to 6 atoms; 1 to 318, 089 reformulations I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 63 / 73

92 Performance Optimized reformulation experiments on three RDBMSs Query answering on LUBM 100 M using System A 28 queries; 2 to 6 atoms; 1 to 318, 089 reformulations I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 64 / 73

93 Performance Optimized reformulation experiments on three RDBMSs Query answering on LUBM 100 M using System B 28 queries; 2 to 6 atoms; 1 to 318, 089 reformulations I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 65 / 73

94 Performance Optimized reformulation experiments on three RDBMSs Optimized reformulation: take-home message 1 Equivalent SQL syntaxes are not equal from the RDBMS optimizer perspective (inside or outside well-supported dialect) 2 Chosing the reformulation with the help of textbook cost model formulas makes queries feasible or efficient when they were not 3 This amounts to enlarging the optimizer s can-do dialect, at a very modest performance overhead. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 66 / 73

95 Performance Optimized reformulation experiments on three RDBMSs Experiments on DL-Lite R reformulation 1 CQ-to-UCQ reformulation 2 C root reformulation 3 Our cover-based greedy reformulation 4 Our cover-based exhaustive reformulation (as a yardstick for the quality of the solution our greedy finds) I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 67 / 73

96 Performance Optimized reformulation experiments on three RDBMSs Query answering on LUBM 20 using PostgreSQL 15 millions facts dataset 13 queries; 2 to 10 atoms, 5.77 on average; 35 to 667 reformulations, on average. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 68 / 73

97 Performance Optimized reformulation experiments on three RDBMSs Algorithms running time on LUBM 20 dataset 15 millions facts 13 queries; 2 to 10 atoms, 5.77 on average; 35 to 667 reformulations, on average. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 69 / 73

98 Part VII Conclusion and perspectives I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 70 / 73

99 Open issues and perspectives Where we stand Constraints (a.k.a. semantics) are crucial for applications, so the push is continuous for chosing the right constraint language I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 71 / 73

100 Open issues and perspectives Where we stand Constraints (a.k.a. semantics) are crucial for applications, so the push is continuous for chosing the right constraint language We considered semantic query answering through FOL reduction (i.e., SQL). I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 71 / 73

101 Open issues and perspectives Where we stand Constraints (a.k.a. semantics) are crucial for applications, so the push is continuous for chosing the right constraint language We considered semantic query answering through FOL reduction (i.e., SQL). Not any SQL query resulting from reformulation is handled well by current RDBMSs! I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 71 / 73

102 Open issues and perspectives Where we stand Constraints (a.k.a. semantics) are crucial for applications, so the push is continuous for chosing the right constraint language We considered semantic query answering through FOL reduction (i.e., SQL). Not any SQL query resulting from reformulation is handled well by current RDBMSs! Vast performance differences between different reformulations of the same query; cost-based approach I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 71 / 73

103 Open issues and perspectives What is ahead Continuous push on the expressivity - efficiency frontier; updates... I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 72 / 73

104 Open issues and perspectives What is ahead Continuous push on the expressivity - efficiency frontier; updates... RDBMSs are highly efficient for some forms of queries (typically conjunctive queries of medium size), not for all FOL reductions of queries under constraints I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 72 / 73

105 Open issues and perspectives What is ahead Continuous push on the expressivity - efficiency frontier; updates... RDBMSs are highly efficient for some forms of queries (typically conjunctive queries of medium size), not for all FOL reductions of queries under constraints RDBMSs have convenient features (indexing, join order, transactions) that make them attractive back-ends for query answering. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 72 / 73

106 Open issues and perspectives What is ahead Continuous push on the expressivity - efficiency frontier; updates... RDBMSs are highly efficient for some forms of queries (typically conjunctive queries of medium size), not for all FOL reductions of queries under constraints RDBMSs have convenient features (indexing, join order, transactions) that make them attractive back-ends for query answering. Novel platforms (MapReduce, NoSQL...) will raise the same query evaluation performance problems anyway. I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 72 / 73

107 Open issues and perspectives What is ahead Continuous push on the expressivity - efficiency frontier; updates... RDBMSs are highly efficient for some forms of queries (typically conjunctive queries of medium size), not for all FOL reductions of queries under constraints RDBMSs have convenient features (indexing, join order, transactions) that make them attractive back-ends for query answering. Novel platforms (MapReduce, NoSQL...) will raise the same query evaluation performance problems anyway. Most attractive languages currently: DL-Lite, Datalog ± I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 72 / 73

108 Open issues and perspectives What is ahead Continuous push on the expressivity - efficiency frontier; updates... RDBMSs are highly efficient for some forms of queries (typically conjunctive queries of medium size), not for all FOL reductions of queries under constraints RDBMSs have convenient features (indexing, join order, transactions) that make them attractive back-ends for query answering. Novel platforms (MapReduce, NoSQL...) will raise the same query evaluation performance problems anyway. Most attractive languages currently: DL-Lite, Datalog ± New relevance of query optimization literature and research! I. Manolescu Semantic Query Optimization SEBD 2015, Gaeta 72 / 73