XML and Databases. Outline. Outline - Lectures. Outline - Assignments. from Lecture 3 : XPath. Sebastian Maneth NICTA and UNSW

Transcription

1 Outline XML and Databases Lecture 10 XPath Evaluation using RDBMS 1. Recall / encoding 2. XPath with and text() 3. XPath with / and -sibling: use / size / level encoding Sebastian Maneth NICTA and UNSW 4. Optimization through tree-aware join: Staircase Join Prune Partition and Skip CSE@UNSW -- Semester 1, Outline - Lectures Outline - Assignments 1. Introduction to XML, Encodings, Parsers 2. Memory Resentations for XML: Space vs Access Speed 3. RDBMS Resentation of XML 4. DTDs, Schemas, Regular Exssions, Ambiguity 5. XML Validation using Automata 6. Node Selecting Queries: XPath 7. Tree Automata for Efficient XPath Evaluation, Parallel Evaluation 8..XPath Properties: backward axes, containment test 9. Streaming Evaluation: how much memory do you need? 10. XPath Evaluation using RDBMS XPath 1. Read XML, using DOM parser. Create document statistics. 2. SAX Parse into memory structure: Tree and DAG 3. Map XML into 4. XPath evaluation 5. XPath into SQL Translation 1. June 11. XSLT stylesheets and transform 12. XQuery XML query language 3 4 from Lecture 3 : from Lecture 3 : Questions What is the benefit of storing the LEVEL of a node in the table? Which accessor functions become easier? Later in this course, we will use the PRE/POST encoding again. We will find a systematic way to map queries on XML (XPath) into XQL queries. It can also be useful to store the size of the subtree at a node in the table. Do you see advantages of doing that? Assignment 5 is about programming this mapping

2 Assignment 3 XML document generate tables, send to DB, run query & print results <book isbn=" " year="1994"> <title>tcp/ip Illustrated</title> <author><last>stevens</last><first>john</first></author> <publisher>addison-wesley</publisher> <price currency="usd">65.95</price> </book> Assignment 3 XML document generate tables, send to DB, run query & print results <book isbn=" " year="1994"> <title>tcp/ip Illustrated</title> <author><last>stevens</last><first>john</first></author> <publisher>addison-wesley</publisher> <price currency="usd">65.95</price> </book> book_tbl: 1 12 book null 2 2 title null 3 1 null TCP/IP Illustrated 4 7 author null 5 4 last null 6 3 null Stevens 7 6 first null 8 5 null John 9 9 publisher null 10 8 null Addison-Wesley price null null book_attr: 1 isbn year currency USD 7 book_tbl: 1 12 book null 2 2 title null 3 1 null TCP/IP Illustrated 4 7 author null 5 4 last null 6 3 null Stevens 7 6 first null 8 5 null John 9 9 publisher null 10 8 null Addison-Wesley price null null Bonus: book_attr: 1 isbn year currency USD Assignment 5 add extra root node (=0) Generate SQL queries from Xpath (1) //, node tests: * and a, etc (2) add / and (3) add filters 8 1. Recall: Pre/Post Encoding Add order b 3 a 10 c b lab a 2 1 b 3 7 a 4 2 c 5 5 d 6 3 c 7 4 c 8 6 b 9 8 b 10 9 c CREATE VIEW AS SELECT r1.,r2. FROM doc_tbl r1, R r2 WHERE r1.<r2. AND r1.>r2. structural join 9 10 Our Tables: XML document Assignment 3 + extra root node <book isbn=" " year="1994"> <title>tcp/ip Illustrated</title> <author><last>stevens</last><first>john</first></author> <publisher>addison-wesley</publisher> <price currency="usd">65.95</price> </book> book_tbl: null null book null title null null TCP/IP Illustrated author null last null null Stevens first null null John publisher null null Addison-Wesley price null null book_attr: 1 isbn year currency USD Add root node with = 0 = #nodes+1 tag = null text = null

3 Our Tables: Assignment 3 + extra root node XML document <book isbn=" " year="1994"> <title>tcp/ip Illustrated</title> <author><last>stevens</last><first>john</first></author> <publisher>addison-wesley</publisher> <price currency="usd">65.95</price> </book> book_tbl: null null book null title null null TCP/IP Illustrated author null last null null Stevens first null null John publisher null null Addison-Wesley price null null book_attr: 1 isbn year currency USD Add root node with = 0 = #nodes+1 tag = null text = null More flexible, of course, to use TYPE-field with values { elem, text, root, com, PI.. } 13 ( pr,po ) = WHERE r1.>pr AND r1.<po ORDERED BY r1. ( pr,po ) = WHERE r1.<pr AND r1.>po ORDERED BY r1. 14 ( pr,po ) = WHERE r1.>pr AND r1.<po ORDERED BY r1. XPath evaluation: need to operate wrt a set of context nodes. Slide by T. Grust and M. van Keulen stored in intermediate table #location steps 2. XPath with and text() //book//author[./text()= Knuth ] location step 1: root node 2. XPath with and text() //book//author[./text()= Knuth ] location step 2: :: book SELECT r3. FROM book_tbl r1, r2, r3 WHERE r1.=0 AND root node =0 Watch out: SELECT r3. FROM book_tbl r1, r2, r3 WHERE r1.=0 AND r2.>r1. AND r2.<r1. AND =2 =1 This must be extra root node XPath data model Formally: do you remember the correct definition of the abbreviation //? // is abbreviation for /-or-self::node()/ //book is abbreviation for /-or-self::node()/child::book 17 :: book 18 3

4 XPath with and text() //book//author[./text()= Knuth ] location step 3: :: author XPath with and text() //book//author[./text()= Knuth ] plus filter SELECT r3. FROM book_tbl r1, r2, r3 WHERE r1.=0 AND r2.>r1. AND r2.<r1. AND r3.>r2. AND r3.<r2. AND r3.tag= author AND SELECT r3. FROM book_tbl r1, r2, r3, r4 WHERE r1.=0 AND r2.>r1. AND r2.<r1. AND r3.>r2. AND r3.<r2. AND r3.tag= author AND r4.=r3.+1 AND r4.<r3. AND r4.text= Knuth 19 Cave Not fully correct!! Works only if author-nodes have a single text node, as first child 20 XPath with and text() //book//author[./text()= Knuth ] XPath with and text() //book//author[@name= Knuth ] plus filter plus filter SELECT r3. FROM book_tbl r1, r2, r3, r4 WHERE r1.=0 AND r2.>r1. AND r2.<r1. AND r3.>r2. AND r3.<r2. AND r3.tag= author AND r4.>r3. AND r4.<r3. AND r4.level=r3.level+1 AND r4.text= Knuth ORDERED BY r3. SELECT r3. FROM book_tbl r1, r2, r3, book_attr a WHERE r1.=0 AND r2.>r1. AND r2.<r1. AND r3.>r2. AND r3.<r2. AND r3.tag= author AND r3.=a. AND a.attr= name AND a.value= Knuth ORDERED BY r3. Correct: returns result nodes in document order. Correct: returns result nodes in document order. What about duplicates? XPath with and text() //book//author[@name= Knuth ] XPath with and text() //book//author[@name= Knuth ] DISTINCT plus filter DISTINCT plus filter SELECT r3. FROM book_tbl r1, r2, r3, book_attr a WHERE r1.=0 AND r2.>r1. AND r2.<r1. AND r3.>r2. AND r3.<r2. AND r3.tag= author AND r3.=a. AND a.attr= name AND a.value= Knuth ORDERED BY r3. What about duplicates? E.g. //author/::* always remove duplicates! 23 SELECT r3. FROM book_tbl r1, r2, r3, book_attr a WHERE r1.=0 AND r2.>r1. AND r2.<r1. AND r3.>r2. AND r3.<r2. AND r3.tag= author AND r3.=a. AND a.attr= name AND a.value= Knuth ORDERED BY r3. Easy to add and axes. What about other axes? E.g., child, parent etc?? 24 4

5 3. XPath with / and -sibling 3. XPath with / and -sibling firstchild( pr, po ) =? firstchild( pr, po ) = left-most node, below and to the right of (pr,po) XPath with / and -sibling XPath with / and -sibling firstchild( pr, po ) = left-most node, below and to the right of (pr,po) firstchild( pr, po ) = left-most node, below and to the right of (pr,po) Not good! But easy: Needs SELECT min() FROM.... AND r2.=r1.+1 AND r2.<r XPath with / and -sibling XPath with / and -sibling How to select ALL children of a node? How to select ALL children of a node? But easy.. AND r2.=r1.+1 AND r2.<r1. maybe, i.e., larger and smaller, AND no other inbetween

6 XPath with / and -sibling XPath with / and -sibling b-node inbetween How to select ALL children of a node? b-node inbetween How to select ALL children of a node? maybe, i.e., larger and smaller, AND no other inbetween can be done in SQL, but is expensive! AND NOT EXIST (SELECT 31 Using LEVEL this is easy: Select all s at LEVEL firstchild( pr, po ) = left-most node, below and to the right of (pr,po) firstchild( pr, po ) = left-most node, below and to the right of (pr,po) nextsibling( pr, po ) = left-most node ( pr2, po2 ), to the right up such that there is no node with value > po and < po2. -sibling Can be done similar as before, using a complicated SQL query. Expensive e.g., not c- and d-node (because b-node is inbetween..) firstchild( pr, po ) = left-most node, below and to the right of (pr,po) -sibling Can be doen similar as before, using a complicated SQL query. Expensive Use other encoding!! PRE / SIZE / LEVEL 35 Question Can you compute the -value of a node, from its (, size, level ) values? size level lab

7 Useful relationships & approximations Useful relationships & approximations THUS (n) + size(n) = (n) + level(n) Useful relationships & approximations Useful relationships & approximations THUS (n) + size(n) = (n) + level(n) THUS (n) + size(n) = (n) + level(n) number x of nodes (n) = x + level(n) number x of nodes (n) = x + level(n) n (n) = x + size(n) n - (n) = x + size(n) (n) - (n) = level(n) - size(n) Useful relationships & approximations Useful relationships & approximations THUS (n) + size(n) = (n) + level(n) size level lab b a c d c c b b c

8 ( pr, si, le ) = ORDERED BY omitted in all queries size level lab 1 9 ( pr, si, le ) = ORDERED BY omitted in all queries size level lab 1 9 WHERE r1. > pr AND r1. <= pr+si WHERE r1. > pr AND r1. <= pr+si ( pr, si, le ) = 43 WHERE r1. < pr AND r1.+r1.size >= pr 44 ( pr, si, le ) = WHERE r1. > pr AND r1. <= pr+si ( pr, si, le ) = WHERE r1. < pr AND r1.+r1.size >= pr size level lab 1 9 child( pr, si, le ) = WHERE 45 ( pr, si, le ) = WHERE r1. > pr AND r1. <= pr+si ( pr, si, le ) = WHERE r1. < pr AND r1.+r1.size >= pr d size level lab 1 9 child( pr, si, le ) = WHERE d AND r1.level = le+1 46 ( pr, si, le ) = WHERE r1. > pr AND r1. <= pr+si ( pr, si, le ) = WHERE r1. < pr AND r1.+r1.size >= pr d a size level lab 1 9 child( pr, si, le ) = WHERE d AND r1.level = le+1 parent( pr, si, le ) = WHERE a AND r1.level = le-1 47 ( pr, si, le ) = WHERE r1.+r1.size < pr size level lab

9 ( pr, si, le ) = WHERE r1.+r1.size < pr size level lab 1 9 ( pr, si, le ) = WHERE r1.+r1.size < pr size level lab 1 9 -sibling( pr, si, le ) = WHERE?? ( pr, si, le ) = WHERE r1. > pr+si 49 ( pr, si, le ) = WHERE r1. > pr+si 50 Sometimes even store Parent with Pre / Size / Level Most of the slides are by Maurice van Keulen (Univ. of Twente, The Netherlands) See Level not needed, if we have parent Context/::*/::* SQL Query SELECT DISTINCT r2. FROM context c, doc_tbl r1, r2 WHERE r1. > c. AND r1. > c. AND r2. > r1. AND r2. < r1. ORDERED BY r2.pr IBM DB2 Query Plan sort unique ixscan / context c ixscan / doc_tbl r2 doc_tbl r

10 Context/::*/::* Avoiding Duplicates SQL Query IBM DB2 Query Plan SELECT DISTINCT r2. FROM context c, doc_tbl r1, r2 WHERE r1. > c. AND r1. > c. AND r2. > r1. AND r2. < r1. ORDERED BY r2.pr sort unique 4 scans Problem 1 might have MANY duplicates here! context c ixscan / doc_tbl r2 ixscan / 3 scans 1 scan doc_tbl r Avoiding Duplicates Pruning Select lowest independent nodes (not of each other) 4 scans 4 scans 3 scans 1 scan 3 scans 1 scan Pruning Pruning How to Prune depends on axis How to Prune depends on axis C ax set of context nodes XPath axis C ax set of context nodes XPath axis (1) If ax=, then Prune(C,ax) = lowest (=bottom-most) independent nodes in C (2) If ax=, then (1) If ax=, then Prune(C,ax) = lowest (=bottom-most) independent nodes in C (2) If ax=, then Prune(C,ax) = highest (=top-most) independent nodes in C

11 Pruning Pruning How to Prune depends on axis C set of context nodes ax XPath axis (1) If ax=, then Prune(C,ax) = lowest independent nodes in C (2) If ax=, then Prune(C,ax) = highest independent nodes in C Hint For any two context nodes Nnd N2: -nodes(n1) -nodes(n2) OR -nodes(n2) -nodes(n1) Why is that? Consider arbitrary N1, N2. Either one is a of the other or Qu: give pruning rule for (3) ax =? Qu: give pruning rule for (3) ax =? Technique 2 Partitioning Technique 2 Partitioning 2 scans Problem Still 2-scan area that generates duplicates 1 scan

12 Technique 2 Partitioning & Skipping More.. Skip empty regions in the plane CAVE: only true if there are no value comparisons in filters More.. Skip empty regions in the plane More.. Skip empty regions in the plane b cannot be a of a node of a! x a b 69 b cannot be a of a node of a! x a b 70 More.. Skip empty regions in the plane a s nodes are also nodes of b. More.. Skip empty regions in the plane a s nodes are also nodes of b. a and b do not have common Descendants. b cannot be a of a node of a! x a b 71 b cannot be a of a node of a! x a b 72 12

13 73 74 END Lecture