ADT 2010 ADT XQuery Updates in MonetDB/XQuery & Other Approaches to XQuery Processing

Transcription

1 1 XQuery Updates in MonetDB/XQuery & Other Approaches to XQuery Processing Stefan Manegold MonetDB/XQuery: Updates

2 Schedule : RDBMS back-end support for XML/XQuery (1/): Document Representation (XPath Accelerator, Pre/Post plane) : XPath navigation (Staircase Join) XQuery to Relational Algebra Compiler: Item- & Sequence- Representation Efficient FLWoR Evaluation (Loop-Lifting) Optimization.11.1: RDBMS back-end support for XML/XQuery (/): Updateable Document Representation Other (DB-) approaches to XML/XQuery processing MonetDB/XQuery: Updates

3 XML/XQuery Updates XQuery Update Facility 1. WC Candidate Recommendation Categorize updates into Value updates Structural updates (MonetDB/XQuery does not yet support the latest syntax changes made by WC; for details see MonetDB/XQuery: Updates

4 4 Value Updates do replace value of fn:doc("bib.xml")/books/book[1]/price with fn:doc("bib.xml")/books/book[1]/price * 1.1 do replace value of fn:doc( bib.xml )/books/book[]/@isbn with do rename fn:doc( bib.xml )/books/book[]/author[1] into primary-author do rename fn:doc( bib.xml )/journals/journal[9]/@isbn into issn => map directly to simple value updates in relational storage MonetDB/XQuery: Updates

5 5 Structural Updates do insert attribute isbn { } into fn:doc("bib.xml")/books/book[17] do delete fn:doc( bib.xml )/books/book[]/@wrong do insert <author>stefan Manegold</author> after fn:doc( bib.xml )/books/book[]/author[last()] do replace fn:doc( bib.xml )/books/book[44]/author[1] with fn:doc( bib.xml )/books/book[]/author[last()] do delete fn:doc( bib.xml )/books/book[author = Kermit ] => How to implement on pre-/post-encoding? MonetDB/XQuery: Updates

6 6 XML/XQuery Updates do insert <k><l/><m/></k> as first into /a/f/g MonetDB/XQuery: Updates

7 7 XML/ XQuery Updates do insert <k><l/><m/></k> as first into /a/f/g MonetDB/XQuery: Updates

8 XML/XQuery Updates MonetDB/XQuery: Updates 8

9 XML/XQuery Updates MonetDB/XQuery: Updates 9

10 1 XML/ XQuery Updates Staircase Join MonetDB/XQuery: Updates

11 11 XML Storage Revisited pre post a b c d e f g h i j pre size level post = pre + size - level pre size level null null null Allow holes MonetDB/XQuery: Updates rid size level nid null null 1 N N1 null null N N N4 N5 N6 N7 N8 N9 Define logical pages

12 1 XML Storage Revisited pre post a b c d e f g h i j pre size level post = pre + size - level pre size level null null null Allow holes ( MonetDB/XQuery: Updates null null 1 N N1 null null N6 N7 N8 N9 N N N4 N5 Define logical pages page map rid = pre.swizzle rid size level nid 1 )

13 1 XML Storage Revisited Update-friendly rid-table is append-only rid-tuples may be unused rid = autoincrement column MonetDB: rid not stored but computed (virtual oid) allows positional lookup/join Opportunity currently not exploited by other RDBMS rid size level nid null null 1 N N1 null null N6 N7 N8 N9 N N N4 N5 Occurs widely in our XQuery translation. MonetDB/XQuery: Updates

14 14 XML Storage Revisited Update-friendly rid-table is append-only rid-tuples may be unused rid = autoincrement column MonetDB: rid not stored but computed (virtual oid) allows positional lookup/join Opportunity currently not exploited by other RDBMS rid size level nid null null 1 N N1 null null N6 N7 N8 N9 N N N4 N5 Occurs widely in our XQuery translation. MonetDB/XQuery: Updates

15 15 MonetDB/XQuery Our own XML DBMS with (almost..) full XQuery support. Built purely on an RDBMS, namely MonetDB XQuery Pathfinder compiler & staircase join : Universität Tübingen (Torsten Grust, et al.) Pathfinder Compiler Technical University Twente (Maurice van Keulen, et. al.) MonetDB High-Performance DBMS CWI Amsterdam (Peter Boncz, Stefan Manegold,...) RDBMS Useful for: (MonetDB) Large XML databases! Querying XML annotations (multimedia, forensic NFI) XML information retrieval... Relational Algebra MonetDB/XQuery: Updates

16 16 Research Projects & Extensions Value indeces Runtime optimization SIGMOD'9 [Abdel Kader, Boncz, v. Keulen Manegold] Algebraic Query Optimization Grust, Rittinger, et al. (Universität Tübingen) Distributed XQuery PP XQuery SOAP group communication, XQuery RPC VLDB'7 [Zhang, Boncz] Benchmarking beyond XMark ExpDB'6 Workshop [Manegold] Support for XML Interval Annotations XIME-P'6 Workshop [Alink et al.] Xquery + Information Retrieval: PF/Tijah MonetDB/XQuery: Updates

17 17 Conclusions Relational approach can be scalable & fast MonetDB/XQuery compares favorably with all other available systems Techniques that made it work Property-driven peephole optimization Order & other properties Loop-lifted XPath steps Evaluate Sets of context nodes in a single pass Support for dense (autoincrement) keys Positional lookup Background Information & Literature MonetDB/XQuery: Updates

18 Schedule : RDBMS back-end support for XML/XQuery (1/): Document Representation (XPath Accelerator, Pre/Post plane) : XPath navigation (Staircase Join) XQuery to Relational Algebra Compiler: Item- & Sequence- Representation Efficient FLWoR Evaluation (Loop-Lifting) Optimization.11.1: RDBMS back-end support for XML/XQuery (/): Updateable Document Representation Other (DB-) approaches to XML/XQuery processing

19 19 Topics Other approaches & techniques (selection, far from complete!) Document storage / tree encoding: ORDPATH DataGuides XPath processing: Tree patterns, holistic twig joins

20 Fixed-Width Tree Encodings & Updates Fixed-width tree encoding (like XPath Accelerator) are Good for read(-only) processing small footprint, positional lookup, staircase join But inherently static Milo et al., PODS : There is a sequence of updates (subtree insertions) for any persistent tree encoding scheme E (where each node keeps its initial encoding label even under updates), such that E needs labels of length (N) to encode the resulting tree of N nodes.

21 1 Fixed-Width Tree Encodings & Updates Fixed-width tree encoding (like XPath Accelerator) are Good for read(-only) processing small footprint, positional lookup, staircase join But inherently static Milo et al., PODS : There is a sequence of updates (subtree insertions) for any persistent tree encoding scheme E (where each node keeps its initial encoding label even under updates), such that E needs labels of length (N) to encode the resulting tree of N nodes.

22 XML/XQuery Updates do insert <k><l/><m/></k> as first into /a/f/g

23 XML/ XQuery Updates MonetDB/XQuery hack: exploit paging & mmap trick but: updating pg off do insert <k><l/><m/></k> as first into /a/f/g is still O(N)

24 4 Fixed-Width Tree Encodings & Updates Fixed-width tree encoding (like XPath Accelerator) are Good for read(-only) processing small footprint, positional lookup, staircase join But inherently static Non-solutions: Gaps in the encoding (never large enough) Encoding based on decimal fractions (limited precision) Possible solution: Variable-width tree encodings: Cheaper updates At the expense of more expensive read(-only) processing

25 5 A Variable-Width Tree Encoding: ORDPATH O'Neil et al., SIGMOD 4.

26 ORDPATH Encoding: Example 6

27 ORDPATH: Insertion Between Siblings 7

28 ORDPATH: Insertion Between Siblings 8

29 ORDPATH: Insertion Between Siblings 9

30 Is ORDPATH suitable for XQuery? Mapping core operations of the XQuery processing model to operations on ORDPATH labels:

31 1 ORDPATH: Variable-Length Node Encoding For a 1 MB XML sample document, the authors of ORDPATH observed label lengths between 6 and 1 bytes. ORDPATH labels encode root-to-node paths => common prefixes. => Label comparisons often need to inspect encoding bits at the far right. MS SQL Server employs further path encodings organized in reverse (node-to-root) order. Note: - Preorder ranks fit into CPU registers. - 4 byte pre's sufficient for = 4G nodes (11 GB XMark fits easily). - 8 byte pre's sufficient for 64 nodes, i.e., the universe...

32 Topics Other approaches & techniques (selection, far from complete!) Document storage / tree encoding: ORDPATH DataGuides XPath processing: Tree patterns, holistic twig joins

33 DataGuides XPath Accelerator, ORDPATH & similar encoding schemes encode the document's tree structure in the node ranks/labels they assign DataGuides Developed in the context of Lore project (DBMS for semi- structured data) Stanford encode University, Goldman & Widom, VLDB 1997 the document's tree structure in relation names Observation: Each node is uniquely identified by its path from the root Paths of siblings with equal tag names can be unified, Provided we keep their relative order (rank) explicitly

34 DataGuides 4 Definition given a semistructured data instance DB, a DataGuide for DB is a graph G s.t.: - every path in DB also occurs in G - every path in G occurs in DB - every path in G is unique

35 DataGuides 5 Example:

36 DataGuides 6 Multiple DataGuides for the same data:

37 7 DataGuides Definition Let p, p be two path expressions and G a graph; we define p G p if p(g) = p (G) i.e., p and p' are indistinguishable on G. Definition G is a strong dataguide for a database DB if G is the same as DB Example: - G1 is a strong dataguide - G is not strong person.project! DB dept.project person.project! G1 dept.project person.project G dept.project

38 DataGuides 8 Constructing the strong DataGuide G: Nodes(G)={{root}} Edges(G)= while changes do choose s in Nodes(G), a in Labels add s ={y x in s, (x -a->y) in Edges(DB)} to Nodes(G) add (x -a->y) to Edges(G) Use hash table for Nodes(G) This is precisely the powerset automaton construction.

39 Monet XML approach Early By 9 attempt to store and query XML data in MonetDB Albrecht Schmidt Not related to Pathfinder & MonetDB/XQuery

40 Monet XML approach 4

41 Monet XML approach 41

42 Monet XML approach Monet XML approach 4

43 Monet XML approach Early By attempt to store and query XML data in MonetDB Albrecht Schmidt Not No 4 related to Pathfinder & MonetDB/XQuery XQuery compiler XMark Child, queries are hand-crafted and -optimized in MIL Descendant, Parent & Ancestor steps become regular expressions on the relation names (i.e., catalog) Open: preceeding & following steps?

44 44 Topics Other approaches & techniques (selection, far from complete!) Document storage / tree encoding: ORDPATH DataGuides XPath processing: Tree patterns, holistic twig joins

45 45 Twig Join Algorithms So far: interpreted XPath expressions in an imperative manner Evaluated XPath expressions step-by-step, as stated in the query Given / 1:: 1/ :: /.../ n:: n, we first evaluated /, then XPath step 1:: 1, then step ::,... This may not always be the best choice: Intermediate results can get very large, even if the final result is small: Database context => think in a declarative manner DBMS optimizer / engine can evaluate query in best order

46 Tree Patterns In fact, XPath is a declarative language. /descendant::timeline/child::event 46 Find all nodes v1, v, and v, such that v1 is a document root, v is a descendant element of v1 and is named timeline, and v is a child element of v and named event. All nodes of type v form the query result. Observe the combination of (a) predicates on single nodes, and (b) structural conditions between these nodes.

47 Tree Patterns 47 Structural conditions: Intuitively expressed as tree patterns: Nodes labeled with node predicates Structural conditions: Double line: ancestor/descendant relationships Single line: parent/child relationships Arbitrary predicates are allowed, but typical are predicate on tag names: Nodes labeled with requested tag name Document timeline root: label / If not /-node specified: search for pattern anywhere in the document event

48 48 Tree Patterns timeline event

49 49 Tree Patterns Not May limited to path patterns also be twig patterns Mapping between tree patterns and XPath is in general not trivial Examples: f a b d c e g h i j

50 PathStack Algorithm 5

51 51 PathStack Algorithm: Path Patterns a b c d e d e

52 PathStack Algorithm: Path Patterns 5

53 PathStack Algorithm: Path Patterns 5

54 PathStack Algorithm: Path Patterns 54

55 PathStack Algorithm: Path Patterns timeline 55 timeline event first timeline node visited timeline timeline event first event node visited second timeline node visited

56 PathStack Algorithm: Path Patterns 56

57 PathStack Algorithm: Path Patterns 57

58 PathStack Algorithm: Path Patterns 58

59 PathStack Algorithm: Path Patterns 59

60 PathStack Algorithm: Path Patterns 6

61 PathStack Algorithm: Path Patterns 61

62 PathStack Algorithm: Path Patterns 6

63 PathStack Algorithm: Path Patterns 6

64 PathStack Algorithm: Path Patterns 64

65 65 PathStack Algorithm: Twig Patterns So far we only considered path patterns Can we extend our ideas for efficient twig pattern evaluation? Idea: Decompose All twig patterns into multiple path patterns. path patterns start from the same root. Use PathStack for each of them and merge their results.

66 PathStack Algorithm: Twig Patterns Example: 66 Decompose twig pattern into path patterns a b c d e

67 PathStack Algorithm: Twig Patterns Example: 67 Decompose twig pattern into path patterns c a a a b b b c d d e e

68 PathStack Algorithm: Twig Patterns 68

69 PathStack Algorithm: Twig Patterns 69

70 7 Summary (1/5) XML Document markup Data exchange Semi-structured Tree model DTDs XML Schema XPath Navigation, location steps, axes, node tests, predicates, functions XQuery Sequences & Iterations (FLWoR expressions)

71 71 Summary (/5) XML Data Management XML file processors XML databases XML integration platforms RDBMS with XML functionality, SQL/XML Relational XML storage: schema-based vs. schema-oblivious

72 7 Summary (/5) Purely Relational XML/XQuery processing: MonetDB/XQuery Document encoding: XPath Accelerator (pre/post plane) XPath navigation: Staircase Join XQuery to Relational Algebra translation Item- & Sequence-representation Iterations: Loop-lifting Loop-lifted staircase join Peephole Optimization Order-awareness, sort avoidance XML/XQuery Update Support

73 7 Summary (4/5) Other approaches & techniques Document storage/encoding: ORDPATH DataGuides XPath processing: Tree patterns, holistic twig joins

74 74 Summary (5/5) Literature Slides Literature references in slides Literature references on website: Tentamen / Exam: Tuesday December 1 1 9: 11: Zaal / Room: A1.14

75 75 Projects: Join the MonetDB Team! Own ideas, suggestions, initiative welcome! Master Student Projects (6 Months) Various projects, each consisting of both research & implementation See monetdb.cwi.nl/development/research/projects/ for a sample list Feel free to come with your own idea(s)! Implementation Projects Both short-term & long-term E.g. open feature requests: sf.net/tracker/?group_id=56967 Become owner/maintainer of some (new) part of MonetDB We are (desperately) looking for Windows SW-development & system experts!

76 76 We Offer... 4x7x65 support & advice Membership in a kind & friendly Family-Team of Experts Chance to participate in & contribute to a large & successful open-source research project Lots of experiences, exiting research & fun Desk & workstation at CWI Fridge, micro-wave, free coffee, free soup, free cake (occasionally) Master Students only (possibly part-time) Limited availability => FCFS! Some pocket money (stage vergoeding) Master Students only Limited availability => FCFS!...