Pathfinder: XQuery Compilation Techniques for Relational Database Targets

Transcription

1 EMPTY_FRAG SERIALIZE (item, pos) ROW# (pos:<pos1>) (pos1, item) X (iter = iter1) ELEM (iter1, item:<iter1, item><iter1, pos, item>) (iter1, item1, pos) ROW# (pos:<pos1>/iter1) (iter1, pos1, item1) access textnode content (item1:<item>) ROW# (pos1:<item>/iter1) / child::text (iter1, item) ATTR (item:<item2, (item2), val: person (item1), val: " " / child::element name { item* } (iter1, item) (item:item1, iter1:iter) (item), val: item (iter1:iter) NMBER (iter) ROW# (pos1:<item1>) (item1) / child::element person { item* } (iter, item1) (iter1, item, pos) ROW# (pos1), val: (pos1), val: #2 (iter1, item) (iter1, item) TEXT (item:<item2>) CAST (item2:<item1>), type: (item1), val: 0 DIFF (iter1) CONT (item1:/iter1) (iter1) (iter, iter1) X (item1 = item) (iter, item1) (iter1, item) access attribute value (item1:<item>) (item, iter:iter2) / attribute::attribute person { atomic* } (iter2, item) / child::element people { item* } (iter, item1) / child::element buyer { item* } (iter2, item) DOC TBL: (iter item1) [#1,"auctionG.xml"] (item:item1, iter2:iter) NMBER (iter) (item1) / child::element closed_auction { item* } (iter, item1) / child::element closed_auctions { item* } (iter, item1) / child::element site { item* } (iter, item1) access attribute value (item:<item1>) (item1, iter1:iter2) / attribute::attribute id { atomic* } (iter2, item1) (item1, iter2:iter) Pathfinder: XQuery Compilation Techniques for Relational Database Targets Jens Teubner Technische niversität München Joint work with: Torsten Grust, Peter Boncz, Martin Kersten, Maurice van Keulen, Stefan Manegold, Sjoerd Mullender, Jan Rittinger, Marc H. Scholl,...

2 Scalable XQuery Processing Challenge: Construction of a Scalable XQuery Processor XQuery: tree-structured XML data ordered sequences of items: (x 1,..., x n ) XQuery explicit iteration: for $v in e 1 return e 2 side effects: element t { e } Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

3 Scalable XQuery Processing Challenge: Construction of a Scalable XQuery Processor XQuery: tree-structured XML data ordered sequences of items: (x 1,..., x n ) XQuery explicit iteration: for $v in e 1 return e 2 side effects: element t { e } Re-use existing RDBMS technology? flat, unordered data model: tables of tuples bulk-oriented processing no side effects RDBMS Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

4 Scalable XQuery Processing Challenge: Construction of a Scalable XQuery Processor XQuery: tree-structured XML data ordered sequences of items: (x 1,..., x n ) XQuery explicit iteration: for $v in e 1 return e 2 side effects: element t { e } Re-use existing RDBMS technology? flat, unordered data model: tables of tuples bulk-oriented processing no side effects RDBMS Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

5 Scalable XQuery Processing This talk bridges the apparent gap. trees, sequences, iteration, side effects XQuery compositional compilation: loop-lifting XQuery Semantics relational step evaluation: staircase join tree encoding: XPath accelerator XPath Evaluation XML Storage tables of tuples, relational algebra, SQL RDBMS Pathfinder is a full open-source implementation of these techniques. Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

6 XPath Accelerator: Relational XML Storage XML Storage Pathfinder s XML Storage is based on XPath Accelerator (Grust 02) 0 a 9 1 b 3 5 f 8 2 c 2 6 g 4 7 h 7 3 d 0 4 e 1 8 i 5 9 j 6 n pre post a 0 9 b 1 3 c 2 2 d 3 0 e 4 1 f 5 8 g post a 9 f 8 h 7 6 j 5 g i 4 b 3 2 c d e pre Any encoding providing node identity/document order suffices. We actually use a variant of this encoding: pre/size/level. Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

7 XPath on Commodity RDBMSs XML Storage Relational XML storage can beat native XPath processors execution time [ms] [SIGMOD 2007] 767 /descendant::open_auction/bidder/increase 1 6 /site/regions/africa 453 /descendant::description [ parent::node() ] 13,534 Relational XML Storage IBM DB2 purexml R 14,595 se B-trees with low-selectivity prefixes (e.g., level, tag names)! Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14 IBM DB2 DB Intel Xeon 3.2 GHz 8 GB RAM

8 Staircase Join: XPath Step Evaluation XPath Evaluation XPath is the backbone of every XQuery processor. XPath: Context is a set of nodes Document order, duplicate-free result Problems: Repeated scans over the same area Expensive sorting and duplicate elimination post c 1 c 2 c 4 c 3 c5 pre Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

9 Staircase Join: XPath Step Evaluation XPath Evaluation XPath is the backbone of every XQuery processor. XPath: Context is a set of nodes Document order, duplicate-free result Problems: Repeated scans over the same area Expensive sorting and duplicate elimination Staircase join: [VLDB 2003] Encapsulates tree awareness in a single join operator Cache-friendly and XPath-compliant post c 4 c5 c 3 c 1 c 2 pre scan skip scan scan Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

10 Staircase Join: XPath Step Evaluation XPath Evaluation We injected staircase join into PostgreSQL 7.3. execution time [ms] orig. XPath accelerator staircase join execution times page misses orig. XPath accelerator staircase join ,118 XML document size [MB] page misses Query: /descendant::age/ancestor::person [VLDB 2004 Demo] Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

11 Loop-Lifting: XQuery Compilation XQuery Semantics XPath is only part the story. Variables and iteration: for $v in e 1 return e 2 Sequence construction: (e 1, e 2 ) Element construction: element { e 1 } { e 2 } Dynamic typing: e 1 instance of e 2 etc. XQuery is a functional language, though. Process independent FLWOR iterations in parallel se bulk-oriented processing capabilities of modern RDBMSs Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

12 Loop-Lifting: XQuery Compilation XQuery Semantics Loop-lifting: Encode independent iterations using a single relation. for $x in (1, 2, 3) return $x to 3 Column iter labels independent iterations. Sequence order is maintained in column pos. This is the loop-lifted encoding of an XQuery item sequence. The compilation procedure operates on loop-lifted sequence representations only. iter pos item Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

13 Loop-Lifting: XQuery Compilation XQuery Semantics Loop-lifting: Encode independent iterations using a single relation. for $x in (1, 2, 3) return $x to 3 Column iter labels independent iterations. Sequence order is maintained in column pos. This is the loop-lifted encoding of an XQuery item sequence. The compilation procedure operates on loop-lifted sequence representations only. iter pos item Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

14 Loop-Lifting: XQuery Compilation XQuery Semantics Loop-lifting: Encode independent iterations using a single relation. for $x in (1, 2, 3) return $x to 3 Column iter labels independent iterations. Sequence order is maintained in column pos. This is the loop-lifted encoding of an XQuery item sequence. The compilation procedure operates on loop-lifted sequence representations only. iter pos item type τ int τ int τ int τ int τ int τ int This representation is highly versatile. Item types to support dynamic type semantics Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

15 Loop-Lifting: XQuery Compilation XQuery Semantics Loop-lifting: Encode independent iterations using a single relation. for $x in (1, 2, 3) return $x to 3 Column iter labels independent iterations. Sequence order is maintained in column pos. This is the loop-lifted encoding of an XQuery item sequence. The compilation procedure operates on loop-lifted sequence representations only. iter pos item type score τ int τ int τ int τ int τ int τ int 1 This representation is highly versatile. Item types to support dynamic type semantics Additional information to support, e.g., XQuery full-text search Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

16 XQuery on DB2 XQuery Semantics XML Storage XPath Evaluation Commodity RDBMSs readily provide all the functionality we need. E.g., SQL on IBM DB2 niversal Database V 8.2. XMark 1 XMark 2 XMark 6 XMark 7 XMark MB 11 MB 110 MB 1.1 GB Intel Xeon 3.2 GHz 8 GB RAM execution time [sec] [VLDB 2004] Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

17 Pathfinder: A Loop-Lifting XQuery Compiler XQuery Semantics XML Storage XPath Evaluation Pathfinder is a full implementation of a loop-lifting compiler. Fully compositional, in line with the XQuery language Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

18 EMPTY_FRAG SERIALIZE (iter:outer, pos:pos1, item) ROW# (pos1:<sort, pos>/outer) ELEM (iter, item:<iter, item><iter, pos, item>) ATTR (res:<item, item1>) X (iter = iter1) (iter1:iter, (item), val: person (iter:outer, pos:pos1, item) ROW# (pos1:<sort, pos>/outer) (iter, pos, item:res) access textnode content (res:<item>) (item), val: " " ELEM_TAG (iter, pos:pos1, (item), val: item (outer:iter, sort:pos, inner) / child::text (iter, item) / child::element name { item* } (iter, item) ROW# (pos1:<ord, (ord), val: (ord), val: #2 (iter, pos, item:res) TEXT (res:<cast>) CAST (cast:<item>), type: (item), val: 0 DIFF CONT (item:/iter) (iter:outer) ROW# (pos1:<sort, pos>/outer) (iter, pos, item) X (iter = iter1) (iter1:iter) SEL (item) (iter, pos, item:res) NOT (item), val: false (iter, pos, item:cast) CAST (cast:<item>), type: ua (outer:iter, sort:pos, inner) (iter, pos, item:res) access attribute value (res:<item>) (iter, (item), val: true DIFF / attribute::attribute id { atomic* } (iter, item) / child::element person { item* } (iter, item) / child::element closed_auction { item* } (iter, item) / child::element people { item* } (iter, item) (item), val: "auctiong.xml" TBL: (iter) [#1] (iter:outer) / child::element closed_auctions { item* } (iter, item) / child::element site { item* } (iter, item) / child::element site { item* } (iter, item) ROW# (pos1:<sort, pos>/outer) (item), val: false (iter:outer) ROW# (pos1:<sort, pos>/outer) (outer:iter, sort:pos, (item), val: 1 SEL (item) (iter, pos, item:res) = (res:<item, item1>) X (iter = (item), val: 1 SEL (item) (iter, pos, item:res) NOT (item), val: true (iter, pos, item:cast) (iter1:iter, item1:cast) CAST (cast:<item>), type: str CAST (cast:<item>), type: str (iter:inner, pos, item) X (iter = outer) ROW# (pos1:<sort, pos>/outer) (iter:outer, pos:pos1, item) (iter:outer, pos:pos1, item) ROW# (pos1:<sort, pos>/outer) (iter, pos, item:cast) CAST (cast:<item>), type: ua (iter, pos, item:res) access attribute value (res:<item>) X (iter = outer) (iter:inner, pos, item) (outer:iter, sort:pos, inner) / attribute::attribute person { atomic* } (iter, item) (outer:iter, sort:pos, inner) X (iter = outer) (outer:iter, sort:pos, inner) (outer:iter, sort:pos, inner) / child::element buyer { item* } (iter, item) X (iter = outer) Pathfinder: A Loop-Lifting XQuery Compiler XQuery Semantics XML Storage XPath Evaluation Pathfinder is a full implementation of a loop-lifting compiler. Fully compositional, in line with the XQuery language The resulting plans can be of significant size, though. XMark Q8 34 MB Intel Xeon 3.2 GHz 8 GB RAM sec Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

19 EMPTY_FRAG ATTR (res:<item, item1>) X (iter = iter1) SERIALIZE (iter:outer, pos:pos1, item) ROW# (pos1:<sort>) ELEM (iter, item:<iter, item><iter, pos, item>) (iter1:iter, (item), val: person (iter:outer, pos:pos1, item) ROW# (pos1:<sort>/outer) (iter, pos, item:res) access textnode content (res:<item>) (item), val: " " ELEM_TAG (iter, pos:pos1, (item), val: item (outer:iter, sort:pos, inner) / child::text (iter, item) / child::element name { item* } (iter, item) ROW# (ord), val: (ord), val: #2 (iter, pos, item:res) TEXT (res:<cast>) CAST (cast:<item>), type: (item), val: 0 DIFF CONT (item:/iter) (iter:outer) ROW# (pos1:<sort, pos>/outer) (iter, pos, item) X (iter = iter1) (iter1:iter) SEL (item) (iter, pos, item:res) NOT (item), val: (item), val: true DIFF (iter, pos, item:cast) CAST (cast:<item>), type: ua (outer:iter, sort:pos, inner) (iter, pos, item:res) access attribute value (res:<item>) (iter, item:res) / attribute::attribute id { atomic* } (iter, item) ROW# (iter), val: #1 (item) ROW# (pos:<item>) / child::element person { item* } (iter, item) ROW# (iter), val: #1 (item) (iter:outer) ROW# (pos1:<sort, pos>/outer) ROW# (pos:<item>) / child::element people { item* } (iter, item) (item), val: "auctiong.xml" TBL: (iter) [#1] / child::element closed_auction { item* } (iter, (item), val: false DIFF / child::element closed_auctions { item* } (iter, item) ROW# (iter), val: #1 / child::element site { item* } (iter, item) (item) ROW# (pos:<item>) / child::element site { item* } (iter, (item), val: 1 SEL (item) (iter, pos, item:res) NOT (item), val: true (iter:outer) ROW# (pos1:<sort, (item), val: 1 SEL (item) (iter, pos, item:res) = (res:<item, item1>) X (iter = iter1) (iter, pos, item:cast) (iter1:iter, item1:cast) CAST (cast:<item>), type: str CAST (cast:<item>), type: str (iter:inner, pos, item) X (iter = outer) ROW# (pos1:<sort>/outer) (outer:iter, sort:pos, inner) (iter:outer, pos:pos1, item) (iter:outer, pos:pos1, item) ROW# (pos1:<sort>/outer) (iter, pos, item:cast) CAST (cast:<item>), type: ua (iter, pos, item:res) access attribute value (res:<item>) X (iter = outer) (iter:inner, pos, item) (outer:iter, sort:pos, inner) / attribute::attribute person { atomic* } (iter, item) (outer:iter, sort:pos, inner) (outer:iter, sort:pos, inner) (outer:iter, sort:pos, inner) / child::element buyer { item* } (iter, item) X (iter = outer) Pathfinder: A Loop-Lifting XQuery Compiler XQuery Semantics XML Storage XPath Evaluation Pathfinder is a full implementation of a loop-lifting compiler. Fully compositional, in line with the XQuery language The resulting plans can be of significant size, though. To optimize relational plans, Pathfinder thus implements 1 constant propagation, XMark Q8 34 MB Intel Xeon 3.2 GHz 8 GB RAM sec 112 sec Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

20 EMPTY_FRAG SERIALIZE (pos:pos1, item) ROW# (pos1:<sort>) ELEM (iter, item:<iter, item><iter, pos, (item), val: person (iter:outer, pos:pos1, item) ROW# (pos1:<sort>/outer) (iter, pos, item:res) access textnode content (res:<item>) ATTR (res:<item, item1>) X (iter = iter1) ELEM_TAG (iter, pos:pos1, (item), val: item (iter1:iter, item1:item) fn:string_join (outer:iter, sort:pos, inner) / child::text (iter, item) / child::element name { item* } (iter, (item), val: " " (iter:inner) ROW# (ord), val: (ord), val: #2 (iter, pos, item:res) (iter, item:res) TEXT (res:<cast>) CAST (cast:<item>), type: (item), val: 0 DIFF CONT (item:/iter) (iter:outer) X (iter = iter1) (iter1:iter) SEL (item) (iter, item:res) NOT (item), val: false (iter:inner) ROW# (pos:<item>) / child::element person { item* } (iter, item) / child::element people { item* } (iter, (item), val: true DIFF (iter, item:cast) CAST (cast:<item>), type: ua (iter, item:res) / child::element closed_auction { item* } (iter, item) / child::element closed_auctions { item* } (iter, item) / child::element site { item* } (iter, item) (item), val: "auctiong.xml" TBL: (iter) [#1] (item), val: false DIFF (outer:iter, sort:pos, inner) / child::element site { item* } (iter, item) SEL (item) (iter, item:res) NOT (item), val: true (iter:outer) SEL (item) (iter, item:res) = (res:<item, item1>) X (iter = iter1) (iter, item:cast) CAST (cast:<item>), type: str X (iter = outer) (iter:outer, item) (outer:iter, inner) access attribute value (res:<item>) / attribute::attribute person { atomic* } (iter, item) / child::element buyer { item* } (iter, item) (iter:inner) (iter1:iter, item1:cast) CAST (cast:<item>), type: str (iter, item:cast) CAST (cast:<item>), type: ua (iter, item:res) access attribute value (res:<item>) (iter:outer, item) (outer:iter, inner) (outer:iter, inner) / attribute::attribute id { atomic* } (iter, item) X (iter = outer) X (iter = outer) (outer:iter, inner) (outer:iter, inner) Pathfinder: A Loop-Lifting XQuery Compiler XQuery Semantics XML Storage XPath Evaluation Pathfinder is a full implementation of a loop-lifting compiler. Fully compositional, in line with the XQuery language The resulting plans can be of significant size, though. To optimize relational plans, Pathfinder thus implements 1 constant propagation, 2 projection pushdown, XMark Q8 34 MB Intel Xeon 3.2 GHz 8 GB RAM 2 51 sec sec 112 sec Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

21 EMPTY_FRAG SERIALIZE (item, pos) ROW# (pos:<pos1>) (pos1, item) X (iter = iter1) ELEM (iter1, item:<iter1, item><iter1, pos, item>) (iter1, item1, pos) ROW# (pos:<pos1>/iter1) (iter1, pos1, item1) access textnode content (item1:<item>) ROW# (pos1:<item>/iter1) / child::text (iter1, item) ATTR (item:<item2, (item2), val: person fn:string_join / child::element name { item* } (iter1, item) (item:item1, (item1), val: " " (item), val: item NMBER (iter) ROW# (pos1:<item1>) (item1) (iter1:iter) / child::element person { item* } (iter, item1) (iter1, item, pos) ROW# (pos1), val: (pos1), val: #2 (iter1, item) (iter1, item) TEXT (item:<item2>) CAST (item2:<item1>), type: (item1), val: 0 DIFF (iter1) CONT (item1:/iter1) (iter1) X (iter = iter2) (iter:iter2) (iter2) SEL (item) (iter2, item) = (item:<item3, item4>) (iter2, item3, item4) CAST (item4:<item>), type: str CAST (item3:<item2>), type: str (iter2, item2, item) CAST (item:<item4>), type: ua access attribute value (item4:<item3>) (item3, iter2, item2) X (iter1 = iter3) (iter1:iter3, item3) / attribute::attribute id { atomic* } (iter3, item3) / child::element people { item* } (iter, item1) (item3:item, iter3:iter1) (iter2, item2:item1, iter3:iter1) NMBER (iter1) (item, iter2, item1) CAST (item1:<item3>), type: ua access attribute value (item3:<item2>) / child::element site { item* } (iter, (iter), val: #1 DOC TBL: (iter item) [#1,"auctionG.xml"] (item1:item) (item2, item, iter2) X (iter = iter3) (iter:iter3, item2) (item, item1, iter1, iter2:iter) / attribute::attribute person { atomic* } (iter3, item2) (item:item1, iter2:iter, iter3:iter) / child::element buyer { item* } (iter3, item2) (item2:item, iter3:iter) (item) NMBER (iter) (item1, iter1:iter) / child::element closed_auction { item* } (iter, item) / child::element closed_auctions { item* } (iter, item) / child::element site { item* } (iter, item) Pathfinder: A Loop-Lifting XQuery Compiler XQuery Semantics XML Storage XPath Evaluation Pathfinder is a full implementation of a loop-lifting compiler. Fully compositional, in line with the XQuery language The resulting plans can be of significant size, though. To optimize relational plans, Pathfinder thus implements 1 constant propagation, 2 projection pushdown, 3 functional dependency and data flow analyses, and XMark Q8 34 MB Intel Xeon 3.2 GHz 8 GB RAM 3 33 sec 2 51 sec sec 112 sec Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

22 EMPTY_FRAG SERIALIZE (item, pos) ROW# (pos:<pos1>) (pos1, item) X (iter = iter1) ELEM (iter1, item:<iter1, item><iter1, pos, item>) (iter1, item1, pos) ROW# (pos:<pos1>/iter1) (iter1, pos1, item1) access textnode content (item1:<item>) ROW# (pos1:<item>/iter1) / child::text (iter1, item) ATTR (item:<item2, (item2), val: person (item1), val: " " / child::element name { item* } (iter1, item) (item:item1, iter1:iter) (item), val: item (iter1:iter) NMBER (iter) ROW# (pos1:<item1>) (item1) / child::element person { item* } (iter, item1) (iter1, item, pos) ROW# (pos1), val: (pos1), val: #2 (iter1, item) (iter1, item) TEXT (item:<item2>) CAST (item2:<item1>), type: (item1), val: 0 DIFF (iter1) CONT (item1:/iter1) (iter1) (iter, iter1) X (item1 = item) (iter, item1) (iter1, item) access attribute value (item1:<item>) (item, iter:iter2) / attribute::attribute person { atomic* } (iter2, item) / child::element buyer { item* } (iter2, item) / child::element people { item* } (iter, item1) DOC TBL: (iter item1) [#1,"auctionG.xml"] (item:item1, iter2:iter) NMBER (iter) (item1) / child::element closed_auction { item* } (iter, item1) / child::element closed_auctions { item* } (iter, item1) / child::element site { item* } (iter, item1) access attribute value (item:<item1>) (item1, iter1:iter2) / attribute::attribute id { atomic* } (iter2, item1) (item1, iter2:iter) Pathfinder: A Loop-Lifting XQuery Compiler XQuery Semantics XML Storage XPath Evaluation Pathfinder is a full implementation of a loop-lifting compiler. Fully compositional, in line with the XQuery language The resulting plans can be of significant size, though. To optimize relational plans, Pathfinder thus implements 1 constant propagation, 2 projection pushdown, 3 functional dependency and data flow analyses, and 4 algebraic join detection. (You saw these optimizations in yesterday s demo session.) XMark Q8 34 MB Intel Xeon 3.2 GHz 8 GB RAM sec 3 33 sec 2 51 sec sec 112 sec Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

23 MonetDB/XQuery Pathfinder targets the main-memory RDMBS MonetDB. Queries over multi-gigabyte XML instances answered in interactive time (XMark: 18 of 20 queries in 1 min on 1.1 GB) nprecedented scalability normalized execution time MB 111 MB 1.1 GB 11 GB Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 [SIGMOD 2006, VLDB 2005 Demo] Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

24 Wrap-p A complete and purely relational XQuery processing stack: XML Storage A relational tree encoding, derived from XPath accelerator, maps XML document trees into relational tables. Re-use of mature storage and indexing techniques Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

25 Wrap-p A complete and purely relational XQuery processing stack: XML Storage A relational tree encoding, derived from XPath accelerator, maps XML document trees into relational tables. Re-use of mature storage and indexing techniques XPath Evaluation Staircase join encapsulates knowledge about our tree encoding in terms of a single join operator. Outstanding XPath performance on any RDBMS Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

26 Wrap-p A complete and purely relational XQuery processing stack: XML Storage A relational tree encoding, derived from XPath accelerator, maps XML document trees into relational tables. Re-use of mature storage and indexing techniques XPath Evaluation Staircase join encapsulates knowledge about our tree encoding in terms of a single join operator. Outstanding XPath performance on any RDBMS XQuery Semantics The loop-lifting compilation procedure maps arbitrary XQuery expressions to primitives of relational algebra. Implementation of iterative XQuery semantics in terms of efficient, bulk-oriented processing Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

27 Ongoing and Future Work Pathfinder is an ongoing, joint research project with CWI Amsterdam, Twente, and Konstanz. Algebraic optimization, cost and result size estimation New functionality: recursion, dynamic typing, and validation Alternative back-ends: Idefix (KN), SQL:1999 [SIGMOD 2007] Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

28 Ongoing and Future Work Pathfinder is an ongoing, joint research project with CWI Amsterdam, Twente, and Konstanz. Algebraic optimization, cost and result size estimation New functionality: recursion, dynamic typing, and validation Alternative back-ends: Idefix (KN), SQL:1999 [SIGMOD 2007] MonetDB/XQuery has started to spread across the world already. X-RPC: XQuery processing in peer-to-peer networks XIRAF: multi-hierarchical XML documents Tijah: full-text retrieval for the MonetDB/XQuery system 150 SourceForge downloads per month (MonetDB/XQuery only) Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14

29 Ongoing and Future Work Pathfinder is an ongoing, joint research project with CWI Amsterdam, Twente, and Konstanz. Algebraic optimization, cost and result size estimation New functionality: recursion, dynamic typing, and validation Alternative back-ends: Idefix (KN), SQL:1999 [SIGMOD 2007] MonetDB/XQuery has started to spread across the world already. X-RPC: XQuery processing in peer-to-peer networks XIRAF: multi-hierarchical XML documents Tijah: full-text retrieval for the MonetDB/XQuery system 150 SourceForge downloads per month (MonetDB/XQuery only) pathfinder ("pa:t n. a person who makes or finds a way, esp. through unexplored areas or fields of knowledge. Collins English Dictionary Pathfinder: XQuery Compilation Techniques for Relational Database Targets BTW / 14