Databases and Information Systems 1. Prof. Dr. Stefan Böttcher

Size: px

Start display at page:

Download "Databases and Information Systems 1. Prof. Dr. Stefan Böttcher"

Lesley Glenn
6 years ago
Views:

1 9. XPath queries on XML data streams Prof. Dr. Stefan Böttcher goals of XML stream processing substitution of reverse-axes an automata-based approach to XPath query processing Processing XPath queries on XML streams Streams of XML data ( e.g. news news ticker GB/min ) further data streams ( traffic, sensor data, wheather, ) <a> <c> </c> </a> <a> <c> </c> </a> Users' queries describe an abonnement using an XPath query, e.g. /a is interested in: children 'a' of the root, e.g. <a> found /a </a> <c> skip </c> <a> found another /a </a> /a/c is interested in: children of a 'c' child of an 'a' child of root, e.g. <a> skip <c> found /a/c </c> </a> skip Problem: we can not go backwards on streams! /5 Prof. Dr. Stefan Böttcher

2 XPath - looking forward () - why? Problem: streams of XML data ( Reuters News GB/minute ) SAX events can not follow reverse axes / descendant :: E / ancestor :: E Solution idea (by Dan Olteanu et. al.): logical rewrite rules to eliminate reverse-axes from XPath expressions /5 XPath - looking forward () - idea use axis-symmetry if node type of c is element: a c / ancestor :: * c a / descendant :: * a c / ancestor-or-self :: * c a / descendant-or-self :: * p c / parent :: * c p / child :: * p c / preceding :: * c p / following :: * p c / preceding-sibling :: * c p / following-sibling :: * More general: p c / reverse-axis :: * c p / reverse-axis - :: * Basic idea: substitute left-hand side with right-hand-side and adjust the paths examples follow /5 Prof. Dr. Stefan Böttcher

3 XPath - looking forward () - parents / descendant :: E / parent :: E / descendant-or-self :: E [./child :: E = /descendant :: E ] E ( parent - = child ) E / descendant :: E / ancestor :: E / descendant-or-self :: E [. / descendant :: E = /descendant :: E ] ( ancestor - = descendant ) 5/5 XPath - looking forward () - parents / descendant :: E / parent :: E / descendant-or-self :: E [./child :: E = /descendant :: E ] E ( parent - = child ) E / descendant :: E / ancestor :: E / descendant-or-self :: E [. / descendant :: E = /descendant :: E ] ( ancestor - = descendant ) 6/5 Prof. Dr. Stefan Böttcher

4 XPath - looking forward () - general / descendant :: E / preceding :: E / descendant::e [./following::e = /descendant::e] ( preceding - = following ) E E / absolute-path :: E / reverse-axis :: E / descendant-or-self :: E [. / reverse-axis - :: E = /absolute-path :: E ] 7/5 XPath - looking forward () - general / descendant :: E / preceding :: E / descendant::e [./following::e = /descendant::e] ( preceding - = following ) E E / absolute-path :: E / reverse-axis :: E / descendant-or-self :: E [. / reverse-axis - :: E = /absolute-path :: E ] + replaces reverse-axis location-step - introduces a join in the filter 8/5 Prof. Dr. Stefan Böttcher

5 XPath - looking forward (5) - no join / descendant :: E / preceding :: E / descendant::e [./following::e = /descendant::e] ( each following :: E is a descendant from root ) E E / descendant :: E / preceding :: E / descendant::e [./following::e ] ( preceding - = following ) + replaces last reverse-axis location-step + avoids the join in the filter 9/5 XPath - looking forward (6) - no join / descendant :: E / ancestor :: E / descendant-or-self :: E [. / descendant :: E = /descendant :: E ] E E ( each descendant :: E is a descendant from root ) Therefore, this rule can be written without a join: / descendant :: E / ancestor :: E / descendant-or-self :: E [. / descendant :: E ] ( ancestor - = descendant ) rule above is applicable to absolute paths only! /5 Prof. Dr. Stefan Böttcher 5

6 XPath - looking forward (7) - no join / descendant :: E / ancestor :: E p E E E / descendant-or-self :: E [. / descendant :: E ] ( ancestor - = descendant ) rule above is applicable to absolute paths only! p / descendant :: E / ancestor :: E p [ descendant :: E ] / ancestor :: E p / descendant-or-self :: E [. / descendant :: E ] /5 XPath - looking forward (8) - filters / descendant :: E [ ancestor :: E ] / descendant :: E [ /descendant-or-self:: E /descendant:: node( ) = self::node( ) ] E ( ancestor - = descendant ) E path::e [ reverse-axis :: E / p ] path::e [ /descendant-or-self:: E [ p ] / reverse-axis - :: node( ) = self::node( ) ] + replaces first reverse-axis location-step in the filter - introduces a join in the filter /5 Prof. Dr. Stefan Böttcher 6

7 XPath - looking forward (9) - summary. Special rules: either remove a reverse-axis or reduce size of XPath expression with reverse-axis some special rules generate unions (" ") of XPath expressions. General rules: remove reverse-axes, but introduce joins XML stream processing possible /5 Summary XPath looking forward () XML data streams like news tickers produce up to GB of XML data per minute can not store document SAX model instead of DOM model can use only forward axes transform XPath expressions using the looking forward approach /5 Prof. Dr. Stefan Böttcher 7

8 Exercise (looking forward) rewrite the following XPath queries: / a / b /.. / c // a /.. / b into equivalent queries that use only forward axes ( transform XPath expressions using the looking forward approach ) 5/5 Conference Contribution on: Evaluating XPath Queries on XML Data Streams Stefan Böttcher, Rita Steinmetz University of Paderborn, Germany BNCOD 7, Glasgow Tuesday, nd July 7 Prof. Dr. Stefan Böttcher 8

9 Problem description Given: Infinite XML Data Stream (SAX) XPath Query Q Result: Return all XML fragments that match Q in document order SAX XPath Result <a></a><a><c></c> <c></c></a> Q <a> </a> <a> </a> <a> </a> <a> </a> 7/5 System components /a/c XPath query a a b c b SAX event stream c first-child :: * self :: a / /* /a next-sibling :: * a a automaton for an XPath query b c b binary SAX event stream c stack-based query evaluation with reservations for filters 8/5 Prof. Dr. Stefan Böttcher 9

10 From SAX events to binary SAX <root> <a> </a> <a> <c> </c> <c> </c> </a> </root> next-sibling::b next-sibling::c start() + start(a) From SAX events to binary SAX <root> <a> </a> <a> <c> </c> <c> </c> </a> </root> next-sibling::b next-sibling::c start() + start(b) Prof. Dr. Stefan Böttcher

11 From SAX events to binary SAX <root> <a> </a> <a> <c> </c> <c> </c> </a> </root> next-sibling::b next-sibling::c start() + end() - From SAX events to binary SAX <root> <a> </a> <a> <c> </c> <c> </c> </a> </root> next-sibling::b next-sibling::c end() + end() Prof. Dr. Stefan Böttcher

12 From SAX events to binary SAX <root> <a> </a> <a> <c> </c> <c> </c> </a> </root> next-sibling::b next-sibling::c end() + start(a) XPath query normalization Remove all backward axes, e.g. Q = / a / b / / c Q = /a[b]/c Remove following-axes following::x ancestor-or-self::* / following-sibling::*/ descendant-or-self::x Result: equivalent XPath query with only few forward-axes /5 Prof. Dr. Stefan Böttcher

13 Further reduction of XPath location steps. /descendant :: Nodetest child ::* / descendant-or-self :: Nodetest. following:: Nodetest. axis:: Nodetest [ Filters ] axis:: Nodetest / self::* [ Filters ] Result: an XPath expression consists only of location steps: descendant-or-self :: * descendant-or-self :: E child :: * child :: E self :: E self :: * [ AFilter ] following-sibling :: * following-sibling :: E 5/5 Automata for XPath location steps /child::a /descendant-or- /following-sibling::a / 6/5 Prof. Dr. Stefan Böttcher

14 Gluing atomic XPath automata together to an XPath automaton Query Q = /child::a/child::c automaton for /a/c is combined from automata for /a and for./c / first-child :: * self :: a /* /a first-child :: * self :: c /a/* /a/c next-sibling :: * next-sibling :: * 7/5 Evaluating filter-free path queries 8/5 Prof. Dr. Stefan Böttcher

15 Evaluating filter-free path queries 9/5 Evaluating filter-free path queries {} /5 Prof. Dr. Stefan Böttcher 5

16 Evaluating filter-free path queries {} /5 Evaluating filter-free path queries {} /5 Prof. Dr. Stefan Böttcher 6

17 Evaluating filter-free path queries {} {,} start output of result /5 Evaluating filter-free path queries {} {,} next-sibling::b {} stop output of result /5 Prof. Dr. Stefan Böttcher 7

18 Evaluating filter-free path queries {} {,} next-sibling::b {} next-sibling::c {,} start output of result 5/5 Evaluating filter-free path queries next-sibling::b next-sibling::c {} {,} {} {,} stop output of result 6/5 Prof. Dr. Stefan Böttcher 8

19 Evaluating filter-free path queries {} {,} next-sibling::b {} next-sibling::c {,} {} 7/5 Evaluating path queries with predicate filters Q = /a[b]/c [b] 5 6 self::b 7 8/5 Prof. Dr. Stefan Böttcher 9

20 Evaluating path queries with predicate filters Q = /a[b]/c [b] 5 6 self::b 7 {} {,-R,5} 9/5 Evaluating path queries with predicate filters Q = /a[b]/c [b] 5 6 self::b 7 {} {,-R,5} {} {,-R,5} {-R,6,7} /5 Prof. Dr. Stefan Böttcher

21 Evaluating path queries with predicate filters Q = /a[b]/c [b] 5 6 self::b 7 {} {,,5} {} {,,5} {,6,7} /5 Evaluating path queries with predicate filters Q = /a[b]/c [b] 5 6 self::b 7 {} {,,5} {} {,,5} {,6,7} {} {,,5} /5 Prof. Dr. Stefan Böttcher

22 Evaluating path queries with predicate filters Q = /a[b]/c [b] 5 6 self::b 7 {} {,,5} {} {,,5} {,6,7} {} {,,5} {} {,-R,5} /5 Evaluating path queries with predicate filters Q = /a[b]/c [b] 5 6 self::b 7 {} {,,5} {} {,,5} {,6,7} {} {,,5} {} {,-R,5} start store results of c {} {,-R,5} {-R,-R,6} /5 Prof. Dr. Stefan Böttcher

23 Evaluating path queries with predicate filters Q = /a[b]/c next-sibling::b [b] 5 6 self::b {} {,,5} {} {,,5} {,6,7} {} {,,5} {} {,-R,5} {} {,-R,5} {-R,-R,6} {} {,-R,5} {-R,6,7} stop store results of c 7 5/5 Evaluating path queries with predicate filters Q = /a[b]/c next-sibling::b [b] 5 {} {,,5} {} {,,5} {,6,7} {} {, {} {,,5},5} output of c result 6 self::b {} {,,5} {,,6} {} {,,5} {,6, 7} 7 filter is satisfied 6/5 Prof. Dr. Stefan Böttcher

24 time (ms) evaluation time (ms) Evaluating path queries with predicate filters Q = /a[b]/c next-sibling::b next-sibling::c [b] 5 {} {,,5} {} {,,5} {,6,7} {} {,,5} {} {,,5} {} {,,5} {,,6} {} {,,5} {,6,7} {} {,,5} {,,6} 6 self::b start output of c result 7 7/5 (a) Q5 Performance, evaluation 5,, 5,, 5,, 5, 5,, 5,, 5, document size (kb) Saxon XPA Yfilter Evaluation time for Q5 on different document sizes Evaluation time for different queries on smallst and largest document size (b) D Q Q Q Q Q5 query Saxon XPA Yfilter 8/5 Prof. Dr. Stefan Böttcher

25 Summary a a /a/c XPath query b c b SAX event stream c / first-child :: * self :: a /* /a a a next-sibling :: * automaton for an XPath query b c b binary SAX event stream c stack-based query evaluation with reservations for filters -R 9/5 Processing XPath queries on XML streams with automata (summary) Abonnement for XPath queries using forward-axes only implemented by automata states describe on which XPath path starting at the root the current context node occurs edges represent pairs SAX parser events ( i.e. pairs of opening and closing element tags detected ) egde labels (next-sibling or first child) describe elemetary location steps performed by pair of SAX parser events that change the state of the the automaton Not applicable to reverse-axes (parent, ancestor, preceding(-sibling)) 5/5 Prof. Dr. Stefan Böttcher 5

26 Conclusions XPath automaton supports all forward axes + XPath query rewriting all axes supported Less time consumption than other approaches Less memory consumption than other approaches Reaching data throughput rates higher than download rate of ADSL+ 5/5 Prof. Dr. Stefan Böttcher 6

Databases and Information Systems 1 - XPath queries on XML data streams -

Databases and Information Systems 1 - XPath queries on XML data streams - Dr. Rita Hartel Fakultät EIM, Institut für Informatik Universität Paderborn WS 211 / 212 XPath - looking forward (1) - why? News