Outline. Approximation: Theory and Algorithms. Ordered Labeled Trees in a Relational Database (II/II) Nikolaus Augsten. Unit 5 March 30, 2009

Transcription

1 Outline Approximation: Theory and Algorithms Ordered Labeled Trees in a Relational Database (II/II) Nikolaus Augsten Experimental Comparison of the Encodings Free University of Bozen-Bolzano Faculty of Computer Science DIS Unit 5 March 30, Importing XML into a Database 5 Conclusion Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 About the Dewey Paths Dewey Decimal Classification: used in libraries to classify books by topics developed by Melvil Dewey in [TVB + 02]: list of nodes each nodes stores path to root Example: tree relational implementation nid dp concatenates a Dewey path dp with an integer i (sibling position) e.g., = Sort order: 1.2 < 1.3, 1.1 < 1.1.2, 1.9 < 1. 1 also Edge Enumeration [Cel04] Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

2 Extending the Dewey encoding implicitly orders trees! Node labeled trees: (v, dp, λ(v)) v N(T) is a node ID dp is the Dewey path to v λ(v) is the label of v Edge labeled trees: (v, dp, λ) v N(T) is a node ID dp is the Dewey path to v λ is the label of the edge from the parent of v to v Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Edit Operations with the Tree relation T (nid, dp, lbl) Rename node: ren(v, l ) update single tuple (v, dp, l) (v, dp, l ) no structure updates Delete node: del(v) remove single tuple (v, dp v, l) update nodes with dp > dp v (descendants of v and descendants v s right-hand siblings) Insert node: ins(v, p, k, m) update nodes with dp dp(p) k (children of p at position k or larger, and all their descendants) insert single tuple (v, dp(p) k, λ(v)) Efficiency: O(n) in the worst case (insert/delete leftmost child of root node) better for nodes with (i) few descendants and (ii) few right siblings O(1) for lonely leaf child of a node Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Example: Delete Node in Preorder d e b 1.1 c 1.2 f g h i a 1 k n l m Delete node v with id(v) = 4 delete single tuple update descendants of v update right siblings of v and their descendants o 1.3 p nid dp lbl 0 1 a b c d e f g h i k l m n o p Tree relation T (nid, dp, lbl) Implementation: sort by attribute dp result is preorder traversal Efficiency: single query with sort on string attribute efficient (especially with index on dp) Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

3 Implementation: Storing the Dewey Path [DTCÖ03, ABG05] Goals: minimize space overhead for Dewey path dp sorting Dewey path should result in preorder traversal Separator character: e.g., 1.2.5, 1.17 overhead: small (separator char) sorting: natural sort order not consistent with preorder (1.2.5 > 1.17) Fixed length: e.g., , overhead: large (small and large numbers require same space) sorting: sort order ok Variable length encoding (UTF-8): UTF-8: 1 byte: 0... (2 7 1), 2 bytes: (2 11 1), etc. overhead: small space overhead sorting: sort order ok (supported by many databases, e.g. PostgreSQL) Idea: Parent contains children, like interval contains other intervals Example: c c a b c d a b c d : assign numbers to interval start and end points store interval start and end point with each node Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, 2009 / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Example Definition () An interval encoding of a tree is a relation T that for each node v of the tree contains a tuple (λ(v), lft, rgt); λ(v) is the label of v, lft and rgt are the endpoints of the interval representing the node. lft and rgt are constrained as follows: lft < rgt for all (lbl, lft, rgt) T, lft a < lft d and rgt a > rgt d if node a is an ancestor of d, and (λ(a), lft a, rgt a ) T, and (λ(d), lft d, rgt d ) T, rgt v < lft w if node v is a left sibling of node w, and (λ(v), lft v, rgt v ) T, and (λ(w), lft w, rgt w ) T, Example algorithm for a valid interval encoding: traverse tree in preorder use an incremental counter assign left interval value lft when node is first visited assign right interval value rgt when node is last visited 1a 2c 3 4 b 9 5c 6 7 d 8 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

4 Edit Operations with the Example: Delete Node in Tree relation T (id, lbl, lft, rgt) Rename node: ren(v, l ) update single tuple (id(v), l, L, R) (id(v), l, L, R) no structure updates Delete node: del(v) remove single tuple (id(v), l, L, R) remaining tree is valid and correct Insert node: ins(v, p, k, m) find left and right interval values L and R if values not free, update ancestors and nodes following in preorder insert single tuple (id(v), λ(v), L, R) Efficiency: rename and delete are very efficient (constant time)! insert may be O(n) in worst case (inefficient) sparse numbering reduces number of updates for insert 4d 5 6 e 15 1b 2 3 c 24 7f 8 9g 14 h i 13 0a 29 16k 21 17l m 20 22n 23 25o 28 26p 27 Delete node v with id(v) = 4 delete single tuple remaining tree is valid and correct id lbl lft rgt 0 a b c d e f g h 11 8 i k l m n o p Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Example: Insert Node in Preorder 4d 5 1b 2 3 c 22 e 6f 7 8g 13 9h 11 i 12 0a 27 14k 19 15l m 18 20n 21 23o 26 24p 25 Insert new node with label e: ins((4, e), 2, 2, 3) update the ancestors of the new node update the nodes following the new node in preorder insert single tuple nid lbl lft rgt 0 a b c d f g h i k l m n o p e 6 15 Tree relation T (id, lbl, lft, rgt) Implementation: sort by attribute lft result is preorder traversal Efficiency: single query with sort on integer attribute very efficient (especially with index on lft) Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

5 Improving Insert/Delete Performance: Sparse Numbering with sparse numbering: leave numbers free for future insert avoids global reordering until gaps are filled node deletions re-open gaps Example: 0a 90 c b 80 40c d 70 Note: Floating-point values do not solve the problem! Sparse numbering using (order, size)-pairs [LM01]: store node position as (order, size)-pair order corresponds to left interval value order + size corresponds to right interval value Example: 0a 90 Node Insertion: How To Deal with Full Gaps? Inserting a node: a) find the correct gap(s) in the tree b) if the/each gap is large enough: insert new node c) otherwise:...? Solution 1: shift left/right values until new node fits cheapest way for inserting a single node but: only a small number of gaps are opened Solution 2: reset all gaps more expensive than shifting but: happens less frequently because all gaps in the tree are opened Shifting and resetting gaps are hard updates c 30 b 50 40c 60 d Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Shifting Gaps is Cheaper than Resetting All Gaps Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 How Often Do We Need a Hard Update Sparse Sparse Time (ms) % of hard updates Tree size (number of nodes) Tree size (number of nodes) Runtime of shifting and resetting: : resets all gaps Sparse : shifts gaps Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Average number of hard updates when a new node is inserted (gap size 0): : resets all gaps Sparse : shifts gaps Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

6 Impact of the Gap Size Experimental Comparison of the Encodings Delete Performance 0 Sparse 00 0 Dewey Adjacency Descendants Fanout % of hard updates 1 Time (ms) 1 0 Nodes Gap size Impact of the gap size on the number of hard updates: : resets all gaps Sparse : shifts gaps Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Experimental Comparison of the Encodings Insert Performance Time (ms) 00 0 Dewey Adjacency Descendants Fanout Tree size (number of nodes) Insert performance of Adjacency List encoding Dewey encoding Interval encoding with sparse numbering () Descendants: maximum number of descendants of a tree node Fanout: maximum fanout in the tree Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / Nodes Tree size (number of nodes) Delete performance of Adjacency List encoding Dewey encoding Interval encoding with sparse numbering () Descendants: maximum number of descendants of a tree node Fanout: maximum fanout in the tree Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Experimental Comparison of the Encodings Efficiency of the Preorder Traversal Time (s) Adjacency List Dewey Tree size (number of nodes) Efficiency of the preorder traversal for Adjacency List encoding Dewey encoding Interval encoding with sparse numbering () Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

7 Experimental Comparison of the Encodings Comparing the Encodings Representing XML as a Tree Adjacency Dewey Interval + update very efficient simple implementation preorder efficient update efficiency: between others preorder very efficient simple implementation Many possibilities we will consider single-label tree double-label tree Pros/cons depend on application! preorder very inefficient update worst case is O(n) space overhead for storing paths insert is O(n) on average (patch: sparse numbering) Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 XML as a Single-Label Tree Example: XML as a Single-Label Tree The XML document is stored as a tree with: XML element: node labeled with element tag name XML attribute: node labeled with attribute name Text contained in elements/attributes: node labeled with the text-value Element nodes contain: nodes of their sub-elements nodes of their attributes nodes with their text values Attribute nodes contain: single node with their text value Text nodes are always leaves Order: sub-element and text nodes are ordered attributes are not ordered (approach: store them before all sub-elements, sort according to attribute name) <article title= pq-grams > <author>augsten</author> <author>boehlen</author> <author>gamper</author> </article> title pq-grams author Augsten article author Boehlen author Gamper Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

8 XML as a Double-Label Tree Example: XML as a Double-Label Tree Node labels are pairs The XML document is stored as a tree with: XML element: node labeled with (tag-name,text-value) XML attribute: node labeled with (attribute-name,text-value) Element nodes contain: nodes of their sub-elements and attributes Attribute nodes are always leaves Element nodes without attributes or sub-elements are leaves Order: sub-element nodes are ordered attributes are not ordered (approach: see previous slide) Limitation: Can represent either elements with sub-elements and/or attributes or elements with a text value <article title= pq-grams > <author>augsten</author> <author>boehlen</author> <author>gamper</author> </article> (article, ε) (title, pq-grams) (author, Augsten) (author, Boehlen) (author, Gamper) Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Example: Single- vs. Double-Label Tree Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 DOM Document Object Model Importing XML into a Database <xhtml> <p>this is <b>bold</b> font.</p> <xhtml> Single-Label Tree This is xhtml b font Double-Label Tree (xhtml, ε) (p,?) W3C 2 standard for accessing and manipulating XML documents Tree-based: represents an XML document as a tree (single-label tree with additional node info, e.g. node type) Elements, attributes, and text values are nodes DOM parsers load XML into main memory random access by traversing tree :-) large XML documents do not fit into main memory :-( bold (b, bold) 2 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

9 SAX Simple API for XML Importing XML into a Database Summary Conclusion de facto standard for parsing XML 3 Event-based: reports parsing events (e.g., start and end of elements) no random access :-( you see only one element/attribute at a time you can parse (arbitrarily) large XML documents :-) Java API available for both, DOM and SAX For importing XML into a database: use SAX! Storing a tree in a relational database Adjacency: store parent Dewey: store path to root Interval: node is an interval, store start and endpoint XML as an ordered, labeled tree single- and double-label model DOM and SAX for parsing XML 3 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 What s Next? Conclusion Edit distance for trees Definition Edit scripts and mappings Dynamic programming algorithm Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten, Michael Böhlen, and Johann Gamper. Approximate matching of hierarchical data using pq-grams. In Proceedings of the International Conference on Very Large Databases (VLDB), pages , Trondheim, Norway, September ACM Press. Joe Celko. Trees and Hierarchies in SQL for Smarties. Morgan Kaufmann Publishers Inc., Eigminas Dagys. Storing XML using interval encoding with sparse numbering. Master thesis, Free University of Bozen-Bolzano, March David DeHaan, David Toman, Mariano P. Consens, and M. Tamer Özsu. A comprehensive XQuery to SQL translation using dynamic interval encoding. Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40 Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40

10 In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages , San Diego, California, June ACM Press. Quanzhong Li and Bongki Moon. Indexing and querying XML data for regular path expressions. In Proceedings of the International Conference on Very Large Databases (VLDB), pages , Roma, Italy, September Morgan Kaufmann Publishers Inc. Igor Tatarinov, Stratis Viglas, Kevin S. Beyer, Jayavel Shanmugasundaram, Eugene J. Shekita, and Chun Zhang. Storing and querying ordered XML using a relational database system. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages , Madison, Wisconsin, June ACM Press. Nikolaus Augsten (DIS) Approximation: Theory and Algorithms Unit 5 March 30, / 40