XML and Databases. Lecture 3 XML into RDBMS. Sebastian Maneth NICTA and UNSW

Transcription

1 XML and Databases Leture 3 XML into RDBMS Sebastian Maneth NICTA and UNSW CSE@UNSW -- Semester 1, 2010

2 2 Leture 3 XML Into RDBMS

3 Memory Representations 3 Fats DOM is easy to use, but memory heavy. in-memory size usually 5-10 times larger than size of original file. SAX is very flexible. Using arrays or binary trees wo bakward pointers, in-memory size is approx. same as size of original file. Can be further improved using DAGs/sharing-Graphs & oding/ompression for data values.

4 Memory Representations 4 Fats DOM is easy to use, but memory heavy. in-memory size usually 5-10 times larger than size of original file. SAX is very flexible. Using arrays or binary trees wo bakward pointers, in-memory size is approx. same as size of original file. Can be further improved using DAGs/sharing-Graphs & oding/ompression for data values. TODAY How an we map XML into a relational DB?

5 Memory Representations 5 Fats DOM is easy to use, but memory heavy. in-memory size usually 5-10 times larger than size of original file. SAX is very flexible. Using arrays or binary trees wo bakward pointers, in-memory size is approx. same as size of original file. Can be further improved using DAGs/sharing-Graphs & oding/ompression for data values. TODAY How an we map XML into a relational DB? but first: Memory effiient tree traversals using e.g. DOM?

6 Traversals and Pre/Post-Enoding 6 1. Pre-Order Traversal (reursively) 2. Post-Order 3. Pre-Order (iteratively) 4. Into RDBMS with Pre/Post-enoding TODAY How an we map XML into a relational DB? but first: Memory effiient tree traversals using e.g. DOM?

7 Tree Traversals Start at root node; want to visit every node. 7 root firstchild a lastchild [ ] hildnodes b a b d b

8 Tree Traversals Start at root node; want to visit every node. 8 (1) reursively root firstchild a lastchild [ ] hildnodes b a b d b

9 Tree Traversals Start at root node; want to visit every node. 9 (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) root firstchild a lastchild [ ] hildnodes b a b d b

10 Tree Traversals Start at root node; want to visit every node. Tr(1) pr(1) Tr(2) pr(2) 10 (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) root firstchild 1 a lastchild 2 [ ] hildnodes b a b d b

11 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) Tr(1) pr(1) Tr(2) pr(2) Tr(3) pr(3) Tr(4) pr(4) 11 root 1 a firstchild lastchild 2 3 b a b [ ] hildnodes 4 d b

12 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) root Tr(1) pr(1) Tr(2) pr(2) Tr(3) pr(3) Tr(4) pr(4) Tr(5) pr(5) Tr(6) pr(6) 12 1 a firstchild lastchild 2 3 b a b [ ] hildnodes 4 5 d b 6

13 firstchild Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) b a b a root reursion depth = 4 lastchild Tr(1) pr(1) Tr(2) pr(2) Tr(3) pr(3) Tr(4) pr(4) Tr(5) pr(5) Tr(6) pr(6) [ ] hildnodes d b 6

14 firstchild Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) b a b 4 5 d 1 a root b lastchild [ ] hildnodes 8 11 reursion depth = 4 12 Tr(1) pr(1) Tr(2) pr(2) Tr(3) pr(3) Tr(4) pr(4) Tr(5) pr(5) Tr(6) pr(6) Tr(7) pr(7) Tr(8) pr(8) Tr(9) pr(9) Tr(10) END pr(10) Tr(11) pr(11) Tr(12) pr(12)

15 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) Memory need proportional to the height of the XML do tree. 15 Should be fine. Usually height (XML do) is small. ( 15)

16 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) Memory need proportional to the height of the XML do tree. 16 Should be fine. Usually height (XML do) is small. ( 15) Problemati 2nd reursion on hildren TR(n:Node){ print(n); if(m=firstchild(n))!=nil then Tr(m); if(m=nextsibling(n))!=nil then Tr(m)

17 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) Memory need proportional to the height of the XML do tree. 17 Should be fine. Usually height (XML do) is small. ( 15) Problemati 2nd reursion on hildren TR(n:Node){ print(n); if(m=firstchild(n))!=nil then Tr(m); if(m=nextsibling(n))!=nil then Tr(m)

18 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) Memory need proportional to the height of the XML do tree. 18 Should be fine. Usually height (XML do) is small. ( 15) Problemati 2nd reursion on hildren A B C D E F TR(n:Node){ print(n); if(m=firstchild(n))!=nil then TR(m); if(m=nextsibling(n))!=nil then TR(m) TR(1);pr(1) TR(2);pr(2) TR(3);pr(3) TR(A);pr(A) TR(B);pr(B)

19 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) Memory need proportional to the height of the XML do tree. 19 Should be fine. Usually height (XML do) is small. ( 15) Problemati 2nd reursion on hildren TR(n:Node){ print(n); if(m=firstchild(n))!=nil then Tr(m); if(m=nextsibling(n))!=nil then Tr(m) A B C D E F TR(1);pr(1) TR(2);pr(2) TR(3);pr(3) TR(A);pr(A) TR(B);pr(B) Memory need proportional to max. length of (firstchild nextsibling)*-path

20 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) Memory need proportional to the height of the XML do tree. 20 Should be fine. Usually height (XML do) is small. ( 15) Problemati 2nd reursion on hildren A B C D E F Tr(n:Node){ print(n); if(m=firstchild(n))!=nil then Tr(m); if(m=nextsibling(n))!=nil then Tr(m) TR(1);pr(1) TR(2);pr(2) TR(3);pr(3) TR(A);pr(A) TR(B);pr(B) Can be HUGE!!! =size(tree) Memory need proportional to max. length of (firstchild nextsibling)*-path

21 root 21 firstchild a lastchild [ ] hildnodes b a b d b Question What is the max reursion depth on this tree? Tr(n:Node){ print(n); if(m=firstchild(n))!=nil then Tr(m); if(m=nextsibling(n))!=nil then Tr(m) Can be HUGE!!! =size(tree) Memory need proportional to max. length of (firstchild nextsibling)*-path

22 Tree Traversals Start at root node; want to visit every node. (1) reursively Traverse(n:Node){ print(n); For m in hildnodes(n) Traverse(m) Memory need proportional to the height of the XML tree. 22 Should be fine. Usually height (XML do) is small. ( 15) Problemati 2nd reursion on hildren Reall binary tree (firstchild/nextsibling) enoding. Question In the binary tree, what orresponds to this number? A B C D E F TR(1);pr(1) TR(2);pr(2) TR(3);pr(3) TR(A);pr(A) TR(B);pr(B) max. length of (firstchild nextsibling)*-path

23 Binary Tree Enoding 23 Reall firstchild/nextsibling enoding. The firstchild The nextsibling beomes the left pointer beomes the right pointer per Node 2 pointers/ids + label info 1 a firstchild 2 b a b d b ID f:ns:lab (2:-:a) 2 (-:3:b) 3 (4:9:a) 4 (-:5:) 5 (6:8:d) 6 (-:7:) 7 (-:-:) 8 (-:-:b) 9 (-:10:b) 10 (-,-:)

24 Tree Traversals 24 both, Traverse and TR an be exeuted on the f/ns-binary tree enoding. 1 a firstchild 2 b a b d b 8 6 7

25 Tree Traversals 25 both, Traverse and Tr an be exeuted on the f/ns-binary tree enoding. For m in hildnodes(n) Traverse(n) if(m=n->left)!=nil { while(m=n->right)!=nil Traverse(m) 1 a firstchild 2 b a b if(m=firstchild(n))!=nil then TR(m); if(m=nextsibling(n))!=nil then TR(m) if(m=n->left)!=nil then TR(m) if(m=n->right)!=nil then TR(m) 4 5 d b 8 7

26 Tree Traversals 26 both, Traverse and Tr an be exeuted on the f/ns-binary tree enoding. For m in hildnodes(n) Traverse(n) if(m=n->left)!=nil { while(m=n->right)!=nil Traverse(m) 1 a firstchild 2 b a b if(m=firstchild(n))!=nil then Tr(m); if(m=nextsibling(n))!=nil then Tr(m) if(m=n->left)!=nil then TR(m) if(m=n->right)!=nil then TR(m) 4 5 d b 7 8 Reursion takes are of the fat that we do not have parent pointers. (true for both, unranked & binary tree)

27 Other Traversals We disussed the Pre-order of the tree. (or, in XML-jargon: doument-order). Realized by Traverse & TR Post-order of a tree = 1. Traverse left subtree in post-order 2. Traverse right subtree in post-order 3. Visit the root Reverse Polish Notation In-order Breadth-First 4 1 (left-to-right) 2 6 level-order Binary Searh Tree (inreasing) q.enq(root); while(not q.empty){ Visit(q.deq); If(q->Left!=NIL) q.enq(q->left) If(q->Right!=NIL) q.enq(q->right)

28 Other Traversals We disussed the Pre-order of the tree. (or, in XML-jargon: doument-order). Realized by Traverse & TR Post-order of a tree = 1. Traverse left subtree in post-order 2. Traverse right subtree in post-order 3. Visit the root Reverse Polish Notation Breadth-First (left-to-right) level-order Memory need proportional to max. #nodes on one level q.enq(root); while(not q.empty){ Visit(q.deq); If(q->Left!=NIL) q.enq(q->left) If(q->Right!=NIL) q.enq(q->right)

29 Other Traversals We disussed the Pre-order of the tree. (or, in XML-jargon: doument-order). Realized by Traverse & TR Post-order of a tree = 1. Traverse left subtree in post-order 2. Traverse right subtree in post-order 3. Visit the root Reverse Polish Notation Can be HUGE!!! Memory need proportional to max. #nodes on one level Breadth-First (left-to-right) level-order q.enq(root); while(not q.empty){ Visit(q.deq); If(q->Left!=NIL) q.enq(q->left) If(q->Right!=NIL) q.enq(q->right);

30 Pre-Order 30 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) 1 a 2 b 3 a 8 b d 9 b 6 7

31 Pre-Order 31 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) b a b a How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; d b

32 Pre-Order 32 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(1) b a b a How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; d b

33 Pre-Order 33 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) pre(1) b a b a How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; d b

34 Pre-Order 34 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) b a b pre(1) d a b How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n;

35 Pre-Order 35 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) b a b pre(1) pre(3) d a b How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n;

36 Pre-Order 36 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) b a b pre(4) pre(1) pre(3) d a b How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n;

37 Pre-Order 37 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) b a b pre(4) pre(5) pre(1) pre(3) d a b How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n;

38 Pre-Order 38 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) b a b pre(4) pre(5) pre(1) pre(3) d a b How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6)

39 Pre-Order 39 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) b a b pre(4) pre(5) pre(1) pre(3) d a b How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6) pre(7)

40 Pre-Order 40 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) b a b pre(4) pre(5) pre(1) pre(3) d a b pre(8) How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6) pre(7)

41 Pre-Order 41 We saw how to ompute Pre-order (and Post and In) (1) reursively memory need: O(max_height) pre(2) b a b pre(4) pre(5) pre(1) pre(3) d a b pre(9) pre(8) pre(10) How to ompute Pre-order (2) iteratively Memory need? i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6) pre(7) if(n=nil)then break;

42 Pre-Order 42 How to ompute Pre-order (2) iteratively Memory need? pre(1) pre(2) pre(3) a pre(10) b a b pre(4) pre(9) d b pre(5) pre(8) No reursion! Needs onstant memory! (only one pointer) i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6) pre(7) if(n=nil)then break;

43 Pre-Order 43 Question Given a binary tree, (top-down, no parent) how muh memory do you need to ompute pre? No reursion! Needs onstant memory! (only one pointer) pre(1) pre(2) pre(3) a pre(10) b a b pre(4) pre(9) d b pre(5) pre(8) i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6) pre(7) if(n=nil)then break;

44 Pre-Order 44 Question Given a binary tree, (top-down, no parent) how muh memory do you need to ompute pre? Can you do it w. onstant memory? pre(2) b a b pre(4) pre(5) pre(1) pre(3) d a b pre(9) pre(8) pre(10) No reursion! Needs onstant memory! (only one pointer) i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6) pre(7) if(n=nil)then break;

45 Pre-Order 45 Question Fun (MS ji) How muh memory you need to hek for yles, in a single-linked (pointer) list? pre(2) b a b pre(4) pre(5) pre(1) pre(3) d a b pre(9) pre(8) pre(10) No reursion! Needs onstant memory! (only one pointer) i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6) pre(7) if(n=nil)then break;

46 Pre-Order 46 Question Do you see how to do Post- and In-oder iteratively? No reursion! Needs onstant memory! (only one pointer) pre(1) pre(2) pre(3) a pre(10) b a b pre(4) pre(9) d b pre(5) pre(8) i=1; n=root; pre(i)=n; repeat { while(firstchild(n)!=nil) { n=firstchild(n); pre(++i)=n; while(nextsibling(n)=nil) { n=parent(n); n=nextsibling(n); pre(++i)=n; pre(6) pre(7) if(n=nil)then break;

47 Pre-Order 47 From pre( ) we an ompute PreFollowing( n ) = { nodes m with pre(m) > n PrePreeding( n ) = { nodes m with pre(m) < n 1 a 2 b 3 a 9 b d 8 b 6 7 pre

48 From pre( ) Pre-Order What is this? 48 we an ompute PreFollowing( n ) = { nodes m with pre(m) > n PrePreeding( n ) = { nodes m with pre(m) < n 1 a 2 b 3 a 9 b d 8 b 6 7 pre

49 From pre( ) Pre-Order What is this? 49 we an ompute PreFollowing( n ) = { nodes m with pre(m) > n PrePreeding( n ) = { nodes m with pre(m) < n 1 a 2 b 3 a 9 b d 8 b 6 7 pre foll pre

50 From pre( ) Pre-Order useful?? 50 we an ompute PreFollowing( n ) = { nodes m with pre(m) > n PrePreeding( n ) = { nodes m with pre(m) < n Not tree (navigation) omplete hild? ns? 1 a 2 b 3 a 8 b d 9 b 6 7 pre

51 XML to RDBMS Enoding 51

52 52

53 53

54 54

55 XML to RDBMS Enoding 55 pre( ) is not enough Other possibilities: (1) large (unparsed) text blok (2) Shema-based enoding (3) Adjaeny-based enoding Dead Ends No good Questions Why is (1) a dead end?

56 56

57 57

58 58

59 59

60 Pre/Post Enoding 60 Add POST order 10 1 a b 3 a 9 b d 8 b PRE POST lab a 2 1 b 3 6 a d b 9 8 b 10 9 CREATE VIEW desendant AS SELECT r1.pre,r2.pre FROM R r1, R r2 WHERE r1.pre<r2.pre AND r1.post>r2.post

61 Pre/Post Enoding 61 Add POST order 10 1 a b 3 a 9 b d 8 b PRE POST lab a 2 1 b 3 6 a d b 9 8 b 10 9 CREATE VIEW desendant AS SELECT r1.pre,r2.pre FROM R r1, R r2 WHERE r1.pre<r2.pre AND r1.post>r2.post strutural join

62 62

63 63

64 Pre/Post Enoding 64 Straightforward how to ompute, for a given node, desendants anestors following preeding Questions How to do lastchild parent hildnodes Can you find orresponding SQL queries?

65 65 END Leture 3