XML and Databases. Sebastian Maneth NICTA and UNSW. Lecture 2 Memory Representations for XML: Space vs Access Speed. -- Semester 1, 2010

Size: px

Start display at page:

Download "XML and Databases. Sebastian Maneth NICTA and UNSW. Lecture 2 Memory Representations for XML: Space vs Access Speed. -- Semester 1, 2010"

Eustace Manning
5 years ago
Views:

1 1 XML n Dtbses Leture 2 Memory Representtions for XML: Spe vs Aess Spee Sebstin Mneth NICTA n UNSW CSE@UNSW -- Semester 1, 2010

2 2 XML n Dtbses Leture 2 Memory Representtions for XML: Spe vs Aess Spee Sebstin Mneth NICTA n UNSW (toy: Kim Nguyen) CSE@UNSW -- Semester 1, 2010

3 Reminer 3 You n freely hoose to progrm your ssignments in C / C++ (xeres- librry not instlle on se, will be fixe shortly) Jv However, your oe must ompile with g / g++, jv, s instlle on CSE linux systems! Assignment 1 is ue Mony 23:59, 15 th of Mrh! Submit your oe using % give s4317 ss1 filenme.pp % give s4317 ss1 filenme.jv

4 4 Leture 2 XML into Memory

5 5 Problem with DOM Uses mssive mounts of memory. Even if pplition touhes only single element noe, the DOM API hs to mintin t struture tht represents the whole XML input oument. Exmple XML size DOM proess size x 2 81M 164M Text only, with one embring element 52M x M Treebnk, eep tree struture with short texts. Usully: more thn 10-times blow up!!

6 6 To remey the memory hunger of DOM Preproess (i.e., filter) the input XML oument to reue its overll size. Use n XPth/XSLT proessor to preselet interesting oument regions. CAVE: no uptes on the input XML oument re possible CAVE: mke sure the XPth/XSLT proessor is not implemente on top of DOM! Use ompletely ifferent pproh to XML proessing ( SAX) esign your own XML t struture n fill it with wht you nee

7 7 To remey the memory hunger of DOM Preproess (i.e., filter) the input XML oument to reue its overll size. Use n XPth/XSLT proessor to preselet interesting oument regions. CAVE: no uptes on the input XML oument re possible CAVE: mke sure the XPth/XSLT proessor is not implemente on top of DOM! Use ompletely ifferent pproh to XML proessing ( SAX) esign your own XML t struture n fill it with wht you nee

8 Outline 8 1. Tree Pointer Strutures 2. Binry Tree Enoings 3. Miniml Unique DAGs 4. How to use SAX

9 1. Tree pointer strutures 9 1. Consier binry trees Type Noe { lbel : String, : Noe, : Noe } book librry ollet.. How muh memory for n-noe binry tree?

10 1. Consier binry trees 1. Tree pointer strutures Type Noe { lbel : String, : Noe, : Noe }. book librry my_tree ollet. 10 How muh memory for n-noe binry tree? r1: librry0 r2: book0 length(lbel_1) length(lbel_2) length(lbel_n) +1 my_tree: r1 tr1 tr2 tr1: r2 tr3 tr4 3 * length(pointer) * n

11 1. Consier binry trees 1. Tree pointer strutures Type Noe { lbel : String, : Noe, : Noe }. book librry my_tree ollet. 11 How muh memory for n-noe binry tree? r1: librry0 r2: book0 length(lbel_1) length(lbel_2) length(lbel_n) +1 my_tree: r1 tr1 tr2 tr1: r2 tr3 tr4 3 * length(pointer) * n typil: 4 bytes

12 1. Consier binry trees Tree pointer strutures Type Noe { lbel : String, : Noe, : Noe }. book librry my_tree ollet. 12 How muh memory for n-noe binry tree? Whtever is neee for the lbels PLUS 12 bytes per noe. length(lbel_1) length(lbel_2) length(lbel_n) +1 3 * length(pointer) * n typil: 4 bytes

13 1. Consier binry trees Tree pointer strutures Type Noe { lbel : String, : Noe, : Noe }. book librry my_tree ollet. 13 How muh memory for n-noe binry tree? Whtever is neee for the lbels PLUS 12 bytes per noe. Cn esily be optimize: E.g., store eh istint string only one! length(lbel_1) length(lbel_2) length(lbel_n) +1 3 * length(pointer) * n typil: 4 bytes

14 1. Consier binry trees Tree pointer strutures Type Noe { lbel : String, : Noe, : Noe }. book librry my_tree ollet. 14 Seriliztion to XML <librry><book>< > </book></librry> ^ ^ ^^ ^ #hrters per noe: * Length(lbel) E.g., one noe w. 4-hrter ASCII lbel: 13 bytes (ssuming UTF-8!)

15 1. Consier binry trees Tree pointer strutures Byte Type Noe { lbel : String, : Noe, : Noe }. book librry my_tree ollet. 15 Often #istint noe lbels is smll, 100. Fits in one Byte Then, only 9 bytes per noe. MEM(n-noe binry tree pointer stru, 256 lbels) = SIZE(n-noe binry tree in XML, verge lbel length=2) #hrters per noe: * Length(lbel) One noe w. 2-hrter ASCI lbel: 9 bytes (ssuming UTF-8!)

16 Tree pointer strutures my_tree 16 Nie Following pointers is fst! muh higher ess spee! (thn on o seen s string..) E.g. t root, get -hil. book librry ollet.. Often #istint noe lbels is smll, 100. Fits in one Byte Then, only 9 bytes per noe. MEM(n-noe binry tree pointer stru, 256 lbels) = SIZE(n-noe binry tree in XML, verge lbel length=2) #hrters per noe: * Length(lbel) One noe w. 2-hrter ASCI lbel: 9 bytes (ssuming UTF-8!)

17 Tree pointer strutures Consier binry trees Plin no ttributes, no text noes, librry Question Using (top-own) pointer struture, s the one bove, how n you implement DOM interfe? Noe noenme : DOMString prentnoe : Noe firstchil : Noe most hil nextsibling : Noe returns NULL for root elem hilnoes : NoeList

18 Tree pointer strutures Consier binry trees Plin no ttributes, no text noes, librry Question Using (top-own) pointer struture, s the one bove, how n you implement DOM interfe? Noe noenme : DOMString prentnoe : Noe firstchil : Noe most hil nextsibling : Noe returns NULL for root elem hilnoes : NoeList At run-time noe is represente s pointer, PLUS stk of pointers of ll its nestors. (Noe, [prent(noe)::prent(prent(noe)):: ::root-noe])

19 Tree pointer strutures Aess spee of prentnoe shoul be pprox sme, s in ntive DOM. Wht bout ess spee of nextsibling? 19 Wht is the run-time size of our binry DOM-tree t struture? (WC/verge) Question Using (top-own) pointer struture, s the one bove, how n you implement DOM interfe? Noe noenme : DOMString prentnoe : Noe firstchil : Noe most hil nextsibling : Noe returns NULL for root elem hilnoes : NoeList At run-time noe is represente s pointer, PLUS stk of pointers of ll its nestors. (Noe, [prent(noe)::prent(prent(noe)):: ::root-noe])

20 interfe Noe { // NoeType onst unsigne short ELEMENT_NODE = 1; onst unsigne short ATTRIBUTE_NODE = 2; onst unsigne short TEXT_NODE = 3; onst unsigne short CDATA_SECTION_NODE = 4; onst unsigne short ENTITY_REFERENCE_NODE = 5; onst unsigne short ENTITY_NODE = 6; onst unsigne short PROCESSING_INSTRUCTION_NODE = 7; onst unsigne short COMMENT_NODE = 8; onst unsigne short DOCUMENT_NODE = 9; onst unsigne short DOCUMENT_TYPE_NODE = 10; onst unsigne short DOCUMENT_FRAGMENT_NODE = 11; onst unsigne short NOTATION_NODE = 12; reonly ttribute DOMString noenme; ttribute DOMString noevlue; // rises(domexeption) on setting // rises(domexeption) on retrievl reonly ttribute unsigne short noetype; reonly ttribute Noe prentnoe; reonly ttribute NoeList hilnoes; reonly ttribute Noe firstchil; reonly ttribute Noe lstchil; reonly ttribute Noe previoussibling; reonly ttribute Noe nextsibling; reonly ttribute NmeNoeMp ttributes; reonly ttribute Doument ownerdoument; Noe insertbefore(in Noe newchil, in Noe refchil) rises(domexeption); Noe replechil(in Noe newchil, in Noe olchil) rises(domexeption); Noe removechil(in Noe olchil) rises(domexeption); Noe ppenchil(in Noe newchil) rises(domexeption); boolen hschilnoes(); Noe lonenoe(in boolen eep); }; 20

21 Tree pointer strutures 21 To slsh memory hunger (of, e.g., DOM ) LESSON 1 Avoi ll bkwr pointers (buil them online, ynmilly) binry trees DOM book librry NIL firstchil &prent firstchil &prent librry book n&psibling hilnoes hilnoes lstchil &prent ollet.

22 Tree pointer strutures Consier binry trees Type Noe { lbel : String, : Noe, : Noe } book librry ollet. How muh memory for n-noe binry tree? How to ttributes n text noes? Text: n be in the lbel, strting with speil hrter (pb: 2 unuse pointers) Attributes: pointer to n ux. Dt-Struture (expensive) Attributes enoe s hilren with speil lbels...

23 Tree pointer strutures Consier unrnke trees firstchil librry lstchil book book book book unrnke = no priori boun on #hilren of noe. Tree struture of XML: unrnke trees! (not binry) Type Noe { lbel : String, hilren : List[Noe] }

24 Tree pointer strutures Consier unrnke trees firstchil librry lstchil book book book book unrnke = no priori boun on #hilren of noe. Tree struture of XML: unrnke trees! (not binry) Type Noe { lbel : String, hilren : List[Noe] } How muh memory for List[Noe] of n noes?

25 Tree pointer strutures Consier unrnke trees firstchil librry lstchil book book book book unrnke = no priori boun on #hilren of noe. Tree struture of XML: unrnke trees! Typilly Type Noe { lbel : String, hilren : List[Noe] } N1 next N2 How muh memory for List[Noe] of n noes?

26 Tree pointer strutures Consier unrnke trees firstchil librry lstchil book book book book unrnke = no priori boun on #hilren of noe. Tree struture of XML: unrnke trees! Typilly Type Noe { lbel : String, hilren : List[Noe] } N1 next N2 How muh memory for List[Noe] of n noes? 2*n pointers

27 Tree pointer strutures Consier unrnke trees In this wy, noe of binry tree nees 5 pointers (plus lbel info/pointer..) unrnke = no priori boun on #hilren of noe. Tree struture of XML: unrnke trees! Typilly Type Noe { lbel : String, hilren : List[Noe] } N1 next N2 How muh memory for List[Noe] of n noes? 2*n pointers

28 Tree pointer strutures Consier unrnke trees In this wy, noe of binry tree nees 5 pointers (plus lbel info/pointer..) More effiient possibilities: (1) Use rrys. Store #hilren. n pointers + (log n) bits (2) Enoe tree s binry tree. Typilly Length of the rry Type Noe { lbel : String, hilren : List[Noe] } N1 next N2 How muh memory for List[Noe] of n noes? 2*n pointers

29 Tree pointer strutures Consier unrnke trees In this wy, noe of binry tree nees 5 pointers (plus lbel info/pointer..) More effiient possibilities: (1) Use rrys. Store #hilren. (2) Enoe tree s binry tree. n pointers + (log ) Bits Typilly Type Noe { lbel : String, hilren : List[Noe] } N1 next N2 How muh memory for List[Noe] of n noes? 2*n pointers

30 2. Binry Tree Enoings 30 Any unrnke tree n be enoe s binry tree. Populr enoing: firstchil/nextsibling enoing. The firstchil beomes the pointer The nextsibling beomes the pointer firstchil N1 N2 N3 N4 N5 N6 N7 N8 N9 N10

31 2. Binry Tree Enoings 31 Any unrnke tree n be enoe s binry tree. Populr enoing: firstchil/nextsibling enoing. The firstchil beomes the pointer The nextsibling beomes the pointer firstchil N1 N1 N2 N3 N4 N5 N2 N3 N4 N5 N6 N7 N8 N6 N7 N8 N9 N10 N9 N10

32 2. Binry Tree Enoings 32 Any unrnke tree n be enoe s binry tree. Populr enoing: firstchil/nextsibling enoing. The firstchil beomes the pointer The nextsibling beomes the pointer firstchil N1 N1 N2 N3 N4 N5 N2 N3 N4 N5 N6 N7 N8 N6 N7 N8 N9 N10 N9 N10

33 2. Binry Tree Enoings 33 Any unrnke tree n be enoe s binry tree. Populr enoing: firstchil/nextsibling enoing. The firstchil beomes the pointer The nextsibling beomes the pointer firstchil N1 N1 N2 N3 N4 N5 N2 N3 N4 N5 N6 N7 N8 N6 N7 N8 N9 N10 N9 N10

34 2. Binry Tree Enoings 34 Any unrnke tree n be enoe s binry tree. Populr enoing: firstchil/nextsibling enoing. The firstchil beomes the pointer The nextsibling beomes the pointer N1 firstchil N1 N2 N3 N2 N3 N4 N5 N6 N4 N6 N7 N8 N9 N7 N8 N5 N9 N10 N10

35 Binry Tree Enoings 35 Any unrnke tree n be enoe s binry tree. Populr enoing: firstchil/nextsibling enoing. The firstchil beomes the pointer The nextsibling beomes the pointer firstchil N1 NIL N2 N3 N1 NIL N2 N3 N4 N5 N6 N7 N8 NIL N9 N6 N7 NIL N8 N4 NIL N5 NIL N9 N10 NIL N10 NIL NIL

36 Binry Tree Enoings 36 Any unrnke tree n be enoe s binry tree. n-noe unrnke tree firstchil/nextsibling enoing eoing n-noe binry tree Questions Time overhe for simulting lstchil ess, on the binry enoing? Cn you think of other binry tree enoings? How to simulte previous-sibling?

37 Binry Tree Enoings 37 Any unrnke tree n be enoe s binry tree. n-noe unrnke tree firstchil/nextsibling enoing eoing n-noe binry tree Goo Property of the firstchil/nextsibling enoing: XML types (e.g., DTD, XML Shem, Relx NG) re preserve when going from unrnke to binry (n vie vers).

38 Binry Tree Enoings 38 Any unrnke tree n be enoe s binry tree. n-noe unrnke tree firstchil/nextsibling enoing eoing n-noe binry tree Goo Property of the firstchil/nextsibling enoing: XML types (e.g., DTD, XML Shem, Relx NG) re preserve when going from unrnke to binry (n vie vers). LESSON 2 ginst memory hunger Use binry trees inste of unrnke trees. ( or use effiient rrys) + Fst hil-m ess - Expensive to upte (insert/elete)

39 Tree Pointer Strutures 39 Question Give ttype for binry trees whih stores only non-nil pointers. Then, n-noe tree: <n pointers Type Noe { lbel : String, : Noe, : Noe }

40 Type Noe { lbel : Byte, : Noe, : Noe } 3. Miniml Unique DAGs binry tree 2 pointers per noe 40 L1: no bkwr pointers L2: use binry trees or effiient rrys Cn we o with even less pointers?

41 Type Noe { lbel : Byte, : Noe, : Noe } 3. Miniml Unique DAGs binry tree 2 pointers per noe 41 L1: no bkwr pointers L2: use binry trees or effiient rrys Cn we o with even less pointers? n-noe tree: 2n pointers

42 3. Miniml Unique DAGs 42 Type Noe { lbel : Byte, : Noe, : Noe } binry tree 2 pointers per noe Cn we o with even less pointers? n-noe tree: 2n pointers YES! Direte Ayli Grph DAG shre ientil subtrees

43 3. Miniml Unique DAGs 43 Type Noe { lbel : Byte, : Noe, : Noe } binry tree 2 pointers per noe 18 pointers relly mens 18 non-nil pointers Cn we o with even less pointers? n-noe tree: 2n pointers YES! Direte Ayli Grph DAG shre ientil subtrees 18/19 pointers/noes 10/6 pointers/noes

44 3. Miniml Unique DAGs 44 A DAG representtion of tree hs lwys Less thn or equl #noes thn the tree Less thn or equl #pointers thn the tree.

45 3. Miniml Unique DAGs 45

46 3. Miniml Unique DAGs 46

47 3. Miniml Unique DAGs 47 (miniml) DAGs hve mny pplitions! CSE (Common Subexpression Elimintion) for effiient evlution of expressions (o term grph rewriting, inste of term rewriting) Moel heking with BDDs Binry Deision Digrms for effiient evlution of logi formuls Effiient XML query evlution

48 3. Miniml Unique DAGs 48 (miniml) DAGs hve mny pplitions! CSE (Common Subexpression Elimintion) for effiient evlution of expressions (o term grph rewriting, inste of term rewriting) Moel heking with BDDs Binry Deision Digrms for effiient evlution of logi formuls Effiient XML query evlution Btw, insie of DAG, you hve referentil ompleteness struturl equlity = equlity of pointers

49 3. Miniml Unique DAGs 49 Every tree hs miniml, unique DAG! The DAG is t most exponentilly smller thn the tree. Builing the miniml unique DAG is esy! Cn be one in (mortize) liner time.

50 3. Miniml Unique DAGs 50 Every tree hs miniml, unique DAG! The DAG is t most exponentilly smller thn the tree. Builing the miniml unique DAG is esy! Cn be one in (mortize) liner time. How?

51 3. Miniml Unique DAGs 51 Every tree hs miniml, unique DAG! The DAG is t most exponentilly smller thn the tree. Builing the miniml unique DAG is esy! Cn be one in (mortize) liner time. How? (even while prsing) Buil hsh tble of ll subtrees seen so fr (we on t wnt to ompre mny trees, noe by noe, lter on..) Question Give simple hsh funtion tht works for the tree bove.

52 3. Miniml Unique DAGs 52 Hsh Tble HT Hsh funtion f 1 2 (,), ((,),) 3 (b,) hsh buket We wnt f istributes trees uniformly into bukets test if tree T is in HT, time O( size(t) ) Question Give simple hsh funtion tht works for the tree bove.

53 Miniml Unique DAGs 53 1: bib [2,3,4,5] 2: book [6,7] 3: rtile [8,9] 4: book [10,11] 5: rtile [12,13] 6: uthor 7: title 8: uthor 9: title 10:prie 11:title 12:prie 13:title <bib> <book> <uthor></uthor> <title></title> </book> <rtile> <uthor></uthor> <title></title> </rtile> <book> <prie></prie> <title></title> </book> <rtile> <prie></prie> <title></title> </rtile> </book>

54 Miniml Unique DAGs 54 1: bib [2,3,4,5] 2: book [6,7] 3: rtile [8,9] 4: book [10,11] 5: rtile [12,13] 6: uthor 7: title 8: uthor 9: title 10:prie 11:title 12:prie 13:title 6 <bib> <book> <uthor></uthor> <title></title> </book> <rtile> <uthor></uthor> <title></title> </rtile> <book> <prie></prie> <title></title> </book> <rtile> <prie></prie> <title></title> </rtile> </book>

55 Miniml Unique DAGs 55 1: bib [2,3,4,5] 2: book [6,7] 3: rtile [8,9] 4: book [10,11] 5: rtile [12,13] 6: uthor 7: title 8: uthor 9: title 10:prie 11:title 12:prie 13:title 7 6 <bib> <book> <uthor></uthor> <title></title> </book> <rtile> <uthor></uthor> <title></title> </rtile> <book> <prie></prie> <title></title> </book> <rtile> <prie></prie> <title></title> </rtile> </book>

56 Miniml Unique DAGs 56 1: bib [2,3,4,5] 2: book [6,7] 3: rtile [8,9] 4: book [10,11] 5: rtile [12,13] 6: uthor 7: title 10 8: uthor 9: title 10:prie 11:title 12:prie 13:title 7 6 <bib> <book> <uthor></uthor> <title></title> </book> <rtile> <uthor></uthor> <title></title> </rtile> <book> <prie></prie> <title></title> </book> <rtile> <prie></prie> <title></title> </rtile> </book>

57 Miniml Unique DAGs 57 1: bib [2,3,4,5] 2: book [6,7] 3: rtile [8,9] 4: book [10,11] 5: rtile [12,13] 6: uthor 7: title 10 8: uthor 9: title 10:prie 11:title 12:prie 13:title 7 6 <bib> <book> <uthor></uthor> <title></title> </book> <rtile> <uthor></uthor> <title></title> </rtile> <book> <prie></prie> <title></title> </book> <rtile> <prie></prie> <title></title> </rtile> </book> miniml unique DAG 8 noes (vs 13 noes in the originl tree)

58 Miniml Unique DAGs 58 Exmple Prse & DAGify 1: strtelement() hsh ontent

59 Miniml Unique DAGs 59 Exmple Prse & DAGify 1: strtelement() 2: strtelement() hsh ontent

60 Miniml Unique DAGs 60 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() hsh ontent

61 Miniml Unique DAGs 61 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 1. p1=hsht.fin() 2. if(p1==null) { p1=new( -noe,null,null) hsht.insert(p1) } hsh ontent

62 Miniml Unique DAGs 62 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 1. p1=hsht.fin() 2. if(p1==null) { p1=new( -noe,null,null) hsht.insert(p1) } hsh ontent 1 2 p1 Memory lotion p1 is DAG with root noe, n hil1-pointer=null hil2-pointer=null

63 Miniml Unique DAGs 63 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 1. p1=hsht.fin() 2. if(p1==null) { p1=new( -noe,null,null) hsht.insert(p1) } must store hilren lists: [ [],[p1] ] hilren of root noe (so fr, none) hilren of - noe (so fr, one) hsh ontent 1 2 p1 Memory lotion p1 is DAG with root noe, n hil1-pointer=null hil2-pointer=null

64 Miniml Unique DAGs 64 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() hsh ontent 1 2 p1

65 Miniml Unique DAGs 65 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() 6: enelement() 1. p2=hsht.fin() 2. if(p2==null) { p2=new( -noe,null,null) hsht.insert(p2) } hsh ontent 1 2 p1

66 Miniml Unique DAGs 66 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() 6: enelement() 1. p2=hsht.fin()=p1 2. if(p2==null) { p2=new( -noe,null,null) hsht.insert(p2) } hsh ontent 1 2 p1

67 Miniml Unique DAGs 67 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() 6: enelement() 1. p2=hsht.fin()=p1 2. if(p2==null) { p2=new( -noe,null,null) hsht.insert(p2) } store hilren lists: [ [],[p1,p1] ] hsh ontent 1 2 p1

68 Miniml Unique DAGs 68 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() 6: enelement() 7: enelement() store hilren lists: [ [],[p1,p1] ] hsh ontent 1 2 p1

69 Miniml Unique DAGs 69 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() 6: enelement() 7: enelement() p1 1. p=hsht.fin() 2. if(p==null) { p=new( -noe,p1,p1) hsht.insert(p) } p1 store hilren lists: [ [],[p1,p1] ] hsh ontent 1 2 p p use hilren list!! Memory lotion p is DAG with root noe, n hil1-pointer=p1 hil2-pointer=p1

70 Miniml Unique DAGs 70 Exmple Prse & DAGify 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() 6: enelement() 7: enelement() p1 1. p=hsht.fin() 2. if(p==null) { p=new( -noe,p1,p1) hsht.insert(p) } p1 store hilren lists: [ [],[p1,p1] ] p (s lst) hsh now upte!! ontent 1 2 p p Memory lotion p is DAG with root noe, n hil1-pointer=p1 hil2-pointer=p1

71 Miniml Unique DAGs 71 Exmple Prse & DAGify New hilren lists: [ [p] ] 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() 6: enelement() 7: enelement() hilren of root noe (so fr, one) 1. p=hsht.fin() 2. if(p==null) { p=new( -noe,p1,p1) hsht.insert(p) } store hilren lists: [ [],[p1,p1] ] p (s lst) hsh now upte!! ontent 1 2 p p Memory lotion p is DAG with root noe, n hil1-pointer=p1 hil2-pointer=p1

72 Miniml Unique DAGs 72 Exmple Prse & DAGify New hilren lists: [ [p] ] 1: strtelement() 2: strtelement() 3: strtelement() 4: enelement() 5: strtelement() 6: enelement() 7: enelement() hilren of root noe (so fr, one) 1. p=hsht.fin() 2. if(p==null) { p=new( -noe,p1,p1) hsht.insert(p) } hsh ontent 1 2 p p Assume 100 element nmes Exmple hsh funtion: (#elementnme * #elementnme(1 st hil) * 100 * #elemnme(2 n hil) + 100^3 * elemnme(1 st hil of 1 st h.) + ) MOD sizeof(hsht)

73 Miniml Unique DAGs 73 DOM interft to the DAG? prentnoe / p&nsibling s before Uptes n be expensive (opying!) How to tth ttribute & text noes to the DAG?

74 Miniml Unique DAGs 74 DOM interft to the DAG? prentnoe / p&nsibling s before Uptes n be expensive (opying!) How to tth ttribute & text noes to the DAG? Store them seprtely in tble. Inex by e.g., Noe number (in o-orer) or number of tr/text noes Store inex in eh DAG noe / or ompute it online. (pre-trversl)

75 Miniml Unique DAGs 75 Wht bout unrnke, vs binry DAGs? librry firstchil lstchil book book book book More preisely, Wht bout size of miniml-unique-unrnke-dag( Tree ) vs size of miniml-unique-binry-dag( fcns-en( Tree ) ) firstchil/nextsibling

76 Miniml Unique DAGs 76 size of miniml-unique-unrnke-dag( Tree ) vs size of miniml-unique-binry-dag( fcns-en( Tree ) ) Questions Give tree for whih first is smller thn the seon. Give tree for whih the seon is smller thn the first.

77 Unrnke vs Binry Trees 77 18/19 3/4 p n s p n s p n s p n s p n s p n s Min. DAG p 6 n s p p p p p p Min. DAG p p 1 p 13/9 n s n s n s n s n s n s p n s p p

78 Unrnke vs Binry Trees 78 18/19 3/4 p n s p n s p n s p n s p n s p n s Min. DAG p 6 n s Cn it be vi vers? (min bin. DAG is smller) YES!! Hs 18 eges DAG of bin.oing only 12 eges u p p p x b b y b b z b b DAG ompression is sensible to rnk/unrnkeness!

79 Unrnke vs Binry Trees 79 Lst omment on binry tree enoings / DAGs YES: the binry trees beome very regulr (eep, to the ) Items (books/resses/et) DAG Items (books/resses/et) DAG w. multipliities Items (books/resses/et) Bin tree w multipliities

80 3. Miniml Unique DAGs 80

81 Effiient XML & Binry XML 81 W3C working groups XML Binry Chrteriztion Working Group The Figure on the next slie is from the EXI Mesurement Note -- new version of the note me out 25 July 2007!)

82 Effiient XML & Binry XML 82 W3C working groups XML Binry Chrteriztion Working Group ene in 2008, merge with the following: Effiient XML Interhnge Working Group (EXI) The Figure on the next slie is from the EXI Mesurement Note -- new version of the note me out 25 July 2007!)

83 83

84 Miniml Unique DAGs 84 Assignment 2 buil miniml DAG for tree (given in XML) For simpliity, ignore ttributes n text vlues. only onsier element noes. Buil the DAG, while prsing the XML! Construt hsh tble whih stores ll (omplete) istint subtrees seen so fr. Clery, we o not wnt to prse into DOM, n then pull things out of there. Inste, we nee more flexible prser tht gives s the freeom of wht extly to store, n how.

85 How to use SAX 85 Remember one of the promises of XML You never nee to write prser gin!

86 How to use SAX 86 Remember one of the promises of XML You never nee to write prser gin! but, of ourse if you wnt to buil up your own (e.g. memory-effiient) t struture, you nee to tlk to the prser. You wnt the prser to Give low level ess to the t: Brket by brket, text-noe by text-noe. In oument orer.

87 How to use SAX 87 Remember one of the promises of XML You never nee to write prser gin! but, of ourse if you wnt to buil up your own (e.g. memory-effiient) t struture, you nee to tlk to the prser. You wnt the prser to Give low level ess to the t: Brket by brket, text-noe by text-noe. In oument orer. SAX

88 88

89 89

90 90

91 91

92 92

93 93

94 94

95 95 Deline: 29 th Mrh For Assignment 2, you only nee to register strtelement n enelement. In tht wy, you utomtilly reeive only element noes.. Of ourse you n use SAX for other things thn builing up t struture. E.g. nswer pth queries while prsing (on strem ) (low memory onsumption!)

96 96 END Leture 2

XML and Databases. Outline. XPath. Outline - Lectures. XPath Data Model. Outline - Assignments. XPath. Sebastian Maneth NICTA and UNSW

XML and Databases. Outline. XPath. Outline - Lectures. XPath Data Model. Outline - Assignments. XPath. Sebastian Maneth NICTA and UNSW Outline XML n Dtses Leture 6 Noe Seleting Queries: XPth 1.0 1. XPth Dt Moel: 7 types of noes 2. Simple Exmples 3. Lotion Steps n Pths 4. Vlue Comprison, n Other Funtions Sestin Mneth NICTA n UNSW CSE@UNSW