Structural Joins: A Primitive for Efficient XML Query Pattern Matching

Transcription

1 Structurl Joins: A Primitive for Efficient XML Query Pttern Mtching Shurug Al-Khlif Univ of Michign shurug@eecs.umich.eu Jignesh M. Ptel Univ of Michign jignesh@eecs.umich.eu H. V. Jgish Univ of Michign jg@eecs.umich.eu Divesh Srivstv AT&T Lbs Reserch ivesh@reserch.tt.com Nick Kous AT&T Lbs Reserch kous@reserch.tt.com Yuqing Wu Univ of Michign yuwu@eecs.umich.eu Abstrct XML queries typiclly specify ptterns of selection preictes on multiple elements tht hve some specifie tree structure reltionships. The primitive tree structure reltionships re prent-chil n ncestor-escennt, n fining ll occurrences of these reltionships in n XML tbse is core opertion for XML query processing. In this pper, we evelop two fmilies of structurl join lgorithms for this tsk: tree-merge n stck-tree. The tree-merge lgorithms re nturl extension of tritionl merge joins n the recently propose multi-preicte merge joins, while the stck-tree lgorithms hve no counterprt in tritionl reltionl join processing. We present experimentl results on rnge of t n queries using the TIMBER ntive XML query engine built on top of SHORE. We show tht while, in some cses, tree-merge lgorithms cn hve performnce comprble to stck-tree lgorithms, in mny cses they re consierbly worse. This behvior is expline by nlyticl results tht emonstrte tht, on sorte inputs, the stck-tree lgorithms hve worst-cse I/O n CPU complexities liner in the sum of the sizes of inputs n output, while the tree-merge lgorithms o not hve the sme gurntee. 1 Introuction XML employs tree-structure moel for representing t. Quite nturlly, queries in XML query lnguges (see, e.g., [10, 7, 6]) typiclly specify ptterns of selection preictes on multiple elements tht hve some specifie tree structure reltionships. For exmple, the XQuery pth expression: book[title XML ]//uthor[. jne ] mtches uthor elements tht (i) hve s content the string vlue jne, n (ii) re escennts of book elements tht hve chil title element whose content is the string vlue XML. This XQuery pth expression cn be represente s noe-lbele tree pttern with elements n string vlues s noe lbels. Such complex query tree pttern cn be nturlly ecompose into set of bsic prent-chil n ncestor-escennt reltionships between pirs of noes. For exmple, the bsic structurl reltionships corresponing to the bove query re the ncestorescennt reltionship (book, uthor) n the prent-chil reltionships (book, title), (title, XML) n (uthor, jne). The query pttern cn then be mtche by (i) mtching ech of the binry structurl reltionships ginst the XML tbse, n (ii) stitching together these bsic mtches. Fining ll occurrences of these bsic structurl reltionships in n XML tbse is clerly core opertion in XML query processing, both in reltionl implementtions of XML tbses, n in ntive XML tbses. There hs been gret el of work one on how to fin occurrences of such structurl reltionships (s well s the query tree ptterns in which they re embee) using reltionl tbse systems (see, e.g., [14, 7, 6]), s well s using ntive XML query engines (see, e.g., [1, 3, ]). These works typiclly use some combintion of inexes on elements n string vlues, tree trversl lgorithms, n join lgorithms on the ege reltionships between noes in the XML t tree. More recently, Zhng et l. [9] propose vrition of the tritionl merge join lgorithm, clle the multi-preicte merge join (MPMGJN) lgorithm, for fining ll occurrences of the bsic structurl reltionships (they refer to them s continment queries). They compre the implementtion of continment queries using ntive support in two commercil tbse systems, n specil purpose inverte list engine bse on the MPMGJN lgorithm. Their results showe tht the MPMGJN lgorithm coul outperform stnr RDBMS join lgorithms by more thn n orer of mgnitue on continment queries. The key to the efficiency of the MPMGJN lgorithm is the (DocI, StrtPos : EnPos, LevelNum) representtion of positions of XML elements, n the (DocI, StrtPos, LevelNum) representtion of positions of string vlues, tht succinctly cpture the structurl reltionships between elements (n string vlues) in the XML tbse (see Section.3 for etils bout this representtion). Checking tht structurl reltionships in the XML tree, like ncestor-escennt n prent-chil (corresponing to continment n irect continment reltionships, respectively, in

2 the XML ocument representtion), re present between elements mounts to checking tht certin inequlity conitions hol between the components of the positions of these elements. While the MPMGJN lgorithm outperforms stnr RDBMS join lgorithms, we show in this pper tht it cn perform lot of unnecessry computtion n I/O for mtching bsic structurl reltionships, especilly in the cse of prent-chil reltionships (or, irect continment queries). In this pper, we tke vntge of the (DocI, StrtPos : EnPos, LevelNum) representtion of positions of XML elements n string vlues to evise novel I/O n CPU optiml join lgorithms for mtching structurl reltionships ginst n XML tbse. Since gret el of XML t is expecte to be store in reltionl tbse systems (ll the mjor DBMS venors incluing Orcle, IBM n Microsoft re proviing system support for XML t), our stuy provies evience tht RDBMS systems nee to ugment their suite of physicl join lgorithms to inclue structurl joins to be competitive on XML query processing. Our stuy is eqully relevnt for ntive XML query engines, since structurl joins provie for n efficient set-t--time strtegy for mtching XML query ptterns, in contrst to the noe-t--time pproch of using tree trversls. 1.1 Outline n Contributions We begin by presenting bckgroun mteril in Section. Our min contributions re s follows: We evelop two fmilies of join lgorithms to perform mtching of the prent-chil n ncestor-escennt structurl reltionships efficiently: tree-merge n stck-tree (Section 3). Given two input lists of tree noes, ech sorte by (DocI, StrtPos), the lgorithms compute n output list of sorte results joine ccoring to the esire structurl reltionship. The tree-merge lgorithms re nturl extension of merge joins n the recently propose MP- MGJN lgorithm [9], while the stck-tree lgorithms hve no counterprt in tritionl reltionl join processing. We present n nlysis of the tree-merge n the stck-tree lgorithms (Section 3). The stck-tree lgorithms re I/O n CPU optiml (in n symptotic sense), n hve worstcse I/O n CPU complexities liner in the sum of sizes of the two input lists n the output list for both ncestorescennt (or, continment) n prent-chil (or, irect continment) structurl reltionships. The tree-merge lgorithms hve worst-cse qurtic I/O n CPU complexities, but on some nturl clsses of structurl reltionships n XML t, they hve liner complexity s well. We show experimentl results on rnge of t n queries using the TIMBER ntive XML query engine built on top of SHORE (Section 4). We show tht while, in some cses, the performnce of tree-merge lgorithms cn be comprble to tht of stck-tree lgorithms, in mny cses they re consierbly worse. This is consistent with the nlysis presente in Section 3. We escribe relte work in Section 5, n iscuss ongoing n future work in Section 6. Bckgroun n Overview.1 Dt Moel n Query Ptterns An XML tbse is forest of roote, orere, lbele trees, ech noe corresponing to n element n the eges representing (irect) element-subelement reltionships. Noe lbels consist of set of (ttribute, vlue) pirs, which suffices to moel tgs, PCDATA content, etc. For the smple XML ocument of Figure 1(), its tree representtion is shown in Figure 1(b). (The utility of the numbers ssocite with the tree noes will be expline in Section.3.) Queries in XML query lnguges like XQuery [6], Quilt [7], n XML-QL [10] mke funmentl use of (noe lbele) tree ptterns for mtching relevnt portions of t in the XML tbse. The query pttern noe lbels inclue element tgs, ttribute-vlue comprisons, n string vlues, n the query pttern eges re either prent-chil eges (epicte using single line) or ncestor-escennt eges (epicte using ouble line). For exmple, the XQuery pth expression in the introuction cn be represente s the roote tree pttern in Figure (). This query pttern woul mtch the ocument in Figure 1. In generl, t ech noe in the query tree pttern, there is noe preicte tht specifies some preicte on the ttributes (e.g., tg, content) of the noe in question. For the purposes of this pper, exctly wht is permitte in this preicte is not mteril. It suffices for our purposes tht there be the possibility of constructing efficient ccess mechnisms (such s inex structures) to ientify the noes in the XML tbse tht stisfy ny given noe preicte.. Mtching Bsic Structurl Reltionships A complex query tree pttern cn be ecompose into set of bsic binry structurl reltionships such s prent-chil n ncestor-escennt between pirs of noes. The query pttern cn then be mtche by (i) mtching ech of the binry structurl reltionships ginst the XML tbse, n (ii) stitching together these bsic mtches. For exmple, the bsic structurl reltionships corresponing to the query tree pttern of Figure () re shown in Figure (b). A strightforwr pproch to mtching structurl reltionships ginst n XML tbse is to use trversl-style lgorithms by using chil-pointers or prent-pointers. Such tuple-t--time processing strtegies re known to be inefficient compre to the set-t--time strtegies use in tbse systems. Pointer-bse joins [8] hve been suggeste s solution to this problem in object-oriente tbses, n shown to be quite efficient. In the context of XML tbses, noes my hve lrge number of chilren, n the query pttern often requires mtching ncestor-escennt structurl reltionships (for exmple, the (book, uthor) ege in the query pttern of Figure ()), in ition to prent-chil structurl reltionships. In this cse, there re two options: (i) explicitly mintining only (prent, chil) noe pirs n ientifying (ncestor, escennt) noe pirs through repete joins; or (ii) explicitly mintining (ncestor, escennt) noe pirs. The former pproch woul tke too much query processing time, while the ltter pproch woul use too

3 book title XML title lluthors uthor jne uthor uthor john uthor lluthors yer 000 yer chpter he Origins he section he... he section... section section section... section chpter chpter... chpter book () book (1,1:70,1) title lluthors yer chpter chpter (1,:4,) (1,5:1,) (1,13:15,) (1,16:40,) (1,41:69,) (1,3,3) XML uthor uthor 000 he section section (1,6:8,3) (1,9:11,3) (1,14,3) (1,17:19,3) (1,0:9,3) (1,30:39,3) jne john Origins he section (1,7,4) (1,10,4) (1,18,4) (1,1:3,4) (1,4:8,4) (b) Figure 1. () A smple XML ocument frgment, (b) Tree representtion title XML book () uthor jne book title title XML (b) book uthor uthor Figure. () Tree pttern, (b) Structurl reltionships jne much (qurtic) spce. In either cse, using pointer-bse joins is likely to be infesible..3 Representing Positions of Elements n String Vlues in n XML Dtbse The key to n efficient, uniform mechnism for set-t--time (join-bse) mtching of structurl reltionships is positionl representtion of occurrences of XML elements n string vlues in the XML tbse (see, e.g., [8, 9, 9]), which extens the clssic inverte inex t structure in informtion retrievl [5]. The position of n element occurrence in the XML tbse cn be represente s the 3-tuple (DocI, StrtPos : EnPos, LevelNum), n the position of string occurrence in the XML tbse cn be represente s the 3-tuple (DocI, StrtPos, LevelNum), where (i) DocI is the ientifier of the ocument; (ii) StrtPos n EnPos cn be generte by counting wor numbers from the beginning of the ocument with ientifier DocI until the strt of the element n en of the element, respectively; n (iii) LevelNum is the nesting epth of the element (or string vlue) in the ocument. Figure 1(b) epicts 3- tuple with ech tree noe, bse on this representtion of position. (The DocI for ech of these noes is chosen to be 1.) Structurl reltionships between tree noes (elements or string vlues) whose positions re recore in this fshion cn be etermine esily: (i) ncestor-escennt: tree noe whose position in the XML tbse is encoe s is escennt of tree noe whose position is encoe s iff n ; 1 (ii) prent-chil: tree noe! whose position in the XML tbse is encoe s "#$ is chil of tree noe % whose position is encoe s & ' ( iff ) *,n +-,. For exmple, in Figure 1(b), the uthor noe with position, /.0 1/3 is escennt of the book noe with position,, 54768,, n the string jne with position, 49:; is chil of the uthor noe with position, /.<18=3. A key point worth noting bout this representtion of noe positions in the XML t tree is tht checking n ncestorescennt structurl reltionship is s esy s checking prentchil structurl reltionship. The reson is tht one cn check for n ncestor-escennt structurl reltionship without knowlege of the intermeite noes on the pth. Also worth noting is tht this representtion of positions of elements n string vlues llow for checking orer n proximity reltionships between elements n/or string vlues; this issue is not explore further in our pper..4 An Overview In the rest of this pper, we tke vntge of the (DocI, StrtPos : EnPos, LevelNum) representtion of positions of XML elements n string vlues to (i) evise novel, I/O n CPU optiml (in n symptotic sense) join lgorithms for mtching bsic structurl reltionships (or, continment queries) ginst n XML tbse; (ii) present n nlysis of these lgorithms; n (iii) show their behvior in prctice using vriety of experiments. The tsk of mtching complex XML query pttern then reuces to tht of evluting join expression with one join opertor for ech binry structurl reltionship in the query pttern. Different join orerings my result in ifferent evlution costs, s usul. Fining the optiml join orering is outsie the scope of this pper, n is the subject of future work in this re. 1 For lef strings, EnPos is the sme s StrtPos.

4 Algorithm Tree-Merge-Anc (AList, DList) /* Assume tht ll noes in AList n DList hve the sme DocI */ /* AList is the list of potentil ncestors, in sorte orer of StrtPos */ /* DList is the list of potentil escennts in sorte orer of StrtPos */ begin-esc DList->firstNoe; OutputList NULL; for ( AList->firstNoe; NULL; ->nextnoe) for ( begin-esc; ( NULL.StrtPos.StrtPos); ->nextnoe) /* skipping over unmtchble s */ begin-esc ; for ( begin-esc; ( NULL.EnPos.EnPos); ->nextnoe) if ((.StrtPos.StrtPos) (.EnPos.EnPos) [ (.LevelNum.LevelNum 1)]) /* the optionl conition is for prent-chil reltionships */ ppen (,) to OutputList; Figure 3. Algorithm Tree-Merge-Anc with output in sorte ncestor/prent orer 3 Structurl Join Algorithms In this section, we evelop two fmilies of join lgorithms for mtching prent-chil n ncestor-escennt structurl reltionships efficiently: tree-merge n stck-tree, n present n nlysis of these lgorithms. Consier n ncestor-escennt (or, prent-chil) structurl reltionship, for exmple, (book, uthor) (or, (uthor, jne)) in our running exmple. Let AList n DList be the lists of tree noes tht mtch the noe preictes 9 n, respectively, ech list sorte by the (DocI, StrtPos) vlues of its elements. There re number of wys in which the AList n the DList coul be generte from the tbse tht stores the XML t. For exmple, ntive XML tbse system coul store ech element noe in the XML t tree s n object with the ttributes: ElementTg, DocI, StrtPos, EnPos, n LevelNum. An inex coul be built cross ll the element tgs, which coul then be use to fin the set of noes tht mtch given element tg. The set of noes coul then be sorte by (DocI, StrtPos) to prouce the lists tht serve s input to our join lgorithms. Given these two input lists, AList of potentil ncestors (or prents) n DList of potentil escennts (resp., chilren), the lgorithms in ech fmily cn output list OutputList of join results, sorte either by (DocI,.StrtPos,.StrtPos) or by (DocI,.StrtPos,.StrtPos). Both vrints re useful, n the vrint chosen my epen on the orer in which n optimizer chooses to compose the structurl joins to mtch the complex XML query pttern. 3.1 Tree-Merge Join Algorithms The lgorithms in the tree-merge fmily re nturl extension of tritionl reltionl merge joins (which use n equlity join conition) to el with the multiple inequlity conitions tht chrcterize the ncestor-escennt or the prent-chil structurl reltionships, bse on the (DocI, StrtPos : EnPos, LevelNum) representtion. The recently propose multi-preicte merge join (MPMGJN) lgorithm [9] is lso member of this fmily. The bsic ie here is to perform moifie merge-join, possibly performing multiple scns through the inner join opern to the extent necessry. Either AList or DList cn be use s the inner (resp., outer) opern for the join: the results re prouce sorte (primrily) by the outer opern. In Figure 3, we present the tree-merge lgorithm for the cse when the outer join opern is the ncestor; this is similr to the MPMGJN lgorithm. Similrly, Figure 4 els with the cse when the outer join opern is the escennt. For ese of unerstning, both lgorithms ssume tht ll noes in the two lists hve the sme vlue of DocI, their primry sort ttribute. Deling with noes from multiple ocuments is strightforwr, requiring the comprison of DocI vlues n the vncement of noe pointers s in the tritionl merge join An Anlysis of the Tree-Merge Algorithms Tritionl merge joins tht use single equlity conition between two ttributes s the join preicte cn be shown to hve time n spce complexities ) +!"#, on sorte inputs, while proucing sorte output. In generl, one cnnot estblish the sme time complexity when the join preicte involves multiple equlity n/or inequlity conitions. In this section, we ientify the criteri uner which tree-merge lgorithms hve symptoticlly optiml time complexity. Algorithm Tree-Merge-Anc for ncestor-escennt Structurl Reltionship: Theorem 3.1 The spce n time complexities of Algorithm Tree-Merge-Anc re ) $%&(') + *%&(') + +-,)./,)%&')0, for the ncestor-escennt structurl reltionship. The intuition is s follows. Consier first the cse where no two noes in AList re themselves relte by n ncestorescennt reltionship. In this cse, the size of OutputList is ) $%&(') + *%&('). Algorithm Tree-Merge-Anc mkes

5 Algorithm Tree-Merge-Desc (AList, DList) /* Assume tht ll noes in AList n DList hve the sme DocI */ /* AList is the list of potentil ncestors, in sorte orer of StrtPos */ /* DList is the list of potentil escennts in sorte orer of StrtPos */ begin-nc AList->firstNoe; OutputList NULL; for ( DList->firstNoe; NULL; ->nextnoe) for ( begin-nc; ( NULL.EnPos.StrtPos); ->nextnoe) /* skipping over unmtchble s */ begin-nc ; for ( begin-nc; ( NULL.StrtPos.StrtPos); ->nextnoe) if ((.StrtPos.StrtPos) (.EnPos.EnPos) [ (.LevelNum.LevelNum 1)]) /* the optionl conition is for prent-chil reltionships */ ppen (,) to OutputList; Figure 4. Algorithm Tree-Merge-Desc with output in sorte escennt/chil orer single pss over the input AList n t most two psses over the input DList. Thus, the bove theorem is stisfie in this cse. Consier next the cse where multiple noes in AList re themselves relte by n ncestor-escennt reltionship. This cn hppen, for exmple, in the (section, he) structurl reltionship for the XML t in Figure 1. In this cse, multiple psses my be me over the sme set of escennt noes in DList, n the size of OutputList my be ) $%&(') *%&(')/, which is qurtic in the size of the input lists. However, we cn show tht the lgorithm still hs optiml time complexity, i.e., ) $%&(') + *%&(') + +-,)./,)%&')0. One cnnot estblish the I/O optimlity of Algorithm Tree-Merge-Anc. In fct, repete pging cn cuse its I/O behvior to egre in prctice, s we shll see in Section 4. Algorithm Tree-Merge-Anc for prent-chil Structurl Reltionship: When evluting prentchil structurl reltionship, the time complexity of Algorithm Tree-Merge-Anc is the sme s if one were performing n ncestor-escennt structurl reltionship mtch between the sme two input lists. However, the size of OutputList for the prent-chil structurl reltionship cn be much smller thn the size of the OutputList for the ncestor-escennt structurl reltionship. In prticulr, consier the cse when ll the noes in AList form (long) chin of length, n ech noe in AList hs two chilren in DList, one on either sie of its chil in AList; this is shown in Figure 5(). In this cse, it is esy to verify tht the size of OutputList is ) $%&(') + *%&(')/, but the time complexity of Algorithm Tree-Merge-Anc is )/ $%&(') + *%&(')/ ; the evlution is pictorilly epicte in Figure 5(b), where ech noe in AList is ssocite with the sublist of DList tht nees to be scnne. The I/O complexity is lso qurtic in the input size in this cse. Algorithm Tree-Merge-Desc: There is no nlog to Theorem 3.1 for Algorithm Tree-Merge-Desc, since the time A clever implementtion tht uses one noe lookhe in AList cn reuce the number of psses over DList to just one. complexity of the lgorithm cn be )/ $%&(') + *%&(') + +-,)./,)%&('-)/ in the worst cse. This hppens, for exmple, in the cse shown in Figure 5(c), when the first noe in AList is n ncestor of ech noe in DList. In this cse, ech noe in DList hs only two ncestors in AList, so the size of OutputList is ) $%&(') + *%&('), but AList is repetely scnne, resulting in time complexity of ) $%&(') *%&(') ; the evlution is epicte in Figure 5(), where ech noe in DList is ssocite with the sublist of AList tht nees to be scnne. While the worst cse behvior of mny members of the treemerge fmily is quite b, on some t sets n queries they perform quite well in prctice. We shll investigte the behvior of Algorithms Tree-Merge-Anc n Tree-Merge-Desc experimentlly in Section Stck-Tree Join Algorithms We observe tht epth-first trversl of tree cn be performe in liner time using stck of size s lrge s the height of the tree. In the course of this trversl, every ncestor-escennt reltionship in the tree is mnifeste by the escennt noe ppering somewhere higher on the stck thn the ncestor noe. We use this observtion to motivte our serch for fmily of stckbse structurl join lgorithms, with better worst-cse I/O n CPU complexity thn the tree-merge fmily, for both prent-chil n ncestor-escennt structurl reltionships. Unfortuntely, the epth-first trversl ie, even though ppeling t first glnce, cnnot be use irectly since it requires trversl of the whole tbse. We woul like to trverse only the cnite noes provie to us s prt of the input lists. We now escribe our stck-tree fmily of structurl join lgorithms; these lgorithms hve no counterprt in tritionl join processing Stck-Tree-Desc Consier n ncestor-escennt structurl reltionship. Let AList n DList be the lists of tree noes tht mtch noe preictes n, respectively, sorte by the (DocI, StrtPos) vlues of its elements.

6 1 3 n 1 3 n () n n-1 n- n+1 AList 1 3 n (b) DList 1 3 n n+1 n- n-1 n n 3 n (c) DList Figure 5. (), (b) Worst cse for Tree-Merge-Anc n (c), () Worst cse for Tree-Merge-Desc 1 3 n () AList n We first iscuss the stck-tree lgorithm for the cse when the output list is sorte by (DocI,.StrtPos,.StrtPos). This is both simpler to unerstn n extremely efficient in prctice. The lgorithm is presente in Figure 6 for the ncestor-escennt cse. The bsic ie is to tke the two input opern lists, AList n DList, both sorte on their (DocI, StrtPos) vlues n conceptully merge (interleve) them. As the merge procees, we etermine the ncestor-escennt reltionship, if ny, between the current top of stck n the next noe in the merge, i.e., the noe with the smllest vlue of StrtPos. Bse on this comprison, we mnipulte the stck, n prouce output. The stck t ll times hs sequence of ncestor noes, ech noe in the stck being escennt of the noe below it. When new noe from the AList is foun to be escennt of the current top of stck, it is simply pushe on to the stck. When new noe from the DList is foun to be escennt of the current top of stck, we know tht it is escennt of ll the noes in the stck. Also, it is gurntee tht it won t be escennt of ny other noe in AList. Hence, the join results involving this DList noe with ech of the AList noes in the stck re output. If the new noe in the merge list is not escennt of the current top of stck, then we re gurntee tht no future noe in the merge list is escennt of the current top of stck, so we my pop stck, n repet our test with the new top of stck. No output is generte when ny element in the stck is poppe. The prent-chil cse of Algorithm Stck-Tree-Desc is even simpler since DList noe cn join only (if t ll) with the top noe on the stck. In this cse, the for loop insie the else cse of Figure 6 nees to be replce with: if (.LevelNum stck->top.levelnum + 1) ppen (stck->top,) to OutputList Exmple 3.1 [Algorithm Stck-Tree-Desc] Some steps uring n exmple evlution of Algorithm Stck-Tree-Desc, for prent-chil structurl reltionship, on the tset of Figure 7(), re shown in Figures 7(b) (e). The 0 s re the noes in AList n the s re the noes in DList. Initilly, the stck is empty, n the conceptul merge of AList n DList is shown in Figure 7(b). In Figure 7(c), hs been put on the stck, n the first new element of the merge list,, is compre with the stck top (t this point is output). Figure 7() illustrtes the stte of the execution severl steps lter, when re ll on the stck, n is being compre with the stck top (fter this point, the OutputList inclues < ). Finlly, Figure 7(e) shows the stte of the execution when the entire input hs lmost been processe. Only remins on the stck (ll the other s hve been poppe from the stck), n is compre with. Note tht ll the esire mtches hve been prouce while mking only single pss through the entire input. Recll tht this is the sme tset of Figure 5(), which illustrte the sub-optimlity of Algorithm Tree-Merge-Anc, for the cse of prent-chil structurl reltionships. 3.. Stck-Tree-Anc We next iscuss the stck-tree lgorithm for the cse when the output list nees to be sorte by (DocI,.StrtPos,.StrtPos). It is not strightforwr to moify Algorithm Stck-Tree-Desc to prouce results sorte by ncestor becuse of the following: if noe from AList on the stck is foun to be n ncestor of some noe in the DList, then every noe from AList tht is n ncestor of (n hence below on the stck) is lso n ncestor of. Since the StrtPos of precees the strt position of, we must ely output of the join pir until fter hs been output. But there remins the possibility of new element fter in the DList joining with s long is on stck, so we cnnot output the pir until the ncestor noe is poppe from stck. Menwhile, we cn buil up lrge join results tht cnnot yet be

7 Algorithm Stck-Tree-Desc (AList, DList) /* Assume tht ll noes in AList n DList hve the sme DocI */ /* AList is the list of potentil ncestors, in sorte orer of StrtPos */ /* DList is the list of potentil escennts in sorte orer of StrtPos */ AList->firstNoe; DList->firstNoe; OutputList NULL; while (the input lists re not empty or the stck is not empty) if ((.StrtPos stck->top.enpos) (.StrtPos stck->top.enpos)) /* time to pop the top element in the stck */ tuple stck->pop(); else if (.StrtPos.StrtPos) stck->push() ->nextnoe else for (1 stck->bottom; 1 NULL; 1 1->up) ppen (1,) to OutputList ->nextnoe Figure 6. Algorithm Stck-Tree-Desc with output in sorte escennt orer output. Our solution to this problem is escribe in Figure 8 for the ncestor-escennt cse. As with Algorithm Stck-Tree-Desc, the stck t ll times hs sequence of ncestor noes, ech noe in the stck being escennt of the noe below it. Now, we ssocite two lists with ech noe on the stck: the first, clle self-list is list of result elements from the join of this noe with pproprite DList elements; the secon, clle inherit-list is list of join results involving AList elements tht were escennts of the current noe on the stck. As before, when new noe from the AList is foun to be escennt of the current top of stck, it is simply pushe on to the stck. When new noe from the DList is foun to be escennt of the current top of stck, it is simply e to the self-lists of the noes in the stck. Agin, s before, if no new noe (from either list) is escennt of the current top of stck, then we re gurntee tht no future noe in the merge list is escennt of the current top of stck, so we my pop stck, n repet our test with the new top of stck. When the bottom element in stck is poppe, we output its self-list first n then its inherit-list. When ny other element in stck is poppe, no output is generte. Inste, we ppen its inherit-list to its self-list, n ppen the result to the inherit-list of the new top of stck. An optimiztion to the lgorithm (incorporte in Figure 8) is s follows: no self-list is mintine for the bottom noe in the stck. Inste, join results with the bottom of the stck re output immeitely. This results in smll spce svings, n reners the stck-tree lgorithm prtilly non-blocking An Anlysis of Algorithm Stck-Tree-Desc Algorithm Stck-Tree-Desc is esy to nlyze. Ech AList element in the input my be exmine multiple times, but these cn be mortize to the element on DList, or the element t the top of stck, ginst which it is exmine. Ech element on the stck is poppe t most once, n when poppe, cuses exmintion of the new top of stck with the current new element. Finlly, when DList element is compre ginst the top element in stck, then it either joins with ll elements on stck or none of them; ll join results re immeitely output. In other wors, the time require for this prt is irectly proportionl to the output size. Thus, the time require for this lgorithm is ) "# + /! # in the worst cse. Putting ll this together, we get the following result: Theorem 3. The spce n time complexities of Algorithm Stck-Tree-Desc re ) $%&(') + *%&(') + +-,)./,)%&')0, for both ncestor-escennt n prent-chil structurl reltionships. Further, Algorithm Stck-Tree-Desc is non-blocking lgorithm. Clerly, no competing join lgorithm tht hs the sme input lists, n is require to compute the sme output list, coul hve better symptotic complexity. The I/O complexity nlysis is strightforwr s well. Ech pge of the input lists is re once, n the result is output s soon s it is compute. Since the mximum size of stck is proportionl to the height of the XML tbse tree, it is quite resonble to ssume tht ll of stck fits in memory t ll time. Hence, we hve the following result: Theorem 3.3 The I/O complexity of Algorithm Stck-Tree-Desc is ) + +, for ncestor-escennt n prent-chil structurl reltionships, where is the blocking fctor An Anlysis of Algorithm Stck-Tree-Anc The key ifference between the nlyses of Algorithms Stck-Tree-Anc n Stck-Tree-Desc is tht join results re ssocite with noes in the stck in Algorithm Stck-Tree-Anc. Obviously, the list of join results t ny noe in the stck is liner in the output size. Wht remins to be nlyze is the ppening of lists ech time the stck is poppe.

8 n n-1 n- 1 1 n n n+1 1 n n n+1 n n n+1 n n n+1 n-1 n 1 n-1 n 1 n-1 n 1 n () (b) (c) () (e) Figure 7. () Dtset (b) (e) Steps uring evlution of Stck-Tree-Desc If the lists re implemente s linke lists (with strt n en pointers), these ppen opertions cn be crrie out in unit time, n require no copying. Thus one comprison per AList input n one per output re ll tht re performe to mnipulte stck. Combine with the nlysis of Algorithm Stck-Tree-Desc, we cn see tht the time require for this lgorithm is still ) + /! # in the worst cse. The I/O complexity nlysis is little more involve. Certinly, one cnnot ssume tht ll the lists of results not yet output fit in memory. Creful buffer mngement is require. It turns out tht the only opertion we ever perform on list is to ppen to it (except for the finl re out). As such, we only nee to hve ccess to the til of ech list in memory s computtion procees. The rest of the list cn be pge out. When list is ppene to list, it is not necessry tht the he of list be in memory, the ppen opertion only estblishes link to this he in the til of. So ll we nee is to know the pointer for the he of ech list, even if it is pge out. Ech list pge is thus pge out t most once, n pge bck in gin only when the list is rey for output. Since the totl number of entries in the lists is exctly equl to the number of entries in the output, we thus hve tht the I/O require on ccount of mintining lists of results is proportionl to the size of output (provie tht there is enough memory to hol in buffer the til of ech list: requiring two pges of memory per stck entry still requirement within reson). All other I/O ctivity is for the input n output. This les to the esire linerity result. Theorem 3.4 The spce n time complexities of Algorithm Stck-Tree-Anc re ) $%&(') + *%&(')/ + +-,)./,)%&(')0, for both ncestor-escennt n prent-chil structurl reltionships. The I/O complexity of Algorithm Stck-Tree-Anc is ) + +, for both ncestor-escennt n prent-chil structurl reltionships, where is the blocking fctor. 4 Experimentl Evlution In this section, we present the results of n ctul implementtion of the vrious join lgorithms for XML t sets. Due to spce limittions, we evlute only the structurl join lgorithms we introuce in this pper, nmely, TREE-MERGE JOIN(TMJ) n STACK-TREE JOIN (STJ). Once more, the output cn be sorte in two wys, bse on the ncestor noe or the escennt noe in the join. Corresponingly, we consier two flvors of these lgorithms, n use the suffix -A n -D to ifferentite between these. The four lgorithms re thus lbele: TMJ-A, TMJ-D, STJ- A n STJ-D. For resons of spce, we omit etile comprison of our structurl join lgorithms with trversl-style lgorithms, n with tritionl reltionl join lgorithms in commercil tbse. As expecte, the performnce of the trversl-style lgorithms egres consierbly with the size of the tset, n yiels very poor performnce compre with our structurl join lgorithms. Also, consistent with the results of [9], structurl join lgorithms (implemente outsie the tbse) perform significntly better thn ntive reltionl DBMS join lgorithms, even in the presence of inexes. 4.1 Experimentl Testbe We implemente the join lgorithms in the TIMBER XML query engine. TIMBER is n ntive XML query engine tht is built on top of SHORE [5]. Since the gol of TIMBER is to efficiently hnle complex XML queries on lrge t sets, we implemente our lgorithms so tht they coul prticipte in complex query evlution plns with pipelining. All experiments using TIMBER were run on 500MHz Intel Pentium III processor running WinowsNT Worksttion v4.0. SHORE ws compile for 8KB pge size. SHORE buffer pool size ws set to 3MB, n the continer size in our implementtion ws 8000 bytes. All numbers presente here re prouce by running the experiments multiple times n verging ll the execution times except for the first run (i.e., these re wrm cche numbers).

9 Algorithm Stck-Tree-Anc (AList, DList) /* Assume tht ll noes in AList n DList hve the sme DocI */ /* AList is the list of potentil ncestors, in sorte orer of StrtPos */ /* DList is the list of potentil escennts in sorte orer of StrtPos */ AList->firstNoe; DList->firstNoe; OutputList NULL; while (the input lists re not empty or the stck is not empty) if ((.StrtPos stck->top.enpos) (.StrtPos stck->top.enpos)) /* time to pop the top element in the stck */ tuple stck->pop(); if (stck->size 0) /* we just poppe the bottom element */ ppen tuple.inherit-list to OutputList else ppen tuple.inherit-list to tuple.self-list ppen the resulting tuple.self-list to stck->top.inherit-list else if (.StrtPos.StrtPos) stck->push() ->nextnoe else for (1 stck->bottom; 1 NULL; 1 1->up) if (1 stck->bottom) ppen (1,) to OutputList else ppen (1,) to the self-list of 1 ->nextnoe Figure 8. Algorithm Stck-Tree-Anc with output in sorte ncestor orer 4. Worklo For our worklo, we use the IBM XML t genertor to generte number of t sets, of vrying sizes n other t chrcteristics, such s the fnout (MxRepets) n the mximum epth, using the Orgniztion DTD presente in Figure 9. We lso use the XMch-1 [1] n XMrk [] benchmrks, n some rel XML t. The results obtine were very similr in ll cses, n in the interest of spce we present results only for the lrgest orgniztion t set tht we generte. This t set consists of 6.3 million element noes, corresponing to pproximtely 800MB of XML ocuments in text formt. The chrcteristics of this t set in terms of the number of occurrences of element tgs re summrize in Tble 1. We evlute the vrious join lgorithms using the set of queries shown in Tble 1. The queries re broken up into two clsses. QS1 to QS6 re simple structurl reltionship queries, n hve n equl mix of prent-chil queries n ncestorescennt queries. QC1 n QC re complex chin queries, n re use to emonstrte the performnce of the lgorithms when evluting complex queries with multiple joins in pipeline. 4.3 Detile Implementtion The focus in the experiments is to chrcterize the performnce of the four structurl join lgorithms, n unerstn their ifferences. Before oing so in the following subsections, we present here some itionl etil regring the mnner in which these were implemente for the experiments reporte. Our choice of implementtion, on top of SHORE n TIMBER, ws riven purely by the nee for sufficient control the lgorithms themselves coul just s well hve been implemente on mny other pltforms, incluing (s new join methos) on reltionl tbses. All join lgorithms were implemente using the opertor itertor moel [15]. In this moel, ech opertor provies n open, next n close interfce to other opertors, n llows the tbse engine to construct n opertor tree with n rbitrry mix of query opertions (ifferent join lgorithms or lgorithms for other opertions such s ggregtion) n nturlly llows for pipeline opertor evlution. To support this itertor moel, we py creful ttention to the mnner in which results re psse from one opertor to nother. Algorithms such s the TMJ lgorithms my nee to repetely scn over one of the inputs. Such repete scns re fesible if the input to TMJ opertor is strem from isk file, but is not fesible if the input strem origintes from nother join opertor (in the pipeline below it). We implemente the TMJ lgorithms so tht the noes in current sweep re store in temporry SHORE file. On the next sweep, this temporry SHORE file is scnne. This solution llows us to limit the memory use by TMJ implementtion, s the only memory use is mnge by the SHORE buffer mnger, which tkes cre of evicting pges of the temporry file from the buffer pool if require. Similrly for the STJ-A lgorithm, the inherit- n self-lists re store in temporry SHORE file, gin limiting the memory use by the lgorithm. In both cses, our implementtion turns logging n locking off for the temporry SHORE files. Note tht STJ-D cn join the two inputs in single pss over both inputs, n, never hs to spool ny noes to temporry file.

10 <!ELEMENT mnger (nme,(mnger eprtment employee)+)> <!ATTLIST mnger i CDATA #FIXED "1"> <!ELEMENT eprtment (nme, emil?, employee+, eprtment*)> <!ATTLIST eprtment i CDATA #FIXED ""> <!ELEMENT employee (nme+,emil?)> <!ATTLIST employee i CDATA #FIXED "3"> <!ELEMENT nme (#PCDATA)> <!ATTLIST nme i CDATA #FIXED "4"> <!ELEMENT emil (#PCDATA)> <!ATTLIST emil i CDATA #FIXED "5"> Figure 9. DTD use in our experiments Noe Count mnger 5,880 eprtment 34,450 employee 574,530 emil 50,530 Query XQuery Pth Expression Result Crinlity QS1 employee/emil 140,700 QS employee//emil 14,958 QS3 mnger/eprtment 16,855 QS4 mnger//eprtment 587,137 QS5 mnger/employee 17,59 QS6 mnger//employee 990,774 QC1 mnger/employee/emil 7,990 QC mnger//employee/emil 3,406 Tble 1. Description of queries n chrcteristics of the t set To mortize the storge n ccess overhe ssocite with ech SHORE object, in our implementtion we group noes into lrge continer object, n crete SHORE object for ech continer. The join lgorithms write noes to continers n when continer is full it is written to the temporry SHORE file s SHORE recor. The performnce benefits of this pproch re substntil; we o not go into etils for lck of spce. 4.4 STJ n TMJ, Simple Structurl Join Queries Here, we compre the performnce of the STJ n the TMJ lgorithms using ll the six simple queries, QS1 QS6, shown in Tble 1. Figure 10 plots the performnce of the four lgorithms. As shown in the Figure, STJ-D outperforms the remining lgorithms in ll cses. The reson for the superior performnce of STJ-D is becuse of its bility to join the two t sets in single pss over the input noes, n it never hs to write ny noes to intermeite files on isk. From Figure 10, we cn lso see tht STJ-A usully hs better performnce thn both TMJ-A n TMJ-D. For queries QS4 n QS6, the STJ-A lgorithms n the two TMJ lgorithms hve comprble performnce. These queries hve lrge result sizes (pproximtely 600K n 1M tuples respectively s shown in Tble 1). Since STJ-A keeps the results in the lists ssocite with the stck, n cn output the results only when the bottommost element of the stck is poppe, it hs to perform mny writes n trnsfers of the lists ssocite with the stck elements (in our implementtion, these lists re mintine in temporry SHORE files). With lrger result sizes this list mngement slows own the performnce of STJ-A in prctice. Figure 10 lso shows tht the two TMJ lgorithms hve comprble performnce. We lso rn these experiments with reuce buffer sizes, but foun tht for this t set the execution time of ll the lgorithms remine firly constnt. Even though the XML t sets tht we use re lrge, fter pplying the preictes, the cnite lists tht we join re not very lrge. Furthermore, the effect of buffer pool size is likely to be criticl when one of the inputs hs noes tht re eeply neste mongst themselves, n the noe tht is higher up in the XML tree hs mny noes tht it joins with. For exmple, consier the TMJ-A lgorithms, n the query mnger/employee. If mny mnger noes re neste below mnger noe tht is higher up in the XML tree, then fter the join of the mnger noe t the top is one, repete scns of the escennt noes will be require for the mnger noes tht re escennts of the mnger noe t the top. Such scenrios re rre in our t set, n, consequently, the buffer pool size hs only mrginl impct on the performnce of the lgorithms. 4.5 Complex Queries Here, we evlute the performnce of the lgorithms using the two complex chin queries, QC1 n QC, from Tble 1. Ech query hs two joins n for this experiment, both join opertions re evlute in pipeline. For ech complex query one cn evlute the query by using only ncestor-bse join lgorithms or using only escennt-bse join lgorithms. These two pproches re lbele with suffixes -A n -D for the ncestor-bse n escennt-bsepproches respectively. The performnce comprison of the STJ n TMJ lgorithms for both query evlution pproches (A n D) is shown in Figure 11. From the figure we see tht STJ-D hs the highest performnce once gin, since it is never hs to spool noes to intermeite files.

11 Response Time (in secons) STJ-D STJ-A TMJ-D TMJ-A QS1 QS QS3 QS4 QS5 QS6 Figure 10. STJ n TMJ, simple queries: QS1 QS6 Response Time (in secons) QC1 STJ-D STJ-A TMJ-D TMJ-A QC Figure 11. STJ n TMJ, complex queries: QC1, QC 5 Relte Work Mtchings between pirs of trees in memory hs been topic of stuy in the lgorithms community for long time (e.g., see [3] n references therein). The lgorithms evelope el with mny vritions of the problem but unfortuntely re of high complexity n lwys ssume tht trees re entirely memory resient. The problem lso hs been consiere in the progrmming lnguge community, s it rises in vrious type checking scenrios but once gin solutions evelope re gere towrs smll t collections processe entirely in min memory. Mny lgorithms re known to be very efficient over tree structures. Most relevnt to us in this literture re lgorithms for checking the presence of sets of eges n pths. Jcobson et l. [16] present liner time merging-style lgorithms for computing the elements of list tht re escennts/ncestors of some elements in secon list, in the context of focusing keywor-bse serches on the Web n in UNIX-style file systems. Jgish et l. [17] present liner time stck-bse lgorithms for computing elements of list tht stisfy hierrchicl ggregte selection conition wrt elements in secon list, for the irectory t moel. However, none of these lgorithms compute join results, which is the focus of our work. Join processing is centrl to tbse implementtion n there is vst mount of work in this re [15]. For inequlity join conitions, bn join [11] lgorithms re pplicble when there exists fixe rithmetic ifference between the vlues of join ttributes. Such lgorithms re not pplicble in our omin s there is no notion of fixe rithmetic ifference. In the context of sptil n multimei tbses, the problem of computing joins between pirs of sptil entities hs been consiere, where commonly the preicte of interest is overlp between sptil entities [18, 4, 19] in multiple imensions. The techniques evelope in this pper re relte to such join opertions. However, the preictes consiere s well s the techniques we evelop re specil to the nture of our structurl join problem. In the context of semistructure n XML tbses, the issue of query evlution n optimiztion hs ttrcte lot of reserch ttention. In prticulr, work one in the context of the Lore tbse mngement system [0, 1], n the Nigr system [3], hs consiere vrious spects of query processing on such t. XML t n vrious issues in their storge s well s query processing using reltionl tbse systems hve recently been consiere in [14, 7, 6, 4, 1, 13]. In [14, 7, 13], the mpping of XML t to number of reltions ws consiere long with trnsltion of select subset of XML queries to reltionl queries. In subsequent work [6, 4, 1], the uthors consiere the problem of publishing XML ocuments from reltionl tbses. Our work is complementry to ll of these since our focus is on the join lgorithms for the primitive (ncestor-escennt n prentchil) structurl reltionships. Our join lgorithms cn be use by these previous works to vntge. The representtion of positions of XML elements use by us, (DocI, StrtPos : EnPos, LevelNum), is essentilly tht of Consens n Milo, who consiere frgment of the PAT text serching opertors for inexing text tbses [8, 9], n iscusse optimiztion techniques for the lgebr. This representtion ws use to compute continment reltionships between text regions in the text tbses. The focus of tht work ws solely on theoreticl issues, without elborting on efficient lgorithms for computing these reltionships. Finlly, the recent work of Zhng et l. [9] is closely relte to ours. They propose the multi preicte merge join (MP- MGJN) lgorithm for evluting continment queries, using the (DocI, StrtPos : EnPos, LevelNum) representtion. The MPMGJN lgorithm is member of our Tree-Merge

12 fmily. Our nlyticl n experimentl results emonstrte tht the Stck-Tree fmily is consierbly superior to the Tree-Merge fmily for evluting continment queries. 6 Conclusions In this pper, our focus hs been the evelopment of new join lgorithms for eling with core opertion centrl to much of XML query processing, both for ntive XML query processor implementtions s well for reltionl XML query processors. In prticulr, the Stck-Tree fmily of structurl join lgorithms ws shown to be both I/O n CPU optiml, n prcticlly efficient. There is gret el more to efficient XML query processing thn is within the scope of this pper. For exmple, XML permits links cross ocuments, n such pointer-bse joins re frequently useful in querying. We o not consier such joins in this pper, since we believe tht they cn be equtely resse using tritionl vlue-bse join methos. There re mny issues yet to be explore, n we currently hve initite efforts to ress some of these, incluing the piecing together of structurl joins n vlue-bse joins to buil effective query plns. Acknowlegements We woul like to thnk Chun Zhng for her helpful comments on n erly version of this pper. References [1] XMch-1. Avilble from [] The XML benchmrk project. Avilble from [3] A. Apostolico n Z. Glil. Pttern mtching lgorithms. Oxfor University Press, [4] M. Crey, J. Kiernn, J. Shnmugsunrm, E. Shekit, n S. Subrmnin. XPERANTO: Milewre for publishing object reltionl t s XML ocuments. Proceeings of VLDB, 000. [5] M. J. Crey, D. J. DeWitt, M. J. Frnklin, N. E. Hll, M. L. McAuliffe, J. F. Nughton, D. T. Schuh, M. H. Solomon, C. K. Tn, O. G. Tstlos, S. J. White, n M. J. Zwilling. Shoring up persistent pplictions. In Proceeings of SIG- MOD, [6] D. Chmberlin, D. Florescu, J. Robie, J. Simeon, n M. Stefnescu. XQuery: A query lnguge for XML. W3C Working Drft. Avilble from Feb [7] D. D. Chmberlin, J. Robie, n D. Florescu. Quilt: An XML query lnguge for heterogeneous t sources. In Proceeings of WebDB, 000. [8] M. P. Consens n T. Milo. Optimizing queries on files. In Proceeings of SIGMOD, [9] M. P. Consens n T. Milo. Algebrs for querying text regions. In Proceeings of PODS, [10] A. Deutsch, M. Fernnez, D. Florescu, A. Levy, n D. Suciu. XML-QL: A query lnguge for XML. Submission to the Worl Wie Web Consortium 19-August Avilble from [11] D. DeWitt, J. Nughton, n D. Schneier. An evlution of non equijoin lgorithms. Proceeings of SIGMOD, [1] M. Fernnez n D. Suciu. SilkRoute: Tring between reltions n XML. WWW9, 000. [13] T. Fiebig n G. Moerkotte. Evluting queries on structure with ccess support reltions. Proceeings of WebDB, 000. [14] D. Florescu n D. Kossmn. Storing n querying XML t using n RDMBS. IEEE Dt Engineering Bulletin, (3):7 34, [15] G. Grefe. Query evlution techniques for lrge tbses. ACM Computing Surveys, 5(), [16] G. Jcobson, B. Krishnmurthy, D. Srivstv, n D. Suciu. Focusing serch in hierrchicl structures with irectory sets. In Proceeings of CIKM, [17] H. V. Jgish, L. V. S. Lkshmnn, T. Milo, D. Srivstv, n D. Vist. Querying network irectories. In Proceeings of SIGMOD, [18] N. Kous n K. C. Sevcik. Size seprtion sptil join. Proceeings of SIGMOD, [19] M.-L. Lo n C. V. Rvishnkr. Sptil hsh-joins. Proceeings of SIGMOD, [0] J. McHugh, S. Abiteboul, R. Golmn, D. Quss, n J. Wiom. Lore: A tbse mngement system for semistructure t. SIGMOD Recor, 6(3), [1] J. McHugh n J. Wiom. Query optimiztion for XML. In Proceeings of VLDB, [] U. of Wshington. The Tukwil system. Avilble from [3] U. of Wisconsin. The Nigr system. Avilble from [4] J. M. Ptel n D. J. DeWitt. Prtition bse sptil-merge join. Proceeings of SIGMOD, [5] G. Slton n M. J. McGill. Introuction to moern informtion retrievl. McGrw-Hill, New York, [6] J. Shnmugsunrm, E. J. Shekit, R. Brr, M. J. Crey, B. G. Linsy, H. Pirhesh, n B. Reinwl. Efficiently publishing reltionl t s XML ocuments. In Proceeings of VLDB, 000. [7] J. Shnmugsunrm, K. Tufte, C. Zhng, G. He, D. J. De- Witt, n J. F. Nughton. Reltionl tbses for querying XML ocuments: Limittions n opportunities. In Proceeings of VLDB, [8] E. Shekit n M. Crey. A performnce evlution of pointer bse joins. Proceeings of SIGMOD, [9] C. Zhng, J. Nughton, D. Dewitt, Q. Luo, n G. Lohmn. On supporting continment queries in reltionl tbse mngement systems. In Proceeings of SIGMOD, 001.