Using Partial Evaluation in Distributed Query Evaluation

Transcription

1 A X x Z z R r y Y B Uing Partial Evaluation in Ditributed Query Evaluation Peter Buneman Gao Cong Univerity of Edinburgh Wenfei Fan Univerity of Edinburgh & Bell Laboratorie Anataio Kementietidi Univerity of Edinburgh {opb, gcong, wenfei, akement}@inf.ed.ac.uk ABSTRACT A baic idea in parallel query proceing i that one i prepared to do more computation than trictly neceary at individual ite in order to reduce the elaped time, the network traffic, or both in the evaluation of the query. We develop thi idea for the evaluation of boolean XPath querie over a tree that i fragmented, both horizontally and vertically over a number of ite. The key idea i to end the whole query to each ite which partially evaluate, in parallel, the query and end the reult a compact boolean function to a coordinator which combine thee to obtain the reult. Thi approach ha everal advantage. Firt, each ite i viited only once, even if everal fragment of the tree are tored at that ite. Second, no prior contraint on how the tree i decompoed are needed, nor i any tructural information about the tree required, uch a a dtd. Third, there i a atifactory bound on the total computation performed on all ite and on the total network traffic. We alo develop a imple incremental maintenance algorithm that require communication only with the ite at which change have taken place; moreover the network traffic depend neither on the data nor on the update. Thee reult, we believe, illutrate the uefulne and potential of partial evaluation in ditributed ytem a well a centralized xml tore for evaluating XPath querie and beyond. 1. INTRODUCTION Partial evaluation (aka. program pecialization [17]) ha been tudied in the context of programming language a a general optimization technique. Intuitively, given a function f(, d) and part of it input, partial evaluation i to pecialize f(, d) with repect to the known input. That i, it perform the part of f computation that depend only on, and generate a partial anwer, i.e., a (reidual) function f that depend on the a yet unavailable input d. Partial evaluation ha been proven ueful in a variety of area including compiler generation, code optimization and dataflow evaluation (ee [17] for a urvey). The lat of thee Supported in part by EPSRC GR/S35/1, GR/T733/1 and BBSRC BB/D73/1. Permiion to copy without fee all or part of thi material i granted provided that the copie are not made or ditributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and it date appear, and notice i given that copying i by permiion of the Very Large Data Bae Endowment. To copy otherwie, or to republih, to pot on erver or to reditribute to lit, require a fee and/or pecial permiion from the publiher, ACM. VLDB, September 1-15,, Seoul, Korea. Copyright VLDB Endowment, ACM //9. (a)tree T b r o k e r F M e r i l l L y n c h N A S D A Q c o d e b u y e l l c o d e b u y e l l G O O G $ 3 7 $ Y H O O $ 3 3 $ 3 5 F p o r t o f o l i o b r o k e r F 3 c h e B a N A S D A Q N Y S E c o d e b u y e l l c o d e b u y e l l c o d e b u y e l l G O O G $ 3 7 $ 3 7 I M $ $ 7 A A P L $ 7 1 $ 5 B (b) A tock portfolio Figure 1: Two fragmented xml tree bear ufficient connection with ditributed query evaluation. Thi ugget that it i worth invetigating it ue in parallel query proceing. We focu here on the evaluation of an important fragment of XPath, Boolean XPath querie, which are commonly ued, for example, in publih-ubcribe ytem [] and ldap directorie [3]. A an example, conider the imple Boolean XPath query Q = [//A //B] over the xml tree T depicted in Fig. 1(a). We defer the dicuion of the phyical repreentation of T, but aume that T conit of four ubtree, or fragment, R, X, Y and Z, and that node with tag A or B only occur in fragment Z and Y a hown in Fig. 1(a). A ophiticated algorithm would do a ingle depth-firt traveral of T computing [//A] and [//B] imultaneouly [1], and viit each node exactly once, omething we cannot expect to improve upon. However, our recurive traveral viit fragment R, X, Z, X, R,Y, R in that order; it make three viit to fragment R and two viit to fragment X. Could we devie an algorithm that make one viit to each fragment? Oberve that the evaluation of the query Q can be readily treated a the Boolean function Q(R,X, Y, Z), i.e., one whoe value depend on all four fragment of the tree T. What doe thi function look like? Suppoe that r A, x A, y A and z A are variable denoting the reult of [//A] repectively evaluated at the root r of fragment R, x of fragment X, y of fragment Y and z of fragment Z, and that r B, x B, y B, z B are defined imilarly. Then, it i not hard to ee that: Q(R,X, Y, Z) = (r A x A y A z A) (r B x B y B z B) We can compute the reult of [//A] and [//B] (the value of the variable) independently, and in parallel, in each fragment by acceing it only once. Each fragment return the value of the correponding variable and each time we 11

2 receive uch value we ue them to compute a partial anwer for Q. Given that node with tag A and B only occur in fragment Z and Y, the execution of Q return (z A, z B) = (1, ) on fragment Z, (y A, y B) = (,1) on Y, (x A, x B) = (z A, z B) on X and (r A, r B) = (x A y A, x B x B) on R. Note that the proce make no aumption about the order in which it receive the value of the variable. A partial anwer i computed each time value are received from ome fragment. One order might compute the anwer fater than ome other but except thi, order i of no conequence. Another remark i that ome of the returned value are truth value while other are Boolean expreion. Irrepectively of the order and type of returned value, the proce ue the returned value from all fragment to compute the anwer to Q which, in thi cae, i true. Thi i a imple example of partial evaluation. There are a number of databae cenario in which partial evaluation could be an effective optimization technique. For example, a in PDOM [15], uppoe a large xml tree i tored in econdary torage and it i plit into fragment. In thi etting, the partial evaluation approach can ave u the cot of two wap of fragment R and one wap of fragment X. The benefit i already evident even though there i no parallelim involved in thi example. A more intereting cenario involve fragmented xml tree that are geographically or adminitratively ditributed over the Internet [], a etting commonly found in, e.g., e- commerce, Web ervice, or while managing large-cale network directorie [1]. More pecifically, let u conider the xml tree hown in Fig. 1(b), which repreent a peron tock portfolio. The peron trade tock through variou broker in poibly overlapping market. For each tock, the tree tore it code, the price paid by the peron to buy the tock, and the price at which the peron can (currently) ell the tock. The ame tock might be traded through a different broker at different period of time and for a different price. For example, the goog tock i purchaed both through Merill Lynch and through Bache. Although conceptually thi i a ingle xml tree, in reality it i inherently ditributed over the Internet. The figure ue dahed line to how one poible fragmentation. For example, fragment F include the root of the tree and all the tock data from broker Bache in the nye market. Thi fragment might be tored locally in the peron dektop. Broker Merill Lynch might require that all trade data are acceed through it own erver and thu fragment i tored there. In turn, the nadaq market might require that all it own data are only remotely acceed and only through recognized broker. Therefore fragment F and F 3 are both tored in it own erver. The nye market impoe no uch retriction and o it trade data can be tored locally by Bache (and Merill Lynch, although not hown in the figure). Notice that we make no aumption about the ize of fragment, their torage location, and the number of fragment aigned to each location. In a variety of application, we have no control over thee parameter and their value are impoed on u by the environment. Aume that the portfolio owner want to know whether the goog tock reache a elling price of $37. To do o, he mut execute the boolean query Q = [//tock[code = goog ell = 37]]. There are two popular alternative to execute uch a query. The firt alternative require for the different ource to create a tream of data to the uer and the query i executed over the received tream. There are two main concern with thi approach: (a) A large part of the tree that i later deemed irrelevant to the query, including the ubtree for the nye market and the information for the yhoo and aapl tock, need to be ent to the uer, cauing increaed network traffic. The uer might want to execute the query and be notified on-the-go, uing a mobile device, like a cell phone. Streaming large data et to the cell phone i particularly unrealitic. (b) Buine or peronal data are typically kept local at truted ite, and are not hipped to other ite for ecurity or privacy concern. In indutry and reearch, imilar concern have generated increaing interet, and a preemergent hift, toward hipping the proceing (querie) to the data, intead of hipping (treaming) the data to the proceing [11]. The econd alternative i to ue a publih-ubcribe ytem. Auming that the tree in Fig. 1(b) i part of uch a ytem, the uer need to iue query Q over the ytem and whenever the query predicate i atified, the uer i notified. Publih-ubcribe ytem are more in-line with moving the proceing to the data. However, any publihubcribe ytem implementing the optimal centralized algorithm in XPath query proceing [1] would require a ingle depth-firt traveral of the document tree viiting, in our example, twice the nadaq erver. In repone to thee, we propoe partial evaluation a a practical query proceing technique ince it enure that each remote ite i viited only once. Moreover, while a depth-firt traveral erialize the proceing of the different fragment, partial evaluation can peed-up query proceing ince fragment at different ite are proceed in parallel. Unlike treaming, partial evaluation execute the querie where the data reide, thu minimizing network traffic and facilitating the execution of (complex) querie over device with limited bandwidth. Our firt contribution conit of everal algorithm for evaluating Boolean XPath querie over a fragmented tree, with the following performance guarantee. (a) Each ite holding a fragment i viited only once. (b) The total network traffic i bounded by the ize of the query and the number of fragment, and i independent of the ize of the xml document. (c) The total amount of computation performed at all ite holding a fragment i comparable to the computation of the optimal centralized algorithm over the whole tree. (d) The algorithm doe not impoe any condition on how the xml document are fragmented, what the ize of thee fragment are, or how they are aigned to ite. Our econd contribution i a imple incremental maintenance algorithm for Boolean XPath view. Cached view and their incremental maintenance are important for efficient query anwering [13, 5]. Our algorithm ha the following feature: (a) The computation i localized o that only the ite where the update take place i viited, and reevaluation i only conducted on the updated fragment; no other ite or fragment are viited. (b) The total network traffic depend on neither the data nor the update. Our third contribution i an experimental tudy evaluating our algorithm veru optimal centralized approache. Our experimental reult how that our technique outperform, by a ubtantial factor in all practical cae, exiting centralized algorithm for evaluating XPath querie. Thee yield the firt Boolean XPath evaluation and incre- 1

3 mental maintenance algorithm with performance guarantee in ditributed ytem. The technique generalize to data election XPath querie, a be een in Section ; we conider Boolean querie in thi paper to focu on the main idea of partial evaluation and to implify the dicuion. A mentioned earlier, the partial evaluation technique i more general than thi application. In fact, other propoal for ditributed query evaluation can be een a related to partial evaluation. We hall dicu thee in Section 7. Organization. Section dicue xml tree fragmentation and review Boolean XPath. Section 3 preent our main algorithm for proceing Boolean XPath querie baed on partial evaluation. Variation of the baic algorithm are given in Section. Section 5 develop our incremental maintenance algorithm for XPath view. An experimental tudy i provided in Section, followed by related work in Section 7 and concluion in Section.. BACKGROUND We next dicu the ditribution of xml document and preent the cla of XPath querie tudied in thi paper..1 XML Tree Fragmentation We allow an xml tree T to be decompoed into a collection F of dijoint tree, or fragment F i, which are ditributed to and tored in different ite. A tree may be fragmented for adminitrative reaon (e.g., different ite are inherently reponible for maintaining different part of the tree), or for efficiency [] (e.g., the tree might be too big to tore in a ingle ite). We do not impoe any contraint on the fragmentation: we allow for an arbitrary neting of fragment. Each fragment can appear at any level of the tree, and different fragment may have different ize (in term of number of node). Furthermore, we do not impoe any contraint on how the fragment are ditributed: thi i determined by the ytem. Hence our fragmentation etting i the mot generic poible, making our olution applicable in almot every conceivable etting. For intance, Fig. (a) how the decompoition of the tree in Fig. 1(b) in four fragment. To the left of Fig. (b) we ummarize thi decompoition in a tructure called the fragment tree. Note that the fragment i itelf fragmented. Next, we define a few ueful notation about fragment. The fragment that contain the root of the tree T, i.e., the fragment at the root of the fragment tree, i called the root fragment. In Fig., thi i fragment F. Given two fragment F j and F k, we ay that F k i a ub-fragment of F j if F k i a child of F j in the fragment tree. If F k i a ubfragment of F j then there exit a node v F j uch that the root node w of F k i a child of v in the original tree T. In Fig. (b), fragment F 3 i a ub-fragment of F and in the tree of Fig. 1(b) there i an edge between the broker node of F and the root node market of F 3. We maintain the relationhip between a fragment and it ub-fragment to preerve the tructure of the original tree T. To do thi, we add to the v node of fragment F j a virtual child node with label F k. While travering fragment F j, we know that if we reach the virtual node F k, we need to jump to fragment F k in order to continue the traveral. In Fig. (a), fragment F ha the virtual node which in turn ha the virtual node F. Finally, we refer to a fragment that ha no ub-fragment a a leaf fragment. In Fig. (b), both fragment F and F 3 are leaf fragment. F r a g m e n t F F r a g m e n t F r a g m e n t F F r a g m e n t F 3 p o r t o f o l i o b r o k e r b r o k e r F N A S D A Q c o d e b u y e l l N A S D A Q c o d e b u y e l l M e r i l l F 3 L y n c h G O O G $ 3 7 $ G O O G $ 3 7 $ 3 7 B a c h e t i t l e c o d e b u y e l l c o d e b u y e l l c o d e b u y e l l Y H O O $ 3 3 $ 3 5 A A P L $ 7 1 $ 5 N Y S E I B M $ $ 7 (a) Four fragment F r a g m e n t T r e e S o u r c e T r e e h ( F ) F F F 3 h ( ) h ( F ) h ( F 3 ) (b) The Fragment tree and the ource tree Figure : Tree fragmentation A fragment are ditributed among ite, it i important to keep track of where the different fragment reide. We aume that there i a mapping function h, which encode the aignment of fragment to ite (ource). To the right of Fig. (b) we how a tructure called the ource tree, which i induced from the fragment tree and the function h. The ource tree of a tree T, denoted by S T, how the name of ite where the fragment of T are tored. From the ource tree we can ee that both fragment F and F 3 are tored in the ame ite S, the nadaq ite. The ource tree S T i the only tructure that our XPath evaluation and incremental maintenance algorithm require. No other information about either the fragmentation or the ditribution of the xml tree i needed. The fragment tree wa introduced to illutrate fragmentation, and i neither maintained nor ued by our ytem.. Boolean XPath We conider a cla of Boolean XPath querie, denoted by X BL. A query [q] in X BL i yntactically defined a follow: q := p p/text() = tr label() = A q q q q q, p := ǫ A p//p p/p p[q], where tr i a tring contant, A i a label (tag),,, are the Boolean negation, conjunction and dijunction operator, repectively; p i a path expreion defined in term of the empty path ǫ (elf), label A, wildcard, the decendantor-elf-axi //, child /, and qualifier [q]. For //, we abbreviate p 1/ // a p 1// and // /p a //p. For example, [//broker[//tock/code/text() = goog (//tock/code/text() = yhoo )]] i a query in X BL. Note that path expreion p in X BL ubume tree pattern querie and beyond, which are commonly tudied in the literature. A mentioned earlier, querie in X BL are widely ued in, e.g., xml data diemination for content-baed filtering and routing of xml document. Although we conider Boolean querie, the technique generalize to a larger cla of querie, which are dicued in the concluion. S 1 S S S 13

4 At a context node v in an xml tree T, the evaluation of a query [q] yield a truth value, denoted by val(q, v), indicating whether or not q i atified at v. Specifically, (a) when q i a path p, val(q, v) i true iff there exit a node reachable from v via p; (b) when q i p/text() = tr, val(q, v) i true iff there i a node u reached from v via p uch that u carrie text value tr; imilarly when q i label() = A; (c) when q i q 1 q, val(q, v) i true iff both val(q 1, v) and val(q, v) are true; imilarly when q i q 1 q or q 1. On a centralized xml tree T, i.e., when T i not decompoed and ditributed, val(q, r) can be computed in O( T q ) time [1], where r i the root of T. To implify the preentation we introduce two notation, which will be ued in our algorithm given in later ection. Firt, we rewrite each path p in an X BL query [q] to a normal form β 1/... /β n, where β i i one of ǫ,, // or ǫ[q ]. Thi normalization can be achieved by uing a linear-time function normalize(q), given inductively a follow: normalize(ǫ) = ǫ; imilarly for, // and label() = A; normalize(a) = /ǫ[label() = A]; normalize(p 1 /p ) = normalize(p 1 )/normalize(p ); normalize(p[q ]) = normalize(p)/ǫ[normalize(q )]; normalize(q 1 q ) = normalize(q 1 ) normalize(q ); imilarly for q 1 q and q 1 ; normalize(p/text() = tr ) = normalize(p)[text() = tr ]; normalize(ǫ[q 1 ]/... /ǫ[q n]) = ǫ[normalize(q 1 ) normalize(q )... normalize(q n)]; where the lat rule i to combine a equence of ǫ into one. In the equel we conider X BL querie in the normal form. Second, we ue QLit(q) to denote the lit of all ub-querie of q. Intuitively, q 1 i a ub-query of q if the pare tree of q 1 i a ubtree of that of q. We ort QLit(q) in a topological order uch that for any ub-querie q 1, q of q, if q 1 i a ub-query of q then q 1 precede q in QLit(q). Example.1: Conider the X BL query [q], where q i //tock[code/text() = yhoo ], then normalize([q]) = ǫ[//ǫ[label() = tock /ǫ[label() = code text() = yhoo ]]], QLit([q]) = [q 1, q, q 3, q, q 5, q, q 7, q, q 9, q 1 ], where q 1 = label() = code, q = (text() = yhoo ), q 3 = q 1 q, q = ǫ[q 3 ], q 5 = /ǫ[q ], q = (label() = tock), q 7 = q 5 q, q = ǫ[q 7 ], q 9 = //ǫ[q ], q 1 = ǫ[q 9 ] Remark. Oberve that both normalize(q) and QLit(q) can be computed in O( q ) time. A a reult, the total ize of ub-querie in the lit QLit(q) i bounded by O( q ). Furthermore, for any XPath evaluation algorithm, to evaluate q it i neceary to evaluate ub-querie in QLit(q). 3. DISTRIBUTED QUERY EVALUATION Conider an X BL query q ubmitted to a ite S, hereafter referred to a the coordinating ite. The query i to be evaluated at the root of a fragmented and ditributed xml tree T. A naïve evaluation i to collect all the fragment of tree T identified by the ource tree S T at the coordinating ite, and ue a centralized algorithm, e.g., the algorithm of [1]. We refer to thi approach a NaiveCentralized. Thi approach i efficient once the coordinating ite get all the data. However, the price i that large fragment need to be ent over the network, each time a query need to be executed. In addition, ince the coordinating ite mut tore thee fragment during the evaluation of q, the benefit gained by our ability to ditribute large xml tree over a network are alleviated. Moreover, privacy and ecurity concern may prevent certain ite from releaing their data to another ite. A better olution, referred to a NaiveDitributed, i to cutomize a centralized evaluation algorithm o that it work in a ditributed fahion. We know that a boolean XPath query can be evaluated on a ingle ite via a ingle traveral of the tree T. We can ue the information from the ource tree S T to perform a ditributed bottom-up traveral of tree T. To do thi, we need to pa certain information between the ite in the ource tree S T, a the ditributed computation i paed forth and back from a fragment F i in ite S j to one of it ub-fragment F k in ite S l. For example, conider the fragment and ource tree in Fig. (b). A we compute the query for fragment F in ite S, we need to pa the control of computation to fragment in ite S 1. At the ame time, ite S ha to wait for thi computation to finih before it continue with fragment F 3 in ite S. While thi ditributed algorithm doe not require any tranmiion of fragment, it ha two hortcoming. Firt, for a ite S i to finih proceing it fragment F j, it ha to wait for all the other ite that hold ub-fragment of F j to finih. Therefore, the ditributed algorithm actually follow a equential execution and doe not take advantage of parallelim. Second, a ite i viited a many time a the number of fragment tored in it. In our example, ite S need to be viited twice, ince it hold fragment F and F 3. For each of thee viit, ite S ha to exchange a number of meage, reulting in increaed network traffic, and it proceor ha to witch context once per fragment. To overcome thee limitation, we propoe next the Parallel Boolean XPath (ParBoX) evaluation algorithm, baed on partial evaluation. The ParBoX Algorithm guarantee the following: (1) Each ite i viited only once, irrepectively of the number of fragment tored in it. () Query proceing i performed in parallel, on all the participating ite. (3) The total computation on all ite i comparable to what i needed by the bet-known centralized algorithm. () The total network traffic, in any practical etting, i determined by the ize of the query rather than the xml tree. 3.1 The ParBoX Algorithm The algorithm i initiated at the coordinating ite which, without lo of generality, we aume to be the ite toring the root fragment of the tree T over which the X BL query q i evaluated. The algorithm conit of three tage: Stage 1: Initially (line 1- of Procedure ParBoX in Fig. 3(a)), the coordinating ite ue the ource tree S T to identify which other ite hold at leat one fragment of tree T. In our example, coordinating ite S ue ource tree in Fig. (b) to identify ite S 1 and S. Stage : The coordinating ite along with all the ite identified in the firt tage evaluate, in parallel, the ame input query q on all their aigned fragment (Procedure evalqual, Fig. 3(b)). Since fragment are part of the tree T, query evaluation on each fragment return a partial anwer to the query q. Stage 3: Finally (line 5-7 of Procedure ParBoX), the coordinating ite collect the partial anwer from all the 1

5 participating ite and all the fragment; it then compoe them to compute the anwer to query q. We now decribe the two crucial component of the algorithm: (a) how to compute partial anwer in parallel (the econd tage), and (b) how to aemble the partial anwer to obtain the anwer to query q (the third tage). Partial evaluation. There i a dependency relation between partial evaluation procee for the query q on different fragment of the xml tree T. To ee thi, conider an efficient evaluation of q over T via a ingle bottom-up traveral of T. During the traveral, at each node v we compute the value at v of all the ub-querie QLit(q) of query q, where QLit(q) i decribed in Section.. Thi computation require the (already computed) value of the QLit(q) ub-querie at the children of v. At the end of the traveral, the anwer to query q i computed by the value of the QLit(q) querie at the root of the tree. Specifically, the anwer to q i the value of the lat query in QLit(q). Conider now Fig. (a) which how the fragment of the xml tree in Fig. 1(b). Thee are the tree over which the ite mut compute the query q. Recall that in thee fragment ome of the leave are virtual node, i.e., they are pointer to other fragment which reide in other ite. For example, in fragment there i a virtual leaf node marked by F, while fragment F ha two virtual leave, one for fragment and one for F 3. In accordance to the trategy given above, at each ite S and for each fragment F, we need to perform a bottom-up evaluation of query q. But how can we compute, during the traveral, the value of the QLit(q) ub-querie at the virtual node? The value of the QLit(q) ub-querie are unknown for thee node and, under normal circumtance, until we learn thee value from another ite we cannot proceed with the evaluation. We propoe a technique to decouple the dependencie between partial evaluation procee and thu avoid unneceary waiting, by introducing Boolean variable, one for each miing value of each QLit(q) ub-query at each virtual node. Uing thee variable, the bottom-up evaluation procedure i given in Fig. 3(b). Procedure bottomup conider the root of a fragment F j and a lit q L of ub-querie which i eentially the QLit(q) of the initial query q. Recurive call of the procedure are ued to perform the bottom-up traveral of the tree F j (line ). At each node v encountered, the procedure compute the value of q L at v and tore the reult of the computation in a vector V v which i of the ame ize a lit q L. Note that thee value are actually Boolean formula with thoe variable introduced at the virtual node. The computation of the q L value at v require the value of q L computed in the children and decendant of v. To cope with thi, we ave thee value (line 3-5) by maintaining only two additional vector, namely vector CV v and DV v, that are of the ame ize a vector V v. Intuitively, for each ub-query q in q L, CV v(q ) i true if and only if there exit ome child u of v uch that V u(q ) i true, and imilarly, DV v(q ) i true if and only if either V v(q ) i true or there exit ome decendant w of v uch that V w(q ) i true. Given a query q i q L at a node v, the computation of the value of q i depend on the tructure of q i. We conider different cae (line -17) of the tructure baed on the normal form given in Section... For example, if query q i i of the form text() = tr (line 1), then it value i true if Procedure ParBoX Input: An X BL query q and a fragmented, ditributed tree T Output: The boolean value an of q over T 1. q L := QLit(q);. S T := retrieve the ource tree of tree T; 3. for each ite S i in the ource tree S T do. execute(s i,evalqual, q L ); 5. for each fragment F j tored in S i do. annotate S T with (V Fj, CV Fj, DV Fj ); 7. an := evalst(s T ); (a) ParBoX algorithm executed at coordinating ite Procedure evalqual Input: A lit q L of orted (ub-)querie Output: A vector F[S i ][F j ] for each fragment F j of ite S i 1. for each ubtree F j aigned to S i do. (V Fj, CV Fj, DV Fj ) = bottomup(root(f j ), q L ); 3. end (V Fj, CV Fj, DV Fj ) to the coordinating ite; Procedure bottomup Input: A node v and a lit q L of (ub-)querie Output: Vector V v, CV v and DV v of formula for node v 1. for each child w of v do. (V w, CV w, DV w) := bottomup(w, q L ); 3. for each query q i in q L do. CV v(q i ) := compfm(cv v(q i ), V w(q i ), OR); 5. DV v(q i ) := compfm(dv v(q i ), DV w(q i ), OR);. for each query q i in q L from left to right do 7. cae q i of. (c) ǫ: V v(q i ) := 1; 9. (c1) label() = l: V v(q i ) := comparestring(label(), l); 1. (c) text() = tr: V v(q i ) := comparestring(text(), tr); 11. (c3) /q j : V v(q i ) := CV v(q j ); 1. (c) ǫ[q j ]/q k : V v(q i ) := compfm(v v(q j ), V v(q k ), AND); 13. (c5) //q j : V v(q i ) := DV v(q j ); 1. (c) q j q k : V v(q i ) := compfm(v v(q j ), V v(q k ), OR); 15. (c7) q j q k : V v(q i ) := compfm(v v(q j ), V v(q k ), AND); 1. (c) q j : V v(q i ) := compfm(v v(q j ), NULL, NEG); 17. DV v(q i ) := compfm(v v(q i ), DV v(q i ), OR); Procedure compfm Input: Two formula f 1 and f and an operator op Output: The compoed formula an := f 1 op f 1. cae (iformula(f 1 )), iformula(f )) of. (c) (,): if op = NEG then an := f 1 ; 3. ele an := f 1 op f ;. (c1) (,1): if op = AND then 5. if f 1 = true then an := f ; ele an := fale;. eleif op = OR then 7. if f 1 = true then an := true; ele an := f ;. (c) (1,): if op = NEG then an := f 1 ; 9. eleif op = AND then 1. if f = true then an := f 1 ; ele an := fale; 11. eleif op = OR then 1. if f = true then an := true; ele an := f 1 ; 13. (c3) (1,1): if op = NEG then an := f 1 ; 1. eleif op = AND then an := f 1 f ; 15. eleif op = AND then an := f 1 f ; (b) ParBoX algorithm executed at participating ite Figure 3: The ParBoX Algorithm 15

6 the text content of node v i equal to the tring tr, and i fale otherwie. More intereting i the cae where q i i of the form /q j (line 11). Then, the value of q i at node v i equal to the dijunction of the value of query q j at the child node of v. A a conequence of recurive evaluation, thi value ha already been accumulated in CV v(q j). Similarly, when q i i //q j (line 13), the value of q i at node v i the dijunction of V v(q j) and DV w(q j) for the children w of v, which have again been computed due to the bottom-up proceing order following the lit q L of ub-querie. Finally, when q i i of the form q j q k, the value of q i i the conjunction of the value of querie q j and q k. If querie q j and q k had imple Boolean value a anwer, then thi computation would be trivial. However, we note that a ditinguihing characteritic of our procedure i that variable are part of our evaluation. Therefore, we compoe Boolean value with variable or compoe Boolean variable with other Boolean variable to create more complex formula. Procedure compfm i reponible for compoing, for each query, the truth value and/or formula neceary to compute the value of the query. Depending on the value of the operator op it compute f 1 op f, which yield either a Boolean value or a Boolean formula. Example 3.1: Recall query q from Example.1. Evaluating the value of the (ub-)querie in q L = QLit(q) (given in Example.1) for the node in fragment reult in the following V v vector: Vname =<,,,,,,,,, > V F =< x 1, x, x 3 = x 1 x, x = x 3, x 5 = cx, x, x 7 = x 5 x, x = x 7, x 9 = dx, x 1 = x 9 > V broker =<,,,, x,,,, dx, dx > We ue and 1 to repreent the fale and true value while x i, cx i and dx i repreent ditinct variable in the V F, CV F and DV F vector, repectively, of virtual node F. Note that for each (ub-)query of node F we introduce a new variable. We ue Procedure bottomup to partially compute the value of the introduced variable, creating a ytem of Boolean equation. Oberve the following. Firt, proceing at each ite invoke Procedure bottomup for each fragment F j tored at the ite (ee Procedure evalqual). For each uch fragment, Procedure bottomup return a ingle triplet (V Fj, CV Fj, DV Fj ) of vector that tore the (ub-)query value for the root of fragment F j, for it children and it decendant, repectively. Each ite end the computed triplet() to the coordinating ite and conclude it computation. Second, in addition to the triplet aociated with virtual node in a fragment, bottomup need only two triplet in total in it proce: one for the current node (V v, CV v, DV v) and one for it children (V w, CV w, DV w), rather than aigning a triplet to each node. Example 3.: Conider the query from our previou example. At the end of the econd phae the following triplet are available to the coordinating ite S : V F =<,,,, y,,,, dy dz, dy dz > CV F =< y 1, y, y 3, y, y 5 z, y, y 7, y, y 9 dz, y 1 dz > DV F =< dy 1 dz 1, dy dz, dy 3 dz 3, dy dz, dy 5 dz 5 z y,1, dy 7 dz 7, dy dz, dy dz dy 9 dz 9, dy dz dy 1 dz 1 > V F1 =<,,,, x,,,, dx, dx > CV F1 =<, x, x 3, x, x 5, x, x 7, x, x 9, x 1 > DV F1 =<, dx, dx 3, dx, x dx 5, dx, dx 7, dx, dx dx 9, dx dx 1 > V F =<,,,,,,,,1, 1 > CV F =<,,,, 1,1, 1, 1,1, 1 > DV F =< 1, 1,1, 1, 1,1, 1, 1,1, 1 > V F3 =<,,,,,,,,, > CV F3 =<,,,,,1,,,, > DV F3 =< 1,,,,,1,,,, > In the triplet, variable x i, cx i and dx i for 1 i 1 are ued in fragment to repreent the value of the virtual node F, while variable y i, cy i, dy i and z i, cz i, dz i are ued in fragment F to repreent the value of the virtual node and F 3, repectively. Compoition of partial anwer. In the third phae of Algorithm ParBoX, the coordinating ite ue the computed triplet from all the fragment to evaluate the anwer to query q. In a nuthell, the computed triplet form a linear ytem of Boolean equation. Uing the computed vector and the ource tree, Procedure evalst (not hown due to the pace contraint) need a ingle bottom-up traveral of the ource tree to olve the ytem of equation and find the anwer to query q. Note that the vector of leaf fragment in the ource tree contain no variable. Thi i the cae for both fragment F and F 3. During the bottom-up traveral of S T, Procedure evalst ue the Boolean value of the leaf fragment to unify the variable of the vector that belong to the parent fragment in S T. The procedure continue in thi fahion until it reache the root of S T. The anwer for query q i the value of V Froot (q lat ), where F root i the root fragment and q lat i the lat query in the q L lit. Example 3.3: Conider the ource tree in Fig. (b) and the vector of the fragment from our previou example. Then, the anwer to query q i the value of the lat query in V F, that i, q = dy dz. A bottom-up evaluation of Procedure evalst ue DV F to unify dx to 1; DV F1 to unify dy to dx ; and DV F3 to unify dz to. Therefore, q = 1 and the query q evaluate to true. 3. Analyi For the complexity of the Algorithm ParBoX, we conider it communication cot a well a the total and parallel computation cot for evaluating a query q on a fragmented and ditributed tree T. The total computation cot i the um of the computation performed at all the ite that participate in the evaluation. In contrat, the parallel computation cot i the time needed for evaluating the query at different ite in parallel. Since a large part of the evaluation i performed in parallel, the parallel computation cot more accurately decribe the performance of the algorithm. We ue the following notation: F denote the et of all fragment of the original tree T, and F j F denote the ubet of fragment of T that are ub-fragment of fragment F j. We ue card(x) to denote the cardinality of a et X. Total network traffic. Oberve that each ite appearing in the ource tree S T of tree T i viited only once, when the coordinating ite end the input query q to thee ite in the firt tage. For each fragment F j in ite S j the algorithm generate three vector, each with O( q ) entrie. Each entry may hold a formula computed by Procedure bottomup, and it ize depend on the number of virtual node in fragment F j, i.e., card(f j), due to the variable introduced by thee virtual node. In the wort cae, the ize of the entry i in O( F j ). Therefore, the communication cot for each fragment F j i O( q card(f j)) and the overall communica- 1

7 B e Algorithm Viit Computation Communication NaiveCentralized 1 O( q T ) O( T ) NaiveDitributed card(f Si ) O( q T ) O( q card(f)) ParBoX 1 T O( q ( T + card(f))) P O( q (max( F Si ) + card(f)) S i O( q card(f)) Hybrid 1 T O( q T ) ParBoX P O( q (max( F Si ) + card(f)) S i O( T ) FullDitParBoX card(f T O( q ( T + card(f))) Si ) P O( q (max( F Si ) + card(f)) S i O( q card(f)) LazyParBoX card(f Si ) T P O( q ( T + card(f))) O( q card(f) max ( F i )) F Figure : Summary of preented algorithm O( q card(f)) tion cot of the algorithm i O( q Σ card(f) j=1 card(f j)), that i, O( q card(f)) (ince fragment are dijoint). Total computation. Site S travere each fragment F j aigned to it only once (through Procedure bottomup). At each node v in a fragment, the procedure take O( q ) time and therefore, the cot of the procedure on fragment F j i O( q F j ). Adding thee up for all fragment of tree T, the total amount of computation in the econd phae of the algorithm i O( q T ). The third phae of the algorithm olve, in linear time, a ytem of Boolean equation which i of ize O( q card(f)). Overall, the total amount of computation of Algorithm ParBoX i O( q ( T + card(f))). Parallel computation. The cot of the econd phae may differ depending on the level of parallelim. Intuitively, a et of fragment are aigned to different ite, the cot of the econd phae i equal to the computation cot at the ite holding the et with the larget aggregated fragment ize. We ue F Si to denote the um of the ize of the fragment in ite S i. Then, the time taken by the econd phae i O( q max S i ( F Si )) and the parallel computation cot of the algorithm i O( q (max S i ( F Si ) + card(f))). In any reaonable etting, we expect that the number of fragment to which a tree i decompoed will be mall compared to the ize of the tree itelf, i.e., card(f) << T. Thu, given a decompoition of a tree T to a et of fragment, our algorithm ha the deirable property that the communication cot of evaluating a query q over T i independent of the ize T of the tree and depend mainly on the ize q of the query. Similarly, the total computation cot of our algorithm become O( q T ), comparable to that of the bet-known centralized algorithm [1, 1] for evaluating an XPath query q over a tree T. Furthermore, the parallel computation cot depend only on the ize of the larget aggregated fragment ize aigned to a ite.. VARIATIONS OF ALGORITHM ParBoX We next adapt Algorithm ParBoX to variou etting. Hybrid ParBoX. Although very unlikely to occur, it i intructive to tudy the pathological cae in which each node v of our document tree T contitute a eparate fragment F and it i aigned to a different ite. Then, we have that card(f) = T. Even in thi pathological cae, the computation cot of our algorithm i till optimal. However, the communication cot become O( q T ), no longer independent of the tree T. Conider now the algorithm NaiveCentralized outlined in Section 3. The computation cot of the naïve al- I n e r t i n g a n e w t o F a c h e p o r t o f o l i o t i t l e t o c k N Y S E c o d e b u y I B M b r o k e r $ e l l $ 7 F 3 t o c k c o d e b u y H P Q $ 3 e l l $ 3 3 n a m p o r t o f o l i o B a c h e S p l i t F N e w f r a g m e n t F b r o k e r F F 3 t i t l e N Y S E t o c k t o c k c o d e b u y e l l c o d e b u y e l l H P Q $ 3 $ 3 3 I B M $ $ 7 Figure 5: Update on data and fragment gorithm i till O( q T ) but it communication cot i only O( T ). Therefore, for the pathological cae conidered, the naïve algorithm ha le communication overhead than Algorithm ParBoX. Thi lead u to conider a hybrid algorithm that, depending on the decompoition of the input tree T, it behave like Algorithm ParBoX for mot decompoition but witche to the naïve algorithm for pathological decompoition. The tipping point in thi witching of behaviour i determined by comparing card(f) and T. A long a q card(f) < T, Algorithm ParBoX ha le communication q overhead than the naïve algorithm. The total computation cot of the hybrid algorithm i O( q T ) while it communication cot i O( T ), in the wort cae, and O( q card(f)) on average. The parallel computation cot of Hybrid ParBoX i the ame a that of Algorithm ParBoX. Full ditribution of computation. When a large number of querie are ubmitted to the ame coordinating ite, the coordinating ite might turn out to be a ytem bottleneck. A the coordinating ite ha to collect and proce the partial anwer of participating ite, it might be overwhelmed by both the amount of incoming traffic and the ize of data to be proceed. We addre thi iue in a new algorithm called FullDitParBoX by ditributing the computation of the third phae of Algorithm ParBoX among all the participating ite. The firt two phae of Algorithm FullDitParBoX are the ame a thoe of ParBoX. During the third phae, Algorithm FullDitParBoX call procedure evalditrst intead of Procedure evalst. In a nuthell, Procedure evalditrst (omitted due to pace contraint) aume that each participating ite hold a copy of the ource tree S T. Given that the ize of S T i 17

8 expected to be much maller than the ize of T, the aumption add minimum torage overhead per ite. The procedure proceed in a bottom-up fahion in S T by conidering initially the ite that appear a leave in S T. Conider uch a ite S and aume that it i reponible for a leaf fragment F. Site S end the triplet of vector correponding to F to it parent ite S in S T. A non-leaf ite, like ite S, conider each local fragment F and, after receiving the triplet of all ub-fragment of F, S execute locally Procedure evalst uing the received triplet along with the triplet for F. Then, it end the reulting triplet for F to it parent ite in S T. Note that S and S till partially evaluate their local fragment in parallel in the econd tage of the algorithm. Alo note that no variable appear in the reulting triplet of vector of F. The proce terminate when it reache the ite at the root of S T. Procedure evalditrst ha the ame total/parallel computation and communication cot a Procedure evalst. Thu Algorithm FullDitParBoX i imilar to Algorithm ParBoX. In practical term we expect that the communication cot of the former algorithm i lower than that of the latter. Indeed, in the former, no variable are ent between ite ince they are alway unified, before any vector i ent. Although Algorithm FullDitParBoX remove the need for a coordinating ite it ha the drawback that a ite might be viited multiple time, once for each time it appear in S T. Lazy computation: Algorithm ParBoX i eager in that it requet all the ite to evaluate the querie in QLit(q) over all their fragment. Thi approach maximize parallelim but it doe, in certain cae, reult in unneceary computation. To ee why thi i o, conider the following query, which check whether there exit in our collection any broker with the name Merill Lynch : [/portofolio/broker/name = Merill Lynch ] Note that although we do not need to compute the query on fragment F and F 3, Algorithm ParBoX will do o. We can ave ome of thi unneceary computation by uing a lazy trategy that evaluate the query in increaing depth of the ite tree S T until it obtain an anwer to the query that doe not depend on any fragment further down the ource tree than the currently evaluated depth. Algorithm LazyParBoX (alo omitted) travere the ource tree S T in pre-order. At the i th traveral tep of the traveral, the coordinating ite identifie all the ite that hold fragment at depth i from the root of the ource tree. For each of thee ite, the coordinator requet the evaluation of Procedure evalqual for the correponding fragment. The coordinating ite collect the evaluated vector for all thee fragment and, along with the vector collected from previou traveral tep, it call Procedure evalst to compute the anwer to the query. Only if no anwer can be computed, due to variable that cannot be unified, the algorithm perform one more tep. The total computation cot of Algorithm LazyParBoX i the ame a that of ParBoX. However, in cae uch a our lat example, the algorithm behave better than ParBoX. In our example, LazyParBoX doe not evaluate the query over fragment F, ince after one tep the given query i evaluated to true. In term of parallel computation, Algorithm LazyParBoX i wore than ParBoX ince in each traveral tep only one fragment i evaluated per ite, and only fragment at the ame level of the ource tree are computed in parallel. Thu the parallel computation cot of the algorithm i O( q card(f) max( Fi )), where max ( Fi ) F F denote the ize of the maximum fragment of tree T. We ummarize the dicuion of thee algorithm in Fig., in which we lit the number of time each ite i viited, the total (T) and parallel (P) computation cot, and the communication cot. Recall that the firt two naïve algorithm do not exploit parallelim and thu we only report their total computation cot. In Fig. we ue card(f Si ) to denote the number of fragment that reide in ite S i. 5. INCREMENTAL VIEW MAINTENANCE A remarked in Section 1, one often want to cache the reult of a query, treat it a a materialized view, and ue it to anwer poible future querie (ee, e.g., [13, 5]). When a new query i iued, the materialized view are ued to provide part of, or the whole of, the anwer to the query. With thi come the iue of view maintenance: when the ource data i updated, the materialized view mut be maintained o a to reflect the current ource content. An approach to maintaining view i by mean of an incremental technique: given a query Q, a databae I, a view V = Q(I) and update I on the ource I, we compute update V on the view uch that V V = Q(I I). Incremental maintenance of view ha proven effective in many application (ee, e.g., [1, ]), ince mall change I to the ource often inflict only mall change V to the view, and thu it i often more efficient to compute V rather than computing the view Q(I I) tarting from cratch. We next preent a mechanim to upport incremental view maintenance that i baed on extenion of our XPath evaluation algorithm. Our incremental algorithm have the following alient feature. (a) The cot of maintaining materialized view depend neither on the ize of the data nor on the ize of the update. (b) The recomputation i localized to the fragment where the update occur. Materialized view. A materialized view M of a query q over a tree T, denoted a M(q, T), i a pair (S T,an), where S T i the ource tree of T and an i the cached anwer of the query q over T. We refer to the pair (S T,an) a the tate of view M. A view i materialized at a ite if the ite maintain it tate. Our maintenance algorithm impoe minimum overhead on the ite, ince only the query, the ource tree and the anwer need to be tored. Update operation. We conider two clae of update that can alter the tate of a materialized view: the firt alter the content of the tree T and the econd alter the fragmentation of T. For each cla, there are two primitive operation, which are lited below. All operation are defined with repect to a fragment F j of tree T. (1) innode(a, v): inert in F j a node labeled A a a child of node v. The operation return the newly inerted node. () delnode(v): delete from F j the node v. (3) plitfragment(v): create a new fragment F k which i the ubtree rooted at node v. The new fragment F k i a ub-fragment of F j and it ubtree i replaced in F j by a virtual node whoe label i F k. () mergefragment(v): merge fragment F j with the ubfragment that correpond to the virtual node v. If v i not virtual, no action i taken. Example 5.1: Conider fragment F in Fig. (a). We can ue a erie of innode operation to inert a new tock in the fragment, yielding the fragment in the left of Fig. 5. The 1

9 F n 1. F n ) F 3 new ubtree i indicated by dotted line. We can continue by applying operation plitfragment(market) to the new fragment and get two fragment: a revied F and a new fragment F hown in Fig. 5. Note that the ubtree rooted at the market node i replaced by a virtual node F. Fragment F can now be aigned to another ite, ay, S 3. Since the firt two operation concern the content of a fragmented tree T, they only affect the an part of the tate of a view M(q, T). Given a erie of inertion and/or deletion in a fragment F j, we extend Algorithm ParBoX to incrementally update an; extenion to the variation of the Algorithm ParBoX are done imilarly. Algorithm outline. To incrementally update an, it uffice to augment the tate of M(q, T) o that we alo maintain the triplet (V Fk, CV Fk, DV Fk ), for each of the fragment F k of T. After the erie of inertion and/or deletion in fragment F j, only the ite toring F j need to call Procedure bottomup and only for fragment F j. The reulting triplet (VF new j, CVF new j, DVF new j ) i ent back to the ite S toring the tate of M(q, T). The triplet i then compared with the one tored in S for the ame fragment F j. If they are identical, incremental evaluation terminate without changing the value of an. Otherwie ite S ue the new triplet, along with the triplet for the other fragment, in Procedure evalst to compute the new value of an. The total (and parallel) computation cot of the incremental algorithm i O( q ( F j + card(f))) while the communication cot i O( q card(f j)), where F j i the ize of the fragment F j while card(f j) and card(f) are the number of ub-fragment of F j and T, repectively. Oberve that the communication cot i independent of both T and the ize of the update. Furthermore, recomputation i localized to fragment F j, in which update take place. Now conider plitfragment(v), which plit a fragment F j into two fragment Fj new and F k. Obviouly, the plitting doe not affect the value of an. However, both the ource tree S T and the correponding fragment vector triplet mut be updated. Thi update i local to the ite S toring the tate of M. The only other ite involved in the proce i ite S where fragment F j ued to reide. Site S need to end to ite S two new vector triplet, one for Fj new and one for F k. It i not hard to ee that the total (and parallel) computation cot for thee operation i O( q F j ), while the communication cot i O( q card(f j)). The analyi for mergefragment(v) i imilar and reult are within the ame bound, where F j now denote the fragment that i the reult of merging.. EXPERIMENTAL STUDY We have implemented the algorithm of Section 3 and. Below we preent four of the conducted experiment. For our experiment we ued 1 Linux machine (ite), ditributed over a local LAN. We generated multiple XMark ite and in each experiment we aigned (fragment of) XMark ite to Linux machine. The reported time are averaged over multiple run of each experiment. Experiment 1: The objective of thi experiment i twofold. Firt, we illutrate the effectivene of Algorithm ParBoX when compared with the NaiveCentralized algorithm. Second, we tudy the effect of the query ize (i.e., it number of ub-querie QLit(q) ) on query evaluation time. The fragment tree ued in thi experiment are imilar to tree F r a g m e n t T r e e ( F T 1 ) F F... F r a g m e n t T r e e ( F T ) F F F 5 F r a g m e n t T r e e 3 ( F T 3 F Figure : Sample of fragment tree ued FT1, hown in Fig.. Any fragment tree topology, with the ame number of fragment could have been ued here without affecting our reult. Thi i becaue both ParBoX and NaiveCentralized communicate directly with all the ite holding fragment, irrepectively of the tructure of the tree and the location of the fragment in the tree. The experiment begin with a ingle fragment F and in each iteration, it increae the number of fragment in the tree by one, by inerting a new fragment a a ub-fragment of F and aigning it to a different machine. Each fragment F j in the tree correpond to a ingle XMark ite and all the fragment have the ame ize within each iteration. The cumulative ize of all the fragment i kept contant acro iteration and equal to 5MB. So, in iteration 1, we have only fragment F of ize 5MB, while in iteration, we have fragment F and, each of ize 5MB. In the lat iteration, there are 1 fragment, each of ize 5MB. Figure 7 compare the evaluation time of ParBoX and NaiveCentralized of a query with QLit(q) =, for each iteration. The query i poed at the (coordinating) ite holding fragment F. The figure clearly demontrate both the benefit of uing ParBoX, and that data tranmiion i prohibitively expenive. Notice that a the number of machine increae in each iteration, o doe parallelim, reulting in reduced evaluation time. Parallelim prove epecially beneficial in the firt iteration, but it gain diminih a increaingly maller fragment are aigned to machine (iteration - 1). Indeed, the evaluation time in iteration (fragment ize.3mb) i not ignificantly different from the one in iteration 1 (fragment ize 5MB). In term of the NaiveCentralized and NaiveDitributed algorithm, note that the query evaluation time of fragment F (. econd) i a lower bound for both of thee algorithm. The additional cot of NaiveCentralized in each iteration i the cot of ending the data to the coordinator. Thi cot i high for the firt few iteration and tart to flatten out after iteration. Indeed, in iteration we ent fragment to the coordinator, that i 5MB, while in iteration 3 we ent fragment and F, that i 3MB (9MB more). However, notice that in iteration, we ent fragment to F 7, that i MB, while in iteration 1 we ent fragment to F 9, that i 5MB (only 3MB more). Figure how the evaluation time of ParBoX for querie whoe ( QLit(q) ) ize are,, 15 and 3. A expected, query evaluation time increae linearly with the ize of the query and the benefit of parallelim are conitent throughout the different query ize. Experiment : The aim of thi experiment i to highlight the difference between ParBoX and it variation. To thi end, we ue fragment tree imilar to tree FT, hown in F F F F 7 19