A Comparison of Heuristics for Scheduling Spatial Clusters to Reduce I/O Cost in Spatial Join Processing

Transcription

1 Edth Cowa Uversty Research Ole ECU Publcatos Pre A Comparso of Heurstcs for Schedulg Spatal Clusters to Reduce I/O Cost Spatal Jo Processg Jta Xao Edth Cowa Uversty 0.09/ICMLC Ths artcle was orgally publshed as: Xao, J. (2006). A Comparso of Heurstcs for Schedulg Spatal Clusters to Reduce I/O Cost Spatal Jo Processg. Proceedgs of Iteratoal Coferece o Mache Learg ad Cyberetcs. (pp ). Dala, Cha. IEEE. Orgal artcle avalable here 2006 IEEE. Persoal use of ths materal s permtted. Permsso from IEEE must be obtaed for all other uses, ay curret or future meda, cludg reprtg/republshg ths materal for advertsg or promotoal purposes, creatg ew collectve works, for resale or redstrbuto to servers or lsts, or reuse of ay copyrghted compoet of ths work other works. Ths Coferece Proceedg s posted at Research Ole.

2 Proceedgs of the Ffth Iteratoal Coferece o Mache Learg ad Cyberetcs, Dala, 3-6 August 2006 A COMPARISON OF HEURISTICS FOR SCHEDULING SPATIAL CLUSTERS TO REDUCE I/O COST IN SPATIAL JOIN PROCESSING JI-TIAN XIAO School of Computer ad Iformato Scece, Edth Cowa Uversty, 2 Bradford Street, Mout Lawley, WA 6050, Australa j.xao@ecu.edu.au Abstract: I spatal jo processg, a commo method to mmze the I/O cost s to partto the spatal objects to clusters, ad the to schedule the processg of the clusters the spatal jo processg such that the umber of tmes the same objects to be fetched to memory ca be mmzed. A key ssue of ths clusterg-ad-schedulg approach s how to produce a better sequece of clusters to gude the cluster schedulg thus to reduce the total I/O cost of spatal jo processg. Ths paper descrbes three cluster sequecg heurstcs. A extesve comparso amog them has bee coducted, ad smulato results have show that, whle usg the cluster sequeces geerated to gude the cluster schedulg ca sgfcat reduce the I/O cost fetchg spatal objects spatal jo processg, ther performace dffers. Keywords: Spatal databases; Spatal jo processg; maxmum spag tree; At coloy optmzato; schedulg, Match. Itroducto I Spatal databases, the cost of spatal jo processg could be very hgh due to the large szes of spatal objects ad the computato-tesve spatal operatos [4]. To reduce the CPU ad I/O costs for spatal jo processg, most spatal jo processg methods are performed two steps (.e., flter-ad-refe approach []). The frst step chooses pars of spatal objects that are lkely to satsfy the jo predcate. The secod step exames the predcate satsfacto for all those pars of objects passg through the flterg step. Durg the flterg step, a coservatve approxmato of each spatal object s used to elmate objects that caot cotrbute to the jo result, ad a weaker codto for the spatal predcate s appled o the approxmatos. Ths step produces a lst of caddates that s a superset of the joable caddates. These caddates are usually represeted as pars of object detfers. All caddates are the checked the refemet step by applyg the spatal operato o the full descrptos of the spatal objects to elmate the ''false drops''. The jo cost ca be reduced because the weaker codto s usually computatoally less expesve to evaluate ad the approxmatos are small sze tha the full geometry of spatal objects. It s very mportat to reduce the I/O cost at the refemet step. Expermets have show that I/O cost (.e., dsk accesses) at the refemet step takes sgfcat amout of tme, as compared wth the CPU tme for spatal jo [3]. If a large umber of spatal objects are volved a spatal jo operato, a commo method to mmze the I/O cost s to partto the spatal objects to clusters, ad the to schedule the processg of the clusters the spatal jo processg such that the umber of tmes the same objects to be fetched to memory ca be reduced. The key ssue of cluster schedulg s how to produce better schedulg sequeces of the clusters such that the total I/O cost s mmzed. However, the problem of fdg a best spatal cluster schedulg sequece s NP-complete [7]. I our prevous work, we have developed a umber of heurstcs [5, 6, 7] to produce spatal cluster schedulg sequeces. Ths paper s to evaluate some dvdual heurstcs ad coduct comprehesve comparso amog them. The rest of the paper s orgazed as follows: I Secto 2, we formalze the spatal cluster schedulg problem. I Secto 3, some prelmary cocepts are preseted ad some cluster sequecg heurstcs are revewed. A example s gve Secto 4 to show the performace of dvdual heurstcs. Smulato results are preseted Secto 5. Ad Secto 6 cocludes the paper. 2. Formulato of cluster schedulg Let S ad T be the two spatal database tables for spatal jo operato, deoted by S χ T. Objects S ad T are dexed by ther uque IDs. The spatal data of these /06/$ IEEE 2455

3 Proceedgs of the Ffth Iteratoal Coferece o Mache Learg ad Cyberetcs, Dala, 3-6 August 2006 objects ca have dfferet szes,.e., they are o-uform szed. The flter operato of the spatal jo produces a set of pars of S ad T objects. Let F be the set of ID pars produced by the flter operato: F = {(sd, td) sd ad td are IDs of objects S ad T, respectvely, that meet the weaker jo codto} where a ID par (sd, td) F s called a caddate. Fgure (a) shows a example of F. Note that F s avalable the ma memory after the flter operato. F cotas oly IDs of the caddates, ot the data objects. S_d A A2 A4 A5 A7 T_d B B B B (a) A caddate set. A A2 A4 A5 A7 B B B (b) A caddate clusterg. Fgure. A example of a caddate set ad ts clusterg The refemet step s to perform S χ T o the pars of objects dexed by F to produce the fal jo results. At ths step, the S ad T objects eed to be fetched to the ma memory for the full spatal jo check. Sce the memory sze s lmted ad t ca ot keep all objects of F memory at the same tme, the objects eed to be parttoed to clusters. Objects the same clusters are brought to the memory together ad processed a batch. For example, Fgure (b) s a parttog of the caddate set show Fgure (a). Assume that the spatal objects refereced F have bee parttoed to clusters. Our goal s to schedule the clusters a way such that the repeatedly fetch of the overlappg objects betwee cosecutve clusters s mmzed. The I/O cost, ths paper, s measured terms of the sze of object data (e.g., umber of vertces of the spatal object) that are fetched to the memory for the refemet operato. Let = {v, v 2,, v k } be the set of objects refereced F, ad V,V 2,, V the clusters of For each ( ), V ={ v v,..., v } (m ), v ( j m). That s, =, 2 m V = ad V φ j for each ( ). For coveece, we defe sze(v ) as the sum of the szes of objects V,.e., sze ( V ) = s( v) where s(v) s the v V sze of object v. We troduce a weghted graph G = (V, E, w), upo, called cluster overlappg (CO) graph, to represet the overlappg relatoshps betwee data clusters. The ode set V = {V, V 2,, V } s a set of clusters, ad the edge set E s defed as: for each par of odes V ad V j ( j), there s a edge E j = (V, V j ) f w(v, V j ) = sze(v V j ) 0. Here w(v, V j ), also deoted as w(e j ), s the weght of edge E j. For stace, based o the object szes gve Fgure 2 (a), Fgure 2 (b) shows the CO graph correspodg to the clusters Fgure (b). od A A2 A4 A5 A7 B obj sze (a) Object sze V A, A2,, B,, , A7,, B,, V3 (b) cluster overlappg graph A4, A5,, B,, V2 Fgure 2. A example of CO graph At refemet step, f the object clusters are processed the sequece of V, V 2,, V (.e., o schedulg), the the total I/O cost s: C 430 = sze( V ) sze( V V ) I / O j = = Whe processg cluster V +, objects V V + are already memory just after processg V. There s o eed to load these objects aga. Geerally, for a schedule π whch determes the processg sequece of V, V 2,, V as Vπ, Vπ,..., V, 2 π where Vπ V ad Vπ V for j, the I/O cost for π j schedule π s C π = ( ) + ( ( ) ( )) sze Vπ sze Vπ sze V V π π = I / O + = sze( Vπ ) sze( V ). π V π + = = = ( V Whe the clusters are gve, sze π ) s a costat. Let y be: = + () (2) y = sze( Vπ Vπ ) (3) The objectve of spatal cluster schedulg s to fd a schedule π such that y s maxmzed, whch s the case that π C / s mmzed. I O 2456

4 Proceedgs of the Ffth Iteratoal Coferece o Mache Learg ad Cyberetcs, Dala, 3-6 August Spatal Cluster Schedulg Heurstcs Gve a CO graph G = (V, E, w) wth V = {V,V 2,, V }, a maxmum overlappg (MO) order amog V,V 2,, V s a sequece ( V V,..., V ) such that ( l V l l+, 2 sze V ) reaches the maxmum amog all permutatos of V [7]. I other words, a MO order a CO graph G s a permutato of odes G such that the total sze of overlappg objects betwee adjacet odes reaches the maxmum. For example, (V, V 2, V 3 ) s a MO order CO graph Fgure 2(a), ad the total sze of overlappg objects betwee adjacet odes the order s 680. The smplest algorthm to fd a MO order s to check all permutatos of V to see whch oe makes the l max { sze ( V V ) }. The complexty of the method l l+ clearly has factoral order ad s certaly ot practcal. Although a MO order exsts for each CO graph G, t s mpossble to fd a MO order polyomal tme. However, the task of fdg a MO order ca be reduced to the case where G s a coected graph [5]. We ow descrbe three heurstcs that produce approxmato of MO (AMO) order gve CO graph G. 3.. Maxmum Spag Tree Based Heurstc A maxmum spag tree (MST) based heurstc was developed [7] to produce a AMO order of relatve hgh overlappg weghts the sese that the weght of the AMO order produced by the algorthm s always greater tha or equal to half the weght of a optmal MO order. The algorthm cossts of three steps: The frst step produces a maxmum spag tree T of the CO graph G; the secod step coducts a depth-frst search (DFS) o T ad, the thrd step, a AMO order s bult, whch s the traversal order of the DFS o T. The complexty of the algorthm s O(m 2 log 2 m), where m = max( V, E ) ACO-based Heurstc The at-coloy optmzato (ACO) based metaheurstc s a populato-based approach to the soluto of dscrete optmzato problems. It mtates real ats searchg for food,.e., by fdg the shortest path from a food source to ther est. Ats use a aromatc essece, called pheromoe, to commucate formato regardg the food source. Whle ats move alog, they lay pheromoe o the groud whch stmulates other ats rather to follow that tral tha to use a ew path. As other ats observe the pheromoe tral ad are attracted to follow t, the pheromoe o the path wll tesfed ad reforced ad wll therefore attract eve more ats. The typcal applcato of ACO s the travellg salesma problem (TSP) [2], defed as follows: A graph G=(V, E, w) wth ode set V ad edge set E s gve; each edge e E has a weght w(e) assocated, represetg the legth of t. The problem s to fd a mmal-legth closed tour that vsts all the odes oce ad oly oce. I the ACO approach each edge of the graph has two assocated measures: the heurstc desrablty η j, whch s defed as the verse of the edge legth ad ever chages for a gve problem stace, ad the pheromoe tral τ j, whch s modfed at rutme by ats. Each at has a startg ode ad ts goal s to buld a soluto, that s, a complete tour. A tour s bult ode by ode: whe at k s ode t chooses to move to ode j usg a probablstc rule that favors odes that are close ad coected by edges wth a hgh pheromoe tral value. Nodes are always chose amog those ot yet vsted order to eforce the costructo of feasble solutos. The pheromoe tral s updated o the edges of the solutos. To apply the ACO approach for fdg a AMO order for a arbtrary CO graph G=(V, E, w), we exteded G to a complete graph G =(V, E, w ) by the followg steps. For ay par of odes v, v j V,, j, add a edge (v, v j ) to E. If (v, v j ) E, the defe w (v, v j ) = /w(v, v j ); otherwse defe w (v, v j ) = w max, where w max s a very large umber. w (v, v j ) s take as the legth betwee odes v ad v j. The the ACO algorthm s appled to G to fd a TSP soluto, whch s a shortest close tour o G. Oce a TSP soluto was foud, a AMO order ca be determed by smply removg a edge of maxmum weght from the soluto ad takg the order of the resultat path as the AMO order [5]. The complexty of the ACO-based algorthm s O(Nc 3 ), where Nc s the predetermed terato parameter of the ACO algorthm (Nc usually takes a value of ) Match-based Heurstc A match of a graph s a set of edges; ay two of them are ot cdet to the same ode. A weghted matchg (WM) problem s, for a gve (edge weghted) graph G, to fd a match of G such that the sum of the edge weghts of the match s maxmal. The WM problem was solved by J. Edmods [2] ad the complexty of hs algorthm s O( 3 ), where s the umber of odes of G. For ay graph, Edmods s algorthm outputs a maxmal match of G. A match s maxmal f ay edge the graph that s ot the match has at least oe of ts edpots matched, ad the 2457

5 Proceedgs of the Ffth Iteratoal Coferece o Mache Learg ad Cyberetcs, Dala, 3-6 August 2006 The basc dea behd the match-based algorthm [6] s frst to dvde the CO graph to sets of dsjot path graphs 2 such that the sum of the edge weghts of the logest paths the path graphs reaches the maxmum, ad the lk these paths usg maxmal match amog the edpots of the logest paths the path graphs. The match-based heurstc s a recursve oe cotag three ma steps: I the frst step, a maxmal match M of G s produced usg Edmods s algorthm (for detals see [2]). Edges M are take as the tal AMO order. The G s dvded to sets of path graphs, each cossts of a par of matched odes ad a edge coectg the matched odes. Each umatched ode of G forms a specal path graph,.e., oe wthout a edge. I the secod step, the graph G s coarseed by collapsg the matchg odes. At ths step, each par of matchg odes are combed to form a sgle ode of the ext level coarser graph G = (V, E, w ). Nodes V are ether the form of v = {v, v j }, where v, v j V are matched M,.e., (v, v j ) M, or v = {v }, where v s a umatched ode of M. That s, V ={{v, v j } v, v j V ad (v, v j ) M} {{v } v s a umatched ode of M}.We refer to ode v of form {v, v j } V a multode. E ad w are the defed such that the edge betwee ay par of multodes v ad v correspods to a edge E whose two edpots are v ad v, respectvely, ad whose weght s maxmal amog all edges coectg odes betwee the multodes v ad v, f such a edge exsts (ote that ay par of umatched odes s ot coected both G ad G ). After graph G s bult, Edmods s algorthm s appled to G aga to produce a maxmal match M. The above matchg ad collapsg process cotues utl o further matchg ca be foud. The the thrd step, the heurstc produces the AMO order accordg to the output of the above procedure. Itutvely, f we coceptually take a par of matchg odes ad the edge betwee them as a path graph at the ed of frst roud of matchg ad collapsg process, the from the secod roud of matchg ad collapsg process o, these path graphs are merged parwsely a way that ther logest paths are lked ed by ed usg a edge of maxmal weght betwee edpots of the paths. Wth the matchg ad collapsg process gog o, paths are lked usg the maxmal matchg o levels of coarser graphs utl a set of dscoected path graphs s reached. At ths sum of the edge weghts of the match s maxmal amog all matches of the graph. 2 A path graph G = (V, E) wth odes s a graph whch all odes V ca be lsted as a sequece v, v 2,..., v such that (v, v 2 ), (v 2, v 3 ),... (v -, v ) are the oly edges of E. stage, a sequece of odes of the logest path for each path graph was output. Ay order of these sequeces ca be take as a AMO, because the produced path graphs are o-jot wth each other, ad each ode of the orgal CO graph belogs oe ad oly oe path graph. 4. Comparso amog heurstcs: A Example Now let us compare the performace of these heurstcs usg the example gve [4], whch s re-produced as below: Let = {a, a 2,, a 36 } be a spatal object set ad V = (V, V 2,, V 6 ) a set of clusters o. The relatoshp betwee a object a ( 36) ad a cluster V j ( j 6) s gve by the followg cdece matrx (m j ),.e., m(, j)= f a j V, ad m(, j)=0 otherwse. V V V 2 V 3 V 4 V 5 V 6 V V V 2 V 3 V 4 V 5 V a 0 0 a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a The correspodg CO graph s gve Fgure 3 (a). I ths example, the szes of all objects are detcal, thus are ot mportat. For smplcty, a object a s expressed by ts dex the fgure, ad the sze of a s take as, for all 36. By applyg the MST based algorthm to the CO graph, a AMO order was produced as show Fgure 3 (b), whch s V, V 4, V 2, V 3, V 6, V 5, wth the total overlappg weght 3. By applyg the match-based algorthm to the CO graph Fgure 3(a), the frst step produces a maxmal match M = {(V, V 3 ), (V 2, V 4 ), (V 5, V 6 )}. That s, the tal set of path graphs cotas three path graphs, wth path sequeces P = (V, V 3 ), P 2 = (V 2, V 4 ) ad P 3 = (V 5, V 6 ), respectvely. After collapsg, the ext roud of matchg produces a maxmal match cotag oe edge,.e., ({V, V 3 }, {V 2, V 4 }), ad a solated ode whch s the multode {V 5, V 6 }. By collapsg the (matched) multodes, two path graphs (.e., those wth path sequeces P ad P 2 ) were 2458

6 Proceedgs of the Ffth Iteratoal Coferece o Mache Learg ad Cyberetcs, Dala, 3-6 August 2006 merged to form a ew path graph, wth loger path sequece V, V 3, V 2, V 4. The edpots (.e., V ad V 4 ) of ths path sequece form a ew multode the ext level coarser graph. The ext matchg ad collapsg process results a sgle multode {V, V 5 } that makes the algorthm stop. Ths step merges the two path sequece (V, V 3, V 2, V 4 ) ad (V 5, V 6 ) together by sertg the edge betwee the matchg odes V 4 ad V 6, leavg the fal edpots of the logest path to be V ad V 5. The fal AMO order s V, V 3, V 2, V 4, V 6, V 5, as show Fgure 4(a). AMO orders to gude the schedulg of processg of clustered jo operatos. Two types of smulatos were coducted. The frst type of smulatos s to show the I/O cost reduced by usg varous cluster sequecg methods, ad the secod s to compare the overlappg weghts o AMOs produced by varous heurstcs. (a) (b) Fgure 4. AMO order produced by (a) match-based ad (b) ACO-based heurstcs. Fgure 3. A CO graph ad ts AMO order produced by the MST based algorthm. By comparg the above AMO orders, we foud that the match based algorthm produced a better AMO order by the fact that the total overlappg weght of the AMO order produced by the match based algorthm s 33, whch s the optmal MO order ths example, whle t s 3 for the AMO order produced by the MST algorthm. Whe applyg the ACO-based heurstc to the CO graph Fgure 3(a), two AMO orders were produced, as show Fgure 4. oe s the same as that Fgure 4(a), ad the other s as Fgure 4(b). Both have a total overlappg weght of 33, the same wth that produced by the match-based algorthm. 5. Smulatos The smulato work s to demostrate the reducto of the I/O costs spatal jo processg by usg dfferet I the frst type of smulatos, AMO orders produced by MST-, match- ad ACO-based heurstcs are appled to the same applcato scearos to gude spatal cluster schedulg. For ease of descrpto, the related schedules are called MST, MB, ad ACO, respectvely, ths secto. These schedules are smulated agast each other usg the followg spatal cluster schedulg strategy: the schedule fetches spatal objects to the memory, cluster by cluster, the AMO orders produced by the MST-based, match-based ad ACO-based heurstcs, respectvely. The overlappg objects betwee two cosecutve clusters are ot fetched to the memory aga whe processg the ext cluster. Although match-based ad ACO-based heurstcs take loger tme tha MST-base heurstc does fdg AMO orders, ther calculato costs ca all be eglected, whe comparg wth the total fetchg cost. I the smulatos, most spatal datasets are geerated whle a small porto of datasets s from real spatal applcatos. The object szes chage from tes to hudreds of vertces. At each smulato pot, the smulato rus 0 tmes. Sce every object eeds to be fetched to the memory for the refemet operato, for smplcty, we measure the I/O cost terms of the total sze of the overlappg objects that are fetched repeatedly to the memory for processg (.e., y value formula (3)). The I/O costs of Y-axs are the mea values of y all rus. Fgure 5 shows the I/O cost versus the cluster umbers. The average sze of clusters ths fgure s 20. We ca see that ACO ad MB are qute smlar performace, ad they perform all the tme better tha MST. O average, ACO ad MB ca acheve over 6% savg whe comparg wth MST. For example, whe the umber of clusters s 50, the average sze of overlappg objects to be 2459

7 Proceedgs of the Ffth Iteratoal Coferece o Mache Learg ad Cyberetcs, Dala, 3-6 August 2006 fetched repeatedly to memory by MST s 2069, whle t s 760 by ACO ad 766 by MB, respectvely. As the umber of clusters grows, the performace ga obtaed by ACO ad MB, respectvely, gets greater. Whe cluster sze chages, smlar treds are acheved our smulatos. Due to space lmtato, the results are omtted here. I/O cost Fgure 5. I/O cost versus cluster umbers (cluster sze=20). Table. A comparso of overlappg weghts amog AMO orders produced by three heurstcs heurstc MST ACO MB Cluster sze = Clustre umber sze (,e) 20/50 30/60 40/00 50/20 00/200 ACO-based Match-based MST-based Table shows the average overlappg weghts amog AMO orders produced by ACO-based, match-based ad MST-based heurstcs, respectvely, for graphs of sze (, e), where s the umber of odes, ad e the umber of edges. We observe that, for ay sze of graphs, the average weght of the match-based AMO orders s always greater tha (or equal to) that of the MST-based AMO orders, ad t s very close to that of ACO-based AMO orders. As ACO produces optmal or ear-optmal soluto for most NP-hard problems, the average weght of the AMO order produced by the ACO-based algorthm s very close to that of the MO order. Therefore, the AMO orders produced by match-based algorthm are very close to the MO order. However, from the smulato, the computato cost of ACO-based AMO order s sestve to the umber of edges of the CO graph, whle MB ad MST are ot. Due to ths, ACO s ot sutable for ole spatal jo servce where spatal jo processg must be completed a reasoable tme lmt. O the other had, for offle spatal jo, ACO performs better tha the other two schedules. 6. Cocluso I spatal jo processg, spatal objects are usually parttoed to clusters ad the are processed cluster by cluster. Sce two clusters may have overlappg, the overlappg objects may be repeatedly loaded to memory. It s mportat to schedule the processg of the clusters such a sequece that two cosecutve clusters the sequece have hgher umber of overlappg objects, thus, there s o eed to load those overlappg objects whe processg the ext cluster because they are already the memory. The I/O cost ca, therefore, be reduced. The key ssue behd the spatal cluster schedulg method s how to produce a better AMO order to gude the schedulg. Ths paper descrbed three cluster-sequecg heurstcs. A comparso amog them has bee coducted to evaluate ther performace. Smulato results have show that, whle ACO-based ad match-based heurstc produce better AMO orders tha the MST-based oe does, ACO s ot sutable for ole spatal jo processg. Refereces [] L. Becker, A. Gese, K. Hrchs ad J. Vahrehold. Algorthms for Performg Polygoal Map Overlay ad Spatal Jo o Massve Data Sets. R. G utg, D. Papadas, F. Lochovsky (Edt.): SSD 99, LNCS 65. pp Sprger-Verlag Berl Hedlberg, 999. [2] E. L. Lawler, Combatoral Optmzato: Networks ad Matrods. Holt, Rehart ad Wsto, New York, 976. [3] M. L. Lo ad C. V. Ravshakar. Spatal Jos Usg Seeded Tree, Proc. ACM SIGMOD It. Cof. o Maagemet of Data, pp , 994 [4] Y. Theodords, E. Stefaaks, T. Sells. Cost Model for Jo Queres Spatal Databases. Proc. of ICDE 98, Orlado, Florda, USA, 998. [5] J. Xao, Applyg the At Coloy Optmzato Algorthm to the Spatal Cluster Schedulg Problem. Proceedgs of the 3rd Iteratoal coferece o Mache Learg ad Cyberetcs (ICMLC04), Shagha, Cha, August 26-29, 2004, pp [6] Jta Xao, Match Based SJP Cluster Sequecg ad Schedulg Spatal Databases, Proceedgs of the 2d Computatoal Itellgece, Robotcs ad Autoomous Systems, Sgapore, Dec.5-8, [7] J. Xao, Y. Zhag, X. Ja ad X. Zhou. A Schedule of Jo Operatos to Reduce I/O Cost Spatal Database Systems, Data & Kowledge Egeerg, Elsever Scece B.V, Vol. 35, 2000, pp