Indexing the Edges A simple and yet efficient approach to high-dimensional indexing

Transcription

1 Inexing the Eges A simple an yet efficient approach to high-imensional inexing Beng Chin Ooi Kian-Lee Tan Cui Yu Stephane Bressan Department of Computer Science National University of Singapore 3 Science Drive 2, Singapore fooibc,tankl,yucui,stephg@comp.nus.eu.sg ABSTRACT In this paper, we propose a new tunable inex scheme, calle (), that maps points in high imensional spaces to single imension values etermine by their maximum or minimum values among all imensions. By varying the tuning \knob", we can obtain ierent family of structures that are optimize for ierent istributions of ata sets. For a -imensional space, a range query nee to be transforme into subqueries. However, some of these subqueries can be prune away without evaluation, further enhancing the eciency of the scheme. Experimental results show that () can outperform the more complex Pyrami technique by a wie margin. 1. INTRODUCTION Many multi-imensional inexing structures have been propose in the literature (see [1] for a survey). In particular, it has been observe that the performance of hierarchical tree inex structures such as R-trees [2] an R -trees [3] eteriorates rapily with the increase in the imensionality of ata. This phenomenon is cause by two factors. First, let us consier the fan-out of internal noes in an R-tree. Suppose we conform to the classic enition of an R-tree where all noes are of a xe size B. The fan-out of an internal noe is clearly boune by B i=1 (2 si ) where s i is the size of ata elements corresponing to imension i. (The expression i=1(2 s i) constitutes the storage neee to ene a minimal bouning region in a - imensional space.) Clearly, the fan-out of an R-tree is inversely proportional to the imensionality of ata objects (). The smaller fan-out contributes not only to increase overlap between noe entries but also the height of the corresponing R-tree. One obvious solution is to reuce the amount ofoverlap by increasing the fan-out of selecte Permission to make igital or har copies of part or all of this work or personal or classroom use is grante without fee provie that copies are not mae or istribute for profit or commercial avantage an that copies bear this notice an the full citation on the first page. To copy otherwise, to republish, to post on servers, or to reistribute to lists, requires prior specific permission an/or a fee. POD 2, Dallas, TX USA ACM x//5...$5. noes. Notwithstaning, the size of a noe cannot be enlarge inenitely, since any increase in noe size contributes ultimately to aitional page accesses an CPU cost, causing the inex to egenerate into a semi-sequential scan within an inex. Secon, as the number of imensions increases, the area covere by the query increases tremenously. Consier a hyper-cube with a selectivity of.1% of the omain space ([,1],[,1],:::,[,1]). This is a relatively small query in two to three-imensional atabases. However, for a 4- imensional space, the query with along each imension works out to be.841, which causes the query to cover a large area of the omain space. Consequently, many leaf noes of a hierarchical inex have to be searche. The above two problems are so severe that the performance is worse o than a simple sequential scan of the inex keys [4] [5]. However, sequential scanning is expensive as it requires the whole atabase to be searche for any range queries, irrespective of query sizes. Therefore, research eorts have been riven to evelop techniques that can outperform sequential scanning. Some of the notable techniques inclue the VA-le [4] an the Pyrami scheme [5]. This paper aopts a slightly ierent approach that reuces high-imensional ata to a single imensional value. It is motivate by two observations. First, ata points in high imensional space can be orere base on the maximum value of all imensions. (We have aopte the maximum value in our iscussion. However, similar observations can be mae with the minimum value.) Secon, if an inex key oes not t in any query range, the ata point will not be in the answer set. The former implies that we can represent high-imension ata in single imensional space, an reuse existing single imensional inexes. The latter provies a mechanism to prune the search space. In this paper, we propose a new tunable inexing scheme, calle (), that aresses the eciency of the simple approach iscusse above. has several nice features. First, () aopts a simple transformation function to map high imension points to a single imension space. Let x min an x max be respectively the smallest an largest values among all the imensions of the ata point (x 1; x 2; :::; x ) x j 1, 1 j. Let the corresponing imension for x min an x max be min an max respectively. The ata point is mappe to y over a single 166

2 imensional space as follows: y = min + x min max + x max if x min + <1, x max otherwise We note that the transformation actually partitions the ata space into ierent partitions base on the imension which has the largest value or smallest value, an provies an orering within each partition. Secon, B + -tree is use to inex the transforme values. Thus, () can be implemente on existing DBMSs without aitional complexity, making it a practical approach. Thir, in (), queries on the original space nee to be transforme to queries on the transforme space. For a given range query, the range of each imension is use to generate a range subqueries on the imension. The union of the answers from all subqueries provies the caniate answer set from which the query answers can be obtaine. 's query mapping function facilitates eective range query processing: (i) the search space on the transforme space contains all answers from the original query, an it cannot be further constraine without the risk of missing some answers; (ii) the number of points within a search space is reuce; an (iii) some of the subqueries can be prune away without being evaluate. Finally, byvarying, we can obtain ierent families of () structures. At the extremes, () maps all high imensional points to the maximum (minimum) value among all imensions; alternatively, it can be tune to map some points to the maximum values, while others to the minimum values. Thus, () can be optimize for ata sets with ierent istributions. Unlike the Pyrami technique, no apriori knowlege or reconstruction is require. We implemente the () an evaluate its performance against the more complex Pyrami technique. Our experimental results on both uniform an skewe ata sets show that the propose scheme can be more ecient than the Pyrami technique by more than 5% of retrieval costs. The rest of our paper is organize as follows: In Section 2, we escribe existing work. In Section 3, we shall present () in etail, an in Section 4, the search algorithms. Section 5 presents the experimental stuy an reports our nings. We conclue in Section RELATED WORK There is a consierable amount of work on high-imensional inexing. In this section, we shall review two of the recently propose inexing structures, namely the Pyrami technique [5] an the VA le [4]. While the Pyrami technique is use for performance comparison in this paper, the VA-le is a potential caniate for further performance stuy. The basic iea of the Pyrami Technique is to transform - imensional ata points into 1-imensional values, an then store an access the values using a conventional inex such as the B + -tree. There are mainly two steps in its partitioning metho. First, it splits the ata space into 2 pyramis, which share the center point of the ata space as their top an have (, 1)- imensional surface of the ata space as their base. h.5 h h 1.5 Pyrami slice Figure 1: Inex key assignment in the Pyrami technique All points locate on the i-th (, 1)-imensional surface of cube (the base of the pyrami) have the common property: either their i-th coorinate is or their (i, )-th coorinate is 1. On the other han, all points locate in the i-th pyrami p i have the furthest istance from the top on the i-th imension. The imension in which the point has the longest istance from the top etermines which pyrami the point lies. Another property of Pyrami Technique is that the location of a point within its pyrami is inicate by a single value, which is the istance from the point to the center point accoring to imension j max (see Figure 1). The ata on the same slice in a pyrami have the same pyrami value. That is, any objects fall on the slice will be represente by the same pyrami value. As a result, many points will be inexe by the same key in a skewe istribution. It has been suggeste that the center point can be shifte to hanle ata skewness. However, this incurs recalculation of all inex values, i.e. reistribution of the points among the pyramis, an reconstruction of the B + -tree. To retrieve apoint q, the pyrami value P of q is compute, an use to serach the B + -tree. All points with P will be checke an retrieve. To perform a range query, the pyramis that intersect the search region are etermine, an for each pyrami, iniviual subquery range is etermine. Each subquery is use to search the B + -tree. For each range query, 2 subqueries may be require, one against each pyrami. The VA-le (vector approximation le) [4] is base on the iea of object approximation by mapping a coorinate to some value that reuces storage requirement. The basic iea is to ivie the ata space into 2 b hyper-rectangular cell where b is the tunable number of bits use for representation. For each imension i, b i bits are use, an 2 b i slices are generate in such away that all slices are equally full. The ata space consists of 2 b hyper-rectangular cell, each of which can be represente by a unique bit string of length b. A ata point is then approximate by the bit string of the cell it falls into. h 2 167

3 To perform a point or range query, the entire approximation le must be sequentially scanne. Objects whose bit string satises the query must be retrieve an checke. Typically, the VA-le is much smaller than the vector le an hence is far more ecient than irect sequential scan of ata le an the variants of R-tree. However, the performance of the VA-le is likely to be aecte by ata istributions an hence the false rop rate; the number of imensions an the volume of ata. 3. INDEXING ON THE EDGES In a multi-imensional range search, all values of all imensions must satisfy the query range along each imension. If any of them fails, the ata point will not be in the answer set. Base on this observation, a straightforwar approach is to inex on a small subset of the imensions. However, the eectiveness of such an approach epens on the ata istribution of the selecte imensions. Our preliminary stuy on inexing one single imension showe that the approach can perform worse than sequential scanning. This le us to examine novel techniques that inex on the \eges". An \ege" of a ata point refers to the maximum or minimum value among all the imensions of the point. The propose technique, () uses either the values of the Max ege (the imension with the maximum value) or the values of the Min ege (the imension with the minimum value) as the representative inex keys for the points. Because the transforme values can be orere an range queries can be performe on the transforme (single imensional) space, we can employ single imensional inexes to inex the transforme values. In this paper, we employ the B + -tree structure since it is supporte by all commercial DBMSs. Thus, () can be reaily aopte for use. In the following iscussion, we consier a unit -imensional space, i.e., points are in the space ([,1],[,1],...,[,1]). We enote an arbitrary ata point in the space as x =(x 1; x 2; :::;x ). Let x max = max i=1 xi an xmin = min i=1 xi be the maximum value an minimum value among the imensions of the point. Moreover, let max an min enote the imensions at which the maximum an minimum values occur. Let the range query be q =([x 11;x 12], [x 21;x 22], :::,[x 1;x 2]). Let ans(q) enote the answers prouce by evaluating a query q. In the following iscussion, we shall present the transformation function that maps a point inaimen- sional space to a single imensional space, an iscuss how range queries can be evaluate. 3.1 Mapping High Dimensional Data to Single Dimension Space () aopts a simple mapping function that is computationally inexpensive. The ata point x is mappe to a point y over a single imensional space as follows: y = min + x min max + x max where is a real number. if x min + <1, x max otherwise First, we note that plays an important role in inuencing the number of points falling on each inex hyperplane. In fact, it is the tuning knob that aects the hyperplane an inex point shoul resie. Take a ata point (.2,.75) in 2-imensional space for example, with = :, the inex point will resie on the Min ege. By setting to.1 will push the inex point to resie on the Max ege. The higher the value of, the biasness the function is expressing towars the Max ege. When =:1, the Max ege has the preference of about 1% more. Similarly, we can \favor" the transformation to the Min ege with <. In fact, at one extreme, when 1:, the transformation maps all points to their Max ege. an by setting,1:, we always pick the value at the Min ege as the inex key. For simplicity, we shall enote the former extreme as imax, the latter extreme as imin, an any other variation as (ropping unless its value is critical). Secon, we note that the transformation actually splits the (single imensional) ata space into ierent partitions base on the imension which has the largest value or smallest value, an provies an orering within each partition. This is eecte by incluing the imension at which the maximum value occurs, i.e., the rst component of the mapping function. Finally, the unique tunable feature facilitates the aaptation of () to ata sets of ierent istributions (uniform or skewe). In cases where ata points are skewe towar certain eges, we may \scatter" these points to other eges to evenly istribute them by making a choice between min an max. Statistical information such as the number of inex points can be use for such purpose. Alternatively, one can either use the information regaring ata istribution or information collecte to categorically ajust the partitioning. 3.2 Mapping Range Queries Range queries on the original -imensional space have tobe transforme to the single imensional space for evaluation. In (), the original query on the -imensional space is mappe into subqueries one for each imension. Let us enote the subqueries as q 1, q 2, :::, q, where q i =[l i;hi] 1 i. For the jth query subrange in q, [x j1;x j2], we have q j as given by the expression in Equation 1. The union of the answers from all subqueries provies the caniate answer set from which the query answers can be obtaine, i.e., ans(q) [ i=1ans(q i). We shall now prove some interesting results. Theorem 1. Uner () scheme, ans(q) [ i=1ans(q i). Moreover, there oes not exist qi =[li;h i], where li >l i or h < h i for which ans(q) [ i=1ans(qi) always hols. In other wors, q i is \optimal" an narrowing its range may miss some of q's answers. Proof: For the rst part, we nee to show that any point x that satises q will be retrieve by some q i; 1 i. For the secon part, we only nee to show that some points that satisfy q may be misse. The proof comprises three parts, corresponing to the three cases in the range query mapping function. Case 1: min i=1 x i1 + 1, max i=1 x i1 In this case, all the answer points that satisfy the query q have been mappe to the Max ege, i.e., a point x that 168

4 q j = ( [j + max i=1 x i1; j+ x j2] if min i=1 xi1 + 1, max i=1 xi1 [j + x j1; j+ min i=1 x i2] if min i=1 x i2 + <1, max i=1 x i2 [j + x j1; j+ x j2] otherwise Equation 1: Transformation function for queries. satises q is mappe to x max, an woul have been mappe to the maxth imension, an has inex key of max + x max. The subquery range for the maxth imension is [ max + max i=1 x i1; max + x max 2]. Since x satises q, wehave x i 2 [x i1;x i2], 8i; 1 i. Moreover, we have x max x i1 8i; 1 i. This implies that x max max i=1 x i1 8i; 1 i. We also have x max x max 2. Therefore, we have x max 2 [max i=1 x i1; x max 2], i.e., x can be retrieve using the maxth subquery. Thus, ans(q) [ i=1 ans(qi). Now, let qi =[li + l;hi, h], for some l > an h >. Consier a point z =(z 1; z 2; :::;z ) that satises q. We note that if l i <zmax <li + l, then, we will miss z if qi has been use. Similarly, if h i, h <zmax < max i=1 x i2, then, we will also miss z if qi has been use. Therefore, no qi provies the tightest boun that guarantees that no points will be misse. Case 2: min i=1 x i2 + <1, max i=1 x i2 This case is the inverse of Case 1, i.e., all points in the query range belongs to the Min ege. As such, we can apply similar logic. Case 3 In case 3, the answers of q may be foun in both the Min ege an the Max ege. Given a point x that satises q, we have x i 2 [x i1;x i2], 8i; 1 i. Wehave two cases to consier. In the rst case, x is mappe to the Min ege, its inex key is min + x min, an it is inex on the minth imension. To retrieve x, we nee to examine the minth subquery, [ min + x min 1; min + x min 2]. Now, we have x min 2 [x min 1;x min 2] (since x is in the answer) an hence the minth subquery will be able to retrieve x. The secon case, which is mapping x onto the Max ege an can be similarly erive. Thus, ans(q) [ i=1 ans(qi). Now, let q i =[l i + l;h i, h], for some l > an h >. Consier a point z =(z 1; z 2; :::;z ) that satises q. We note that if l i <zmax <li + l, then, we will miss z if q i has been use. Similarly, if h i, h <zmax <h i, then, we will also miss z if q i has been use. Therefore, no q i provies the tightest boun that guarantees that no points will be misse. We woul like to point out that in an actual implementation, the leaf noes of the B + -tree will contain the highimensional point, i.e., even though the inex key on the B + -tree is only single imension, the leaf noe entries contain the triple (x key;x;ptr) where xkey is the single imensional inex key of point x an ptr is the pointer to the ata page containing other information that may be relate to the high-imensional point. Therefore, the false rop of Theorem 1 aects only the vectors use as inex keys, rather than the actual ata itself. 2 T heorem 2. Given a query q, an the subqueries q 1;q 2; :::; q, q i nee not be evaluate if any of the followings hols: (i) min (ii) min x j1 + 1, x j2 + <1, max max x j1 an h i < x j2 an l i > max x j1 min x j2 Proof: Consier the rst case: min x j1+ 1,max x j1 an h i < max x j1. The rst expression implies that all the answers for q can only be foun in the Max ege. We note that the point with the smallest maximum value that satises q is max x j1. This implies that if h i < max xj1, then the answer set for qi will be an empty set. Thus, q i nee not be evaluate. The secon expression means that all the answers for q are locate in the Min ege. The point with the largest minimum value that satises q is min xj2. This implies that if l i > min x j2, then the answer set for q i will be empty. Thus, q i nee not be evaluate. Example 1. Let = :5. Consier the range query ([.2,.3], [.4,.6]) in 2-imensional space. Since :2 + :5 > 1, :4 =:6, we know that all points that satisfy the query falls on the Max ege. This means that the lower boun for the subqueries shoul be.4, i.e., the two subqueries are respectively [.4,.3] an [.4,.6]. Since the rst subquery's upper boun (i.e,.3) is smaller than.4, it nee not be evaluate because no points will satisfy the query. T heorem 3. Given a query q, an the subqueries q 1;q 2;:::;q, at most subqueries nee to be evaluate. Proof: The proof is straightforwar, an follows from Theorem 2. From theorems 2 an 3 we have a glimpse of the eectiveness of (). In fact, for very high imension spaces, we can expect signicant savings from the pruning of subqueries. Example 2. In this example, we illustrate how can keep out points from the search space. Figure 2 shows the example. Here, we have two points A(.2,.5) an B(.87,.25) in 2-imensional space. If we employ either imax or imin, at least one false rop will occur. On the other han, using (.5) eectively keeps both points out of the search space

5 y y 1 1 A (.2,.5) A B(.87,,25) B x x 1 (a) imax (b) imin Figure 2: Sample search space for 2-imensional space. 4. IMINMAX() SEARCH ALGORITHMS In our implementation of (), we have aopte the B + -tree [6] as the unerlying single imensional inex structure. However, for greater eciency, leaf noes also store the high-imensional key, i.e., leaf noe entries are of the form (key, v, ptr) where key is the single imensional key, v is the high-imensional vector whose transforme value is key, an ptr is the pointer to the ata page containing information relate to v. Keeping v at the leaf noes can minimize page accesses to non-matching points. We note that multiple high-imensional keys may be mappe to a single key value. The search, insert an elete algorithms are similar to the B + -tree algorithms. The aitional complexity arises as we have to eal with the aitional high-imensional key (besies the single imensional key value). In this paper, we shall present the search algorithms (for both point an range queries). Insert an elete algorithms are similar to those of B + -tree, an so we omit them. 4.1 Point Search Algorithm In point search, a point p is issue an all matching tuples are to be retrieve. Suppose has been tune for performance purposes, the maximum an minimum values of p have to be use to search the tree. In this case, a that will cause the search to be one on the other ege can be use to call the algorithm. The algorithm is summarize in Figure 3. Base on, the search algorithm rst maps p to the single-imensional key, x p, using the function transform(point, ) (line 1). For each query, the B + -tree is traverse (line 2) to the leaf noe where x p may be store. If the point oes not exist, then a NULL value is returne (lines 3-4). Otherwise, for every matching x p value, the high-imensional key of the ata is compare with p for a match. Those that match are accesse using the pointer value (lines 7-13); otherwise, they are ignore. We note that it is possible for a sequence of leaf noes to contain matching key values an hence they all have to be examine. The nal answers are then returne (line 14). 4.2 Range Search Algorithm Range queries are slightly more complicate than point search. Figure 4 shows the algorithm. Unlike point queries, a - imensional range query r is transforme into subqueries (line 2,3). The ith subquery is enote as r i =[l i;hi]. Next, routine prunesubquery is invoke to check if r i can be prune (line 4). If it can be, then it is ignore. Otherwise, the subquery is evaluate as follows (lines 5-12). The B + -tree is traverse to the appropriate leaf noe. If there are no points in the range of r i, then the subquery stops. Otherwise, for every x 2 [l i;hi], the high-imensional key of the ata is compare with p for a match. Those that match are accesse using the pointer value. As in point search, multiple leaf pages may have to be examine. Once all subqueries have been evaluate, the nal answers are then returne (line 13). 5. PERFORMANCE STUDY We implemente () an the Pyrami technique [5] in C, an use the B + -tree as the single imensional inex structure. Each inex page is 4 KB pages. We i not buer any ata pages in this stuy. Therefore, every page touche incurs an I/O. However, it shoul be note that the traversal paths of the subqueries generate by () o not overlap an hence share very few common internal noes. This is also true for the subqueries generate by the Pyrami technique. For the performance stuy reporte, we i not use Theorem 2 to prune any subqueries. We conucte many experiments. Here, we report some of the more interesting results on range queries. A total of 5 range queries are use. Each query is a hyper-cube an has a efault selectivity of.1% of the omain space ([,1],[,1],:::,[,1]). The query with is the -th root of the selectivity: p :1. As an inication on how large the with of a fairly low selectivity can be, the query with for 4-imensional space is.841, which ismuch larger than half of the extension of the ata space along each imension. Different query size will be use for non-uniform istributions. The efault number of imensions use is 3. Each I/O correspons to the retrieval of a 4 KB page. The average I/O cost of the queries is use as the performance metrics. 17

6 Algorithm PointSearch Input: point p,, root of the B + -tree R Output: tuples matching p 1. x p transform(p, ) 2. l traverse(x p, R) 3. if x p is not foun in l 4. return (NULL) 5. else 6. S ; 7. for every entry in l with key x p,(x p; v; ptr) 8. if v == p 9. tuple access(ptr) 1. S S[ tuple 11. if l's last entry contains key x p 12. l l's right sibling 13. goto return (S) Algorithm RangeSearch Figure 3: Point search algorithm. Input: range query r =([x 11;x 12]; [x 21;x 22];:::), root of the B + -tree R Output: answer tuples to the range query 1. S ; 2. for (i =1to) 3. r i transform(r, i) 4. if NOT(pruneSubquery(r i, r)) 5. l traverse(l i, R) 6. for every entry in l with key x 2 [l i;hi], (x; v; ptr) 7. if v == P 8. tuple access(ptr) 9. S S[ tuple 1. if l's last entry contains key x<h i 11. l l's right sibling 12. goto return (S) Figure 4: Range search algorithm. 5.1 Effect of Dimensions In the rst set of experiments, wevary the number of imensions from 8 to 5. The ata set is uniformly istribute over the omain space. There are a total of 1K points. In the rst experiment, besies the Pyrami scheme, we also compare against the MAX scheme an the sequential scan (seq-scan) technique. The MAX scheme is the simple scheme that maps each point to its maximum value. However, the transforme space is not partitione. Moreover, two variations of () are use, namely imax (i.e., = 1) an ( = :) (enote as ). Figure 5 shows the results. First, we note that both the MAX an seq-scan techniques perform poorly, an their I/O cost increases with the higher number of imensions. MAX performs slightly worse because of the aitional internal noes to be accesse an the high number of false rops. Secon, while the number of I/Os for, imax an Pyrami also increases with increasing number of imensions, it is growing at a much slower rate. Thir, we see that performs the best, with Pyrami following closely, an imax performing worse than Pyrami. imin- Max outperforms imax an Pyrami since its search space touches fewer points. In a typical application, apart from the inex attributes, there are many more large attributes which make sequential scan of an entire le not cost eective. Instea, a feature le which consists of vectors of inex attribute values is use to lter out objects (recors) that o not match the search conition. However, we note that for queries which entail retrieval of a large proportion of objects, irect sequential scan may still be cost eective. The ata le of the technique can be clustere base on the leaf noes of its B + - tree to reuce ranom reas. The clusters can be forme in 171

7 MAX imax Pyrami Seq-Scan Dimension Data Set Size (K) Pyrami Figure 5: Eect of imensions on uniformly istribute ata set Dimension Pyrami Figure 6: Comparing an Pyrami schemes. such a way they allow easy insertion of objects an expansion of their extent. Other optimizations such as those at the physical level are possible to make the B + -tree behave like an inex sequential le. Base on above argument an experimental results, for all subsequent experiments, we shall restrict our stuy to an Pyrami techniques. We further evaluate Pyrami an an the results are shown in Figure 6. We observe that remains superior, an can outperform Pyrami by up to 25%. 5.2 Effect of Data Set Sizes an Query Sizes In this set of experiments, we stuy several ierent factors - the ata set sizes, the query selectivities. For both stuies, we xe the number of imensions at 3. Figure 7 shows the results when we vary the ata set sizes from 1K to 5K points. Figure 8 shows the results when we vary the query selectivities from.1% to 1%. Figure 7: Eect of varying ata set sizes Pyrami Selectivity of Query (%) Figure 8: Eect of varying query selectivity (1K ataset). As expecte, both an Pyrami incurre higher I/O cost with increasing ata set sizes as well as the query selectivities. As before remains superior over the Pyrami scheme. It is interesting to note that the relative ierence between the two schemes seems to be unaecte by the ata set sizes an query selectivities. Upon investigation, we foun that both an Pyrami return the same caniate answer set. The improvement of stems from its reuce number of subqueries compare to the Pyrami scheme. 5.3 Effect of Data Distributions In this experiment, we stuy the relative performance of an Pyrami on skewe ata istributions. Here, we show the results on two istributions, namely skewe normal an skewe exponential. Figure 9 illustrates two skewe ata istributions in a 2-imensional space. The rst set of experiments stuies the eect of on skewe 172

8 norm (a) Normal istribution.1 exp (b) Exponential istribution Figure 9: Skewe ata istribution. normal istribution. For normal istribution, the closer the ata center is to the cluster center, the more we can keep points evenly assigne to each ege. For queries that follow the same istribution, the ata points will have the same probability of being kept far from the query cube. In these experiments, we x each imension of the query to have a with f.4. Figure 1(a) shows the results for 1K 3-imensional points. First, we observe that for, there exists a certain optimal value that leas to the best performance. Essentially, \looks out" for the center of the cluster. Secon, can outperform the Pyrami technique by a wie margin (more than 5%!). Thir, we note that can perform worse than the Pyrami scheme. This occurs when the istribution of points to the eges become skewe, an a larger numberofpoints have to be searche. Because of the above points, we note that it is important to ne tune for ierent ata istributions in orer to obtain optimal performance. The nice property is that this tuning can be easily performe by varying. In Figure 1(b), we have the results for 5K 3-imensional points. As in the earlier experiment, 's eectiveness epens on the value set. We observe that performs better than Pyrami over a wier range of tuning factors, an over a wier margin (more than 66%) Pyrami Tuning value of (a) 1K points Pyrami Tuning value of (b) 5K points Figure 1: Skewe normal ata set. The secon set of experiments looks at the relative performance of the schemes for skewe exponential ata sets. As above, we x each imension of the query to have a with of.4. For exponential istribution (we choose to be exponential to small value), many imensions will have small values, an a small number of them will have large values. Thus, lots of ata points will have at least one big value. Because many of the imensions are with small values, the ata points ten to lie close along the eges of ata space. We note that exponential ata istribution can be far ifferent from each other. They are more likely to be close along the eges, or close to the ierent corners epening 173

9 on the number of imensions that are skewe to be large, or small. A range query, if it is with exponentially istribution characteristic, its subqueries will mostly be close to the low corner. Therefore, tuning the keys to choose large values is likely to keep away more points from the query. Figure 11 shows the results for 5K 3-imensional points on skewe exponential istribution. The results are similar to that of the normal istribution experiments is optimal at certain values. [4] R. Weber, H. Schek, S. Blott. A Quantitative Analysis an Performance Stuy for Similarity-Search Methos in High-Dimensional Spaces. VLDB'98, , [5] S. Berchtol, C. B}ohm, H-P. Kriegel. The Pyrami-Technique: Towars Breaking the Curse of Dimensionality. SIGMOD'98, , [6] D. Comer. The Ubiquitous B-tree. ACM Computing Surveys. 11(2), 121{137, Pyrami Tuning value of Figure 11: Skewe exponential ata sets (5K points). 6. CONCLUSION In this paper, we have propose a simple an yet very ecient metho for inexing very high imensional ata base on eges. We have shown by experiments that the metho is signicantly more ecient an ynamic than the Pyrami technique. Performance ierence is expecte to increase as the ata volume an imensionality increase, an for skewe ata istributions. We are currently comparing against the VA-le [4] an looking at how to generalize () for nearest neighbor search. We are also looking at the possibility an the eect of maintaining i for each imension. Finally, we are exploring how to etermine aaptively. 7. ACKNOWLEDGEMENT This work is partially supporte by the Global-Atlas project fune by the National University of Singapore. 8. REFERENCES [1] E. Bertino, et. al. Inexing Techniques for Avance Database Systems. Chapters 2, , Kluwer Acaemic Publishers, Boston, [2] A. Guttman. R-tree: A ynamic Inex Structure for Spatial Searching. SIGMOD'84, 47-54, [3] N. Beckmann, H-P. Kriegel R. Schneier, B. Seeger. The R -tree: An Ecient an Robust Access Metho for Points an Rectangles. SIGMOD'9, ,