[5] R. A. Dwyer. Higher-dimensional Voronoi diagrams in linear expected time. Discrete

Transcription

1 [5] R. A. Dwyer. Higher-dimensional Voronoi diagrams in linear expected time. Discrete and Computational Geometry, 6(4):343{367, [6] J. H. Friedman and L. C. Rafsky. Multivariate generalizations of the Wald-Wolfowitz and Smirnov two-sample tests. Annals of Statistics, 7:697{717, [7] J. H. Friedman and L. C. Rafsky. Graph-theoretic measures of multivariate association and prediction. Annals of Statistics, 11:377{391, [8] J. W. Pratt and J. D. Gibbons. Concepts of onparametric Theory. Springer-Verlag, ew York, [9] F. P. Preparata and M. I. Shamos. Computational Geometry: An Introduction. Springer- Verlag, [10] M. F. Schilling. Multivariate two-sample tests based on nearest neighbors. J. Amer. Stat. Assoc., 81(395<):799{806, [11] R. Seidel. Constructing higher-dimensional convex hulls at logarithmic cost per face. In Proc. 18th ACM Symp. on Theory of Computing, pages 404{413. ACM, May [12] A. C. Yao. On constructing minimum spanning trees in k-dimensional space and related problems. SIAM J. Comput., 11(4):721{736,

2 The MST and tests are applicable with moderate n in any dimension because running times for constructing these graphs grows only linearly with the dimension the cost of a single distance computation increases, but the number of distance computations is stable. The Voronoi test, on the other hand, is computationally very demanding in high dimensions even when n is relatively small. Friedman and Rafsky's and Schilling's articles both present simulation results for 10- and 20-dimensional data. Our Voronoi dual algorithm was not able to nish simulations in these dimensions. The reason is clear in light of the following fact: [5, p. 355] If the points are uniformly distributed, the expected number of (d + 1)-cliques that must be enumerated by the algorithm for large n is approximately 2 (d+1)=2 e 1=4 (d01)=2 d (d02)=2 n: d + 1 (This estimate is accurate to 1% for d 5.) For d = 5, 10, and 20, this number is about 189n, ( )n, and ( )n respectively. evertheless, the Voronoi test is both statistically powerful and computationally feasible for small dimensions. References [1] M. Bern, D. Eppstein, and F. Yao. The expected extremes in a Delaunay triangulation. Int. J. Comput. Geom. Appl., 1(1):79{92, March [2] K. L. Clarkson and P. W. Shor. Applications of random sampling in computational geometry, II. Discrete and Computational Geometry, 4:387{421, [3] H. E. Daniels. The relation between measures of correlation in the universe of sample permutations. Biometrika, 33:120{135, [4] L. P. Devroye. Lecture otes on Bucket Algorithms. Birkhauser, Boston,

3 parameter; rather, it reects the geometric realities of the data. 5 Computational Issues The simplest algorithms for computing minimum spanning trees and nearest neighbor subgraphs in R d require O(d 2 ) time in the worst case. Subquadratic algorithms exist [12], but are relatively complicated. In two dimensions, O(n log n) time suces. [9] The faster algorithms work by rst constructing the Voronoi dual. KMST and K graphs can be computed in O(kd 2 ) time. In the worst case, the Voronoi dual may have 2(n b(d+1)=2c ) cliques. Since existing algorithms must enumerate these cliques to nd the edges, at least this much time is apparently required to construct a Voronoi dual. However, the number of cliques appears to be linear in n in more \typical" cases, and it is known that the Voronoi diagram of n independent points uniform in the unit ball can be constructed in O(n) time on average for xed d. [5] The Voronoi dual of a set of points in R d can be found by computing the convex hull in R d+1 of the images of the points under the mapping (x 1 ; : : : ; x d ) 7! (x 1 ; : : : ; x d ; x 2 i ): The downward-facing facets of this convex hull are in one-to-one correspondence with the vertices of the Voronoi diagram and the (d+1)-cliques of its dual. Thus, all the algorithms for this well-studied problem are applicable. Among them are the gift-wrapping and beneathbeyond algorithms [9], Seidel's shelling algorithm [11], and Clarkson and Shor's randomized incremental algorithm [2]. To carry out the simulations described in the preceding section, we used an algorithm that combines techniques of gift-wrapping with randomized divideand-conquer. We conjecture that this algorithm requires only O(n log n) time on average in any xed dimension when points are chosen from any absolutely continuous distribution. 11 d i=1

4 the Wald-Wolfowitz test using the KMST for k = 1; 2; 3 are recorded in the tables. For Smirnov's test, the MST can rooted at an arbitrarily chosen vertex, and a rank is assigned to each vertex based upon the length of the MST path from the root to the vertex. The eccentricity of a node is the length (in edges) of the longest simple path in the MST beginning at that node. Friedman and Rafsky rooted the MST at nodes of maximum and minimum eccentricity; these correspond to the rst and second table entries respectively. Schilling [10] analyzes a two-sample test based on the K graph. He shows that power can be gained against specic alternatives by weighting edges based on the location of the sample values. He also derives asymptotic values for the vertex degrees in the K graph and develops tests that require no conditioning on the graph. We include his conditioned results because conditioning results in a more powerful, if less general, test, and because our test also requires conditioning on the graph. The results of Schilling's K test are reprinted for k = 1; 2; 3. The Voronoi test compares very favorably with the others. The older tests demonstrate power only against location or scale alternatives. The Voronoi test exhibits power against both alternatives. As the dimension increases, so does the Voronoi test's relative power. When d = 5, the Voronoi results clearly dominate the others. However, this power comes at a high computational price. Computational issues are discussed in the next section. The Voronoi test's high power is a result of the graph's density, which increases steeply with d. During the simulations, the maximum vertex degree observed in the Voronoi dual was 12 for d = 2 and 78 for d = 5. The average degree was 6 for d = 2 and 49 for d = 5. It was never necessary to apply step 2 to delete edges. One might argue that a 49-MST or 49- test would do just as well when d = 5. This is perhaps true, but merely serves to highlight another strength of the test: The density of the graph is not determined by an arbitrary 10

5 4 mpirical Comparisons In this section, we present results of simulations comparing the power of our two-sample test to earlier tests. These results are summarized in Tables 1 and 2. Each table entry reports the number of trials for which the test was able to reject the null hypothesis at a 5% signicance level. One hundred trials were carried out for each table entry, with m = n = 100. Table 1 exhibits the power of the tests against location alternatives. One sample was drawn from the standard normal distribution, (0; I d ), and the other from (1e 1 ; I d ). Table 2 deals with scale alternatives. (0; I d ) and (0; 2 I d ) were used as the sample distributions. Dimension and distribution were chosen to coincide with previously published results [6, 10]. d = 2; 1 = 0:5 d = 5; 1 = 0:75 Voronoi MST WW 1,2, MST Smirnov K 1,2, Table 1: Location Alternatives d = 2; = 1:2 d = 5; = 1:2 Voronoi MST WW 1,2, MST Smirnov K 1,2, Table 2: Scale Alternatives MST results are reprinted from Friedman & Rafsky's work [6] describing multivariate Wald- Wolfowitz and Smirnov tests. They extend both tests by computing the MST of the pooled sample. To generalize the Wald-Wolfowitz test, Friedman and Rafsky count the number of edges (X i ; Y j ) between the samples in the MST of the pooled sample. Results of applying 9

6 (2d + k + 2) ln d d01 e 0(2d+k+2) ln d+1 = 0k02 : d+1 The second step in the derivation above holds whenever is large enough that ( 0 (2d + k + 2) ln ) 1. Therefore, the probability that Z j is a neighbor of Z 1 in the Voronoi dual given that it lies outside of B (Z 1 ) is less than 1= k+2. Since there are fewer than sites outside of B (Z 1 ), the probability that any site outside of B (Z 1 ) is adjacent to Z 1 most 1= k+1. 2 is at Finally, we can show that the second step of our test will rarely be used in practice. It is included only to ensure that S is asymptotically normal. Theorem. Suppose g is continuous on its support, and Z 1 ; : : : ; Z are i.i.d. with density g. Then given k > 0, we can choose c k such that the probability that the Voronoi dual contains a vertex with degree greater than c k ln is O( 0k ). Proof. Let c k be as in Lemma 1. If vertex Z 1 has more than c k ln neighbors in the Voronoi dual, either it has a neighbor outside of B (Z i ), or there are more than c k ln neighbors inside B (Z i ). By Lemmas 1 and 2, the probability of either occurring is at most 2= k+1. Since the sites are i.i.d., the probability is clearly the same for the other 01 sites. Therefore, the probability that any site has high degree is at most 2= k. 2 Bern, Eppstein, and Yao [1] found that the expected maximum degree of the Voronoi dual is 2(ln = ln ln ) for a Poisson process in R d. This conrms that our bound of c ln is not very restrictive. 8

7 k + 2)2 d+2 + k + 1. The probability content of B (x), denoted by P, is P 2v da x g(x) ln (2d + k + 2)2d+2 ln : Applying Cherno's bound PrfX xg e 0x E(e X ) [4], we see that the probability of having c k ln or more sites in B (x) is ic k ln i P i (1 0 P ) 0i 1= k+1 : 2 Lemma 2. Suppose g is continuous on the interior of its support. Then for all large, the probability that a neighbor of Z 1 in the Voronoi dual lies outside of B (Z 1 ) is 1= k+1. Proof. The probability that Z 1 is on the boundary of the support of g is zero, so we may ignore this case. For Z 1 in the interior, continuity implies that y 2 B (Z 1 ) g(y) g(z 1 )=2 for all large. ow x j and suppose that Z j does not lie inside B (Z 1 ). If Z 1 and Z j are neighbors in the Voronoi dual, then least one of the 02 d01 balls dened by Z 1, Z j, and d 0 1 other sites must be empty. Consider one such ball, B. Let v denote the unit vector in the direction from Z 1 to the center of B, and let r denote the radius of B (Z 1 ). Then the ball centered at Z 1 + rv=2 with radius r=2 lies inside B (Z 1 )\B, and must be empty. Its probability content is P v dg(x) a x ln 1=d d = v da x g(x) ln 2 d+1 = (2d + k + 2) ln ; and it is empty with probability (1 0 P ) 0d01. Summing over all 02 d01 balls, the probability that Z 1 and Z j are neighbors is 0 2 d 0 1 (2d + k + 2)) ln d01

8 2. If any vertex has degree greater than c ln, delete an edge of the graph at random. Repeat until no vertex has degree greater than c ln. 3. Compute E, the number of edges; S, the number of X-Y edges; and, the number of edge pairs sharing an endpoint. 4. se equations 1 and 2 and the asymptotic normality of the test to accept or reject the null hypothesis based on the given signicance level. Since we use a graph having no degree greater than c ln, Condition 4 is satised, and S is asymptotically normally distributed. The main reason for using the Voronoi dual is that it is denser than the other graphs, but still tends to have edges only between geometrically close points. Randomly deleting edges as in step 2 of the test would appear to defeat this purpose. We will show that we can choose c so that this step will rarely be invoked. Let Z 1 ; : : : ; Z be i.i.d. with density g in R d. A d-ball of radius r has volume v d r d, where v d = d=2 = (d=2 + 1) depends only on d. When g(x) > 0, dene a x = (2d + k + 2)2d+1 ; v d g(x) and denote by B (x) the hypersphere with center x and radius (a x ln =) 1=d. The following two lemmas will be used to prove that the probability of invoking step 2 of our test can be made arbitrarily small. Lemma 1. Let x be xed, and suppose g is continuous and positive at x. Then for any xed k > 0, there exists c k such that for all large, the probability that c k ln or more sites lie in B (x) is 1= k+1. Proof. By continuity, y 2 B (x) g(y) 2g(x) for all large. Dene c k = (e 0 1)(2d + 6

9 all possible ways of assigning X to n vertices and Y to the other m, and calculate the test statistic in each case. The expectation and variance are: ar[s j E; ] = 2mn ( 0 1) E + + [S j E] = 2mnE ( 0 1) 2(m 0 1)(n 0 1) 2mnE2 [E(E 0 1) 0 2 ] 0 ( 0 2)( 0 3) ( 0 1) (1) (2) sing Daniels's generalized indicator variable approach [3], they also show that S is asymptotically normal if either of the following conditions holds: lim!1 i=1 d 2 i 3 (3) ( lim i=1 d 3 i ) 2!1 ( i=1 d 2 i ) = 0 (4) 3 The rst condition is satised by many dense graphs, while the second condition holds for many sparse graphs. The Voronoi dual does not necessarily satisfy either of these conditions. The maximum degree in the Voronoi dual is not bounded by a constant as it is for the KMST and K graphs. For example, suppose we had one point in the center of a sphere, and 0 1 points uniformly distributed on its surface. Then the center vertex would be adjacent to every other point, while the other points would have a relatively low degree. In this case, neither condition holds. Fortunately, the lack of degree bounds in the Voronoi dual does not disqualify it for twosample testing. Shortly, we will show that under very mild conditions, we can choose a constant c such that the probability that d i c ln is very small. Thus, pathological cases like the one just mentioned are negligible. The two-sample test using the Voronoi dual takes the following form: 1. Construct the Voronoi dual of the pooled sample. 5

10 d dimensions, d + 1 sites form a clique of the Voronoi dual if and only if the hypersphere passing through them is \empty". The convex hull of the points is partitioned into simplices by the Voronoi dual, with each simplex being the convex hull of the points in a (d+1)-clique. It is possible to construct sets of n points for which the Voronoi dual is just the complete graph. [9, p. 247] However, if the points are uniform in a unit ball, the expected number of edges approaches dn, where d depends exponentially on the dimension but is independent of n. [5] We argue in Section 3 that, subject to rather mild conditions on the population distribution, the number of edges is approximately linear with high probability. An interesting property of the Voronoi dual is that it contains a minimum spanning tree as a subgraph. In fact, MST Voronoi dual. (Preparata & Shamos [9] discuss the many interesting properties of these graphs in some detail.) eoretical roperties of t e oronoi est Let X = fx 1 ; : : : ; X n g and Y = fy 1 ; : : : ; Y m g be our two sets of sample observations. Dene the pooled sample Z, such that Z i = X i for 1 i n and Z i = Y i0n for n < i n+m =. Each Z i sample value has d components, so each observation is viewed as a point in d- dimensional space. The general two-sample test will test the null hypothesis that the two sets were drawn from the same population versus all alternatives. Let be any graph formed on the pooled sample. Clearly has vertices. Let E denote the number of edges in. Further, dene = d i=1 i(d i 01)=2, where d i is the degree of the ith vertex in. The value represents the number of pairs of edges that share a common endpoint. The test statistic, S, is the number of edges (X i ; Y j ) joining points in dierent sets. Friedman and Rafsky [7] derived the expected value and variance of S by conditioning on the particular graph constructed on the pooled sample. Given a xed, they consider 4

11 to the Euclidean distance d(z i ; Z j ). We denote this graph by. A subgraph of any graph is a k-clique if it has k vertices, every pair of which are joined by an edge. A subgraph spans a graph if its edges include a path (not necessarily a single edge) between every pair of vertices in the larger graph. A tree is a connected, acyclic graph. The weight of a subgraph is the sum of the weights of its edges. We consider three spanning subgraphs of having the property that edges with smaller weights tend to be retained, while those edges between distant nodes tend to be eliminated. These graphs are the nearest-neighbor subgraph, the minimum spanning tree, and the Voronoi dual. The nearest neighbor subgraph of, denoted by, is a graph in which each node of is joined to its closest neighbor. The k-nearest-neighbor subgraph of, K, joins each node to its k nearest neighbors. The number of edges E in such a graph is not xed, but satises k=2 E k. A minimum spanning tree (MST) of a graph is a subgraph of least weight among all the connected spanning subgraphs. Let 1 be a MST of the complete graph, and let 2 be a minimum spanning tree of 0 1. Friedman and Rafsky call 2 a 2-MST. A k-mst, k, is a minimum spanning tree of k01. The KMST is dened to be the union of 1; : : : ; k. The KMST has ( 0 1) edges. The Voronoi diagram of a set of sites is a partition of R d into regions, i, such that d(y; Z i ) d(y; Z j ) for all y 2 i. Region i contains all points in R d that are closer to Z i than to any other of the sites. We dene a dual graph by including an edge (Z i ; Z j ) if and only if i \ j has dimension d 0 1. In two dimensions, this graph is known as the Delaunay triangulation. Three sites form a triangle in the Delaunay triangulation if and only if the (unique) circle passing through the sites contains no other points of the set. Likewise, in 3

12 The concept of ranking does not have an obvious generalization in higher dimensions. Friedman & Rafsky [6] propose treating the two samples as two sets of points in Euclidean space. We may think of one set colored red and the other, blue. The two samples are pooled, and a graph is constructed in which the presence of an edge indicates geometric proximity of its endpoints. We call such a graph a proximity graph. If, for example, there are equal numbers of red and blue points, and the samples are indeed drawn from the same population, then we expect about 1=2 of the edges to join a red point to a blue point, 1=4 to join two red points, and the remaining 1=4 to join two blue points. If, on the other hand, the samples are from distinct populations, we expect the proportion of monochrome edges to be larger. Thus, we choose the number of red-blue edges in the graph as the test statistic, and calculate its distribution under the null hypothesis that the populations are identical. Graphs used previously for this test include the minimum spanning tree [6] and various nearest-neighbor graphs [10]. To improve upon these tests, we use a denser proximity graph, the dual of the Voronoi diagram. We derive the theoretical properties of a multivariate two-sample test based on this graph, and present empirical evidence that the test is more powerful than earlier proximity-graph tests. In Section 2, we review necessary graph-theoretic concepts, including the Voronoi diagram and its dual. We describe our two-sample test and its theoretical properties in Section 3. Simulation results are presented in Section 4 and compared to tests proposed by others. Finally, in Section 5, we briey discuss computational issues. ro imit rap s Let Z = fz 1 ; ; Z g be a set of sites (points, observations) in R d. The complete weighted graph on Z includes an edge between each of the ( 01)=2 pairs (Z i ; Z j ) with weight equal 2

13 - e. e e. e 3 ; e e e e e, Intro uction Suppose two sets of observations of a random variable are given. A two-sample test is a method for determining whether the two sets were drawn from the same population. The Smirnov and Wald-Wolfowitz tests [8] are well-known nonparametric univariate twosample tests. These tests rely on the ability to rank, or sort, the pooled sample values. Given a partition of the ranks between the two samples, one can calculate the probability that the ranking would occur under the null hypothesis that the two samples are drawn from the same population.,,,,, 1