Approximation Algorithms for Outlier Removal in Convex Hulls

Transcription

1 Approximation Algorithms for Outlier Removal in Convex Hulls Michael Biro Justine Bonanno Roozbeh Ebrahimi Lynda Montgomery Abstract Given n points in R 2, we give approximation algorithms to find a subset of k of the points that has minimum-area or minimum-perimeter convex hull. We give algorithms that individually yield constantfactor approximations to the minimum-perimeter problem with different running times for differing values of k. Together, these algorithms provide a constant-factor approximation of the minimumperimeter problem in linear time. We also show a 2-approximation for the minimum-area problem in sub-cubic time, as well as an analysis of a collection of heuristics that work well in practice. 1 Introduction Identifying outliers and clustering points are fundamental problems whenever dealing with noisy data and has been considered extensively in a wide variety of fields [6]. In this paper, we consider the problem where we are given noisy geometric data and wish to remove a fixed number of outliers, so that the remaining points are more closely clustered. Our objective will be to remove a fixed number n k outliers so that the convex hull of the remaining k points is minimized; either with minimum perimeter or minimum area. This type of outlier removal and clustering analysis has been studied previously, with many exact algorithms published in the computational geometry literature [13, 1, 4, 12, 9, 8, 10, 3]. However, instead of exact solutions, we give more efficient algorithms for finding approximate solutions to these problems [5]. The problem of finding a subset of size k from a point set of size n that has the least-perimeter convex hull was previously considered in several papers [13, 1, 4, 12, 9], going back to 1983, with results improving from the original O(k 2 n log n + k 5 n) in Dobkin et al. [13] to the current best known algorithm of O(n log n + k 3 n) in Datta et al. [12] and Dept. Applied Mathematics and Statistics, Stony Brook University, mbiro@ams.stonybrook.edu, justine.bonanno@stonybrook.edu, lynda.montgomery@stonybrook.edu Dept. Computer Science, Stony Brook University, rebrahimi@cs.stonybrook.edu Eppstein et al. [9]. Finding the subset of k points with the minimumarea convex hull was also previously considered by Eppstein [8], and Eppstein et al. [10], where they give O(kn 3 ) and O(n 2 log n+k 3 n 2 ) exact algorithms to find the subset of k points whose convex hull has minimum area. These algorithms are based on a dynamic programming approach, and the O(kn 3 ) algorithm from [10] was implemented in the process of testing the heuristics analyzed in this paper. The above algorithms give exact solutions, but their running times can be Ω(n 4 ), or worse, for large k. Recently, for k = n c, Atanassov, et al, [3] gave exact O(n log n + ( 4c 2c) (3c) c+1 n) algorithms for both problems, however this still leaves a difficulty of finding exact solutions for k sufficiently far from n. In many situations, one expects a fixed error rate in generating outliers, yielding k = Ω(n), yet k not n c for small constant c. In order to efficiently deal with situations such as these, efficient approximation algorithms are developed in this paper. We give a linear-time, constant-factor approximation to the minimum-perimeter problem, as well as a 2-approximation algorithm for the minimum-area problem that runs in O(min(n 3 / log n, n 2 log n + kn(n k)(n k + log k))) time. In addition, we describe and analyze a group of heuristics that run in near or sub-quadratic time, and approximate the minimum area convex hull well in practice, but with no possible constant-factor bound. 2 Minimum-Perimeter Convex Hull We approximate the k-outlier minimum-perimeter convex hull by approximating the shape of the convex hull as either a rectangle or circle. Lemma 1 Let P be a convex polygon in R 2. Then the perimeter of P is at most a factor 2 away from the perimeter of the minimum-perimeter axisparallel rectangle enclosing P. Proof. Suppose R is the minimum-perimeter axisparallel rectangle enclosing P. Each edge of R is pinned by a vertex of P, that together form a quadrilateral Q in R (possibly degenerate). The

2 25 th Canadian Conference on Computational Geometry, 2013 perimeter of P is at least as large as the perimeter of Q, and we show that the perimeter of R is at most 2 times larger than the perimeter of Q, thereby proving the claim. Each edge of Q separates a vertex of P, forming a right triangle. Each length of perimeter of R is charged to the corresponding edge length of Q. If the triangle has sides a, b, c and angle θ, the sum of the lengths of the legs a + b is bounded by: a + b = c(sin(θ) + cos(θ)) = 2c sin(θ + π 4 ) 2c. Since the sum of the length of the legs of a right triangle is at most 2 times the length of the hypotenuse, this shows that the perimeter of R is at most 2 times the perimeter of Q and therefore, the periemter of R is at most 2 times the perimeter of P. Lemma 2 Let P be a convex polygon in R 2. Then the perimeter of P is at most a factor of π 2 away from the perimeter of the minimum-radius disc enclosing P. Proof. Suppose D is the minimum-radius disc enclosing P. In general, D is pinned by 2 or 3 vertices of P forming a triangle T (possibly degenerate) containing D s center point, O. The perimeter of P is at least the perimeter of T, as P is convex, and we show that the circumference of D is at most π 2 times the perimeter of T, thereby proving the claim. Say O has radius r, and connecting the vertices of T to O gives three angles about O, say θ 1, θ 2, and θ 3. Then, the perimeter of T is given by 2r(sin( θ 1 2 ) + sin(θ 2 2 ) + sin(θ 3 2 )) 2r( 2θ 1 2π + 2θ 2 2π + 2θ 2 2π ) 2r π (θ 1 + θ 2 + θ 3 ) = 2r (2π) = 4r. π Where the inequality is due to the fact that sin x 2x π for 0 x π 2. Since the circumference of D is only 2πr, we have that D has circumference at most π 2 times the perimeter of T, and hence the perimeter of P. We now use recent results of Ahn et al. [2], that finds the minimum-perimeter axis-parallel rectangle in time O(n+k 3 ), and results of Har-Peled et al. [14] that finds a (1 + ɛ)-approximation to the minimum enclosing disk in time O(n+n min( 1 kɛ log 2 ( 1 2 ɛ ), k)). Then we have, Theorem 3 The k-outlier minimum-perimeter convex hull problem can be approximated by a factor of 2 in time O(n + k 3 ), and a factor of π 2 (1 + ɛ) in time O(n + n min( 1 kɛ 2 log 2 ( 1 ɛ ), k)). These algorithms give constant-factor approximations to the minimum-perimeter problem for a variety of values of k. If k = O(n 1/3 ) then the rectangle algorithm yields a 2 approximation in linear time, and if k = Ω(n 1/3 ), then the disk algorithm gives a π 2 (1 + ɛ)-approximation in linear time (for constant values of ɛ). Therefore, for each value of k, we give constant-factor approximations to the k-outlier minimum-perimeter problem that run in linear time. Corollary 4 For each k, the k-outlier minimumperimeter convex hull problem can be approximated by a constant factor in linear time. 3 Minimum-Area Convex Hull We approximate the k-outlier minimum-area convex hull problem by approximating the shape of the convex hull as a rectangle with arbitrary orientation. Lemma 5 Let P be a convex set in R 2. Then the area of P is at most a factor of 2 away from the area of the minimum-area rectangle enclosing P. Proof. Take the longest diagonal D of P, and construct a minimal enclosing rectangle R with two sides parallel to D. Take R along with the D and the two points defining the perpendicular edges to D of R. The area of P is at least the area of the two triangles thus defined, and the two triangles take up exactly half of R. Therefore, R has area at most twice the area of P, and as the minimum-area rectangle has area at most the area of R, it has area at most twice the area of P. [15] Using the idea in the above proof, we construct an algorithm that checks every possible longest diagonal D of the point set P, then computes the minimum-area rectangle containing at least k points among all such diagonals, in time O(n 3 / log n). 1. Examine every pair of points (p, q) and look at the strip defined by lines perpendicular to pq through p and q respectively. Find the points of P that lie in the strip. 2. Sort the points in the strip by perpendicular distance to pq, and for every point in the strip

3 CCCG 2013, Waterloo, Ontario, August 8 10, 2013 find the corresponding point so that the rectangle defined by the four points contains k points of P. 3. Take the minimum-area rectangle among all rectangles constructed. We combine this with the recent result of Das et al. [11], that finds a minimum-area rectangle in time O(n 2 log n + kn(n k)(n k + log k)) time. Theorem 6 The k-outlier minimum-area convex hull problem can be 2-approximated in time O(min(n 3 / log n, n 2 log n+kn(n k)(n k+log k))). This approximation is useful if k is Θ(n), so the optimal solution in [8] runs in time Ω(n 5 ), but not n c for constant c, as the optimal solution in [3] runs in time exponential in c. Note that if k = 3, solving the minimum area problem exactly, or to within a constant factor, would allow us to determine if there exists a triple of collinear points in the data set. Since this problem is a well known 3SUM-hard problem [16], there may not be an exact or constant-factor approximation algorithm for the minimum area problem that runs in subquadratic time for all k. 4 Heuristic Algorithms In this section, we describe multiple heuristic algorithms for the minimum-area convex hull problem. Then, we present the experimental outcomes of these algorithms on different types of input distributions. 4.1 Heuristic A: k-nearest Convex Hull This algorithm tries to capture the min-area convex hull by trying a few subsets of points. The subsets are the k 1 nearest points to each point in the input. The heuristic is described in detail in Algorithm 1. To compute the k-nearest neighbors for all the points, we can use the O(n log n+kn log k) algorithm due to Dickerson et al. in [7]. Each iteration of the loop takes O(k log k) time and there are n iterations. Therefore, Heuristic A can be computed in O(n log n + kn log k) time. 4.2 Heuristic B: Bucket Removal This heuristic creates divides the plane into different sections. Based on location of points with respect to the sections, points are considered to inliers/outliers. Outliers points are removed and the procedure is repeated to have exactly n k Algorithm 1 Heuristic A: k-nearest Convex Hull minarea =. Pre-compute the k 1 nearest neighbors, N k 1 (p), to all points p S. for point p in the input do Compute the convex hull of N k 1 (p) and its area A(CH(N k 1 (p)). minarea = min{minarea, A(CH(N k 1 (p))}. end for return minarea. outliers removed. The division into different sections is termed bucketing. The heuristic is described in detail in Algorithm 2. The running time is O(n(n k) log n). Algorithm 2 Heuristic B: Bucket Removal i 0. Number of deleted Outliers while i < n k do Bucketing: Let AB = diameter{s}. Let O be the center of AB, and d = AB. Let the lune L = B d (A) B d (B). We have that S L. (B x (y) is a ball of radius x centered at y.) Let the inner disk ID = B d/4 (O). Let E 1, E 2 be two perpendicular lines intersecting at O, where E 1 OB = π/4. Divide L \ ID evenly into 4 buckets (sections) created by the intersection of E 1, E 2. Associate each point in S with its respective bucket. Solve a constant size knapsack to decide which combination of buckets (if any) should be removed. If so, remove buckets and update iand S. if i < n k then Remove points in order of decreasing distance from O until either i points are removed, or one of A, B (or both of them) is removed. Update i and S. end if end while return A(CH(S)). 4.3 Heuristic C: Greedy Removal The greedy heuristic removes points from the input set one by one trying to maximize the area gain at each step. The heuristic is described in detain in Algorithm 3. A simple implementation requires O((n k)n 2 log n) of time.

4 Algorithm 3 Heuristic C: Greedy Removal i = 0 Number of deleted outliers. while i < n k do Find the convex hull of the input set S. bestgain = 0, bestp oint = null. Let A S be the area of convex hull of S. for point p in CH(S) do Let A p = A(CH(S \ {p})). gain p = A S A p. if bestgain < gain p then bestgain = gain p. bestp oint = p. end if end for S S \ {bestp oint}. end while return A(CH(S)). Heuristic Perf. in the 2-Box Dist. A: k-nearest µ = 1.14, σ = 0.16, O 2 = 0.0 B: Bucket µ = 1.52, σ = 0.59, O 2 = 0.19 Min(A,B) µ = 1.11, σ = 0.15, O 2 = 0.0 C: Greedy µ = 2.95, σ = 1.80, O 2 = 0.58 Table 1: Comparison of Heuristic Algorithms in the 2-Box Distribution. Heuristic Perf. in the 2-Cluster Dist. A: k-nearest µ = 1.13, σ = 0.13, O 2 = 0.0 B: Bucket µ = 1.42, σ = 0.48, O 2 = 0.04 Min(A,B) µ = 1.13, σ = 0.12, O 2 = 0.0 C: Greedy µ = 2.77, σ = 0.15, O 2 = 0.4 Table 2: Comparison of Heuristic Algorithms in the 2-Cluster Distribution. 5 Comparison of Heuristic Algorithms In order to measure the performance of these algorithms we needed to have access to the optimal solution. Therefore, we implemented the complicated dynamic programming solution of [8]. To the best knowledge of authors, this is the first implementation of the algorithm in [8]. Due to its complexity O(kn 3 ), the input sets we could test on were of limited size (< 100 points). 5.1 Setting We tested the described algorithms on different distributions of input points. 1. Two Boxes. Input was generated randomly in one small box B 1 and an annulus between a larger coccentric box B 2 and B 1. The inner more populated B 1, meant to represent the inlier points and the emptier B 2 meant to represent the outliers. The number of points in the B 2 was chosen to be a random number between 1 and n/4. 2. Two Gaussian Clusters. Input was generated in two Gaussian clusters. The first more populated cluster C 1 meant to represent the inlier points and the emptier cluster C 2 meant to represent the outliers. Again, the number of points in the C 2 was chosen to be a random number between 1 and n/4. 3. Uniform Distribution. Points were uniformly distributed in a range. The data was generated for a wide range of different parameters of these different distributions. For example, different sizes and ratios of boxes, and different mean/variances of clusters were considered. For each setting of these parameters, 100 input points were generated and the final results were computed on all the different settings of parameters. 5.2 Analysis Tables 1, 2 and 3 present the results of running the heuristic algorithms on these different input distributions. The measurements are the mean and standard deviation of approximation factor and the percentile where this approximation factor goes above 2. We let O 2 represent the percentile. The results indicate a very good performance for Heuristic A (k-nearest Convex Hull) in all settings and a very bad performance for Heuristic C in most settings. A nice property of Heuristic A was that its approximation factor never went above 2 in our tests. The low cost of implementation, combined with the low average/stdev approximation factor (µ < 1.14, σ < 0.16) in all distributions make Heuristic A a very good candidate for implementations in practical settings.. Heuristic B (Bucket Removal), however, was good only in uniform distribution and O.K. in the other two. Despite this, there were input instances were Heuristic B was better than Heuristic A. Therefore, taking the minimum of the two solution poses a fourth better heuristic in all scenarios. The results for this heuristic min(a,b) is also presented in the tables.

5 CCCG 2013, Waterloo, Ontario, August 8 10, 2013 Heuristic Perf. in the Uniform Dist. A: k-nearest µ = 1.08, σ = 0.07, O 2 = 0.0 B: Bucket µ = 1.2, σ = 0.16, O 2 = Min(A,B) µ = 1.07, σ = 0.07, O 2 = 0.0 C: Greedy µ = 1.31, σ = 0.25, O 2 = 0.0 Table 3: Comparison of Heuristic Algorithms in the Uniform Distribution. 6 Conclusion For future work, we would like to find a PTAS for minimum-perimeter that runs in linear or nearlinear time, as well as improve the running time of the approximation for minimum-area convex hulls. References [1] A. Aggarwal, H. Imai, N. Katoh, S. Suri. Finding k points with minimum diameter and related problems. Journal of Algorithms, 12:38-56, [2] H. Ahn, S. Bae, E. Demaine, M. Demaine, S. Kim, M. Korman, I. Reinbacher, W. Son. Covering points by disjoint boxes with outliers. Computational Geometry, 44(3): , [3] R. Atanassov, P. Bose, M. Couture, A. Maheshwari, P. Morin, M. Paquette, M. Smid, S. Wuhrer. Algorithms for optimal outlier removal. J. Discrete Algorithms, 7(2): [4] E. Arkin, S. Khuller, J. S. B. Mitchell. Geometric Knapsack Problems. Algorithmica, 10(5): , [5] M. Biro, J. Bonanno, R. Ebrahimi, L. Montgomery. Approximation Algorithms for Outlier Removal in Convex Hulls. Proceedings of the 22nd Fall Workshop on Computational Geometry (FWCG 2012). [6] F. Dehne, H. Noltmeier. A computational geometry approach to clustering problems. Proceedings of the first annual Symposium on Computational Geometry, pages , [7] M. T. Dickerson, R. Drysdale, J. Sack. Simple algorithms for enumerating interpoint distances and finding k-nearest neighbors. International Journal of Computational Geometry & Applications, 2(03): , [8] D. Eppstein. New algorithms for minimum area k- gons. Proceedings of the 3rd ACM-SIAM Symp. Discrete Algorithms, pages 83-88, [9] D. Eppstein, J. Erickson. Iterated nearest neighbors and finding minimal polytopes. Discrete & Computational Geometry, 11(1): , [10] D. Eppstein, M. Overmars, G. Rote, G. Woeginger. Finding minimum area k-gons. Discrete & Computational Geometry, 7:45-58, [11] S. Das, P. Goswami, S. Nandy. Smallest k-point enclosing rectangle and square of arbitrary orientation. Information Processing Letters, 94(6): , [12] A. Datta, H. P. Lenhof, C. Schwarz, M. Smid. Static and Dynamic Algorithms for k-point Clustering Problems. J. Algorithms, 19(3): , [13] D. Dobkin, R. Drysdale, L. Guibas. Finding smallest polygons. Computational Geometry, 1: , [14] S. Har-Peled, S. Mazumdar. Fast algorithms for computing the smallest k-enclosing disc. Proceedings of the 11th Annual European Symposium on Algorithms, pages , [15] A. King (mathoverflow.net/users/4580), Area ratio of a minimum bounding rectangle of a convex polygon, (version: ) [16] A. Gajentaan, M. H. Overmars. On a class of O(n 2 ) problems in computational geometry. Computational Geometry, 5(3): , 1995.