Efficient O(N log N) algorithms for scattered data interpolation

Transcription

1 Efficient O(N log N) algorithms for scattered data interpolation Nail Gumerov University of Maryland Institute for Advanced Computer Studies Joint work with Ramani Duraiswami February Fourier Talks 2007 Papers to read AC Faul, G Goodsell, and MJD Powell, A Krylov subspace algorithm for multiquadric interpolation in many dimensions, IMA Journal of Numerical Analysis, Vol 25, pp 1-24, 2005 NA Gumerov & R Duraiswami Fast radial basis function interpolation via preconditioned Krylov iteration, SIAM Journal of Scientific Computing, Vol 29, No 5, pp ,

2 Contents Introduction (multidimensional RBF interpolation) Faul, Goodsell, and Powell (FGP 05) algorithm Fast algorithm for construction of L-sets Algorithm for fast matrix-vector product (FMM) Performance and Examples Conclusion Gumerov & Duraiswami, Fast Radial Basis Function Interpolation Via Preconditioned Krylov Iteration, SIAM J Scientific Computing, 2007, Introduction 2

3 Introduction Multidimensional RBF fitting RBF polynomial of degree at most K-1 other functions can be also OK Introduction Applications Computer graphics: Image rendering Repairing (filling in, etc) Implicit surfaces PDE solvers: Non-homogeneous equations (eg Poisson equation) Nonuniform data fitting to grid: Isosurfaces Multidimensional FFT 3

4 Introduction Algebraic problem K is low: For multiquadric RBF it is sufficient to have K=1 (P is constant) Publications Introduction Much work by Powell, Beatson and co-workers; Culmination of this work: an iterative Krylov subspace algorithm Faul et al (2005) (FGP 05): A C Faul, G Goodsell, M J D Powell, "A Krylov subspace algorithm for multiquadric interpolation in many dimensions," IMA Journal of Numerical Analysis (2005) 25,

5 Introduction Motivation for this work: Direct solution requires O(N 3 ) operations and O(N 2 ) memory and is applicable for small problems only (N 10 4 ) Iterative solution (such as FGP 05) is much faster: it requires O(N iter N 2 ) while still O(N 2 ) memory The size of the problems is limited approximately by the same max N More broadly, goal is to develop efficient and robust preconditioners for Fast Multipole Methods The FGP 05 algorithm converges for low N iter; the method is robust We are looking for two basic algorithms, which brings the method complexity to O(N iter N log N) in number of operations and O(N log N) in memory This is achieveable via Algorithm for construction of the L-sets (preconditioner); FMM for matrix-vector multiplication FGP 05 algorithm 5

6 FGP 05 algorithm Cardinal functions FGP 05 algorithm Approximate cardinal functions L-set 6

7 Preconditioner via approximate cardinal functions Beatson and Powell (1992) created preconditioners via approximate cardinal functions Preconditioner must be close to the inverse Cardinal function has value one at the point and zeros at all other points Thus a cardinal function c satisfies Ac = [ ] t If we stack cardinal functions for each row, we obtain A -1 f Idea: require cardinality properties at each point and several other points in the domain (not the whole set) Selection of which points to include and which points to exclude, and their influence on preconditioner performance is the key difference L-set construction FGP 05 algorithm (For a proof that this works see FGP 05 and Faul s dissertation) 1 Find 2 closest points in the total set, and assign one of them (j) as a center of the L j -set; 2 Find (q 1) closest points to the center They form the L j set (plus center) 3 Delete point j from the total set and repeat the procedure, until less than q points remains 4 The remaining L-sets are build similarly, but the number of points there are q - 1,,2 Total number of L-sets is N - 1 7

8 FGP 05 algorithm Semi-norm and Krylov subspace two functions semi-norm inner product Krylov subspace is generated by powers of operator X Projection in this subspace of s is FGP 05 algorithm Preconditioned conjugate gradient iterations One matrix-vector multiplication per step 8

9 Fast algorithm for construction of L-sets Fast L-set construction Facts and problems There exist O(log N) algorithms to find closest neighbors for a point in d dimensions (based on k-d trees) For d = 2 and 3 we use quadtrees and octrees; The problem is to determine a pair of closest points for O(log N), as there are O(N) L-sets; Initiation of the process takes O(N log N) operations (spatial data sorting and finding the pair of closest points 9

10 Fast L-set construction Main idea Initially find for each point its closest neighbor and organize them in the heap data structure according to rank r = -d, where d is the distance to the closest neighbor; In this case the top ranked point can be deleted from the data structure for the cost O(log N); Create an O(log N) procedure for recursive updating of the heap data structure changed due to deletion of the top ranked point; This includes updating of the quad- or octree structures and updating of the closest neighbor/rank of the points Updating of quad- or octree structures can be performed using linked lists of points, boxes, and boxes in the neighborhood Updating of the point ranks is a special (new) procedure Heap data structure Fast L-set construction R2>R4, R2>R5 R2 R1 R1>R2, R1>R3 R3 R4 R5 R6 R7 R3>R6, R3>R7 Repairing of the heap data structure after insertion or deletion of any entrée is O(log N) procedure R4>R3? may be yes, may be no 10

11 Updating of ranks: 2 Lists Fast L-set construction List of closest neighbors (C) Point # Closest neighbor # Inverse list of closest neighbors (C*) Closest neighbor # Point # Top Rank , ,1,2 Heap NA ,15 As the top ranked point is deleted, the inverse list provides indices of points whose rank should be updated Updating of ranks: main theorem Fast L-set construction Theorem: Remark 1: Remark 2: 11

12 Algorithm Fast L-set construction Set initial data structure: O(N log N) Do untill all L-sets are constructed Get top ranked point (j top ) from the heap and find its q-1 neighbors: O(log N); This is the next L-set; Get the set C top * of points from list C* corresponding to j top : O(log N); Delete j top (repair the heap, broken links in the linked lists, and entries in C* which contain j top ): O(log N); Find closest neighbors to points from C top * and determine their ranks ): O(log N); Update the heap according these ranks, and update lists C and C* ): O(log N) Total complexity O(N log N) Remark: There exist weird distributions when the closest neighbor cannot be found in O(log N) operations In this case the algorithm complexity is larger Performance Fast L-set construction N (A) d=2: Present FGP (19900) (221100) (A) d=3: Present FGP (19900) (221100) (E) d=2: Present FGP (19900) (221100) (E) d=3: Present FGP (19900) (221100) Case A: uniformly random distribution; Case E: uniformly random distribution on a lower dimensionality manifold (32 GHz Intel Xeon, 35 GB RAM) 12

13 Time to build preconditioner (s) 11/8/2011 Performance Fast L-set construction 1E+04 1E+03 A, d=2 A, d=3 E, d=2 E, d=3 FGP'05 y=ax 2 y=bx 1E+02 Present q=30 1E+01 1E+00 1E-01 1E+02 1E+03 1E+04 1E+05 1E+06 Number of points, N (32 GHz Intel Xeon, 35 GB RAM) Algorithm for fast matrix-vector multiplication (Fast Multipole Method) 13

14 Some facts on the FMM Fast Multipole Method Originated by publications of Rokhlin in Greengard ( ) for harmonic functions in 2D and 3D; Achieves O(N) or O(N log N) memory and operation complexity for approximate products of matrices of special structure with vectors; Listed (along with the FFT) as one of the 10 top algorithms of 20 th century; Several versions exist today for Laplace, biharmonic, polyharmonic, Helmholtz equations; Applied to accelerate Gauss, Legendre, and Non-uniform Fourier transforms; Usually employs local and far field kernel factorization, hierarchical space subdivision (quadtrees and octrees), and translation operators to transform expansion bases Our FMM involvement Fast Multipole Algorithms for Helmholtz, Laplace, Maxwell, nonuniform Fast Fourier transforms, Gaussians, logistic functions, error functions Fast Multipole Accelerated boundary element methods Speed up of of FMM on CPU/GPU clusters Also a course taught by us as part of the Wiener center s graduate program 14

15 Fast Multipole Method Factorization in the FMM FMM Fast Multipole Method y x i x c (n,l) 15

16 CPU Time (s) 11/8/2011 Fast Multipole Method Translation of biharmonic functions Translation operator Conversion operator Gumerov & Duraiswami, Fast multipole method for the biharmonic equation in three dimensions J Comput Phys, Volume 215, Issue 1, 10 June 2006, Pages Fast Multipole Method Performance of 3D FMM for biharmonic kernel 1E+03 1E+02 Direct y=bx FMM y=ax 1E+01 p=4 1E+00 1E-01 Direct p=4 p=9 p=19 1E-02 1E+03 1E+04 1E+05 1E+06 1E+07 Number of Sources, N (32 GHz Intel Xeon, 35 GB RAM) 16

17 Fast Multipole Method M-v product with multiquadric kernel in 2D via biharmonic kernel in 3D Computationally efficient factorization of the multiquadric kernel is a problem In 2D our solution is: y Set of sources = Set of receivers A r B x z B Sources y r c B A Receivers x Sources and receivers are separated f (r) = (r 2 +c 2 ) 1/2 = (x 2 + y 2 + c 2 ) 1/2 f (r) = r = (x 2 + y 2 + c 2 ) 1/2 In this case the number of data structure operations and translations in 3D is the same as in 2D Performance and Examples 17

18 CPU Time (s) 11/8/2011 Performance of the present algorithm (total timing (s)) Examples N (A) d=2,c=0: Present FGP (49320) (547700) (A) d=2,c=n -1/2 : Present FGP (49320) (547700) (A) d=3,c=0: Present FGP (49320) (547700) (E) d=3,c=0: Present FGP (49320) (547700) (32 GHz Intel Xeon, 35 GB RAM) Performance of the present algorithm (total timing (s)) 1E+04 1E+03 1E+02 Preconditioner+ Matrix-Vector Product FGP'05 y=ax 2 Present 14 y=cx y=dx 115 Examples y=bx 1E+01 A, d=2, c=0 A, d=2, c=c(n) A, d=3, c=0 E, d=3, c=0 1E+00 1E+03 1E+04 1E+05 1E+06 Number of Points, N (32 GHz Intel Xeon, 35 GB RAM) 18

19 3D biharmonic #Sources = 104,502 #Receivers = 8,120,601 Examples Original unstructured data Time/Preconditioner = 310 s, Time/Iterative process = 283 s, Time/Final evaluation = 395 s (32 GHz Intel Xeon, 35 GB RAM) Isosurface from RBF/FMM interpolation to regular spatial grid 201 x 201 x 201 2D multiquadric (c = N -1/2 ) #Sources = 51,204 #Receivers = 323,846 3 channels (RGB) Examples Original data 86% random data loss RBF interpolation 375,050 pixels 51,204 pixels 375,050 pixels Time/Preconditioner = 30 s, Time/Iterative process = 75 s (per channel), Time/Final evaluation = 19 s (32 GHz Intel Xeon, 35 GB RAM) 19

20 Convergence for the last example 10 4 Examples 10 2 Error Iteration # Conclusion The developed algorithms show a good performance and enable computation of 2D and 3D million size problems on a desktop PC Optimization of the algorithms is required as many parameters including the size of the L-sets, number of points, accuracy of the approximate matrix-vector product, tolerance for iteration, and parameter of multiquadric are connected; We plan to map this algorithm onto CPU/GPU computer architectures, which may increase the speed times; The software implementing this algorithm was licensed from the University of Maryland to Fantalgo, LLC Contact info@fantalgocom for further information 20