Iterative methods for use with the Fast Multipole Method

Size: px

Start display at page:

Download "Iterative methods for use with the Fast Multipole Method"

Georgina Margaret Quinn
5 years ago
Views:

1 Iterative methods for use with the Fast Multipole Method Ramani Duraiswami Perceptual Interfaces and Reality Lab. Computer Science & UMIACS University of Maryland, College Park, MD Joint work with Nail A. Gumerov

2 Fast Multipole Methods Follows from seminal work of Rokhlin and Greengard (1987) General method for accelerating large classes of dense matrix vector products Apply to matrices whose entries are associated with computations of pairs of points Reduce computational/memory complexity from O(N 2 ) to O(N log N) Potentially reduce many O(N 2 )/O(N 3 ) calculations to O(N log N) Nail Gumerov and I are applying it to many areas Acoustics (JASA 2002,2005), Room impulse responses (IEEE Trans. SAP, 2006), book (Elsevier, 2005) Maxwell s equations in 3D and scattering (submitted, 2006) Fast statistics (NIPS 2004), similarity measures (CVPR 2005), image processing, segmentation (ICCV 2003), tracking, learning (Data Mining 2006) Non uniform fast Fourier transforms and reconstruction Biharmonic equation (JCP 2006), fitting thin-plate splines

3 Solving Linear Systems Typically the FMM is used with an iterative method some preliminary work aims at directly constructing the inverse approximately (Rokhlin, Martinsson, Gimbutas, Cheng) FMM reduces the cost of the matrix-vector product step in the iterative solution of linear systems, and for an iteration requiring N iter steps, the complexity is O(N iter NlogN). To bound N iter appropriate pre-conditioning strategies must be used with the dense matrix. However, many conventional pre-conditioning strategies rely on sparsity in the matrix, or are ineffective

4 Preconditioner cost To be effective with the FMM the preconditioner cost should be O(N log N) or less Otherwise we negate the advantage of the FMM We present results for two preconditioners we have used First, for highly oscillatory kernel --- multiple scattering with the Helmholtz equation Second, we accelerate a highly successful preconditioner developed by Faul, Powell and Goodsell (2005), but which has a complexity of O(N 2 ) to O(N log N).

5 Radial basis function (RBF) interpolation Construct interpolating function for scattered data, with the interpolating function expressed as a sum of RBFs centered at the data points. Much work by Powell, Beatson and coworkers Culmination of this work: an iterative Krylov subspace algorithm Faul et al. (2005) A. C. Faul, G. Goodsell, M. J. D. Powell, "A Krylov subspace algorithm for multiquadric interpolation in many dimensions," IMA Journal of Numerical Analysis (2005) 25, 1--24

6 RBF Interpolation x are points in R d f are values of the function to be interpolated s is the interpolating function λ are the interpolation coefficients Interpolating RBFs are multiquadric or biharmonic

7 Linear system \Phi is the RBF

8 Preconditioner via cardinal functions Beatson and Powell (1992) created preconditioners via approximate cardinal functions Preconditioner must be close to the inverse Cardinal function has value one at the point and zeros at all other points Thus a cardinal function c satisfies Ac = [ ] t If we stack cardinal functions for each row, we obtain A -1 f BP92, proposed to use the smoothness of the interpolant and required the cardinality properties at each point and several other points in the domain (not the whole set) Selection of which points to include and which points to exclude, and their influence on preconditioner performance is the subject of several subsequent papers by the groups of Powell and Beatson.

9 Empirical comparison of iterative methods We compared several strategies proposed by the authors, and found that the proposal of Faul et al. 2005, works best For some of the proposed strategies, for very large (10 6 or more points), some of the proposed preconditioners do not converge within 1000 iterations. However Faul et al. s preconditioned iterative algorithm converges robustly within about 50 iterations.

10 Accelerating the Faul et al. algorithm Matrix vector product can be accelerated via the FMM (e.g., Gumerov and Duraiswami, 2006, JCP) Choice of points for the approximate cardinal functions is very particular in Faul et al. Chooses the point that is a member of the closest pair of points, and its q closest neighbors and builds an interpolant The first point is eliminated from future consideration Algorithm is recursively applied and a fine-to-coarse structure of the sets of approximate cardinal functions are obtained Preconditioned Conjugate Direction algorithm is applied along the directions of these functions Search for q closest neighbors with removal of points from the set is O(N 2 ).

11 Converting it to O(N log N) Use FMM data structure to build lists of closest neighbors Use heaps to develop a dynamic data structure that allows deletions Employed a lemma that bounds the number of closest neighbors a point can have is a fixed number that is a function of the dimension Achieve a point selection that is O(N log N). Details in a preprint being prepared. For a 110,000 point problem, time of the preset set went down from seconds to 115 seconds in 3D biharmonic interpolation

Original unstructured data 104502 points

12 Original unstructured data points RBF/FMM interpolation to regular spatial grid FMM algorithm cost per iteration:9.6 seconds on a Pentium IV 3.2 GHz desktop

13 Error Iteration number

Original Data: surface mesh with 34834 vertices and

14 Original Data: surface mesh with vertices and faces RBF interpolation to the spatial grid + isosurface (size of the grid is shown near each figure) 5x5x5 11x11x11 26x26x26 51x51x51 101x101x x201x201

0 100 200 300 400 500 Original Image 0 100 200 300 400 500 10 percent samples drawn

15 Original Image percent samples drawn from image FMM algorithm cost per iteration:6.2 seconds on a Pentium IV 3.2 GHz desktop

16 Multiquadric Interpolation of a 2D image 10 4 Error Iteration number

17 Interpolated Result Difference Image 14 seconds to interpolate whole image from the fit data

18 Equations and Boundary Conditions Formulation Wave Equation Fourier Transform Helmholtz Equation Impedance Boundary Conditions 3 Field Decomposition Sommerfield Radiation Condition 2 Incident Wave 6

19 Scattered Field Decomposition T-Matrix Method Scattered field expressed in terms of singular (multipole) wave functions that satisfy the Sommerfeld condition Singular Basis Functions Hankel Functions Expansion Coefficients Spherical Harmonics Vector Form: dot product

20 T-Matrix Method Solution of Multiple Scattering Problem Write a system of equations relating expansion coefficients for each scatterer Effective Incident Field Coupled System of Equations: Scattered Wave (S R)-Translation Matrix Incident Wave 6 Linear system of size p 2 N p 2 N

T-matrix Method Three Spheres Comparisons of BEM & MultisphereHelmholtz HRTF (db) 12 9 6 3 0-3 60 o 30 o θ 1 = 0 o 90 o 60 180 o BEM MultisphereHelmholtz 30 o o 90 o 120

21 T-matrix Method Three Spheres Comparisons of BEM & MultisphereHelmholtz HRTF (db) o 30 o θ 1 = 0 o 90 o o BEM MultisphereHelmholtz 30 o o 90 o 120 o o 150 o o Three Spheres, ka 1 = Angle φ 1 (deg) BEM: 5184 triangular elements MH: N trunc = 9 (100 coefficients for each sphere)

22 Iterative Methods Krylov Subspace Method (GMRES) For larger systems use an iterative method Diagonal Matrix The product of this matrix by An arbitrary input vector can be done fast with the FMM

23 Iterative Methods FGMRES Unpreconditioned LA = E To converge requires N iter multiplications LG, where G is an input vector Cost: C N iter Right Preconditioner LM -1 (MA)=E 1). Internal Loop: Solve M -1 F=E Requires N (1) iter multiplications MG 2). External Loop: Requires N (2) iter multiplications LG Cost: C (1) N (1) iter+ C N (2) iter Substantial speed up if M L and C (1) «C

24 MLFMM FMM splits influence of scatterers in to far and near Fields Neighborhood (Near Field) Far Field

25 MLFMM Computation of the Far Field (1) 1). Set Data Structure (hierarchically subdivide space with an oct-tree) 2). multipole to multipole translate S- expansions for all scatterers in a box at the finest level to the center of the box and sum up (determine contribution to Far Field for each box at the finest level). 3). Recursively multipole to multipole translate S-expansions to center of the parent box and sum up (determine contribution to Far Field for each box at all coarser levels). y x i x c (n,l) Upward Pass (From the finest to the coarsest level)

26 MLFMM Computation of the Far Field (2) Steps 4 and 5 performed one after the other recursively 4). Multipole-to-local translate S- expansions for boxes which are inside the parent neighborhood but outside the box neighborhood to the center of the box (convert S-expansion to R- expansion). Downward Pass (From the coarsest to the finest level) 5). Local-to-local translate R-expansions from the center of the box to the center of its children boxes (determine Far Field for each box at all levels). 6). local translation of R-expansions from the center of the boxes at the finest level to the centers of the spheres.

27 Preconditioned FGMRES FGMRES Choice of preconditioning operator: near (sparse) or far (dense)? Experimentally we found that the far (dense) operator works well in all tests

28 Range of Parameters Results Number of Spheres: ; ka: ; Random and regularly spaced grids of spheres; Polydispersity: (ratio to the mean radius); Volume fractions:

29 Results 100 random spheres (MLFMM) Plane Wave Plane Wave ka = 1.6 ka = 2.8

30 1000 random spheres Results ka=1

31 10000 random spheres Results ka = 0.75 kd 0 = 90

32 A posteriori Error Check Results

33 Results GMRES vs FGMRES Volume Fraction = 0.2 ka = 0.5, D 0 /a = 60, p 2 = GHz, Xeon, 3.5 MB RAM

34 Performance Test Results Volume fraction = 0.2, ka = 0.5, p 2 = FMM+FGMRES y=cx 1.25 CPU Time (s) y = cx 1.25 Total y = bx External Loop y = ax 3.2GHz, Xeon, 3.5 MB RAM Matrix-Vector Multiplication 1 Internal Loop Number of Scatterers

35 Conclusions Need for preconditioners for dense matrices for FMM based algorithms For a highly oscillatory kernel, preconditioner based on the FMM algorithm itself gives good results Preconditioner theoretical performance remains to be proved Approach that interpolates the cardinal function approximately on carefully chosen data appears very promising for smoother kernels Geometrical computations involved in this method reduced to O(N log N) Extension to other kernels of this approach is the subject of

Efficient O(N log N) algorithms for scattered data interpolation

Efficient O(N log N) algorithms for scattered data interpolation Nail Gumerov University of Maryland Institute for Advanced Computer Studies Joint work with Ramani Duraiswami February Fourier Talks 2007