9. Efficient Many-Body Algorithms: Barnes-Hut and Fast Multipole

Transcription

1 9. Efficient Many-Body Algorithms: Barnes-Hut and Fast Multipole Perlen der Informatik II, Hans-Joachim Bungartz page 1 of 47

2 9.1. Simulation on the Molecular Level Hierarchy of Models Different points of view for simulating human beings: issue level of resolution model basis (e.g.!) global increase in countries, regions population dynamics population local increase in villages, individuals population dynamics population man circulations, organs system simulator blood circulation pump/channels/valves network simulator heart blood cells continuum mechanics cell macro molecules continuum mechanics macro molecules atoms molecular dynamics atoms electrons or finer quantum mechanics Perlen der Informatik II, Hans-Joachim Bungartz page 2 of 47

3 Scales an Important Issue length scales in simulations: from 10 9 m (atoms) to m (galaxy clusters) time scales in simulations: from s to s mass scales in simulations: from g (atoms) to g (galaxies) obviously impossible to take all scales into acount in an explicit and simultaneous way first molecular dynamics simulations reported in 1957 Perlen der Informatik II, Hans-Joachim Bungartz page 3 of 47

4 Simulation on the Molecular Level Physical Model Mathematical Model Long-Range Potentials Applications for Micro and Nano Simulations Lab-on-a-chip, used in brewing technology (Siemens) Flow through a nanotube (where the assumptions of continuum mechanics are no longer valid) Perlen der Informatik II, Hans-Joachim Bungartz page 4 of 47

5 Applications for Micro and Nano Simulations Material science: hexagonal crystal grid of Bornitrid Protein simulation: actin, important component of muscles (overlay of macromolecular model with electron density obtained by X-ray crystallography (brown) and simulation (blue)) Perlen der Informatik II, Hans-Joachim Bungartz page 5 of 47

6 Applications for Micro and Nano Simulations Protein simulation: human haemoglobin (light blue and purple: alpha chains; red and green: beta chains; yellow, black, and dark blue: docked stabilizers or potential docking positions for oxygen) Perlen der Informatik II, Hans-Joachim Bungartz page 6 of 47

7 A Prominent Recent Example: Gordon-Bell-Prize 2005 (most important annual supercomputing award) phenomenon studied: solidification processes in Tantalum and Uranium method: 3D molecular dynamics, up to 524,000,000 atoms simulated machine: IBM Blue Gene/L, 131,072 processors (world s #1 in November 2005) performance: more than 101 TeraFlops (almost 30% of the peak performance) Perlen der Informatik II, Hans-Joachim Bungartz page 7 of 47

8 9.2. Molecular Dynamics the Physical Model Classical Molecular Dynamics Quantum mechanics approximation classical Molecular Dynamics classical Molecular Dynamics is based on Newton s equations of motion molecules are modelled as particles; simplest case: point masses there are interactions between molecules multibody potential functions describe the potential energy of the system; the velocities of the molecules (kinetic energy) are a composition of the Brownian motion (high velocities, no macroscopic movement), flow velocity (for fluids) Perlen der Informatik II, Hans-Joachim Bungartz page 8 of 47

9 Fundamental Interactions Classification of the fundamental interactions: strong nuclear force electromagnetic force weak nuclear force gravity interaction potential energy total potential of N particles is the sum of multibody potentials: U := 0<i<N U 1(r i ) + 0<i<N i<j<n U 2(r i, r j ) + 0<i<N i<j<n j<k<n U 3(r i, r j, r k ) +... there are ( N n ) = N! n!(n n)! O(Nn ) n-body potentials U n, particulary N one-body and 1 2N(N 1) two-body potentials force F = gradu r j r i r k O Perlen der Informatik II, Hans-Joachim Bungartz page 9 of 47

10 Lennard Jones Potential r ij r i r j O a widespread family of intermolecular two-body potentials ( ( ) ( ) n ( ) ) m Lennard Jones potential: U LJ rij = αɛ σ r ij σ r ij with n > m and α = 1 n m n n 1 n m m m continuous and differentiable (C ), since r ij > 0 Perlen der Informatik II, Hans-Joachim Bungartz page 10 of 47

11 LJ 12-6 potential ( ( ) ( ) 12 ( ) ) 6 U LJ rij = 4ɛ σ r ij σ r ij m = 6: van der Waals attraction (van der Waals potential) n = 12: Pauli repulsion (softsphere potential): heuristic application: simulation of inert gases (e.g. Argon) force between 2 molecules: ( ) F ij = U(r ij) 12 ( ) ) 6 r ij = 24ɛ σ r ij (2 r ij σ r ij very fast fade away short range (m = 6 > 3 = d dimension) Perlen der Informatik II, Hans-Joachim Bungartz page 11 of 47

12 9.3. Molecular Dynamics the Mathematical Model System of ODE resulting force acting on a molecule: F i = j i acceleration of a molecule (Newton s 2nd law): F r i = U( r i, r j ) i j i F ij j i r = = ij (1) m i m i m i system of dn coupled ordinary differential equations of 2nd order transferable (as compared to Hamilton formalism) to 2dN coupled ordinary differential equations of 1st order (N: number of molecules, d: dimension), e.g. independent variables q := r and p with p i := m i r i p i = F i Perlen der Informatik II, Hans-Joachim Bungartz page 12 of 47 F ij (2a) (2b)

13 Boundary Conditions Initial Value Problem: position of the molecules and velocities have to be given; initial configuration e.g.: molecules in crystal lattice (body-/face-centered cell) initial velocity random direction absolute value dependent of the temperature (normal distribution or uniform), e.g. 3 P N i=1 mv 2 2 Nk BT = 1 2 v 0 := q 3kB T m i with v i := v 0 resp. v 0 := 3T t time discretisation: t := t 0 + i t time integration procedure Perlen der Informatik II, Hans-Joachim Bungartz page 13 of 47

14 Short-Range Interactions dim. red. Lennard Jones 12 6 Potential 0.5 dim. red. Lennard Jones 12 6 Force U * 0.2 F * r * F ij r * Force matrix/interaction-graph - F 12 F 13 F 14 F 15 F 12 - F 23 F 24 F 25 F 13 F 23 - F 34 F 35 F 14 F 24 F 34 - F 45 F 15 F 25 F 35 F 45 - fast decay of force contributions with increasing distance dense force matrix with O(n 2 ), mostly very small, entries Perlen der Informatik II, Hans-Joachim Bungartz page 14 of 47

15 Short-Range Interactions dim. red. finites Lennard Jones 12 6 Potential (rc=2) 0.5 dim. red. finite Lennard Jones 12 6 Force (rc=2) U * 0.2 F * r * F ij r * Force matrix/interaction-graph - F 12 F 13 F 14 0 F 12-0 F 24 F 25 F F 34 0 F 14 F 24 F 34 - F 45 0 F 25 0 F 45 - cut-off radius leads to a reduction of the computational effort sparse force matrix with O(n) entries Perlen der Informatik II, Hans-Joachim Bungartz page 15 of 47

16 Nucleation Nucleation process of supersaturated Argon t = t = t = t = nucleation process for an oversaturated Argon vapour at 0.97 Mol/l and 80k the simulation program automatically detects clusters (droplets) Perlen der Informatik II, Hans-Joachim Bungartz page 16 of 47

17 Clusters in a supersaturated Argon vapor, 80 K, 0.97 Mol/l f 1 (x) = x f 2 (x) = x f 3 (x) = x cluster size > 20 cluster size > 30 cluster size > 40 f 1 (x) f 2 (x) f 3 (x) number of clusters timesteps (10 3 ) counting and grouping clusters of certain size ranges, a statistic can be generated the growth of the clusters (slope) is known as nucleation rate and important for macroscopic simulations Perlen der Informatik II, Hans-Joachim Bungartz page 17 of 47

18 9.4. Numerical Methods for Long-Range Potentials Numerical Methods for Long-Range Potentials so far: focus on short-range potentials such as Lennard-Jones resulting mutual interactions are restricted to particles in some local neighbourhood facilitates numerical treatment and algorithmic organization: no quadratic complexity induced by an each-with-each behaviour now: tackle long-range potentials, too examples: Coulomb or gravitation potential interactions between remote particles must not be neglected simple cut-off not possible nevertheless need for approaches that avoid quadratic complexity Perlen der Informatik II, Hans-Joachim Bungartz page 18 of 47

19 what is long-range? intuitively: potential function U(r) does not decrease rapidly with increasing r formally (one possibility): for d > 2, potentials not decreasing faster than r d for increasing r (criterion: integrability over R d ) typical potentials in applications have both a short-range part (to be dealt with according to the previous sections) and a long-range part, represented as two additive components: U(r) := U short + U long Perlen der Informatik II, Hans-Joachim Bungartz page 19 of 47

20 Tree-Based Methods based on integral representation of the potential Φ(x) = 1 1 ϱ(y) 4πɛ 0 y x dy hierarchical decompositions of the domain of simulation adaptive approximation of the particle distribution widespread scheme: octrees allow for separation of near-field and far-field influences log-linear or even linear complexity can be obtained Perlen der Informatik II, Hans-Joachim Bungartz page 20 of 47

21 high flexibility with respect to more general potentials (as needed for special applications, such as biomolecular problems) advantageous especially for heterogeneous particle distributions (frequent in astrophysics, relevant also for molecular dynamics) examples: panel clustering Barnes-Hut method (fast) multipole methods Perlen der Informatik II, Hans-Joachim Bungartz page 21 of 47

22 Series Expansion of the Potential general (integral) representation of the potential: Φ(x) = G(x, y)ϱ(y)dy (general kernel G, particle density ϱ(y), and domain Ω) Ω Taylor expansion of the kernel G (if sufficiently smooth apart from the singularity in x = y) in y around y 0 : G(x, y) = j 1 p 1 j! G 0,j(x, y 0 )(y y 0 ) j + R p (x, y) (multi-index j = (j 1, j 2, j 3 ), j! = j 1!j 2!j 3!, G k,j (x, y) mixed (k, j)-th derivative (k-th w.r.t. x, j-th w.r.t. y), remainder R p (x, y)) Perlen der Informatik II, Hans-Joachim Bungartz page 22 of 47

23 leads to expansion (and approximation) of the potential: Φ(x) j 1 p 1 j! M j(ω, y 0 )G 0,j (x, y 0 ) with the so-called moments M j (Ω, y 0 ) := Ω ϱ(y)(y y 0 ) j dy Perlen der Informatik II, Hans-Joachim Bungartz page 23 of 47

24 Subdivision of the Domain separation of near-field and far-field for given x: Ω = Ω near Ω far, Ω near Ω far = decomposition of the far-field into disjoint, convex subdomains: Ω far = i Ω far i note: this decomposition depends on x, i.e. it is done for each particle position x (efficient derivation possible from one hierarchical tree structure) each Ω far i has an associated point y i 0 how to choose the subdivision? diam x y i 0 sup y y y Ωfar i 0 i := x y0 i θ for some suitable constant 0 < θ < 1 Perlen der Informatik II, Hans-Joachim Bungartz page 24 of 47

25 resulting approximation for Φ(x): Φ(x) = = = ϱ(y)g(x, y)dy Ω ϱ(y)g(x, y)dy + ϱ(y)g(x, y)dy Ω near Ω far ϱ(y)g(x, y)dy + ϱ(y)g(x, y)dy Ω near i Ω far i Ω near ϱ(y)g(x, y)dy + i j 1 p 1 j! M j(ω far i, y i 0)G 0,j (x, y i 0) Perlen der Informatik II, Hans-Joachim Bungartz page 25 of 47

26 Error Estimates error characteristics for one fix particle position x Ω: local relative approximation error for one Ω far i can be shown to be of order O(θ p+1 ) global relative approximation error (summation over whole far-field) can be shown to be of order O(θ p+1 ) this clarifies the role of θ: allows to control the global approximation error in x geometric requirement to the far-field subdivision: the closer Ω far i is located to x, the smaller it has to be to fulfil the θ-condition hence: a typical level of detail the closer, the higher resolved cf. terrain representation in flight simulators Perlen der Informatik II, Hans-Joachim Bungartz page 26 of 47

27 Tree Structures central question: how can we construct all these necessary separations of near-fields and far-fields and subdivisions of far-fields in an efficient way? idea: recursive decomposition of Ω (a square in 2D, a cube in 3D without loss of generality) in cells of different size, terminating the subdivision process if a cell is either empty or contains just one particle concepts: kd-tree: alternate subdivision in coordinate direction (x, y, and z), such that the separation produces two subdomains that roughly contain the same number of particles each quadtree (2D) or octree (3D): subdivision into four congruent subsquares or eight congruent subcubes, respectively the following algorithms (Barnes-Hut etc.) use the octree approach Perlen der Informatik II, Hans-Joachim Bungartz page 27 of 47

28 kd-trees Example Perlen der Informatik II, Hans-Joachim Bungartz page 28 of 47

29 Quadtrees and Octrees Examples Perlen der Informatik II, Hans-Joachim Bungartz page 29 of 47

30 Recursive Computation of the Far-Field starting point: create the octree corresponding to the set of particles each node of the octree represents a subdomain of Ω or one cell for each cell i, define some y0 i (the centre point or the centre of gravity of all particles contained, e.g.) for doing the Taylor expansion for each cell i, let the parameter diam just denote the diameter of the smallest surrounding sphere, e.g. objective: for each particle position x, use as few cells as possible (i.e. as big cells as possible) for fulfilling the diam-θ rule hence: start from root node, check diam x y i 0 θ, stop if fulfilled (no need for further subdivision) and proceed if not yet fulfilled Perlen der Informatik II, Hans-Joachim Bungartz page 30 of 47

31 note that for each x, we typically get a different subdivision but note also that all these subdivisions are just subtrees of our constructed octree Perlen der Informatik II, Hans-Joachim Bungartz page 31 of 47

32 Recursive Computation of the Moments now: use this subdivision for the efficient calculation of the local moments M j (Ω far i, y i 0 ) direct (numerical) integration or direct summation are not efficient therefore: use hierarchical tree structure to calculate all moments for all cells in one run and store them crucial property for that: M j (Ω 1 Ω 2, y 0 ) = M j (Ω 1, y 0 ) + M j (Ω 2, y 0 ), if the point of expansion y 0 is the same in the (standard) case of different y0 1 and y 0 2, there are simple conversion formulas: M j (Ω 1, ŷ 0 ) = ( ) j (y 0 ŷ 0 ) j k M k (Ω 1, y 0 ) k k j (k j component-wise, multiplicative binomial coefficients) Perlen der Informatik II, Hans-Joachim Bungartz page 32 of 47

33 this allows for a bottom-up calculation of the moments from the leaves to the root in the leaves: if no particle present: zero if one particle of mass m there in x: m(x y i 0 )j Perlen der Informatik II, Hans-Joachim Bungartz page 33 of 47

34 Using these Building Blocks still to be done for a numerical routine: how to construct the tree, starting from a given set of particles? how to store the tree? how to choose cells and expansion points? how to determine far-field and near-field? several algorithmic variants to be discussed in the following Perlen der Informatik II, Hans-Joachim Bungartz page 34 of 47

35 The Barnes-Hut Method the so-called Barnes-Hut method is the oldest, simplest, and most widespread hierarchical tree-based approach, dates back to 1986 original (and still main) target applications: astrophysics (high particle numbers and a typically very heterogeneous density distribution) particle-particle interaction via gravitation potential (or modifications): U(r ij ) = G Grav m i m j r ij uses octrees: leaves of the tree represent empty cells or one particle, inner nodes represent clusters of particles or pseudo particles main underlying idea: gravitation (or other force) induced by many particles in one remote cell can be approximated by the influence induced by one pseudo particle of the accumulated mass in the cell s centre of gravity Perlen der Informatik II, Hans-Joachim Bungartz page 35 of 47

36 three main steps: construction of the octree: refinement, until just one or no particle per cell (top-down recursion) calculation of pseudo particles: mass is just sum of the cell s particles masses, associated point is the mass-weighted average of the particles positions (bottom-up recursion) calculation of forces (N incomplete top-down traversals for N particles) Perlen der Informatik II, Hans-Joachim Bungartz page 36 of 47

37 Calculation of the Forces remember: to be done for each particle assume a particle with position x Ω start in root node proceed to son nodes, until the θ rule: diam r θ is fulfilled, where r denotes the distance of the corresponding pseudo particle to x then, add the resulting interaction influence to the overall result implicit separation into near-field and far-field: near-field: all leaves reached during this traversal (single-particle influence) far-field: all inner nodes where the process stops, i.e. cells representing pseudo particles Perlen der Informatik II, Hans-Joachim Bungartz page 37 of 47

38 several variants concerning the determination of the parameter diam Barnes-Hut method can be interpreted as a special case of the Taylor approximation discussed above for p = 0 (for the calculation of the pseudo particles, we just sum up masses, i.e. zero-th moments) Perlen der Informatik II, Hans-Joachim Bungartz page 38 of 47

39 Example of the Tree Construction and Force Calculation x i x i Perlen der Informatik II, Hans-Joachim Bungartz page 39 of 47

40 Accuracy and Complexity accuracy: depends on control parameter θ the smaller we choose θ, the larger gets the near-field and the smaller, hence, gets the error resulting from the corresponding far-field approximation however, slow O(θ) convergence due to p = 0 complexity: increases for decreasing θ for roughly homogeneous particle distributions: number of active cells bounded by C log N/θ 3 for some constant C and N particles overall cost of order O(θ 3 N log N) for limit case θ 0: method degenerates to the quadratic complexity of the original each-with-each approach without far-field approximation Perlen der Informatik II, Hans-Joachim Bungartz page 40 of 47

41 Some Remarks on the Implementation use a standard data structure for each (pseudo) particle standard tree implementation with pointers: each node contains one (pseudo) particle at most subdomain corresponding to a node/cell can be either stored explicitly or calculated on-the-fly during a tree traversal four (2D) or eight (3D) pointers from a node to its sons linearisation is possible, too (i.e. no need for pointers) tree traversal typically by recursion pre-order (top-down) post-order (bottom-up) tree construction: successive insertion of particles starting from the empty tree (i.e. the octree consisting of the root node only) refine, when necessary (i.e. two particles in one node) Perlen der Informatik II, Hans-Joachim Bungartz page 41 of 47

42 Some Remarks on the Implementation (2) calculation of pseudo particles: bottom-up (post-order) traversal sum of masses, weighted average of positions as described before calculation of forces: outer loop: traversal visiting each leaf (to get the far-field approximation for each particle) inner loop: top-down (pre-order) traversal until the θ-rule is fulfilled parameter diam on-the-fly time integration: essentially as before, now as another tree traversal motion: either via constructing a new octree or via modifying the existing octree (generally preferred) Perlen der Informatik II, Hans-Joachim Bungartz page 42 of 47

43 The Fast-Multipole Method Main Idea idea of Barnes-Hut: replace particle-particle interactions by interactions between particles and pseudo particles effect is an improved complexity: O(N log N) instead of O(N 2 ) for almost homogeneous particle distributions, and still a significant effect otherwise next step now: avoid calculation of interactions with remote pseudo particles on the origin side for each particle on the effect side individually instead, combine particles to pseudo particles on the effect side also and calculate interactions of pseudo particles with remote pseudo particles, only cells representing pseudo particles with their contained particles are called clusters from such a clucter-cluster interaction, the particle-cluster interactions can be derived for all of the cluster s particles if done properly, this fast multipole method provides a linear O(N) complexity Perlen der Informatik II, Hans-Joachim Bungartz page 43 of 47

44 Fundamentals starting point again: Taylor expansion of the kernel G(x, y), but now in x and y around x l 0 Ω l and y i 0 Ω i: G(x, y) = k 1 p j 1 p 1 k!j! (x x l 0) k (y y i 0) j G k,j (x l 0, y i 0) + ˆR p (x, y) leads to expansion (and approximation) of the potential (here: the part due to the subdomain Ω i ): Φ i (x) k 1 p 1 k! (x x 0) l k (moments defined as before) j 1 p 1 j! G k,j(x l 0, y i 0)M j (Ω i, y i 0) Perlen der Informatik II, Hans-Joachim Bungartz page 44 of 47

45 hence: interaction between Ω i and Ω l is calculated once and can, afterwards, be used for deriving the particle-cluster interactions for all x Ω l (using suitable transformation rules) this principle is now applied hierarchically or recursively: first, calculate interactions between large clusters these are inherited to the son level, where now the resulting interactions are determined finally, get particle-x interactions in the leaves Perlen der Informatik II, Hans-Joachim Bungartz page 45 of 47

46 Barnes-Hut vs. Fast Multipole Comparison x i 0 y x l 0 i 0 y Perlen der Informatik II, Hans-Joachim Bungartz page 46 of 47

47 Subdivision instead of a recursive subdivision of Ω, we have to do it for Ω Ω now same octree structure for origin and effect side typically, same midpoints : x i 0 = y i 0 Ω i (again, geometrical midpoint or centre of gravity) now, also a double θ-criterion: x x l 0 x l 0 y i 0 θ and y y i 0 x l 0 y i 0 θ x Ω l, y Ω i hierarchy can be used for an efficient calculation of the interactions (for details, we refer to the literature) accuracy: as before, an O(θ p+1 )-characteristics can be shown for the relative error parallelisation: only slightly more complicated than for Barnes-Hut Perlen der Informatik II, Hans-Joachim Bungartz page 47 of 47