Load Balancing Techniques for Asynchronous Spacetime Discontinuous Galerkin Methods

Size: px

Start display at page:

Download "Load Balancing Techniques for Asynchronous Spacetime Discontinuous Galerkin Methods"

Constance Ryan
5 years ago
Views:

1 Load Balancing Techniques for Asynchronous Spacetime Discontinuous Galerkin Methods Aaron K. Becker Robert B. Haber Laxmikant V. Kalé University of Illinois, Urbana-Champaign Parallel Programming Lab Center for Process Simulation and Design UNSCCM 09 NSF: ITR/AP DMR ITR/AP DMR NSF: ITR/AP DMR

2 Fixed Timestep 1D Algorithm Time!t Space 2

3 Tentpitcher: Causal Spacetime Mesh Advancing-Front Solution Strategy Time Space 3

4 Tentpitcher: patch by patch solution & meshing 1 2 4

5 Crack-tip Wave Scattering 5

6 Parallelizing Tentpitcher Approach take advantage of local decision-making algorithm to avoid global communication and promote scalability build in latency tolerance to support large grain sizes Decompose and distribute space mesh All non-boundary operations are purely local Perform boundary communication on-demand using a message driven approach 6

7 Message-driven SDG Over-decomposition and virtualization multiple mesh partitions per processor computation on one partition can be overlapped with blocking communication on another local partition User View System View 7

8 System Overview Tentpitcher Algorithm Partitioning Incremental Adaptivity ParFUM Ghost Layer Maintenance Element Migration Virtualization Migration Scheduling Charm++ Runtime System

9 Code Structure Partition and Distribute Mesh Pitch Local Vertex Local Adaptivity Pitch Local Vertex Local Adaptivity... Pitch Local Vertex Local Adaptivity Virtual Processors Combine Results 9

10 Performance Effects of Virtualization Virtualized, Non-adaptive Virtualized, Adaptive Non-virtualized, Non-adaptive Non-virtualized, Adaptive Perfect Scaling, Non-adaptive Perfect Scaling, Adaptive Pitches/s Processors 10

11 SDG Cluster Performance (Abe) Non-adaptive, Weak scaling Adaptive, Weak scaling Perfect Scaling Pitches/s Processors 11

12 Dealing with Load Imbalance Aside from load imbalance, few barriers to scalability This method naturally tolerates small imbalances But, for some problems we expect large imbalances 12

13 Partition Migration Idea: take advantage of virtualization: there are multiple partitions per processor, so they can be rearranged to improve load balance Standard approach in virtualized environments: Charm++ supports a variety of algorithms for relocating partitions Advantages built-in support, requires little modification of application effective for moderate imbalances Disadvantages global, synchronous approach is a poor fit for tentpitcher really large imbalances may not be fixable--the presence of dramatically overloaded partitions cannot be covered up without unacceptable overhead 13

14 Diffusion Load Balancing Idea: apply purely local decision making process to load balance by migrating individual mesh elements across partition boundaries once load imbalance crosses a particular threshold value If neighboring partitions i and j have loads λi and λj, choose r >1 and migrate elements from i to j when r λi > λj Advantages: requires only local synchronization and communication 14

15 Code Structure Partition and Distribute Mesh Pitch Local Vertex Local Adaptivity Pitch Local Vertex Local Adaptivity... Pitch Local Vertex Local Adaptivity Virtual Processors Load Balancing Load Balancing Load Balancing Combine Results 15

16 Diffusion Load Balancing Partition i Partition j Initially, λi λj so no load balancing is needed. 16

17 Diffusion Load Balancing Partition i Partition j After local refinement, λi > rλj so boundary elements will move from i to j 17

18 Diffusion Load Balancing Partition i Partition j After local refinement, λi > rλj so boundary elements will move from i to j 18

19 Diffusion Load Balancing Partition i Partition j After local refinement, λi > rλj so boundary elements will move from i to j 19

20 Diffusion Load Balancing Partition i Partition j After local refinement, λi > rλj so boundary elements will move from i to j 20

21 Diffusion Load Balancing Partition i Partition j After local refinement, λi > rλj so boundary elements will move from i to j 21

22 Diffusion Load Balancing Partition i Partition j We attempt to migrate elements in a way that maintains or improves boundary quality. 22

23 Diffusion Load Balancing Issues Maintaining boundary quality Maintaining accurate load estimates Choosing r to avoid unneeded transfers while still avoiding serious imbalance Determining the right termination condition for the load balancing step Minimizing lock contention on boundary elements 23

24 Load Balancing Techniques for Asynchronous Spacetime Discontinuous Galerkin Methods Aaron K. Becker Robert B. Haber Laxmikant V. Kalé University of Illinois, Urbana-Champaign Parallel Programming Lab Center for Process Simulation and Design UNSCCM 09

Scalable Dynamic Adaptive Simulations with ParFUM

Scalable Dynamic Adaptive Simulations with ParFUM Terry L. Wilmarth Center for Simulation of Advanced Rockets and Parallel Programming Laboratory University of Illinois at Urbana-Champaign The Big Picture