Penalized Graph Partitioning for Static and Dynamic Load Balancing

Transcription

1 Penalized Graph Partitioning for Static and Dynamic Load Balancing Tim Kiefer, Dirk Habich, Wolfgang Lehner Euro-Par 06, Grenoble, France,

2 Task Allocation Challenge Application (Workload) = Set of Tasks Consolidation Task Allocation (Load Balancing) Combine various tasks on a single infrastructure node Infrastructure = Set of Nodes

3 Modeling Problems as Graph GRAPH Problem reduced to smaller, independent tasks Processing cost of tasks are modelled as vertex weights (possibly multiple per vertex) Communication between tasks is modelled as edge weights TASK ALLOCATION CHALLENGE Given a workload graph and an infrastructure, find a valid and balanced allocation that minimizes communication costs. (0.,0GB) (,0GB) (0.,7GB) (0.,0GB) (0.,0GB) (0.,0GB) (0.,GB) (0.,GB) (,0GB) (0.,7GB) Graph partitioning is an appropriate technique for that area (0.,5GB) (,0GB) (0.,5GB) (,0GB)

4 Classical Performance Model of Nodes COMBINED LOAD PREDICTOR Tasks are consolidated on nodes (i.e., tasks share nodes) Linear model: combined load is sum of all individual loads [Schaffner0]

5 Penalized Performance Model COMBINED LOAD PREDICTOR Tasks are consolidated on nodes (i.e., tasks share nodes) Linear model: combined load is sum of all individual loads Non-linear model: combined load is greater than sum of individual loads (congestion) Special case: linearly combined load and non-linear penalty [Schaffner0] 5

6 Motivating Example WORKLOAD GRAPH workload graph contains,000 heterogeneous tasks with weights following a Zipf distribution each task is communicating with 0 to 0 other tasks (again Zipf distributed). INFRASTRUCTURE nodes and each node can execute 6 parallel tasks before contention occurs 6

7 Overview Introduction Task Allocation Challenge Penalized Resource Model Graph Partitioning Multi-Level Graph Partitioning Penalized Graph Partitioning Static Case Dynamic Case Evaluation Scalability Experiments 7

8 Graph Partitioning Definitions by Example BALANCED K-WAY MIN-CUT GRAPH PARTITIONING Find k partitions Minimize the sum of edges that are cut Keep the sizes of all parts in balance NP-complete problem [Hyafil and Rivest, 97] NOTATION Vertex Identifiers Vertex Weights Vertex weights Edge weights Graph partitioning Cut edge Total cut Balance constraint Edge Weights Partitioning 8

9 Multilevel Graph Partitioning Framework. COARSENING input graph match and contract vertices refine partitioning extract vertices final partitioning. UNCOARSENING AND REFINEMENT. INITIAL PARTITIONING [Karypis and Kumar 995, Buluç et. al 0] 9

10 Multilevel Graph Partitioning Framework. COARSENING Find matching and contract vertices Balanced cut in the coarse graph equals balanced cut in the fine graph with same cut value E.g., heavy edge heuristic 0

11 Multilevel Graph Partitioning Framework. COARSENING Find matching and contract vertices Balanced cut in the coarse graph equals balanced cut in the fine graph with same cut value E.g., heavy edge heuristic. INITIAL PARTITIONING Many possible algorithms/heuristics Region growing Repeated bisection or direct k-way

12 Multilevel Graph Partitioning Framework. COARSENING Find matching and contract vertices Balanced cut in the coarse graph equals balanced cut in the fine graph with same cut value E.g., heavy edge heuristic. INITIAL PARTITIONING Many possible algorithms/heuristics Region growing Repeated bisection or direct k-way. UNCOARSENING AND REFINEMENT Uncoarsen by extracting collapsed vertices Refinement with local vertex-swapping heuristics Kernighan-Lin (KL) [Kenighan and Lin 970] Fiduccia-Mattheyses (KL/FM) [Fiduccia and Mattheyses 98] Karypis and Kumar [Karypis and Kumar 995 and 998]????

13 Overview Introduction Task Allocation Challenge Penalized Resource Model Graph Partitioning Multi-Level Graph Partitioning Penalized Graph Partitioning Static Case Dynamic Case Evaluation Scalability Experiments

14 Non-Linear Graph Partitioning PENALIZED GRAPH PARTITIONING no penalty: p( V ) = 0 Penalized partition weight: sum of vertex weights plus penalty w V $ w v ' ) + p V sum of vertex weights penalty partition weight total cut V V V V V V linear penalty: p( V ) = V sum of vertex weights penalty partition weight total cut V V square penalty: p( V ) = V V V sum of vertex weights penalty partition weight total cut V V

15 Non-Linear Graph Partitioning PENALIZED GRAPH PARTITIONING total vertex weight total partition weight Penalized partition weight: sum of vertex weights plus penalty Penalized partition weights are not additive: w(v È V ) w(v ) + w(v ) Total vertex weight vs. total partition weigh Total partition weight is not constant w ) $ w ) v ' ) V + p V w, $ w ) V -. -/0 V V V V V sum of vertex weights square penalty partition weight 76 total vertex weight total partition weight 5

16 Non-Linear Graph Partitioning PENALIZED GRAPH PARTITIONING Penalized partition weight: sum of vertex weights plus penalty Penalized partition weights are not additive: w(v È V ) w(v ) + w(v ) Total vertex weight vs. total partition weigh Total partition weight is not constant MOVING A VERTEX w ' V 0 v = $ 5 ) 6 ' w ) u + p V 0 v = $ w ) u w ) v + p V 0 5 ) 6 = w ) V 0 w ) v p V 0 + p V 0 w ) V 9 v = w ) V 9 + w ) v p V 9 + p V 9 + MODIFIED GRAPH PARTITIONING FRAMEWORK Modifications in. Coarsening. Initial partitioning. Refinement New operations/implementations for. Moving a vertex. Combining partitions COMBINING PARTITIONS w ' V 0 V 9 = $ w ) v 5 ) 6 ) ; + p V 0 V 9 = $ w ) v ' ) 6 + $ w ) v + p V 0 + V 9 ' ) ; = w ) V 0 + w ) V 9 + p V 0 + V 9 p V 0 p V 9 6

17 Dynamic Load Balancing CHANGES IN WORKLOAD OR INFRASTRUCTURE Change weights Add or remove vertices or edges MIGRATION COSTS Incremental update is trade off between partition quality and migration costs HYBRID STRATEGY Local Refinement: use KL/FM method to balance/refine partitioning Complete Repartitioning: partition from scratch and try to map partitions such that migration costs are minimal Local refinement if possible, complete repartitioning if not 7

18 Overview Introduction Task Allocation Challenge Penalized Resource Model Graph Partitioning Multi-Level Graph Partitioning Penalized Graph Partitioning Static Case Dynamic Case Evaluation Scalability Experiments 8

19 Partitioning Time Comparison IMPLEMENTATION OF OUR PENALIZED GRAPH PARTITIONING PenMETIS based on METIS (v5.) Walshaw Benchmark: Graph Partitioning Archive ( graphs and best known partitioning) 6 partitions, max. % imbalance V = E =..6 V =.69 E =.07.9 V =.765 E =

20 Scalability GRAPH SIZE PenMETIS scales linearly with the number of vertices and the number of edges 0

21 Scalability GRAPH SIZE PenMETIS scales linearly with the number of vertices and the number of edges PARTITION COUNT PenMETIS scales linearly with the number of partitions

22 Summary and Conclusion Application (Workload) = Set of Tasks Task Allocation (Load Balancing) Penalized Graph Partitioning Infrastructure = Set of Nodes