Graph Operators for Coupling-aware Graph Partitioning Algorithms

Transcription

1 Graph Operators for Coupling-aware Graph Partitioning lgorithms Maria Predari, urélien Esnard To cite this version: Maria Predari, urélien Esnard. Graph Operators for Coupling-aware Graph Partitioning lgorithms. CIMI Workshop on Innovative clustering methods for large graphs and block methods, Jul 2015, Toulouse, France. < <hal > HL Id: hal Submitted on 22 Sep 2015 HL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 Graph Operators for Coupling-aware Graph Partitioning lgorithms 1/32 Graph Operators for Coupling-aware Graph Partitioning lgorithms Maria Predari urélien Esnard Université de ordeaux INRI HiePCS & LRI CIMI15 July 2015

3 Graph Operators for Coupling-aware Graph Partitioning lgorithms 2/32 Outline 1 Introduction 2 lgorithms 3 Preliminary Experimental Results 4 Conclusion & Prospects

4 Graph Operators for Coupling-aware Graph Partitioning lgorithms 3/32 Outline 1 Introduction 2 lgorithms 3 Preliminary Experimental Results 4 Conclusion & Prospects

5 Context Efficiency of numerical simulations requires a distribution of the computational mesh to several processors Load alancing Problem Distribute equally the load is crucial for the performance of simulations common approach is based on graph representation: vertex process edge dependency Graph Partitioning Problem Divide the graph in equal parts and assign them to different processors [Teresco, J. D., Devine, K. D., and Flaherty, J. E. (2006)] NP-hard problem heuristic algorithms raph Operators for Coupling-aware Graph Partitioning lgorithms 4/32

6 raph Operators for Coupling-aware Graph Partitioning lgorithms 5/32 Multilevel Graph Partitioning Most common graph partitioning techiques use a multilevel framework to simplify the partitioning 3 phases of a V-cycle: 1 Coarsening Phase vertex contraction: HEM 2 Initial Partitioning Phase perform partitioning with any known algorithm: bisection, spectral, etc. 3 Uncoarsening Phase Projection of the initial partitioning back to the initial graph (refinement)

7 raph Operators for Coupling-aware Graph Partitioning lgorithms 6/32 Multilevel Graph Partitioning Multilevel Recursive isection (MLR) Multilevel k-way Partitioning (MLKW) k-1 V-cycles for a k-way partitioning 1 V-cycle with direct k-way partitioning

8 Graph Operators for Coupling-aware Graph Partitioning lgorithms 7/32 Challenges emerging complex simulations where traditional load balancing strategies are not adequate multi-scale, multi-physics, coupled simulations Domain Simulation Type Components astrophysics structure and Newtonian gravitational interactions model multiplysics evolution of stars star aging model biology modeling thrombus macroscopic dynamics of bload flow multiscale development microscopic interactions between cells aerospace multiphysics, Navier-Stokes equations for fluid flow engineering combustion chamber coupled heat tranfer solver for solid atmospheric model environemental multiphysics, ocean model climate modeling science coupled sea-ice model land model

9 Graph Operators for Coupling-aware Graph Partitioning lgorithms 8/32 Coupled Simulation Model coupled simulation consists of a number of component simulations that interact periodically Regular Phase 1. Components solve individual systems 2. Parallel execution of components 3. Synchronization step (T idle ) Coupling Phase 1. Domains intersect - coupling interface 2. Data exchange between components Coupled simulation model between components and

10 Graph Operators for Coupling-aware Graph Partitioning lgorithms 9/32 Related Work We identify two sub-problems of coupled simulations: 1 Resource Distribution Problem: find nb of processors for the components on both phases 2 Data Distribution Problem: find a good data distribution for both phases of the coupled simulation

11 Graph Operators for Coupling-aware Graph Partitioning lgorithms 9/32 Related Work We identify two sub-problems of coupled simulations: 1 Resource Distribution Problem: find nb of processors for the components on both phases empiric approach as VP-VTP [ F. Duchaine, et al. (2012)] dynamic adaptation of processors as CESM [Graig,.P., et al. (2012)] 2 Data Distribution Problem: find a good data distribution for both phases of the coupled simulation NIVE approach: partition each component independently imbalance in coupling phase multi-constraint graph partitioning [Karypis, G. (2003)] no control on the nb of processors in different phases

12 raph Operators for Coupling-aware Graph Partitioning lgorithms 10/32 Motivation With a classic graph partitioning, during a coupling phase: Load is not well balanced ad inter-components communications Coupling-aware partitioning (co-partitioning) multi-scale simulation in material physics Partitioning that takes into account the coupling phase explicitly as a trade-off to the partitioning in regular phase

13 Graph Operators for Coupling-aware Graph Partitioning lgorithms 11/32 Outline 1 Introduction 2 lgorithms 3 Preliminary Experimental Results 4 Conclusion & Prospects

14 raph Operators for Coupling-aware Graph Partitioning lgorithms 12/32 Classic Graph Partitioning k-way graph partitioning divide graph in k parts. ssign it to k processors Objectives: balance weight between parts minimize number of edge cut 3 parts of weight= 4 and edgecut= 8 Coupled simulations enrich the classic graph model to take into account multiple components and the coupling phase

15 Graph Operators for Coupling-aware Graph Partitioning lgorithms 13/32 Definition of Co-partitioning Graph Model G X : component in regular phase G cpl X : component in coupling phase I XY : inter-component communication Coupling interface with interedges k-way co-partition with k = k + k k-way co-partitioning Objectives For each X component: 1. k X partitioning in regular phase 2. k cpl X partitioning in coupling interface For each coupling phase XY : 3. minimize inter-component communication

16 raph Operators for Coupling-aware Graph Partitioning lgorithms 14/32 lgorithms State-of-the-art approach: each component is partitioned independently (NIVE) coupling interfaces of mesh and mesh Problem: Components are not aware of their coupling interfaces Imbalance during coupling phase! naive partitions of mesh and mesh

17 Graph Operators for Coupling-aware Graph Partitioning lgorithms 14/32 lgorithms State-of-the-art approach: each component is partitioned independently (NIVE) coupling interfaces of mesh and mesh Problem: Components are not aware of their coupling interfaces Imbalance during coupling phase! naive partitions of mesh and mesh We propose two k-way co-partitioning algorithms: WRE PROJREPRT

18 Graph Operators for Coupling-aware Graph Partitioning lgorithms 15/32 Graph Operators lgorithmic description as a sequence of graph operators: Partition, Restriction, Projection, Repartition, Extension [Predari, M., Esnard,. (14)] Partition computes a partition of a graph with any algorithm Restriction finds the subgraph in coupling phase Projection computes a partition similar to another one

19 raph Operators for Coupling-aware Graph Partitioning lgorithms 16/32 Graph Operators II Extension: extends an initially smaller partition to the rest of a graph Input: initial partition P clp Constraint: vertices in P clp on a subgraph should remain fixed

20 raph Operators for Coupling-aware Graph Partitioning lgorithms 16/32 Graph Operators II Extension: extends an initially smaller partition to the rest of a graph Input: initial partition P clp Constraint: vertices in P clp on a subgraph should remain fixed Output: partition P of the whole graph

21 raph Operators for Coupling-aware Graph Partitioning lgorithms 16/32 Graph Operators II Extension: extends an initially smaller partition to the rest of a graph Input: initial partition P clp Constraint: vertices in P clp on a subgraph should remain fixed Output: partition P of the whole graph Repartition: changes the numbers of part in a partition [Vuchener, C., Esnard,. (2012)] Input: initial partition P on the whole graph Objective: good migration matrix (greedy strategy)

22 raph Operators for Coupling-aware Graph Partitioning lgorithms 16/32 Graph Operators II Extension: extends an initially smaller partition to the rest of a graph Input: initial partition P clp Constraint: vertices in P clp on a subgraph should remain fixed Output: partition P of the whole graph Repartition: changes the numbers of part in a partition [Vuchener, C., Esnard,. (2012)] Input: initial partition P on the whole graph Objective: good migration matrix (greedy strategy) Output: balanced partition P on the whole graph

23 raph Operators for Coupling-aware Graph Partitioning lgorithms 16/32 Graph Operators II Extension: extends an initially smaller partition to the rest of a graph Input: initial partition P clp Constraint: vertices in P clp on a subgraph should remain fixed Output: partition P of the whole graph Repartition: changes the numbers of part in a partition [Vuchener, C., Esnard,. (2012)] Input: initial partition P on the whole graph Objective: good migration matrix (greedy strategy) Output: balanced partition P on the whole graph Partitioning implementation should handle fixed vertices!

24 Graph Operators for Coupling-aware Graph Partitioning lgorithms 17/32 Graph Partitiong Tools for Initial Fixed Vertices List of graph partitioning tools that handle fixed vertices. Tools Type Fixed Parallel Multilevel Scheme Initial Part. vailable Scotch graph yes no MLKW R source RM-Metis graph only k no MLKW greedy no HMetis hypergraph yes no MLR binary PaToH hypergraph yes no MLR binary KPaToH hypergraph yes no MLKW R no Zoltan(PHG) hypergraph yes yes MLR source Limitations of Recursive isection (R) based lgorithms Initial fixed vertices scheme Partitioning quality of R before refinements after refinements R fails to partition a simple graph in 4 parts with initial fixed vertices inherent numbering constraint VS fixed vertex constraint

25 Graph Operators for Coupling-aware Graph Partitioning lgorithms 18/32 k-way Greedy Graph Growing Partitioning (KGGGP) KGGGP: initial partitioning algorithm inside SCOTCH s framework: extension of greedy bisection to k parts at each step, selection of the best global displacement (v, k) selection based on an edgecut minimization criterion use of FM-like structures Time Complexity: O(k E ) Optimization: local selection of best displacement ([Karypis, G., Kumar, V. (1998)]) Initial fixed vertices scheme Partitioning quality of KGGGP before refinements after refinements

26 Graph Operators for Coupling-aware Graph Partitioning lgorithms 19/32 WRE lgorithm Each component is aware of it s coupling interface Input: G, G, K, K, K cpl, K cpl Output: P, P 1. Rest(G ) G cpl 2. Rest(G ) G cpl 3. Part(G cpl 4. Part(G cpl 5. Ext(G, G cpl 6. Ext(G, G cpl ) Pcpl ) Pcpl, Pcpl, Pcpl ) P ) P mesh coupling overview with interedges mesh

27 Graph Operators for Coupling-aware Graph Partitioning lgorithms 19/32 WRE lgorithm Each component is aware of it s coupling interface Input: G, G, K, K, K cpl, K cpl Output: P, P 1. Rest(G ) G cpl 2. Rest(G ) G cpl 3. Part(G cpl 4. Part(G cpl 5. Ext(G, G cpl 6. Ext(G, G cpl ) Pcpl ) Pcpl, Pcpl, Pcpl ) P ) P mesh coupling overview with interedges mesh

28 Graph Operators for Coupling-aware Graph Partitioning lgorithms 19/32 WRE lgorithm Each component is aware of it s coupling interface Input: G, G, K, K, K cpl, K cpl Output: P, P 1. Rest(G ) G cpl 2. Rest(G ) G cpl 3. Part(G cpl 4. Part(G cpl 5. Ext(G, G cpl 6. Ext(G, G cpl ) Pcpl ) Pcpl, Pcpl, Pcpl ) P ) P mesh coupling overview with interedges mesh

29 Graph Operators for Coupling-aware Graph Partitioning lgorithms 20/32 PROJREPRT lgorithm Components are also aware of other component s coupling interface Input: G, G, K, K, K cpl, K cpl Output: P, P 1. Rest(G ) G cpl 2. Rest(G ) G cpl 3. Part(G cpl 4. Proj(G cpl 5. Repart(G cpl ) Pcpl, G cpl 6. Ext(G, G cpl 7. Ext(G, G cpl, P cpl, Pcpl, Pcpl, Pcpl ) P cpl ) Pcpl ) P ) P mesh coupling overview with interedges mesh

30 Graph Operators for Coupling-aware Graph Partitioning lgorithms 20/32 PROJREPRT lgorithm Components are also aware of other component s coupling interface Input: G, G, K, K, K cpl, K cpl Output: P, P 1. Rest(G ) G cpl 2. Rest(G ) G cpl 3. Part(G cpl 4. Proj(G cpl 5. Repart(G cpl ) Pcpl, G cpl 6. Ext(G, G cpl 7. Ext(G, G cpl, P cpl, Pcpl, Pcpl, Pcpl ) P cpl ) Pcpl ) P ) P mesh coupling overview with interedges mesh

31 Graph Operators for Coupling-aware Graph Partitioning lgorithms 20/32 PROJREPRT lgorithm Components are also aware of other component s coupling interface Input: G, G, K, K, K cpl, K cpl Output: P, P 1. Rest(G ) G cpl 2. Rest(G ) G cpl 3. Part(G cpl 4. Proj(G cpl 5. Repart(G cpl ) Pcpl, G cpl 6. Ext(G, G cpl 7. Ext(G, G cpl, P cpl, Pcpl, Pcpl, Pcpl ) P cpl ) Pcpl ) P ) P mesh coupling overview with interedges mesh

32 Graph Operators for Coupling-aware Graph Partitioning lgorithms 20/32 PROJREPRT lgorithm Components are also aware of other component s coupling interface Input: G, G, K, K, K cpl, K cpl Output: P, P 1. Rest(G ) G cpl 2. Rest(G ) G cpl 3. Part(G cpl 4. Proj(G cpl 5. Repart(G cpl ) Pcpl, G cpl 6. Ext(G, G cpl 7. Ext(G, G cpl, P cpl, Pcpl, Pcpl, Pcpl ) P cpl ) Pcpl ) P ) P mesh coupling overview with interedges mesh

33 Graph Operators for Coupling-aware Graph Partitioning lgorithms 20/32 PROJREPRT lgorithm Components are also aware of other component s coupling interface Input: G, G, K, K, K cpl, K cpl Output: P, P 1. Rest(G ) G cpl 2. Rest(G ) G cpl 3. Part(G cpl 4. Proj(G cpl 5. Repart(G cpl ) Pcpl, G cpl 6. Ext(G, G cpl 7. Ext(G, G cpl, P cpl, Pcpl, Pcpl, Pcpl ) P cpl ) Pcpl ) P ) P mesh coupling overview with interedges mesh

34 Graph Operators for Coupling-aware Graph Partitioning lgorithms 21/32 Outline 1 Introduction 2 lgorithms 3 Preliminary Experimental Results 4 Conclusion & Prospects

35 Graph Operators for Coupling-aware Graph Partitioning lgorithms 22/32 Experimental Description for KGGGP algorithm Graph Description (DIMCS 10 Collection) collection graph # vtx # edges avg d min d max d walshaw fe rotor walshaw walshaw wave walshaw m14b matrix audikw matrix ecology matrix thermal matrix af shell numerical NC numerical 333SP numerical nlr numerical channel numerical adaptive Metrics: partitioning quality, time performance our implementations : KGGGP G, KGGGP L tools: SCOTCH 6.0.3, kmetis 5.1.0, PaToH 3.0, Zoltan 3.81 imbalance tolerance = 5% results from 3 experiments on average values of each graph

36 Graph Operators for Coupling-aware Graph Partitioning lgorithms 23/32 Exp1:Evaluation of KGGGP without fixed vertices Relative results to SCOTCH matrix numerical walshaw edgecut nb parts 2.0 matrix numerical walshaw 1.5 runtime nb parts Methods SCOTCH KGGGP_G KGGGP_L KMETIS

37 Graph Operators for Coupling-aware Graph Partitioning lgorithms 24/32 Exp2:Evaluation of KGGGP with fixed vertices vertices are fixed based on a bubble scheme (seeds & levelset) matrix numerical walshaw edgecut nb parts 2.0 matrix numerical walshaw 1.5 runtime nb parts Methods SCOTCH KGGGP_G KGGGP_L PTOH ZOLTN

38 Graph Operators for Coupling-aware Graph Partitioning lgorithms 25/32 Exp3:Evaluation of KGGGP with fixed vertices vertices are fixed based on a repartition scheme (initial imbalance partition) matrix numerical walshaw edgecut nb parts 2.0 matrix numerical walshaw 1.5 runtime nb parts Methods SCOTCH KGGGP_G KGGGP_L PTOH ZOLTN

39 raph Operators for Coupling-aware Graph Partitioning lgorithms 26/32 Experimental Description for co-partitioning synthetically generated mesh structures use of KGGGP implementation Description of the experiments Exp. Graph Graph exp1 cube-hexa-25x25x25 cube-hexa-100x100x100 exp2 cube-hexa-25x25x25 cube-hexa-70x70x70 exp3 cube-tetra cube-hexa-100x100x100 exp4 cube-tetra cube-tetra Metrics: load balancing in coupling phase global edgecut number of messages exchanged Results shown for k = 16 and k = 48

40 Graph Operators for Coupling-aware Graph Partitioning lgorithms 27/32 Load alancing in Coupling Phase Load balance during the coupling phase of mesh WRE and PROJREPRT respect the imbalance tolerance (typically 5% of the ideal weight) NIVE highly imbalanced during coupling phase

41 Graph Operators for Coupling-aware Graph Partitioning lgorithms 28/32 Edgecut Total edgecut of mesh NIVE is used as the reference WRE and PROJREPRT do not severely degrade the total egdecut

42 Graph Operators for Coupling-aware Graph Partitioning lgorithms 29/32 Number of Messages mesh mesh well-aligned mesh mesh Total number of messages exchanged during coupling interface bad-aligned exp1 - exp2: hexaedric elements are well aligned exp3 - exp4: elements of different discretization misaligned

43 Graph Operators for Coupling-aware Graph Partitioning lgorithms 30/32 Outline 1 Introduction 2 lgorithms 3 Preliminary Experimental Results 4 Conclusion & Prospects

44 Graph Operators for Coupling-aware Graph Partitioning lgorithms 31/32 Conclusion and Future Work Conclusion 1. new algorithm for graph partitioning with fixed vertices KGGGP 2. Two new algorithms for graph partitioning of coupled simulations: WRE and PROJREPRT Experimental results: 2. Good load balancing during the coupling phase for both methods at a slight increase of edge-cut 3. In simple experiments the number of messages exchanged between the two graph is minimized Prospects for co-partitioning More experiments with real-life and larger graphs Parallel version

45 Graph Operators for Coupling-aware Graph Partitioning lgorithms 32/32 End Thank you for you attention. ny questions?