Batched Factorization and Inversion Routines for Block-Jacobi Preconditioning on GPUs
|
|
- Daniel Daniel
- 5 years ago
- Views:
Transcription
1 Workshop on Batched, Reproducible, and Reduced Precision BLAS Atlanta, GA 02/25/2017 Batched Factorization and Inversion Routines for Block-Jacobi Preconditioning on GPUs Hartwig Anzt Joint work with Goran Flegar, Enrique Quintana-Orti
2 Workshop on Batched, Reproducible, and Reduced Precision BLAS Atlanta, GA 02/25/2017 Batched Factorization and Inversion Routines for Block-Jacobi Preconditioning on GPUs Hartwig Anzt Joint work with Goran Flegar, Enrique Quintana-Orti Pitch from the sparse community: Small size (<30), somewhat flexible Add inversion routine to Batched-Blas Go for Gauss-Jordan Elimination instead of LU
3 Iterative linear system solvers Goal: solve linear system Ax = b, A 2 R n n Use iterative Krylov method for generating solution approximation Convergence typically benefits from using a preconditioner We focus on Jacobi preconditioners based on diagonal scaling P =(diag(a)) 1 Provide a high level of parallelism Attractive for many-core accelerators like GPUs Scalar Jacobi acts on the main diagonal block-jacobi acts on the block-diagonal 3
4 Block-Jacobi preconditioner Can reflect the block-structure of sparse matrices (e.g. Higher-Order FEM) Often superior to scalar Jacobi preconditioner Light-weight preconditioner (compared to ILU) Parallel preconditioner application ( batched trsv / SpMV + vector scaling ) Inversion of diag. blocks in prec. setup + SpMV in prec. application Factorization of diag. blocks in prec. setup + trsv in prec. application 4
5 Block-Jacobi preconditioner Can reflect the block-structure of sparse matrices (e.g. Higher-Order FEM) Often superior to scalar Jacobi preconditioner Light-weight preconditioner (compared to ILU) Parallel preconditioner application ( batched trsv / SpMV + vector scaling ) Inversion of diag. blocks in prec. setup + SpMV in prec. application Factorization of diag. blocks in prec. setup + trsv in prec. application 5
6 Block-Jacobi preconditioner Can reflect the block-structure of sparse matrices (e.g. Higher-Order FEM) Often superior to scalar Jacobi preconditioner Light-weight preconditioner (compared to ILU) Parallel preconditioner application ( batched trsv / SpMV + vector scaling ) Inversion of diag. blocks in prec. setup + SpMV in prec. application Factorization of diag. blocks in prec. setup + trsv in prec. application Preconditioner setup (inversion of diagonal blocks) can become a challenge Batched inversion of many small systems Diagonal blocks are typically of small size 6
7 Block-Jacobi preconditioner generation 1. extract diagonal block from sparse data structure invert diagonal block via Gauss-Jordan elimination insert solution as diagonal block into the preconditioner matrix 7
8 Block-Jacobi preconditioner generation 1. extract diagonal block from sparse data structure invert diagonal block via Gauss-Jordan elimination insert solution as diagonal block into the preconditioner matrix 8
9 Batched Gauss-Jordan elimination (BGJE) on GPUs Flexible size up to 32 x 32 One warp per system Inversion in registers Communication via shfl() Implicit pivoting: accumulate row/col swaps realize them when writing results Advantages over LU-based approach: No operations on triangular systems Implicit pivoting Implementation can use static system size 9
10 NVIDIA Pascal architecture (P100) NVIDIA whitepaper: 56 SMX, 5.3 TF DP, 768 GB/s ( 6 : 1 ) 2n 3 operations for inversion Flexible-size inversion up to size 32x32, one warp per system. 600 BGJE 600 BGJE GFlop/s GFlop/s Batch size Matrix size Batched Gauss-Jordan Elimination (BGJE) achieves >10% of DP peak! Anzt et al. Batched Gauss-Jordan Elimination for Block-Jacobi Preconditioner Generation on GPUs. PMAM'17, 8th International Workshop on Programming Models and Applications for Multicores and Manycores, pp 1-10,
11 Block-Jacobi preconditioner generation 1. extract diagonal block from sparse data structure invert diagonal block via Gauss-Jordan elimination insert solution as diagonal block into the preconditioner matrix 11
12 Challenge: diagonal block extraction from sparse data structures 1. extract diagonal block from sparse data structure invert diagonal block via Gauss-Jordan elimination 2. Matrix in CSR format Diagonal block extraction costly Assign one thread per row Unbalanced nonzero distribution results in load imbalance Dense rows become bottleneck Non-coalescent memory access 3. insert solution as diagonal block into the preconditioner matrix 12
13 Challenge: diagonal block extraction from sparse data structures memory requests cached extraction Matrix in CSR format 1 Diagonal block extraction costly Assign one thread per row 2 Unbalanced nonzero distribution results in load imbalance Dense rows become bottleneck 3 Non-coalescent memory access 4 5 row-ptr col-indices r 13
14 Challenge: diagonal block extraction from sparse data structures memory requests row-ptr cached extraction col-indices row-ptr shared memory extraction col-indices shared_mem Matrix in CSR format Diagonal block extraction costly Assign one thread per row Unbalanced nonzero distribution results in load imbalance Dense rows become bottleneck Non-coalescent memory access Add intermediate step: Extract diag. blocks to shared mem. Multiple threads for each row Coalescent access to data in CSR Col-major storage 14
15 Challenge: diagonal block extraction from sparse data structures memory requests row-ptr cached extraction col-indices row-ptr shared memory extraction col-indices shared_mem Matrix in CSR format Diagonal block extraction costly Assign one thread per row Unbalanced nonzero distribution results in load imbalance Dense rows become bottleneck Non-coalescent memory access Add intermediate step: Extract diag. blocks to shared mem. Multiple threads for each row Coalescent access to data in CSR Col-major storage 15
16 Challenge: diagonal block extraction from sparse data structures memory requests row-ptr cached extraction col-indices row-ptr shared memory extraction col-indices shared_mem Matrix in CSR format Diagonal block extraction costly Assign one thread per row Unbalanced nonzero distribution results in load imbalance Dense rows become bottleneck Non-coalescent memory access Add intermediate step: Extract diag. blocks to shared mem. Multiple threads for each row Coalescent access to data in CSR Col-major storage Merge kernels vs. 3 separate kernels: [ extraction + inversion + insertion ] [ extraction ] + [ inversion ] + [ insertion ] 16
17 Diagonal block extraction from sparse data structures US: unmerged + shared extraction UC: unmerged + cached extraction MS: merged + shared extraction MC: merged + cached extraction Results show: Runtime block-jacobi vs. BGJE 56 test matrices (SuiteSparse) Data arranged w.r.t. performance of (avg.) best strategy BJ generation time / BGJE time BJ generation time / BGJE time US UC MS MC Block K40 size Test matrices US UC MS MC Block size Test matrices 17
18 Diagonal block extraction from sparse data structures US: unmerged + shared extraction UC: unmerged + cached extraction MS: merged + shared extraction MC: merged + cached extraction Tridiagonal-type matrix structure: - balanced nnz/row Arrow-type matrix structure: - few rows with many elements - corresponding threads are bottleneck for cached extraction BJ generation time / BGJE time BJ generation time / BGJE time US UC MS MC Matrix size US UC MS MC Tridiagonal-type Arrow-type Matrix size
19 Block-Jacobi preconditioner setup cost Block Jacobi generation time [s] Block size 4 Block size 8 Block size 16 Block size 24 Block size 32 Chebyshev2 dw2048 dw1024 LeGresley_2508 Chebyshev3 saylr4 sts4098 olm5000 nasa2910 s2rmq4m1 s2rmt3m1 s3rmt3m1 s3rmt3m3 s3rmq4m1 s1rmt3m1 dw8192 dw4096 Kuu bcsstk38 CurlCurl_0 linverse nemeth15 bcsstk18 piston bcsstk17 nd3k cbuckle Pres_Poisson msc10848 ABACUS_shell_ud nd6k Preconditioner setup needs up to 0.01 s on P100 GPU. Irrelevant for overall Iterative solver execution time. spmsrtls cz40948 gridgena ibm_matrix_2 sme3dc 3D_51448_3D 2D_54019_highK nd12k gas_sensor rail_79841 crankseg_1 matrix_9 dc3 matrix-new_3 nd24k ship_003 CurlCurl_1 F1 af_shell3 ecology2 Emilia_923 G3_circuit Hook_1498 ML_Geer Flan_1565 K40
20 Block-Jacobi preconditioner Can reflect the block-structure of sparse matrices (e.g. Higher-Order FEM) Often superior to scalar Jacobi preconditioner Light-weight preconditioner (compared to ILU) Parallel preconditioner application ( batched trsv / SpMV + vector scaling ) Inversion of diag. blocks in prec. setup + SpMV in prec. application Factorization of diag. blocks in prec. setup + trsv in prec. application 20
21 Factorization-based block-jacobi Preconditioner setup Preconditioner application 2. Generate factorization and pivoting. Apply trsv using pivoting information to turn the right-hand side sub-vector into the solution of the diagonal-block linear problem. 1. extract diagonal block from sparse data structure. 3. Insert factorization as block into the preconditioner matrix and store pivoting. *ipiv 21
22 Factorization-based block-jacobi Batched routines: Small-Size LU, Gauss-Huard, Gauss-Huard-T. Gauss-Huard-T writes the factors in access-friendly fashion. Flexible size up to limit of 32 x 32. Use one warp per system, factorization in registers. GFlop/s Small-Size LU Gauss-Huard Gauss-Huard-T cublas LU dgetrf GFlop/s Small-Size LU Gauss-Huard Gauss-Huard-T cublas LU dgetri Matrix size Matrix size NVIDIA P100 GPU 22
23 NVIDIA Pascal architecture (P100) Gauss-Jordan elimination (BGJE) MAGMA dgetrf-inv (BLU) Gauss-Huard (BGH) Gauss-Huard + transp (BGHT) Time/matrix [ms] Batch size 10 4 BGJE and BLU invert diagonal blocks vs. BGH / BGHT factorizing only. BGHT writes factors in non-coalescent fashion for faster preconditioner application. NVIDIA P100 GPU
24 Inversion-based block-jacobi vs. factorization-based block-jacobi Convergence of BiCGSTAB with block-jacobi preconditioner for 300 matrices Test cases block size 4 block size 8 block size 16 block size 24 block size BGH needs more iterations BGJE needs more iterations Explicit inversion better. Factorization better. NVIDIA P100 GPU 24
25 25 Block-Jacobi preconditioning: Performance goal Runtime (BGJE) [s] Runtime (BGH) [s] Test matrices 10-3 solve setup solve setup Test matrices Execution time of BiCGSTAB with Jacobi(24) preconditioner Runtime [s] BGJE BGH BGHT Test matrices Almost no difference in total execution time. BGH better if setup is significant to iterative solver time. BGJE better for long solver runs. NVIDIA P100 GPU
26 Pitch from the sparse community: Small size (<30) somewhat flexible Add inversion routine to Batched-Blas Go for Gauss-Jordan Elimination instead of LU This research is based on a cooperation between Hartwig Anzt, Jack Dongarra (University of Tennessee), Goran Flegar and Enrique Quintana-Orti (University of Jaume I), and partly funded by the Department of Energy. 26
Variable-Size Batched Gauss-Huard for Block-Jacobi Preconditioning
Available online at www.sciencedirect.com ScienceDirect Procedia Computer Science 18C (217) 1783 1792 International Conference on Computational Science, ICCS 217, 12-14 June 217, Zurich, Switzerland Variable-Size
More informationFlexible Batched Sparse Matrix-Vector Product on GPUs
ScalA'17: 8th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems November 13, 217 Flexible Batched Sparse Matrix-Vector Product on GPUs Hartwig Anzt, Gary Collins, Jack Dongarra,
More informationIterative Sparse Triangular Solves for Preconditioning
Euro-Par 2015, Vienna Aug 24-28, 2015 Iterative Sparse Triangular Solves for Preconditioning Hartwig Anzt, Edmond Chow and Jack Dongarra Incomplete Factorization Preconditioning Incomplete LU factorizations
More informationAccelerating the Conjugate Gradient Algorithm with GPUs in CFD Simulations
Accelerating the Conjugate Gradient Algorithm with GPUs in CFD Simulations Hartwig Anzt 1, Marc Baboulin 2, Jack Dongarra 1, Yvan Fournier 3, Frank Hulsemann 3, Amal Khabou 2, and Yushan Wang 2 1 University
More informationGTC 2013: DEVELOPMENTS IN GPU-ACCELERATED SPARSE LINEAR ALGEBRA ALGORITHMS. Kyle Spagnoli. Research EM Photonics 3/20/2013
GTC 2013: DEVELOPMENTS IN GPU-ACCELERATED SPARSE LINEAR ALGEBRA ALGORITHMS Kyle Spagnoli Research Engineer @ EM Photonics 3/20/2013 INTRODUCTION» Sparse systems» Iterative solvers» High level benchmarks»
More informationUsing Jacobi Iterations and Blocking for Solving Sparse Triangular Systems in Incomplete Factorization Preconditioning
Using Jacobi Iterations and Blocking for Solving Sparse Triangular Systems in Incomplete Factorization Preconditioning Edmond Chow a,, Hartwig Anzt b,c, Jennifer Scott d, Jack Dongarra c,e,f a School of
More informationAnalysis and Optimization of Power Consumption in the Iterative Solution of Sparse Linear Systems on Multi-core and Many-core Platforms
Analysis and Optimization of Power Consumption in the Iterative Solution of Sparse Linear Systems on Multi-core and Many-core Platforms H. Anzt, V. Heuveline Karlsruhe Institute of Technology, Germany
More informationExploiting GPU Caches in Sparse Matrix Vector Multiplication. Yusuke Nagasaka Tokyo Institute of Technology
Exploiting GPU Caches in Sparse Matrix Vector Multiplication Yusuke Nagasaka Tokyo Institute of Technology Sparse Matrix Generated by FEM, being as the graph data Often require solving sparse linear equation
More informationAdministrative Issues. L11: Sparse Linear Algebra on GPUs. Triangular Solve (STRSM) A Few Details 2/25/11. Next assignment, triangular solve
Administrative Issues L11: Sparse Linear Algebra on GPUs Next assignment, triangular solve Due 5PM, Tuesday, March 15 handin cs6963 lab 3 Project proposals Due 5PM, Wednesday, March 7 (hard
More informationHYPERDRIVE IMPLEMENTATION AND ANALYSIS OF A PARALLEL, CONJUGATE GRADIENT LINEAR SOLVER PROF. BRYANT PROF. KAYVON 15618: PARALLEL COMPUTER ARCHITECTURE
HYPERDRIVE IMPLEMENTATION AND ANALYSIS OF A PARALLEL, CONJUGATE GRADIENT LINEAR SOLVER AVISHA DHISLE PRERIT RODNEY ADHISLE PRODNEY 15618: PARALLEL COMPUTER ARCHITECTURE PROF. BRYANT PROF. KAYVON LET S
More informationIterative Sparse Triangular Solves for Preconditioning
Iterative Sparse Triangular Solves for Preconditioning Hartwig Anzt 1(B), Edmond Chow 2, and Jack Dongarra 1 1 University of Tennessee, Knoxville, TN, USA hanzt@icl.utk.edu, dongarra@eecs.utk.edu 2 Georgia
More informationLeveraging Matrix Block Structure In Sparse Matrix-Vector Multiplication. Steve Rennich Nvidia Developer Technology - Compute
Leveraging Matrix Block Structure In Sparse Matrix-Vector Multiplication Steve Rennich Nvidia Developer Technology - Compute Block Sparse Matrix Vector Multiplication Sparse Matrix-Vector Multiplication
More informationUsing Jacobi Iterations and Blocking for Solving Sparse Triangular Systems in Incomplete Factorization Preconditioning
Using Jacobi Iterations and Blocking for Solving Sparse Triangular Systems in Incomplete Factorization Preconditioning Edmond Chow a,, Hartwig Anzt b,c, Jennifer Scott d, Jack Dongarra c,e,f a School of
More informationOn Level Scheduling for Incomplete LU Factorization Preconditioners on Accelerators
On Level Scheduling for Incomplete LU Factorization Preconditioners on Accelerators Karl Rupp, Barry Smith rupp@mcs.anl.gov Mathematics and Computer Science Division Argonne National Laboratory FEMTEC
More informationOn the Parallel Solution of Sparse Triangular Linear Systems. M. Naumov* San Jose, CA May 16, 2012 *NVIDIA
On the Parallel Solution of Sparse Triangular Linear Systems M. Naumov* San Jose, CA May 16, 2012 *NVIDIA Why Is This Interesting? There exist different classes of parallel problems Embarrassingly parallel
More informationStudy and implementation of computational methods for Differential Equations in heterogeneous systems. Asimina Vouronikoy - Eleni Zisiou
Study and implementation of computational methods for Differential Equations in heterogeneous systems Asimina Vouronikoy - Eleni Zisiou Outline Introduction Review of related work Cyclic Reduction Algorithm
More informationHartwig Anzt, Edmond Chow, Daniel B. Szyld, and Jack Dongarra. Report Novermber Revised February 2016
Domain Overlap for Iterative Sparse Triangular Solves on GPUs Hartwig Anzt, Edmond Chow, Daniel B. Szyld, and Jack Dongarra Report 15-11-24 Novermber 2015. Revised February 2016 Department of Mathematics
More informationMAGMA a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures
MAGMA a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures Stan Tomov Innovative Computing Laboratory University of Tennessee, Knoxville OLCF Seminar Series, ORNL June 16, 2010
More informationHartwig Anzt, Edmond Chow, Daniel Szyld, and Jack Dongarra. Report Novermber 2015
Domain Overlap for Iterative Sparse Triangular Solves on GPUs Hartwig Anzt, Edmond Chow, Daniel Szyld, and Jack Dongarra Report 15-11-24 Novermber 2015 Department of Mathematics Temple University Philadelphia,
More informationGPU-based Parallel Reservoir Simulators
GPU-based Parallel Reservoir Simulators Zhangxin Chen 1, Hui Liu 1, Song Yu 1, Ben Hsieh 1 and Lei Shao 1 Key words: GPU computing, reservoir simulation, linear solver, parallel 1 Introduction Nowadays
More informationAdaptive precision in block-jacobi preconditioning for iterative sparse linear system solvers
Received: 19 September 2017 Revised: 29 January 2018 Accepted: 29 January 2018 DOI: 10.1002/cpe.4460 SPECIAL ISSUE PAPER Adaptive precision in block-jacobi preconditioning for iterative sparse linear system
More informationScheduling Strategies for Parallel Sparse Backward/Forward Substitution
Scheduling Strategies for Parallel Sparse Backward/Forward Substitution J.I. Aliaga M. Bollhöfer A.F. Martín E.S. Quintana-Ortí Deparment of Computer Science and Engineering, Univ. Jaume I (Spain) {aliaga,martina,quintana}@icc.uji.es
More informationEfficient Finite Element Geometric Multigrid Solvers for Unstructured Grids on GPUs
Efficient Finite Element Geometric Multigrid Solvers for Unstructured Grids on GPUs Markus Geveler, Dirk Ribbrock, Dominik Göddeke, Peter Zajac, Stefan Turek Institut für Angewandte Mathematik TU Dortmund,
More informationMixed Precision Methods
Mixed Precision Methods Mixed precision, use the lowest precision required to achieve a given accuracy outcome " Improves runtime, reduce power consumption, lower data movement " Reformulate to find correction
More informationMAGMA. Matrix Algebra on GPU and Multicore Architectures
MAGMA Matrix Algebra on GPU and Multicore Architectures Innovative Computing Laboratory Electrical Engineering and Computer Science University of Tennessee Piotr Luszczek (presenter) web.eecs.utk.edu/~luszczek/conf/
More informationCUDA Accelerated Compute Libraries. M. Naumov
CUDA Accelerated Compute Libraries M. Naumov Outline Motivation Why should you use libraries? CUDA Toolkit Libraries Overview of performance CUDA Proprietary Libraries Address specific markets Third Party
More informationA Standard for Batching BLAS Operations
A Standard for Batching BLAS Operations Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester 5/8/16 1 API for Batching BLAS Operations We are proposing, as a community
More informationTowards a complete FEM-based simulation toolkit on GPUs: Geometric Multigrid solvers
Towards a complete FEM-based simulation toolkit on GPUs: Geometric Multigrid solvers Markus Geveler, Dirk Ribbrock, Dominik Göddeke, Peter Zajac, Stefan Turek Institut für Angewandte Mathematik TU Dortmund,
More informationWhitepaper: Software-Defined Events (SDEs) in MAGMA-Sparse
Whitepaper: Software-Defined Events (SDEs) in MAGMA-Sparse ECP PEEKS: Production-ready, Exascale-Enabled, Krylov Solvers for Exascale Computing Heike Jagode Anthony Danalis Hartwig Anzt Ichitaro Yamazaki
More informationGPU Implementation of Elliptic Solvers in NWP. Numerical Weather- and Climate- Prediction
1/8 GPU Implementation of Elliptic Solvers in Numerical Weather- and Climate- Prediction Eike Hermann Müller, Robert Scheichl Department of Mathematical Sciences EHM, Xu Guo, Sinan Shi and RS: http://arxiv.org/abs/1302.7193
More informationAutomatic Development of Linear Algebra Libraries for the Tesla Series
Automatic Development of Linear Algebra Libraries for the Tesla Series Enrique S. Quintana-Ortí quintana@icc.uji.es Universidad Jaime I de Castellón (Spain) Dense Linear Algebra Major problems: Source
More informationS0432 NEW IDEAS FOR MASSIVELY PARALLEL PRECONDITIONERS
S0432 NEW IDEAS FOR MASSIVELY PARALLEL PRECONDITIONERS John R Appleyard Jeremy D Appleyard Polyhedron Software with acknowledgements to Mark A Wakefield Garf Bowen Schlumberger Outline of Talk Reservoir
More informationAccelerated ANSYS Fluent: Algebraic Multigrid on a GPU. Robert Strzodka NVAMG Project Lead
Accelerated ANSYS Fluent: Algebraic Multigrid on a GPU Robert Strzodka NVAMG Project Lead A Parallel Success Story in Five Steps 2 Step 1: Understand Application ANSYS Fluent Computational Fluid Dynamics
More informationMAGMA: a New Generation
1.3 MAGMA: a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures Jack Dongarra T. Dong, M. Gates, A. Haidar, S. Tomov, and I. Yamazaki University of Tennessee, Knoxville Release
More informationBatch Linear Algebra for GPU-Accelerated High Performance Computing Environments
Batch Linear Algebra for GPU-Accelerated High Performance Computing Environments Ahmad Abdelfattah, Azzam Haidar, Stanimire Tomov, and Jack Dongarra SIAM Conference on Computational Science and Engineering
More informationFrequency Scaling and Energy Efficiency regarding the Gauss-Jordan Elimination Scheme on OpenPower 8
Frequency Scaling and Energy Efficiency regarding the Gauss-Jordan Elimination Scheme on OpenPower 8 Martin Köhler Jens Saak 2 The Gauss-Jordan Elimination scheme is an alternative to the LU decomposition
More informationMatrix-free multi-gpu Implementation of Elliptic Solvers for strongly anisotropic PDEs
Iterative Solvers Numerical Results Conclusion and outlook 1/18 Matrix-free multi-gpu Implementation of Elliptic Solvers for strongly anisotropic PDEs Eike Hermann Müller, Robert Scheichl, Eero Vainikko
More informationHow to perform HPL on CPU&GPU clusters. Dr.sc. Draško Tomić
How to perform HPL on CPU&GPU clusters Dr.sc. Draško Tomić email: drasko.tomic@hp.com Forecasting is not so easy, HPL benchmarking could be even more difficult Agenda TOP500 GPU trends Some basics about
More informationHarnessing CUDA Dynamic Parallelism for the Solution of Sparse Linear Systems
Harnessing CUDA Dynamic Parallelism for the Solution of Sparse Linear Systems José ALIAGA, a,1 Davor DAVIDOVIĆ b, Joaquín PÉREZ a, and Enrique S. QUINTANA-ORTÍ a, a Dpto. Ingeniería Ciencia de Computadores,
More informationA GPU Sparse Direct Solver for AX=B
1 / 25 A GPU Sparse Direct Solver for AX=B Jonathan Hogg, Evgueni Ovtchinnikov, Jennifer Scott* STFC Rutherford Appleton Laboratory 26 March 2014 GPU Technology Conference San Jose, California * Thanks
More informationGPU-Accelerated Algebraic Multigrid for Commercial Applications. Joe Eaton, Ph.D. Manager, NVAMG CUDA Library NVIDIA
GPU-Accelerated Algebraic Multigrid for Commercial Applications Joe Eaton, Ph.D. Manager, NVAMG CUDA Library NVIDIA ANSYS Fluent 2 Fluent control flow Accelerate this first Non-linear iterations Assemble
More informationPorting the NAS-NPB Conjugate Gradient Benchmark to CUDA. NVIDIA Corporation
Porting the NAS-NPB Conjugate Gradient Benchmark to CUDA NVIDIA Corporation Outline! Overview of CG benchmark! Overview of CUDA Libraries! CUSPARSE! CUBLAS! Porting Sequence! Algorithm Analysis! Data/Code
More informationAccelerating GPU kernels for dense linear algebra
Accelerating GPU kernels for dense linear algebra Rajib Nath, Stanimire Tomov, and Jack Dongarra Department of Electrical Engineering and Computer Science, University of Tennessee, Knoxville {rnath1, tomov,
More informationMAGMA Library. version 0.1. S. Tomov J. Dongarra V. Volkov J. Demmel
MAGMA Library version 0.1 S. Tomov J. Dongarra V. Volkov J. Demmel 2 -- MAGMA (version 0.1) -- Univ. of Tennessee, Knoxville Univ. of California, Berkeley Univ. of Colorado, Denver June 2009 MAGMA project
More informationSolving Dense Linear Systems on Graphics Processors
Solving Dense Linear Systems on Graphics Processors Sergio Barrachina Maribel Castillo Francisco Igual Rafael Mayo Enrique S. Quintana-Ortí High Performance Computing & Architectures Group Universidad
More informationNEW ADVANCES IN GPU LINEAR ALGEBRA
GTC 2012: NEW ADVANCES IN GPU LINEAR ALGEBRA Kyle Spagnoli EM Photonics 5/16/2012 QUICK ABOUT US» HPC/GPU Consulting Firm» Specializations in:» Electromagnetics» Image Processing» Fluid Dynamics» Linear
More informationESPRESO ExaScale PaRallel FETI Solver. Hybrid FETI Solver Report
ESPRESO ExaScale PaRallel FETI Solver Hybrid FETI Solver Report Lubomir Riha, Tomas Brzobohaty IT4Innovations Outline HFETI theory from FETI to HFETI communication hiding and avoiding techniques our new
More informationPerformance Strategies for Parallel Mathematical Libraries Based on Historical Knowledgebase
Performance Strategies for Parallel Mathematical Libraries Based on Historical Knowledgebase CScADS workshop 29 Eduardo Cesar, Anna Morajko, Ihab Salawdeh Universitat Autònoma de Barcelona Objective Mathematical
More informationPARDISO Version Reference Sheet Fortran
PARDISO Version 5.0.0 1 Reference Sheet Fortran CALL PARDISO(PT, MAXFCT, MNUM, MTYPE, PHASE, N, A, IA, JA, 1 PERM, NRHS, IPARM, MSGLVL, B, X, ERROR, DPARM) 1 Please note that this version differs significantly
More informationTechnical Report Performance Analysis of CULA on different NVIDIA GPU Architectures. Prateek Gupta
Technical Report 2014-02 Performance Analysis of CULA on different NVIDIA GPU Architectures Prateek Gupta May 20, 2014 1 Spring 2014: Performance Analysis of CULA on different NVIDIA GPU Architectures
More informationLarge Displacement Optical Flow & Applications
Large Displacement Optical Flow & Applications Narayanan Sundaram, Kurt Keutzer (Parlab) In collaboration with Thomas Brox (University of Freiburg) Michael Tao (University of California Berkeley) Parlab
More informationImplicit Low-Order Unstructured Finite-Element Multiple Simulation Enhanced by Dense Computation using OpenACC
Fourth Workshop on Accelerator Programming Using Directives (WACCPD), Nov. 13, 2017 Implicit Low-Order Unstructured Finite-Element Multiple Simulation Enhanced by Dense Computation using OpenACC Takuma
More informationEfficient Multi-GPU CUDA Linear Solvers for OpenFOAM
Efficient Multi-GPU CUDA Linear Solvers for OpenFOAM Alexander Monakov, amonakov@ispras.ru Institute for System Programming of Russian Academy of Sciences March 20, 2013 1 / 17 Problem Statement In OpenFOAM,
More informationHighly Parallel Multigrid Solvers for Multicore and Manycore Processors
Highly Parallel Multigrid Solvers for Multicore and Manycore Processors Oleg Bessonov (B) Institute for Problems in Mechanics of the Russian Academy of Sciences, 101, Vernadsky Avenue, 119526 Moscow, Russia
More informationMulti-GPU simulations in OpenFOAM with SpeedIT technology.
Multi-GPU simulations in OpenFOAM with SpeedIT technology. Attempt I: SpeedIT GPU-based library of iterative solvers for Sparse Linear Algebra and CFD. Current version: 2.2. Version 1.0 in 2008. CMRS format
More informationAccelerating GPU Kernels for Dense Linear Algebra
Accelerating GPU Kernels for Dense Linear Algebra Rajib Nath, Stan Tomov, and Jack Dongarra Innovative Computing Lab University of Tennessee, Knoxville July 9, 21 xgemm performance of CUBLAS-2.3 on GTX28
More informationPerformance Analysis of BLAS Libraries in SuperLU_DIST for SuperLU_MCDT (Multi Core Distributed) Development
Available online at www.prace-ri.eu Partnership for Advanced Computing in Europe Performance Analysis of BLAS Libraries in SuperLU_DIST for SuperLU_MCDT (Multi Core Distributed) Development M. Serdar Celebi
More informationOptimizing GPU Kernels for Irregular Batch Workloads: A Case Study for Cholesky Factorization
Optimizing GPU Kernels for Irregular Batch Workloads: A Case Study for Cholesky Factorization Ahmad Abdelfattah, Azzam Haidar, Stanimire Tomov, and Jack Dongarra Innovative Computing Laboratory University
More informationUnveiling the performance-energy trade-off in iterative linear system solvers for multithreaded processors
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. (2014) Published online in Wiley Online Library (wileyonlinelibrary.com)..3341 SPECIAL ISSUE PAPER Unveiling the
More informationFigure 6.1: Truss topology optimization diagram.
6 Implementation 6.1 Outline This chapter shows the implementation details to optimize the truss, obtained in the ground structure approach, according to the formulation presented in previous chapters.
More informationCSE 599 I Accelerated Computing - Programming GPUS. Parallel Pattern: Sparse Matrices
CSE 599 I Accelerated Computing - Programming GPUS Parallel Pattern: Sparse Matrices Objective Learn about various sparse matrix representations Consider how input data affects run-time performance of
More informationEnhanced Oil Recovery simulation Performances on New Hybrid Architectures
Renewable energies Eco-friendly production Innovative transport Eco-efficient processes Sustainable resources Enhanced Oil Recovery simulation Performances on New Hybrid Architectures A. Anciaux, J-M.
More informationOptimizing the SVD Bidiagonalization Process for a Batch of Small Matrices
This space is reserved for the Procedia header, do not use it Optimizing the SVD Bidiagonalization Process for a Batch of Small Matrices Tingxing Dong 1, Azzam Haidar 2, Stanimire Tomov 2, and Jack Dongarra
More informationACCELERATING CFD AND RESERVOIR SIMULATIONS WITH ALGEBRAIC MULTI GRID Chris Gottbrath, Nov 2016
ACCELERATING CFD AND RESERVOIR SIMULATIONS WITH ALGEBRAIC MULTI GRID Chris Gottbrath, Nov 2016 Challenges What is Algebraic Multi-Grid (AMG)? AGENDA Why use AMG? When to use AMG? NVIDIA AmgX Results 2
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 20: Sparse Linear Systems; Direct Methods vs. Iterative Methods Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 26
More informationS4289: Efficient solution of multiple scalar and block-tridiagonal equations
S4289: Efficient solution of multiple scalar and block-tridiagonal equations Endre László endre.laszlo [at] oerc.ox.ac.uk Oxford e-research Centre, University of Oxford, UK Pázmány Péter Catholic University,
More informationContents. F10: Parallel Sparse Matrix Computations. Parallel algorithms for sparse systems Ax = b. Discretized domain a metal sheet
Contents 2 F10: Parallel Sparse Matrix Computations Figures mainly from Kumar et. al. Introduction to Parallel Computing, 1st ed Chap. 11 Bo Kågström et al (RG, EE, MR) 2011-05-10 Sparse matrices and storage
More informationIntel MKL Sparse Solvers. Software Solutions Group - Developer Products Division
Intel MKL Sparse Solvers - Agenda Overview Direct Solvers Introduction PARDISO: main features PARDISO: advanced functionality DSS Performance data Iterative Solvers Performance Data Reference Copyright
More informationOutline. Parallel Algorithms for Linear Algebra. Number of Processors and Problem Size. Speedup and Efficiency
1 2 Parallel Algorithms for Linear Algebra Richard P. Brent Computer Sciences Laboratory Australian National University Outline Basic concepts Parallel architectures Practical design issues Programming
More informationDense Linear Algebra on Heterogeneous Platforms: State of the Art and Trends
Dense Linear Algebra on Heterogeneous Platforms: State of the Art and Trends Paolo Bientinesi AICES, RWTH Aachen pauldj@aices.rwth-aachen.de ComplexHPC Spring School 2013 Heterogeneous computing - Impact
More informationDAG-Scheduled Linear Algebra Using Template-Based Building Blocks
DAG-Scheduled Linear Algebra Using Template-Based Building Blocks Jonathan Hogg STFC Rutherford Appleton Laboratory 1 / 20 19 March 2015 GPU Technology Conference San Jose, California * Thanks also to
More informationTomonori Kouya Shizuoka Institute of Science and Technology Toyosawa, Fukuroi, Shizuoka Japan. October 5, 2018
arxiv:1411.2377v1 [math.na] 10 Nov 2014 A Highly Efficient Implementation of Multiple Precision Sparse Matrix-Vector Multiplication and Its Application to Product-type Krylov Subspace Methods Tomonori
More informationA MATLAB Interface to the GPU
A MATLAB Interface to the GPU Second Winter School Geilo, Norway André Rigland Brodtkorb SINTEF ICT Department of Applied Mathematics 2007-01-24 Outline 1 Motivation and previous
More informationAccelerating the Iterative Linear Solver for Reservoir Simulation
Accelerating the Iterative Linear Solver for Reservoir Simulation Wei Wu 1, Xiang Li 2, Lei He 1, Dongxiao Zhang 2 1 Electrical Engineering Department, UCLA 2 Department of Energy and Resources Engineering,
More informationSolution of the Transport Equation Using Graphical Processing Units
Solution of the Transport Equation Using Graphical Processing Units Gil Gonçalves Brandão October - 2009 1 Introduction Computational Fluid Dynamics (CFD) always have struggled for faster computing resources
More informationEfficient multigrid solvers for strongly anisotropic PDEs in atmospheric modelling
Iterative Solvers Numerical Results Conclusion and outlook 1/22 Efficient multigrid solvers for strongly anisotropic PDEs in atmospheric modelling Part II: GPU Implementation and Scaling on Titan Eike
More informationEFFICIENT SOLVER FOR LINEAR ALGEBRAIC EQUATIONS ON PARALLEL ARCHITECTURE USING MPI
EFFICIENT SOLVER FOR LINEAR ALGEBRAIC EQUATIONS ON PARALLEL ARCHITECTURE USING MPI 1 Akshay N. Panajwar, 2 Prof.M.A.Shah Department of Computer Science and Engineering, Walchand College of Engineering,
More informationGPU ACCELERATION OF CHOLMOD: BATCHING, HYBRID AND MULTI-GPU
April 4-7, 2016 Silicon Valley GPU ACCELERATION OF CHOLMOD: BATCHING, HYBRID AND MULTI-GPU Steve Rennich, Darko Stosic, Tim Davis, April 6, 2016 OBJECTIVE Direct sparse methods are among the most widely
More informationGPU Acceleration of the Longwave Rapid Radiative Transfer Model in WRF using CUDA Fortran. G. Ruetsch, M. Fatica, E. Phillips, N.
GPU Acceleration of the Longwave Rapid Radiative Transfer Model in WRF using CUDA Fortran G. Ruetsch, M. Fatica, E. Phillips, N. Juffa Outline WRF and RRTM Previous Work CUDA Fortran Features RRTM in CUDA
More informationHydrodynamic Computation with Hybrid Programming on CPU-GPU Clusters
Hydrodynamic Computation with Hybrid Programming on CPU-GPU Clusters Tingxing Dong, Veselin Dobrev, Tzanio Kolev, Robert Rieben, Stanimire Tomov, Jack Dongarra Innovative Computing Laboratory, University
More informationGPU ACCELERATION OF WSMP (WATSON SPARSE MATRIX PACKAGE)
GPU ACCELERATION OF WSMP (WATSON SPARSE MATRIX PACKAGE) NATALIA GIMELSHEIN ANSHUL GUPTA STEVE RENNICH SEID KORIC NVIDIA IBM NVIDIA NCSA WATSON SPARSE MATRIX PACKAGE (WSMP) Cholesky, LDL T, LU factorization
More informationBlocked-Based Sparse Matrix-Vector Multiplication on Distributed Memory Parallel Computers
The International Arab Journal of Information Technology, Vol. 8, No., April Blocked-Based Sparse Matrix-Vector Multiplication on Distributed Memory Parallel Computers Rukhsana Shahnaz and Anila Usman
More informationPARALUTION - a Library for Iterative Sparse Methods on CPU and GPU
- a Library for Iterative Sparse Methods on CPU and GPU Dimitar Lukarski Division of Scientific Computing Department of Information Technology Uppsala Programming for Multicore Architectures Research Center
More informationMAGMA. LAPACK for GPUs. Stan Tomov Research Director Innovative Computing Laboratory Department of Computer Science University of Tennessee, Knoxville
MAGMA LAPACK for GPUs Stan Tomov Research Director Innovative Computing Laboratory Department of Computer Science University of Tennessee, Knoxville Keeneland GPU Tutorial 2011, Atlanta, GA April 14-15,
More informationLecture 15: More Iterative Ideas
Lecture 15: More Iterative Ideas David Bindel 15 Mar 2010 Logistics HW 2 due! Some notes on HW 2. Where we are / where we re going More iterative ideas. Intro to HW 3. More HW 2 notes See solution code!
More informationReport of Linear Solver Implementation on GPU
Report of Linear Solver Implementation on GPU XIANG LI Abstract As the development of technology and the linear equation solver is used in many aspects such as smart grid, aviation and chemical engineering,
More informationWHILE linear algebra software has achieved high efficiency
2036 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 27, NO. 7, JULY 2016 Implementation and Tuning of Batched Cholesky Factorization and Solve for NVIDIA GPUs Jakub Kurzak, Hartwig Anzt, Mark
More informationOptimizing Memory-Bound Numerical Kernels on GPU Hardware Accelerators
Optimizing Memory-Bound Numerical Kernels on GPU Hardware Accelerators Ahmad Abdelfattah 1, Jack Dongarra 2, David Keyes 1 and Hatem Ltaief 3 1 KAUST Division of Mathematical and Computer Sciences and
More informationLinear Equation Systems Iterative Methods
Linear Equation Systems Iterative Methods Content Iterative Methods Jacobi Iterative Method Gauss Seidel Iterative Method Iterative Methods Iterative methods are those that produce a sequence of successive
More informationDistributed Dense Linear Algebra on Heterogeneous Architectures. George Bosilca
Distributed Dense Linear Algebra on Heterogeneous Architectures George Bosilca bosilca@eecs.utk.edu Centraro, Italy June 2010 Factors that Necessitate to Redesign of Our Software» Steepness of the ascent
More informationApplication of GPU-Based Computing to Large Scale Finite Element Analysis of Three-Dimensional Structures
Paper 6 Civil-Comp Press, 2012 Proceedings of the Eighth International Conference on Engineering Computational Technology, B.H.V. Topping, (Editor), Civil-Comp Press, Stirlingshire, Scotland Application
More informationScheduling of QR Factorization Algorithms on SMP and Multi-core Architectures
Scheduling of Algorithms on SMP and Multi-core Architectures Gregorio Quintana-Ortí Enrique S. Quintana-Ortí Ernie Chan Robert A. van de Geijn Field G. Van Zee quintana@icc.uji.es Universidad Jaime I de
More informationExperiences with the Sparse Matrix-Vector Multiplication on a Many-core Processor
Experiences with the Sparse Matrix-Vector Multiplication on a Many-core Processor Juan C. Pichel Centro de Investigación en Tecnoloxías da Información (CITIUS) Universidade de Santiago de Compostela, Spain
More informationME964 High Performance Computing for Engineering Applications
ME964 High Performance Computing for Engineering Applications Outlining Midterm Projects Topic 3: GPU-based FEA Topic 4: GPU Direct Solver for Sparse Linear Algebra March 01, 2011 Dan Negrut, 2011 ME964
More informationPerformance engineering for hardware-, numericaland energy-efficiency in FEM frameworks: modeling and control
Performance engineering for hardware-, numericaland energy-efficiency in FEM frameworks: modeling and control Hardware-oriented Numerics in the FEAT software family Markus Geveler Higher-order DG methods
More informationEvaluation of sparse LU factorization and triangular solution on multicore architectures. X. Sherry Li
Evaluation of sparse LU factorization and triangular solution on multicore architectures X. Sherry Li Lawrence Berkeley National Laboratory ParLab, April 29, 28 Acknowledgement: John Shalf, LBNL Rich Vuduc,
More informationDistributed NVAMG. Design and Implementation of a Scalable Algebraic Multigrid Framework for a Cluster of GPUs
Distributed NVAMG Design and Implementation of a Scalable Algebraic Multigrid Framework for a Cluster of GPUs Istvan Reguly (istvan.reguly at oerc.ox.ac.uk) Oxford e-research Centre NVIDIA Summer Internship
More informationSolving the heat equation with CUDA
Solving the heat equation with CUDA Oliver Meister January 09 th 2013 Last Tutorial CSR kernel - scalar One row per thread No coalesced memory access Non-uniform matrices CSR kernel - vectorized One row
More informationLecture 17: More Fun With Sparse Matrices
Lecture 17: More Fun With Sparse Matrices David Bindel 26 Oct 2011 Logistics Thanks for info on final project ideas. HW 2 due Monday! Life lessons from HW 2? Where an error occurs may not be where you
More informationHPC with Multicore and GPUs
HPC with Multicore and GPUs Stan Tomov Electrical Engineering and Computer Science Department University of Tennessee, Knoxville COSC 594 Lecture Notes March 22, 2017 1/20 Outline Introduction - Hardware
More information