High Performance Linear Algebra
|
|
|
- Milo Shepherd
- 5 years ago
- Views:
Transcription
1 High Performance Linear Algebra Hatem Ltaief Senior Research Scientist Extreme Computing Research Center King Abdullah University of Science and Technology 4th International Workshop on Real-Time Control for Adaptive Optics Observatoire de Paris Dec H. Ltaief 1 / 37
2 Outline 1 Motivations 2 Real Scientific Applications 3 Cholesky-based Matrix Computations 4 The HiCMA Library 5 Algorithmic Perspectives H. Ltaief 2 / 37
3 Motivations Outline 1 Motivations 2 Real Scientific Applications 3 Cholesky-based Matrix Computations 4 The HiCMA Library 5 Algorithmic Perspectives H. Ltaief 3 / 37
4 Motivations Hardware Landscape: The TOP500 List Ranks the 500 fastest computers in the world (twice a year) High Performance LINPACK benchmark (HPL) Solves a dense system of linear equations Ax = b using LU Compute intensive benchmark (Matrix-Matrix multiplication) Performance reported in Xflop/s: rate of execution, i.e., number of floating point operations per second H. Ltaief 4 / 37
5 Motivations The Top500 List, November 16: Rank Name Vendor Cores R max (Pflop/s) Power (MW) 1 TaihuLight NRCPC Sunway SW ,649, ,3 2 Tianhe-2 NUDT Intel Xeon E5 3,120,000 33,8 17,8 3 Titan Cray AMD Opteron + NVIDIA K20x 560,640 17,6 8,21 4 Sequoia IBM BG/Q 1,572,864 17,2 7,89 5 Cori Cray Intel Xeon Phi 622,336 14,01 3,93 6 Oakforest Fujitsu Intel Xeon Phi 556,104 13,55 2,72 7 K Computer Fujitsu SPARC64 705,024 10,51 12,66 8 PizDaint Cray Intel Xeon E5 + NVIDIA P ,720 9,78 1,31 9 Mira IBM BG/Q 786,432 8,59 3,95 10 Trinity Cray Intel Xeon E5 301,056 8,1 4,23 H. Ltaief 5 / 37
6 Motivations The Top500 List, November 16: Rank Name Vendor Cores R max (Pflop/s) Power (MW) 1 TaihuLight NRCPC Sunway SW ,649, ,3 2 Tianhe-2 NUDT Intel Xeon E5 3,120,000 33,8 17,8 3 Titan Cray AMD Opteron + NVIDIA K20x 560,640 17,6 8,21 4 Sequoia IBM BG/Q 1,572,864 17,2 7,89 5 Cori Cray Intel Xeon Phi 622,336 14,01 3,93 6 Oakforest Fujitsu Intel Xeon Phi 556,104 13,55 2,72 7 K Computer Fujitsu SPARC64 705,024 10,51 12,66 8 PizDaint Cray Intel Xeon E5 + NVIDIA P ,720 9,78 1,31 9 Mira IBM BG/Q 786,432 8,59 3,95 10 Trinity Cray Intel Xeon E5 301,056 8,1 4, Shaheen 2 Cray Intel Xeon E5 196,608 5,53 2,83 H. Ltaief 5 / 37
7 Motivations The Top500 List, November 16: Rank Name Vendor Cores R max (Pflop/s) Power (MW) 1 TaihuLight NRCPC Sunway SW ,649, ,3 2 Tianhe-2 NUDT Intel Xeon E5 3,120,000 33,8 17,8 3 Titan Cray AMD Opteron + NVIDIA K20x 560,640 17,6 8,21 4 Sequoia IBM BG/Q 1,572,864 17,2 7,89 5 Cori Cray Intel Xeon Phi 622,336 14,01 3,93 6 Oakforest Fujitsu Intel Xeon Phi 556,104 13,55 2,72 7 K Computer Fujitsu SPARC64 705,024 10,51 12,66 8 PizDaint Cray Intel Xeon E5 + NVIDIA P ,720 9,78 1,31 9 Mira IBM BG/Q 786,432 8,59 3,95 10 Trinity Cray Intel Xeon E5 301,056 8,1 4, Shaheen 2 Cray Intel Xeon E5 196,608 5,53 2,83 Ever increasing hardware complexity H. Ltaief 5 / 37
8 Motivations The Top500 List, November 16: Rank Name Vendor Cores R max (Pflop/s) Power (MW) 1 TaihuLight NRCPC Sunway SW ,649, ,3 2 Tianhe-2 NUDT Intel Xeon E5 3,120,000 33,8 17,8 3 Titan Cray AMD Opteron + NVIDIA K20x 560,640 17,6 8,21 4 Sequoia IBM BG/Q 1,572,864 17,2 7,89 5 Cori Cray Intel Xeon Phi 622,336 14,01 3,93 6 Oakforest Fujitsu Intel Xeon Phi 556,104 13,55 2,72 7 K Computer Fujitsu SPARC64 705,024 10,51 12,66 8 PizDaint Cray Intel Xeon E5 + NVIDIA P ,720 9,78 1,31 9 Mira IBM BG/Q 786,432 8,59 3,95 10 Trinity Cray Intel Xeon E5 301,056 8,1 4, Shaheen 2 Cray Intel Xeon E5 196,608 5,53 2,83 Human brain: 20 PetaFLOPS! (cf Kurzweil) H. Ltaief 6 / 37
9 Motivations The Never Ending Race... c/o J. Dongarra H. Ltaief 7 / 37
10 Motivations Just in case you are wondering whats beyond ExaFlops... H. Ltaief 8 / 37
11 Motivations Recipe for High Performance Linear Algebra H. Ltaief 9 / 37
12 Real Scientific Applications Outline 1 Motivations 2 Real Scientific Applications 3 Cholesky-based Matrix Computations 4 The HiCMA Library 5 Algorithmic Perspectives H. Ltaief 10 / 37
13 Real Scientific Applications Geospatial Statistics Arising from multivariate large spatial data sets in climate/weather modeling to improve prediction for temperature, wind speeds, dust storm, etc. (a) Problem Definition. (b) Temperature prediction. Figure: Modeling climate/weather forecasting. H. Ltaief 11 / 37
14 Real Scientific Applications Computational Ground-Based Astronomy Enhancing the observed image quality using MOAO by filtering out the noise coming from the adaptive optics instrumentation and the atmospheric turbulence. (a) Problem Definition. (b) The E-ELT. Figure: Finding new galaxies. H. Ltaief 12 / 37
15 Real Scientific Applications Computing the Eigenspectrum for Symmetric Hierarchical Low Rank Matrix Arising structural and vibrational analysis to problems in computational physics and chemistry like electronic and band structure calculations (a) Problem Definition. (b) Electronic structure. Figure: Design of new materials. H. Ltaief 13 / 37
16 Cholesky-based Matrix Computations Outline 1 Motivations 2 Real Scientific Applications 3 Cholesky-based Matrix Computations 4 The HiCMA Library 5 Algorithmic Perspectives H. Ltaief 14 / 37
17 Cholesky-based Matrix Computations Matrix Form The Cholesky factorization of an N N real symmetric, positive-definite matrix A has the form A = LL T, where L is an N N real lower triangular matrix with positive diagonal elements. H. Ltaief 15 / 37
18 Cholesky-based Matrix Computations A Look Back... Software infrastructure and algorithmic design follow hardware evolution in time: 70 s - LINPACK, vector operations: Level-1 BLAS operation H. Ltaief 16 / 37
19 Cholesky-based Matrix Computations A Look Back... Software infrastructure and algorithmic design follow hardware evolution in time: 70 s - LINPACK, vector operations: Level-1 BLAS operation 80 s - LAPACK, block, cache-friendly: Level-3 BLAS operation H. Ltaief 16 / 37
20 Cholesky-based Matrix Computations A Look Back... Software infrastructure and algorithmic design follow hardware evolution in time: 70 s - LINPACK, vector operations: Level-1 BLAS operation 80 s - LAPACK, block, cache-friendly: Level-3 BLAS operation 90 s - ScaLAPACK, distributed memory: PBLAS Message passing H. Ltaief 16 / 37
21 Cholesky-based Matrix Computations A Look Back... Software infrastructure and algorithmic design follow hardware evolution in time: 70 s - LINPACK, vector operations: Level-1 BLAS operation 80 s - LAPACK, block, cache-friendly: Level-3 BLAS operation 90 s - ScaLAPACK, distributed memory: PBLAS Message passing 00 s: PLASMA, MAGMA, many x86/cuda cores friendly: DAG scheduler, tile data layout, some extra kernels H. Ltaief 16 / 37
22 Cholesky-based Matrix Computations Block Algorithms Panel-Update Sequence Transformations are blocked/accumulated within the Panel (Level 2 BLAS) Transformations applied at once on the trailing submatrix (Level 3 BLAS) Parallelism hidden inside the BLAS Fork-join Model H. Ltaief 17 / 37
23 Cholesky-based Matrix Computations Block Algorithms: Fork-Join Paradigm H. Ltaief 18 / 37
24 Cholesky-based Matrix Computations Tile Data Layout Format LAPACK: column-major format PLASMA: tile format H. Ltaief 19 / 37
25 Cholesky-based Matrix Computations PLASMA: Tile Algorithms PLASMA: Parallel Linear Algebra for Scalable Multi-core Architectures = Parallelism is brought to the fore May require the redesign of linear algebra algorithms Tile data layout translation Remove unnecessary synchronization points between Panel-Update sequences DAG execution where nodes represent tasks and edges define dependencies between them Dynamic runtime system environment QUARK H. Ltaief 20 / 37
26 Cholesky-based Matrix Computations MAGMA: x86 + MultiGPUs MAGMA: Matrix Algebra on GPU and Multicore Architectures = Lessons Learned from PLASMA! CUDA-based hybrid systems New high performance numerical kernels StarPU (INRIA, Bordeaux), OmpSs (BSC, Barcelona), PaRSEC (UTK, Knoxville) Both: x86 and GPUs = Hybrid Computations Similar to LAPACK in functionality H. Ltaief 21 / 37
27 Cholesky-based Matrix Computations Dynamic Runtime System Conceptually similar to out-of-order processor scheduling because it has: Dynamic runtime DAG scheduler Out-of-order execution flow of fine-grained tasks Task scheduling as soon as dependencies are satisfied Producer-Consumer H. Ltaief 22 / 37
28 Cholesky-based Matrix Computations State-of-the-art Performance Comparisons Tile Hybrid CPU GPU MAGMA PLASMA LAPACK Gflop/s Matrix size x 10 4 H. Ibeid, D. Kaushik, D. Keyes and H. Ltaief, HIPC 11, India H. Ltaief 23 / 37
29 Cholesky-based Matrix Computations MOAO performance in Tflop/s on 16 SDB cores + 8 K40 GPUs Tflops/s Async - 8GPUs + 16 cores Sync - 8GPUs + 16 cores Async - 4GPUs + 16 cores Sync - 4GPUs + 16 cores Async - 2GPUs + 16 cores Sync - 2GPUs + 16 cores Async - 1GPU + 16 cores Sync - 1GPU + 16 cores Total # of Measurements H. Ltaief, D. Gratadour, A. Charara and E. Gendron, PASC 16, Switzerland 6min to simulate 50 PSFs! H. Ltaief 24 / 37
30 The HiCMA Library Outline 1 Motivations 2 Real Scientific Applications 3 Cholesky-based Matrix Computations 4 The HiCMA Library 5 Algorithmic Perspectives H. Ltaief 25 / 37
31 The HiCMA Library Geospatial Statistics H. Ltaief 26 / 37
32 The HiCMA Library Computational Astronomy H. Ltaief 27 / 37
33 The HiCMA Library HiCMA DPOTRF: batch algorithms, matrix structure and data-sparsity exploitation H. Ltaief 28 / 37
34 The HiCMA Library HiCMA s methodology 1 Redesign the numerical algorithm so that it operates on the low rank data structure. 2 Formulate the new task-based low rank matrix computation using a DAG representation. 3 Employ a dynamic runtime system to ensure proper scheduling, asynchronous execution and load balancing. 4 Ensure a separation of concerns by abstracting the hardware complexity from users. H. Ltaief 29 / 37
35 The HiCMA Library HiCMA DPOTRF for Climate Apps on KNL Figure: Memory footprint depending on the truncation rank. H. Ltaief 30 / 37
36 The HiCMA Library HiCMA DPOTRF for Climate Apps on KNL Figure: Performance in time (seconds) depending on the truncation rank. H. Ltaief 31 / 37
37 Algorithmic Perspectives Outline 1 Motivations 2 Real Scientific Applications 3 Cholesky-based Matrix Computations 4 The HiCMA Library 5 Algorithmic Perspectives H. Ltaief 32 / 37
38 Algorithmic Perspectives New data structures needed Redesign the algorithm from tile-centric to kernel-centric Increase hardware occupancy Refactor the code around batch BLAS (incomplete vendor support) Use contiguous memory buffers Expose opportunities for HW/SW prefetching H. Ltaief 33 / 37
39 Algorithmic Perspectives Batch BLAS Operations: KBLAS H-Matrix computations: Level 3 BLAS: SYRK, TRSM, GEMM Factorization: POTRF Solve: POTRS, POSV Inversion: LAUUM, TRTRI, POTRI, POTI Truncation: QR, SVD H. Ltaief 34 / 37
40 Algorithmic Perspectives KBLAS Performance Results: Level 3 BLAS on NVIDIA P100 GFlop / s cublas-dgemm KBLAS-DSYRK KBLAS-DTRMM KBLAS-DTRSM cublas-dtrsm Matrix size (M=N, batch=10240) H. Ltaief 35 / 37
41 Algorithmic Perspectives KBLAS Performance Results: Factorization on NVIDIA K GFlops/s cublas-dgemm KBLAS-DPOTRF MAGMA-DPOTRF MKL-DPOTRF Matrix Size (batch=10240) H. Ltaief 36 / 37
42 Acknowledgments Students/Collaborators/Vendors Extreme Computing Research KAUST: S. Abdullah, K. Akbudak, W. Boukaram, A. Charara, G. Chávez, M. Genton, D. Keyes, A. Mikhalev, D. Sukkari, G. Turkiyyah and Y. Sun L Observatoire de Paris, LESIA: R. Dembet, N. Doucet, E. Gendron, D. Gratadour, A. Sevin and F. Vidal Innovative Computing UTK: PLASMA/MAGMA/PaRSEC Teams INRIA/INP/LaBRI Bordeaux, France: Runtime/HiePACS Teams Max-Planck Leipzig, Germany: R. Kriemann Barcelona Supercomputing Center, Spain: OmpSs Team KAUST Supercomputing Lab and IT Research Computing support NVIDIA GPU Research Center Intel Parallel Computing Center Cray Center of Excellence H. Ltaief 37 / 37
Power Profiling of Cholesky and QR Factorizations on Distributed Memory Systems
International Conference on Energy-Aware High Performance Computing Hamburg, Germany Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Sept Profiling, DLA Algorithms ENAHPC / 6 Power Profiling of Cholesky and
MAGMA a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures
MAGMA a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures Stan Tomov Innovative Computing Laboratory University of Tennessee, Knoxville OLCF Seminar Series, ORNL June 16, 2010
MAGMA. Matrix Algebra on GPU and Multicore Architectures
MAGMA Matrix Algebra on GPU and Multicore Architectures Innovative Computing Laboratory Electrical Engineering and Computer Science University of Tennessee Piotr Luszczek (presenter) web.eecs.utk.edu/~luszczek/conf/
Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester
Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester 11/20/13 1 Rank Site Computer Country Cores Rmax [Pflops] % of Peak Power [MW] MFlops /Watt 1 2 3 4 National
Efficient Observations Forecast for the World s Biggest Eye Using DGX-1
Efficient Observations Forecast for the World s Biggest Eye Using DGX-1 Damien Gratadour 1 and Hatem Ltaief 2 1 LESIA, Observatoire de Paris and Université Paris Diderot, France 2 Extreme Computing Research
Making Dataflow Programming Ubiquitous for Scientific Computing
Making Dataflow Programming Ubiquitous for Scientific Computing Hatem Ltaief KAUST Supercomputing Lab Synchronization-reducing and Communication-reducing Algorithms and Programming Models for Large-scale
MAGMA: a New Generation
1.3 MAGMA: a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures Jack Dongarra T. Dong, M. Gates, A. Haidar, S. Tomov, and I. Yamazaki University of Tennessee, Knoxville Release
Distributed Dense Linear Algebra on Heterogeneous Architectures. George Bosilca
Distributed Dense Linear Algebra on Heterogeneous Architectures George Bosilca bosilca@eecs.utk.edu Centraro, Italy June 2010 Factors that Necessitate to Redesign of Our Software» Steepness of the ascent
Dense Linear Algebra on Heterogeneous Platforms: State of the Art and Trends
Dense Linear Algebra on Heterogeneous Platforms: State of the Art and Trends Paolo Bientinesi AICES, RWTH Aachen pauldj@aices.rwth-aachen.de ComplexHPC Spring School 2013 Heterogeneous computing - Impact
Hierarchical DAG Scheduling for Hybrid Distributed Systems
June 16, 2015 Hierarchical DAG Scheduling for Hybrid Distributed Systems Wei Wu, Aurelien Bouteiller, George Bosilca, Mathieu Faverge, Jack Dongarra IPDPS 2015 Outline! Introduction & Motivation! Hierarchical
A Standard for Batching BLAS Operations
A Standard for Batching BLAS Operations Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester 5/8/16 1 API for Batching BLAS Operations We are proposing, as a community
A Linear Algebra Library for Multicore/Accelerators: the PLASMA/MAGMA Collection
A Linear Algebra Library for Multicore/Accelerators: the PLASMA/MAGMA Collection Jack Dongarra University of Tennessee Oak Ridge National Laboratory 11/24/2009 1 Gflop/s LAPACK LU - Intel64-16 cores DGETRF
GPU ACCELERATION OF WSMP (WATSON SPARSE MATRIX PACKAGE)
GPU ACCELERATION OF WSMP (WATSON SPARSE MATRIX PACKAGE) NATALIA GIMELSHEIN ANSHUL GUPTA STEVE RENNICH SEID KORIC NVIDIA IBM NVIDIA NCSA WATSON SPARSE MATRIX PACKAGE (WSMP) Cholesky, LDL T, LU factorization
Redesigning Triangular Dense Matrix Computations on GPUs
Redesigning Triangular Dense Matrix Computations on GPUs Ali Charara, Hatem Ltaief, and David Keyes Extreme Computing Research Center, King Abdullah University of Science and Technology, Thuwal, Jeddah,
Presentations: Jack Dongarra, University of Tennessee & ORNL. The HPL Benchmark: Past, Present & Future. Mike Heroux, Sandia National Laboratories
HPC Benchmarking Presentations: Jack Dongarra, University of Tennessee & ORNL The HPL Benchmark: Past, Present & Future Mike Heroux, Sandia National Laboratories The HPCG Benchmark: Challenges It Presents
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Waiting for Moore s Law to save your serial code started getting bleak in 2004 Source: published SPECInt
MAGMA. LAPACK for GPUs. Stan Tomov Research Director Innovative Computing Laboratory Department of Computer Science University of Tennessee, Knoxville
MAGMA LAPACK for GPUs Stan Tomov Research Director Innovative Computing Laboratory Department of Computer Science University of Tennessee, Knoxville Keeneland GPU Tutorial 2011, Atlanta, GA April 14-15,
How to perform HPL on CPU&GPU clusters. Dr.sc. Draško Tomić
How to perform HPL on CPU&GPU clusters Dr.sc. Draško Tomić email: drasko.tomic@hp.com Forecasting is not so easy, HPL benchmarking could be even more difficult Agenda TOP500 GPU trends Some basics about
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Waiting for Moore s Law to save your serial code start getting bleak in 2004 Source: published SPECInt data Moore s Law is not at all
Thinking Outside of the Tera-Scale Box. Piotr Luszczek
Thinking Outside of the Tera-Scale Box Piotr Luszczek Brief History of Tera-flop: 1997 1997 ASCI Red Brief History of Tera-flop: 2007 Intel Polaris 2007 1997 ASCI Red Brief History of Tera-flop: GPGPU
An Overview of High Performance Computing and Challenges for the Future
An Overview of High Performance Computing and Challenges for the Future Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester 6/15/2009 1 H. Meuer, H. Simon, E. Strohmaier,
Hybrid KAUST Many Cores and OpenACC. Alain Clo - KAUST Research Computing Saber Feki KAUST Supercomputing Lab Florent Lebeau - CAPS
+ Hybrid Computing @ KAUST Many Cores and OpenACC Alain Clo - KAUST Research Computing Saber Feki KAUST Supercomputing Lab Florent Lebeau - CAPS + Agenda Hybrid Computing n Hybrid Computing n From Multi-Physics
Towards Exascale Computing with the Atmospheric Model NUMA
Towards Exascale Computing with the Atmospheric Model NUMA Andreas Müller, Daniel S. Abdi, Michal Kopera, Lucas Wilcox, Francis X. Giraldo Department of Applied Mathematics Naval Postgraduate School, Monterey
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Waiting for Moore s Law to save your serial code started getting bleak in 2004 Source: published SPECInt
Fast and reliable linear system solutions on new parallel architectures
Fast and reliable linear system solutions on new parallel architectures Marc Baboulin Université Paris-Sud Chaire Inria Saclay Île-de-France Séminaire Aristote - Ecole Polytechnique 15 mai 2013 Marc Baboulin
Parallel Computing & Accelerators. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
Parallel Computing Accelerators John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Purpose of this talk This is the 50,000 ft. view of the parallel computing landscape. We want
Automatic Development of Linear Algebra Libraries for the Tesla Series
Automatic Development of Linear Algebra Libraries for the Tesla Series Enrique S. Quintana-Ortí quintana@icc.uji.es Universidad Jaime I de Castellón (Spain) Dense Linear Algebra Major problems: Source
Mixed Precision Methods
Mixed Precision Methods Mixed precision, use the lowest precision required to achieve a given accuracy outcome " Improves runtime, reduce power consumption, lower data movement " Reformulate to find correction
Trends in HPC (hardware complexity and software challenges)
Trends in HPC (hardware complexity and software challenges) Mike Giles Oxford e-research Centre Mathematical Institute MIT seminar March 13th, 2013 Mike Giles (Oxford) HPC Trends March 13th, 2013 1 / 18
Heterogenous Acceleration for Linear Algebra in Mulit-Coprocessor Environments
Heterogenous Acceleration for Linear Algebra in Mulit-Coprocessor Environments Azzam Haidar 1, Piotr Luszczek 1, Stanimire Tomov 1, and Jack Dongarra 1,2,3 1 University of Tennessee Knoxville, USA 2 Oak
Heterogenous Acceleration for Linear Algebra in Mulit-Coprocessor Environments
Heterogenous Acceleration for Linear Algebra in Mulit-Coprocessor Environments Azzam Haidar 1, Piotr Luszczek 1, Stanimire Tomov 1, and Jack Dongarra 1,2,3 1 University of Tennessee Knoxville, USA 2 Oak
Solving Dense Linear Systems on Graphics Processors
Solving Dense Linear Systems on Graphics Processors Sergio Barrachina Maribel Castillo Francisco Igual Rafael Mayo Enrique S. Quintana-Ortí High Performance Computing & Architectures Group Universidad
Introduction to Parallel and Distributed Computing. Linh B. Ngo CPSC 3620
Introduction to Parallel and Distributed Computing Linh B. Ngo CPSC 3620 Overview: What is Parallel Computing To be run using multiple processors A problem is broken into discrete parts that can be solved
CRAY XK6 REDEFINING SUPERCOMPUTING. - Sanjana Rakhecha - Nishad Nerurkar
CRAY XK6 REDEFINING SUPERCOMPUTING - Sanjana Rakhecha - Nishad Nerurkar CONTENTS Introduction History Specifications Cray XK6 Architecture Performance Industry acceptance and applications Summary INTRODUCTION
Confessions of an Accidental Benchmarker
Confessions of an Accidental Benchmarker http://bit.ly/hpcg-benchmark 1 Appendix B of the Linpack Users Guide Designed to help users extrapolate execution Linpack software package First benchmark report
NEW ADVANCES IN GPU LINEAR ALGEBRA
GTC 2012: NEW ADVANCES IN GPU LINEAR ALGEBRA Kyle Spagnoli EM Photonics 5/16/2012 QUICK ABOUT US» HPC/GPU Consulting Firm» Specializations in:» Electromagnetics» Image Processing» Fluid Dynamics» Linear
High performance matrix inversion of SPD matrices on graphics processors
High performance matrix inversion of SPD matrices on graphics processors Peter Benner, Pablo Ezzatti, Enrique S. Quintana-Ortí and Alfredo Remón Max-Planck-Institute for Dynamics of Complex Technical Systems
A scalable approach to solving dense linear algebra problems on hybrid CPU-GPU systems
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. () Published online in Wiley Online Library (wileyonlinelibrary.com)..33 A scalable approach to solving dense linear
HPC with Multicore and GPUs
HPC with Multicore and GPUs Stan Tomov Electrical Engineering and Computer Science Department University of Tennessee, Knoxville COSC 594 Lecture Notes March 22, 2017 1/20 Outline Introduction - Hardware
An Extension of the StarSs Programming Model for Platforms with Multiple GPUs
An Extension of the StarSs Programming Model for Platforms with Multiple GPUs Eduard Ayguadé 2 Rosa M. Badia 2 Francisco Igual 1 Jesús Labarta 2 Rafael Mayo 1 Enrique S. Quintana-Ortí 1 1 Departamento
Runtime Systems and Out-of-Core Cholesky Factorization on the Intel Xeon Phi System
Runtime Systems and Out-of-Core Cholesky Factorization on the Intel Xeon Phi System Allan Richmond R. Morales, Chong Tian, Kwai Wong, Eduardo D Azevedo The George Washington University, The Chinese University
Steve Scott, Tesla CTO SC 11 November 15, 2011
Steve Scott, Tesla CTO SC 11 November 15, 2011 What goal do these products have in common? Performance / W Exaflop Expectations First Exaflop Computer K Computer ~10 MW CM5 ~200 KW Not constant size, cost
SciDAC CScADS Summer Workshop on Libraries and Algorithms for Petascale Applications
Parallel Tiled Algorithms for Multicore Architectures Alfredo Buttari, Jack Dongarra, Jakub Kurzak and Julien Langou SciDAC CScADS Summer Workshop on Libraries and Algorithms for Petascale Applications
Linear Algebra Libraries: BLAS, LAPACK, ScaLAPACK, PLASMA, MAGMA
Linear Algebra Libraries: BLAS, LAPACK, ScaLAPACK, PLASMA, MAGMA Shirley Moore svmoore@utep.edu CPS5401 Fall 2012 svmoore.pbworks.com November 8, 2012 1 Learning ObjecNves AOer complenng this lesson, you
Chapter 5 Supercomputers
Chapter 5 Supercomputers Part I. Preliminaries Chapter 1. What Is Parallel Computing? Chapter 2. Parallel Hardware Chapter 3. Parallel Software Chapter 4. Parallel Applications Chapter 5. Supercomputers
PORTING PARALLEL APPLICATIONS TO HETEROGENEOUS SUPERCOMPUTERS: LIBRARIES AND TOOLS CAN MAKE IT TRANSPARENT
PORTING PARALLEL APPLICATIONS TO HETEROGENEOUS SUPERCOMPUTERS: LIBRARIES AND TOOLS CAN MAKE IT TRANSPARENT Jean-Yves VET, DDN Storage Patrick CARRIBAULT, CEA Albert COHEN, INRIA CEA, DAM, DIF, F-91297
Toward a supernodal sparse direct solver over DAG runtimes
Toward a supernodal sparse direct solver over DAG runtimes HOSCAR 2013, Bordeaux X. Lacoste Xavier LACOSTE HiePACS team Inria Bordeaux Sud-Ouest November 27, 2012 Guideline Context and goals About PaStiX
Overview. CS 472 Concurrent & Parallel Programming University of Evansville
Overview CS 472 Concurrent & Parallel Programming University of Evansville Selection of slides from CIS 410/510 Introduction to Parallel Computing Department of Computer and Information Science, University
Dealing with Asymmetry for Performance and Energy Efficiency
Dealing with Asymmetryfor Performance and Energy Efficiency Enrique S. QUINTANA-ORTÍ Motivation Moore s law is alive, but Dennard s scaling is over Motivation Welcome dark silicon and asymmetric architectures
Optimizing Memory-Bound Numerical Kernels on GPU Hardware Accelerators
Optimizing Memory-Bound Numerical Kernels on GPU Hardware Accelerators Ahmad Abdelfattah 1, Jack Dongarra 2, David Keyes 1 and Hatem Ltaief 3 1 KAUST Division of Mathematical and Computer Sciences and
Titan - Early Experience with the Titan System at Oak Ridge National Laboratory
Office of Science Titan - Early Experience with the Titan System at Oak Ridge National Laboratory Buddy Bland Project Director Oak Ridge Leadership Computing Facility November 13, 2012 ORNL s Titan Hybrid
INTERNATIONAL ADVANCED RESEARCH WORKSHOP ON HIGH PERFORMANCE COMPUTING AND GRIDS Cetraro (Italy), July 3-6, 2006
INTERNATIONAL ADVANCED RESEARCH WORKSHOP ON HIGH PERFORMANCE COMPUTING AND GRIDS Cetraro (Italy), July 3-6, 2006 The Challenges of Multicore and Specialized Accelerators Jack Dongarra University of Tennessee
Exploiting Data Sparsity for Large-Scale Matrix Computations
Exploiting Data Sparsity for Large-Scale Matrix Computations Item Type Technical Report Authors Akbudak, Kadir; Ltaief, Hatem; Mikhalev, Aleksandr; Charara, Ali; Keyes, David E. Download date 06/07/2018
swsptrsv: a Fast Sparse Triangular Solve with Sparse Level Tile Layout on Sunway Architecture Xinliang Wang, Weifeng Liu, Wei Xue, Li Wu
swsptrsv: a Fast Sparse Triangular Solve with Sparse Level Tile Layout on Sunway Architecture 1,3 2 1,3 1,3 Xinliang Wang, Weifeng Liu, Wei Xue, Li Wu 1 2 3 Outline 1. Background 2. Sunway architecture
Jack Dongarra University of Tennessee Oak Ridge National Laboratory
Jack Dongarra University of Tennessee Oak Ridge National Laboratory 3/9/11 1 TPP performance Rate Size 2 100 Pflop/s 100000000 10 Pflop/s 10000000 1 Pflop/s 1000000 100 Tflop/s 100000 10 Tflop/s 10000
ECE 574 Cluster Computing Lecture 2
ECE 574 Cluster Computing Lecture 2 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 24 January 2019 Announcements Put your name on HW#1 before turning in! 1 Top500 List November
Batch Linear Algebra for GPU-Accelerated High Performance Computing Environments
Batch Linear Algebra for GPU-Accelerated High Performance Computing Environments Ahmad Abdelfattah, Azzam Haidar, Stanimire Tomov, and Jack Dongarra SIAM Conference on Computational Science and Engineering
Intel Performance Libraries
Intel Performance Libraries Powerful Mathematical Library Intel Math Kernel Library (Intel MKL) Energy Science & Research Engineering Design Financial Analytics Signal Processing Digital Content Creation
Cray XC Scalability and the Aries Network Tony Ford
Cray XC Scalability and the Aries Network Tony Ford June 29, 2017 Exascale Scalability Which scalability metrics are important for Exascale? Performance (obviously!) What are the contributing factors?
Memory Footprint of Locality Information On Many-Core Platforms Brice Goglin Inria Bordeaux Sud-Ouest France 2018/05/25
ROME Workshop @ IPDPS Vancouver Memory Footprint of Locality Information On Many- Platforms Brice Goglin Inria Bordeaux Sud-Ouest France 2018/05/25 Locality Matters to HPC Applications Locality Matters
Parallel Linear Algebra in Julia
Parallel Linear Algebra in Julia Britni Crocker and Donglai Wei 18.337 Parallel Computing 12.17.2012 1 Table of Contents 1. Abstract... 2 2. Introduction... 3 3. Julia Implementation...7 4. Performance...
High-performance Cholesky factorization for GPU-only execution
High-performance Cholesky factorization for GPU-only execution Azzam Haidar Ahmad Abdelfatah Stanimire Tomov University of Tennessee, U.S.A. haidar,ahmad,tomov@icl.utk.edu Jack Dongarra University of Tennessee,
Intel Math Kernel Library 10.3
Intel Math Kernel Library 10.3 Product Brief Intel Math Kernel Library 10.3 The Flagship High Performance Computing Math Library for Windows*, Linux*, and Mac OS* X Intel Math Kernel Library (Intel MKL)
Achieving numerical accuracy and high performance using recursive tile LU factorization with partial pivoting
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. (213) Published online in Wiley Online Library (wileyonlinelibrary.com)..311 Achieving numerical accuracy and high
Matrix Computations. Enrique S. Quintana-Ortí. September 11, 2012, Hamburg, Germany
Doing Nothing to Save Energy in Matrix Computations Enrique S. Quintana-Ortí quintana@icc.uji.esuji eeclust Workshop, Energy efficiency Motivation Doing nothing to save energy? Why at Ena-HPC then? Energy
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Moore's Law abandoned serial programming around 2004 Courtesy Liberty Computer Architecture Research Group
Top500
Top500 www.top500.org Salvatore Orlando (from a presentation by J. Dongarra, and top500 website) 1 2 MPPs Performance on massively parallel machines Larger problem sizes, i.e. sizes that make sense Performance
COMPUTING ELEMENT EVOLUTION AND ITS IMPACT ON SIMULATION CODES
COMPUTING ELEMENT EVOLUTION AND ITS IMPACT ON SIMULATION CODES P(ND) 2-2 2014 Guillaume Colin de Verdière OCTOBER 14TH, 2014 P(ND)^2-2 PAGE 1 CEA, DAM, DIF, F-91297 Arpajon, France October 14th, 2014 Abstract:
HPC Algorithms and Applications
HPC Algorithms and Applications Intro Michael Bader Winter 2015/2016 Intro, Winter 2015/2016 1 Part I Scientific Computing and Numerical Simulation Intro, Winter 2015/2016 2 The Simulation Pipeline phenomenon,
Solving Dense Linear Systems on Platforms with Multiple Hardware Accelerators
Solving Dense Linear Systems on Platforms with Multiple Hardware Accelerators Francisco D. Igual Enrique S. Quintana-Ortí Gregorio Quintana-Ortí Universidad Jaime I de Castellón (Spain) Robert A. van de
Hierarchical Low-Rank Approximation Methods on Distributed Memory and GPUs
JH160041-NAHI Hierarchical Low-Rank Approximation Methods on Distributed Memory and GPUs Rio Yokota(Tokyo Institute of Technology) Hierarchical low-rank approximation methods such as H-matrix, H2-matrix,
MAGMA Library. version 0.1. S. Tomov J. Dongarra V. Volkov J. Demmel
MAGMA Library version 0.1 S. Tomov J. Dongarra V. Volkov J. Demmel 2 -- MAGMA (version 0.1) -- Univ. of Tennessee, Knoxville Univ. of California, Berkeley Univ. of Colorado, Denver June 2009 MAGMA project
QR Factorization on a Multicore Node Enhanced with Multiple GPU Accelerators
QR Factorization on a Multicore Node Enhanced with Multiple GPU Accelerators Emmanuel Agullo, Cédric Augonnet, Jack Dongarra, Mathieu Faverge, Hatem Ltaief, Samuel Thibault and Stanimire Tomov INRIA, LaBRI,
COMPUTATIONAL LINEAR ALGEBRA
COMPUTATIONAL LINEAR ALGEBRA Matrix Vector Multiplication Matrix matrix Multiplication Slides from UCSD and USB Directed Acyclic Graph Approach Jack Dongarra A new approach using Strassen`s algorithm Jim
Automatically Tuned Linear Algebra Software (ATLAS) R. Clint Whaley Innovative Computing Laboratory University of Tennessee.
Automatically Tuned Linear Algebra Software (ATLAS) R. Clint Whaley Innovative Computing Laboratory University of Tennessee Outline Pre-intro: BLAS Motivation What is ATLAS Present release How ATLAS works
BLASFEO. Gianluca Frison. BLIS retreat September 19, University of Freiburg
University of Freiburg BLIS retreat September 19, 217 Basic Linear Algebra Subroutines For Embedded Optimization performance dgemm_nt 5 4 Intel Core i7 48MQ HP OpenBLAS.2.19 MKL 217.2.174 ATLAS 3.1.3 BLIS.1.6
Hybrid Architectures Why Should I Bother?
Hybrid Architectures Why Should I Bother? CSCS-FoMICS-USI Summer School on Computer Simulations in Science and Engineering Michael Bader July 8 19, 2013 Computer Simulations in Science and Engineering,
Some notes on efficient computing and high performance computing environments
Some notes on efficient computing and high performance computing environments Abhi Datta 1, Sudipto Banerjee 2 and Andrew O. Finley 3 July 31, 2017 1 Department of Biostatistics, Bloomberg School of Public
A Scalable Approach to Solving Dense Linear Algebra Problems. on Hybrid CPU-GPU Systems
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. 2014; 00:2 35 Published online in Wiley InterScience (www.interscience.wiley.com). A Scalable Approach to Solving
Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory
Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory Quinn Mitchell HPC UNIX/LINUX Storage Systems ORNL is managed by UT-Battelle for the US Department of Energy U.S. Department
Towards an efficient tile matrix inversion of symmetric positive definite matrices on multicore architectures
Towards an efficient tile matrix inversion of symmetric positive definite matrices on multicore architectures Emmanuel Agullo, Henricus Bouwmeester, Jack Dongarra, Jakub Kurzak, Julien Langou, Lee Rosenberg
Achieving Numerical Accuracy and High Performance using Recursive Tile LU Factorization
Achieving Numerical Accuracy and High Performance using Recursive Tile LU Factorization Jack J. Dongarra, Mathieu Faverge, Hatem Ltaief, Piotr Luszczek To cite this version: Jack J. Dongarra, Mathieu Faverge,
Parallel Computing & Accelerators. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
Parallel Computing Accelerators John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Purpose of this talk This is the 50,000 ft. view of the parallel computing landscape. We want
On Level Scheduling for Incomplete LU Factorization Preconditioners on Accelerators
On Level Scheduling for Incomplete LU Factorization Preconditioners on Accelerators Karl Rupp, Barry Smith rupp@mcs.anl.gov Mathematics and Computer Science Division Argonne National Laboratory FEMTEC
Towards an Efficient Tile Matrix Inversion of Symmetric Positive Definite Matrices on Multicore Architectures
Towards an Efficient Tile Matrix Inversion of Symmetric Positive Definite Matrices on Multicore Architectures Emmanuel Agullo 1, Henricus Bouwmeester 2, Jack Dongarra 1, Jakub Kurzak 1, Julien Langou 2,andLeeRosenberg
GPU ACCELERATION OF CHOLMOD: BATCHING, HYBRID AND MULTI-GPU
April 4-7, 2016 Silicon Valley GPU ACCELERATION OF CHOLMOD: BATCHING, HYBRID AND MULTI-GPU Steve Rennich, Darko Stosic, Tim Davis, April 6, 2016 OBJECTIVE Direct sparse methods are among the most widely
Parallel Reduction to Condensed Forms for Symmetric Eigenvalue Problems using Aggregated Fine-Grained and Memory-Aware Kernels
Parallel Reduction to Condensed Forms for Symmetric Eigenvalue Problems using Aggregated Fine-Grained and Memory-Aware Kernels Azzam Haidar University of Tennessee 1122 Volunteer Blvd Knoxville, TN haidar@eecs.utk.edu
Adrian Tate XK6 / openacc workshop Manno, Mar
Adrian Tate XK6 / openacc workshop Manno, Mar6-7 2012 1 Overview & Philosophy Two modes of usage Contents Present contents Upcoming releases Optimization of libsci_acc Autotuning Adaptation Asynchronous
Speedup Altair RADIOSS Solvers Using NVIDIA GPU
Innovation Intelligence Speedup Altair RADIOSS Solvers Using NVIDIA GPU Eric LEQUINIOU, HPC Director Hongwei Zhou, Senior Software Developer May 16, 2012 Innovation Intelligence ALTAIR OVERVIEW Altair
Parallel Reduction to Condensed Forms for Symmetric Eigenvalue Problems using Aggregated Fine-Grained and Memory-Aware Kernels
Parallel Reduction to Condensed Forms for Symmetric Eigenvalue Problems using Aggregated Fine-Grained and Memory-Aware Kernels Azzam Haidar University of Tennessee 1122 Volunteer Blvd Knoxville, TN haidar@eecs.utk.edu
Accelerating HPL on Heterogeneous GPU Clusters
Accelerating HPL on Heterogeneous GPU Clusters Presentation at GTC 2014 by Dhabaleswar K. (DK) Panda The Ohio State University E-mail: panda@cse.ohio-state.edu http://www.cse.ohio-state.edu/~panda Outline
QR Decomposition on GPUs
QR Decomposition QR Algorithms Block Householder QR Andrew Kerr* 1 Dan Campbell 1 Mark Richards 2 1 Georgia Tech Research Institute 2 School of Electrical and Computer Engineering Georgia Institute of
Achieving Numerical Accuracy and High Performance using Recursive Tile LU Factorization
Achieving Numerical Accuracy and High Performance using Recursive Tile LU Factorization Jack Dongarra 1, Mathieu Faverge 1, Hatem Ltaief 2, and Piotr Luszczek 1 1 Department of Electrical Engineering and
Portable and Productive Performance on Hybrid Systems with libsci_acc Luiz DeRose Sr. Principal Engineer Programming Environments Director Cray Inc.
Portable and Productive Performance on Hybrid Systems with libsci_acc Luiz DeRose Sr. Principal Engineer Programming Environments Director Cray Inc. 1 What is Cray Libsci_acc? Provide basic scientific
Piz Daint: Application driven co-design of a supercomputer based on Cray s adaptive system design
Piz Daint: Application driven co-design of a supercomputer based on Cray s adaptive system design Sadaf Alam & Thomas Schulthess CSCS & ETHzürich CUG 2014 * Timelines & releases are not precise Top 500
Automatic Tuning of the High Performance Linpack Benchmark
Automatic Tuning of the High Performance Linpack Benchmark Ruowei Chen Supervisor: Dr. Peter Strazdins The Australian National University What is the HPL Benchmark? World s Top 500 Supercomputers http://www.top500.org
Performance Analysis and Optimization of the Tiled Cholesky Factorization on NUMA Machines
Performance Analysis and Optimization of the Tiled Cholesky Factorization on NUMA Machines Emmanuel Jeannot Inria Bordeaux Sud-Ouest, LaBRI, France Email: emmanuel.jeannot@inria.fr Abstract We discuss
Hybrid Multicore Cholesky Factorization with Multiple GPU Accelerators
Hybrid Multicore Cholesky Factorization with Multiple GPU Accelerators Hatem Ltaief 1, Stanimire Tomov 1, Rajib Nath 1, and Jack Dongarra 1,2,3 1 Department of Electrical Engineering and Computer Science,
Mathematical computations with GPUs
Master Educational Program Information technology in applications Mathematical computations with GPUs Introduction Alexey A. Romanenko arom@ccfit.nsu.ru Novosibirsk State University How to.. Process terabytes
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Moore's Law abandoned serial programming around 2004 Courtesy Liberty Computer Architecture Research Group