Power Profiling of Cholesky and QR Factorizations on Distributed Memory Systems
|
|
|
- Laurel Gilmore
- 6 years ago
- Views:
Transcription
1 International Conference on Energy-Aware High Performance Computing Hamburg, Germany Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Sept Profiling, DLA Algorithms ENAHPC / 6 Power Profiling of Cholesky and QR Factorizations on Distributed s George Bosilca Hatem Ltaief Jack Dongarra 4 Innovative Computing Laboratory University of Tennessee Knoxville KAUST Supercomputing Laboratory Thuwal, Saudi Arabia Oak Ridge National Lab 4 University of Manchester
2 Outline Motivations From LAPACK to PLASMA From PLASMA to DPLASMA 4 Power Measurements Technique 5 Power Measurements Results 6 Summary and Future Work Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
3 The Top5 List Motivations Rank Name Vendor Cores R max (Tflop/s) Power (KW) Sequoia IBM BG/Q (8%) 789. K Computer Fujitsu SPARC (9%) Mira IBM BG/Q (8%) SuperMUC Intel Xeon E (9%) Tianhe-A Intel Xeon X567 + M (55%) Jaguar Cray XK6 Opteron + M (74%) Fermi IBM BG/Q (8%) JuQUEEN IBM BG/Q (8%) Curie thin nodes Intel Xeon E (8%) 5. Nebulae Intel Xeon X567 + M (4%) 58. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
4 The Top5 List Motivations Rank Name Vendor Cores R max (Tflop/s) Power (KW) Sequoia IBM BG/Q (8%) 789. K Computer Fujitsu SPARC (9%) Mira IBM BG/Q (8%) SuperMUC Intel Xeon E (9%) Tianhe-A Intel Xeon X567 + M (55%) Jaguar Cray XK6 Opteron + M (74%) Fermi IBM BG/Q (8%) JuQUEEN IBM BG/Q (8%) Curie thin nodes Intel Xeon E (8%) 5. Nebulae Intel Xeon X567 + M (4%) 58. Human brain: PetaFLOPS! (cf Kurzweil) Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 4 / 6
5 Motivations Today s Special Meal 8 MW needed to feed the baby Exascale roadmap says up to MW Power Envelope Huge challenge: achieving orders of magnitude in performance by only doubling the power rate High level of concurrency Ingredients: Fine-grain parallelism, Dynamic runtime systems, Power Efficiency Flops are cheap, Data movement is expensive Co-designed Hardware and Software solutions Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 5 / 6
6 From LAPACK to PLASMA Block Algorithms Panel-Update Sequence Transformations are blocked/accumulated within the Panel (Level BLAS) Transformations applied at once on the trailing submatrix (Level BLAS) Parallelism hidden inside the BLAS Fork-join Model Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 6 / 6
7 From LAPACK to PLASMA One-Sided Block Algorithms: LU Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 7 / 6
8 From LAPACK to PLASMA Block Algorithms: Fork-Join Paradigm Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 8 / 6
9 From LAPACK to PLASMA Tile Data Layout Format LAPACK: column-major format PLASMA: tile format Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 9 / 6
10 From LAPACK to PLASMA PLASMA: Tile Algorithms PLASMA: Parallel Linear Algebra for Scalable Multi-core Architectures = Parallelism is brought to the fore May require the redesign of linear algebra algorithms Tile data layout translation Remove unnecessary synchronization points between Panel-Update sequences DAG execution where nodes represent tasks and edges define dependencies between them Dynamic runtime system environment QUARK Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
11 From LAPACK to PLASMA Dynamic Runtime QUARK Basic Ideas: Conceptually similar to out-of-order processor scheduling Dynamic runtime DAG scheduler Out-of-order execution flow of fine-grained tasks Task scheduling as soon as dependencies are satisfied Producer-Consumer Similar projects: SuperMatrix, OMPSs, StarPU Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
12 From LAPACK to PLASMA DataFlow Programming Model Five decades OLD concept Programming paradigm that models a program as a directed graph of the data flowing between operations (cf. Wikipedia) Think "how things connect" rather than "how things happen" Assembly line Inherently parallel Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
13 From PLASMA to DPLASMA D Block Cyclic Distribution (a) Column-major data layout format within a block. (b) Tile data layout format within a block. Figure: Two-Dimensional Block Cyclic Data Distribution. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
14 From PLASMA to DPLASMA DAGuE Dynamic Runtime Scheduler Bosilca et. al, UTK Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 4 / 6
15 From PLASMA to DPLASMA DAGuE Dynamic Runtime Scheduler Bosilca et. al, UTK Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 5 / 6
16 Power Measurements Technique The PowerPack Framework Dual-socket quad-core Intel Xeon system from Virginia Tech, clocked at.8ghz with 8GB of memory Measurements from power meters attached to the hardware of the system Fine-grain measurement (ms) allows power consumption to be measured on a per-component basis, memory, hard disk, motherboard and (as a whole) N = 4 for all experiments Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 6 / 6
17 Power Measurements Technique The PowerPack Framework K. Cameron et. al, Virginia Tech Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 7 / 6
18 Power Measurements Results (a) block size =. (a) tile size = (b) block size = 8. (b) tile size = (c) block size = 5. (c) tile size = 768. Figure: Impact of the block size on the power profiles (Watts) of the ScaLAPACK Cholesky. Figure: Impact of the tile size on the power profiles (Watts) of the DPLASMA Cholesky. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 8 / 6
19 Power Measurements Results (a) block size =. (a) tile size = (b) block size = 8. (b) tile size = (c) block size = 5. (c) tile size = 768. Figure: Impact of the block size on the power profiles (Watts) of the ScaLAPACK QR Factorization. Figure: Impact of the tile size on the power profiles (Watts) of the DPLASMA QR Factorization. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 9 / 6
20 Power Measurements Results (a) Number of cores = 8. (a) Number of cores = (b) Number of cores = (b) Number of cores = 5. Figure: Impact of the number of Figure: Impact of the number of cores on the power profiles (Watts) cores on the power profiles (Watts) of the ScaLAPACK Cholesky of the DPLASMA Cholesky Factorization. Factorization. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
21 Power Measurements Results (a) Number of cores = 8. (a) Number of cores = (b) Number of cores = (b) Number of cores = 5. Figure: Impact of the number of Figure: Impact of the number of cores on the power profiles (Watts) cores on the power profiles (Watts) of the ScaLAPACK QR. of the DPLASMA QR. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
22 Power Measurements Results (a) ScaLAPACK (b) DPLASMA. Figure: Power Profiles of the Cholesky Factorization. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
23 Power Measurements Results (a) ScaLAPACK (b) DPLASMA. Figure: Power Profiles of the QR Factorization. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC / 6
24 Power Measurements Results # Cores Library Cholesky QR 8 ScaLAPACK 9 67 DPLASMA ScaLAPACK 4 86 DPLASMA ScaLAPACK 5 DPLASMA Figure: Total amount of energy (joule) used for each test based on the number of cores Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 4 / 6
25 Summary and Future Work Conclusion Stressing the system s components DPLASMA Cholesky algorithms decrease the energy consumption up to 6% compared to ScaLAPACK Cholesky DPLASMA QR algorithms decrease the energy consumption up to 4% compared to ScaLAPACK QR Asynchronous execution runtime and adapted algorithms can lead to significantly improved efficiencies and power saving Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 5 / 6
26 Summary and Future Work What s next? Power analysis of advanced numerical algorithms on distributed systems (two-sided transformations, tree reduction, mixed precisions) Comparisons with other DLA libraries: Elemental, Eigen-K, ELPA Distributed heterogeneous architectures Scheduler interaction through DVFS/Intel RAPL technology Running on IBM BG P/Q and exploit embedded power collection hardware tools. Bosilca, Ltaief, Dongarra (KAUST, UTK) Power Profiling of DLA Algorithms ENAHPC 6 / 6
High Performance Linear Algebra
High Performance Linear Algebra Hatem Ltaief Senior Research Scientist Extreme Computing Research Center King Abdullah University of Science and Technology 4th International Workshop on Real-Time Control
Making Dataflow Programming Ubiquitous for Scientific Computing
Making Dataflow Programming Ubiquitous for Scientific Computing Hatem Ltaief KAUST Supercomputing Lab Synchronization-reducing and Communication-reducing Algorithms and Programming Models for Large-scale
Distributed Dense Linear Algebra on Heterogeneous Architectures. George Bosilca
Distributed Dense Linear Algebra on Heterogeneous Architectures George Bosilca bosilca@eecs.utk.edu Centraro, Italy June 2010 Factors that Necessitate to Redesign of Our Software» Steepness of the ascent
Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester
Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester 11/20/13 1 Rank Site Computer Country Cores Rmax [Pflops] % of Peak Power [MW] MFlops /Watt 1 2 3 4 National
MAGMA. Matrix Algebra on GPU and Multicore Architectures
MAGMA Matrix Algebra on GPU and Multicore Architectures Innovative Computing Laboratory Electrical Engineering and Computer Science University of Tennessee Piotr Luszczek (presenter) web.eecs.utk.edu/~luszczek/conf/
MAGMA a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures
MAGMA a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures Stan Tomov Innovative Computing Laboratory University of Tennessee, Knoxville OLCF Seminar Series, ORNL June 16, 2010
MAGMA: a New Generation
1.3 MAGMA: a New Generation of Linear Algebra Libraries for GPU and Multicore Architectures Jack Dongarra T. Dong, M. Gates, A. Haidar, S. Tomov, and I. Yamazaki University of Tennessee, Knoxville Release
Thinking Outside of the Tera-Scale Box. Piotr Luszczek
Thinking Outside of the Tera-Scale Box Piotr Luszczek Brief History of Tera-flop: 1997 1997 ASCI Red Brief History of Tera-flop: 2007 Intel Polaris 2007 1997 ASCI Red Brief History of Tera-flop: GPGPU
A Linear Algebra Library for Multicore/Accelerators: the PLASMA/MAGMA Collection
A Linear Algebra Library for Multicore/Accelerators: the PLASMA/MAGMA Collection Jack Dongarra University of Tennessee Oak Ridge National Laboratory 11/24/2009 1 Gflop/s LAPACK LU - Intel64-16 cores DGETRF
Dense Linear Algebra on Heterogeneous Platforms: State of the Art and Trends
Dense Linear Algebra on Heterogeneous Platforms: State of the Art and Trends Paolo Bientinesi AICES, RWTH Aachen pauldj@aices.rwth-aachen.de ComplexHPC Spring School 2013 Heterogeneous computing - Impact
Jack Dongarra University of Tennessee Oak Ridge National Laboratory
Jack Dongarra University of Tennessee Oak Ridge National Laboratory 3/9/11 1 TPP performance Rate Size 2 100 Pflop/s 100000000 10 Pflop/s 10000000 1 Pflop/s 1000000 100 Tflop/s 100000 10 Tflop/s 10000
Achieving numerical accuracy and high performance using recursive tile LU factorization with partial pivoting
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. (213) Published online in Wiley Online Library (wileyonlinelibrary.com)..311 Achieving numerical accuracy and high
Trends in HPC (hardware complexity and software challenges)
Trends in HPC (hardware complexity and software challenges) Mike Giles Oxford e-research Centre Mathematical Institute MIT seminar March 13th, 2013 Mike Giles (Oxford) HPC Trends March 13th, 2013 1 / 18
Presentations: Jack Dongarra, University of Tennessee & ORNL. The HPL Benchmark: Past, Present & Future. Mike Heroux, Sandia National Laboratories
HPC Benchmarking Presentations: Jack Dongarra, University of Tennessee & ORNL The HPL Benchmark: Past, Present & Future Mike Heroux, Sandia National Laboratories The HPCG Benchmark: Challenges It Presents
A Standard for Batching BLAS Operations
A Standard for Batching BLAS Operations Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester 5/8/16 1 API for Batching BLAS Operations We are proposing, as a community
MAGMA. LAPACK for GPUs. Stan Tomov Research Director Innovative Computing Laboratory Department of Computer Science University of Tennessee, Knoxville
MAGMA LAPACK for GPUs Stan Tomov Research Director Innovative Computing Laboratory Department of Computer Science University of Tennessee, Knoxville Keeneland GPU Tutorial 2011, Atlanta, GA April 14-15,
Steve Scott, Tesla CTO SC 11 November 15, 2011
Steve Scott, Tesla CTO SC 11 November 15, 2011 What goal do these products have in common? Performance / W Exaflop Expectations First Exaflop Computer K Computer ~10 MW CM5 ~200 KW Not constant size, cost
An Overview of High Performance Computing and Challenges for the Future
An Overview of High Performance Computing and Challenges for the Future Jack Dongarra University of Tennessee Oak Ridge National Laboratory University of Manchester 6/15/2009 1 H. Meuer, H. Simon, E. Strohmaier,
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Waiting for Moore s Law to save your serial code start getting bleak in 2004 Source: published SPECInt data Moore s Law is not at all
Efficient Observations Forecast for the World s Biggest Eye Using DGX-1
Efficient Observations Forecast for the World s Biggest Eye Using DGX-1 Damien Gratadour 1 and Hatem Ltaief 2 1 LESIA, Observatoire de Paris and Université Paris Diderot, France 2 Extreme Computing Research
Managing Hardware Power Saving Modes for High Performance Computing
Managing Hardware Power Saving Modes for High Performance Computing Second International Green Computing Conference 2011, Orlando Timo Minartz, Michael Knobloch, Thomas Ludwig, Bernd Mohr timo.minartz@informatik.uni-hamburg.de
Top500
Top500 www.top500.org Salvatore Orlando (from a presentation by J. Dongarra, and top500 website) 1 2 MPPs Performance on massively parallel machines Larger problem sizes, i.e. sizes that make sense Performance
Self Adapting Numerical Software (SANS-Effort)
Self Adapting Numerical Software (SANS-Effort) Jack Dongarra Innovative Computing Laboratory University of Tennessee and Oak Ridge National Laboratory 1 Work on Self Adapting Software 1. Lapack For Clusters
Achieving Numerical Accuracy and High Performance using Recursive Tile LU Factorization
Achieving Numerical Accuracy and High Performance using Recursive Tile LU Factorization Jack J. Dongarra, Mathieu Faverge, Hatem Ltaief, Piotr Luszczek To cite this version: Jack J. Dongarra, Mathieu Faverge,
A scalable approach to solving dense linear algebra problems on hybrid CPU-GPU systems
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. () Published online in Wiley Online Library (wileyonlinelibrary.com)..33 A scalable approach to solving dense linear
SciDAC CScADS Summer Workshop on Libraries and Algorithms for Petascale Applications
Parallel Tiled Algorithms for Multicore Architectures Alfredo Buttari, Jack Dongarra, Jakub Kurzak and Julien Langou SciDAC CScADS Summer Workshop on Libraries and Algorithms for Petascale Applications
Parallel Reduction to Condensed Forms for Symmetric Eigenvalue Problems using Aggregated Fine-Grained and Memory-Aware Kernels
Parallel Reduction to Condensed Forms for Symmetric Eigenvalue Problems using Aggregated Fine-Grained and Memory-Aware Kernels Azzam Haidar University of Tennessee 1122 Volunteer Blvd Knoxville, TN haidar@eecs.utk.edu
Parallel Reduction to Condensed Forms for Symmetric Eigenvalue Problems using Aggregated Fine-Grained and Memory-Aware Kernels
Parallel Reduction to Condensed Forms for Symmetric Eigenvalue Problems using Aggregated Fine-Grained and Memory-Aware Kernels Azzam Haidar University of Tennessee 1122 Volunteer Blvd Knoxville, TN haidar@eecs.utk.edu
The Mont-Blanc approach towards Exascale
http://www.montblanc-project.eu The Mont-Blanc approach towards Exascale Alex Ramirez Barcelona Supercomputing Center Disclaimer: Not only I speak for myself... All references to unavailable products are
Parallel and Distributed Systems. Hardware Trends. Why Parallel or Distributed Computing? What is a parallel computer?
Parallel and Distributed Systems Instructor: Sandhya Dwarkadas Department of Computer Science University of Rochester What is a parallel computer? A collection of processing elements that communicate and
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Waiting for Moore s Law to save your serial code started getting bleak in 2004 Source: published SPECInt
Energy Footprint of Advanced Dense Numerical Linear Algebra using Tile Algorithms on Multicore Architectures
12 Second International Conference on Cloud and Green Computing Energy Footprint of Advanced Dense Numerical Linear Algebra using Tile Algorithms on Multicore Architectures Jack Dongarra, Hatem Ltaief,
Mixed Precision Methods
Mixed Precision Methods Mixed precision, use the lowest precision required to achieve a given accuracy outcome " Improves runtime, reduce power consumption, lower data movement " Reformulate to find correction
Optimizing Memory-Bound Numerical Kernels on GPU Hardware Accelerators
Optimizing Memory-Bound Numerical Kernels on GPU Hardware Accelerators Ahmad Abdelfattah 1, Jack Dongarra 2, David Keyes 1 and Hatem Ltaief 3 1 KAUST Division of Mathematical and Computer Sciences and
Using R for HPC Data Science. Session: Parallel Programming Paradigms. George Ostrouchov
Using R for HPC Data Science Session: Parallel Programming Paradigms George Ostrouchov Oak Ridge National Laboratory and University of Tennessee and pbdr Core Team Course at IT4Innovations, Ostrava, October
Achieving Numerical Accuracy and High Performance using Recursive Tile LU Factorization
Achieving Numerical Accuracy and High Performance using Recursive Tile LU Factorization Jack Dongarra 1, Mathieu Faverge 1, Hatem Ltaief 2, and Piotr Luszczek 1 1 Department of Electrical Engineering and
Performance Analysis of BLAS Libraries in SuperLU_DIST for SuperLU_MCDT (Multi Core Distributed) Development
Available online at www.prace-ri.eu Partnership for Advanced Computing in Europe Performance Analysis of BLAS Libraries in SuperLU_DIST for SuperLU_MCDT (Multi Core Distributed) Development M. Serdar Celebi
A Scalable Approach to Solving Dense Linear Algebra Problems. on Hybrid CPU-GPU Systems
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. 2014; 00:2 35 Published online in Wiley InterScience (www.interscience.wiley.com). A Scalable Approach to Solving
CRAY XK6 REDEFINING SUPERCOMPUTING. - Sanjana Rakhecha - Nishad Nerurkar
CRAY XK6 REDEFINING SUPERCOMPUTING - Sanjana Rakhecha - Nishad Nerurkar CONTENTS Introduction History Specifications Cray XK6 Architecture Performance Industry acceptance and applications Summary INTRODUCTION
Heterogenous Acceleration for Linear Algebra in Mulit-Coprocessor Environments
Heterogenous Acceleration for Linear Algebra in Mulit-Coprocessor Environments Azzam Haidar 1, Piotr Luszczek 1, Stanimire Tomov 1, and Jack Dongarra 1,2,3 1 University of Tennessee Knoxville, USA 2 Oak
Emerging Heterogeneous Technologies for High Performance Computing
MURPA (Monash Undergraduate Research Projects Abroad) Emerging Heterogeneous Technologies for High Performance Computing Jack Dongarra University of Tennessee Oak Ridge National Lab University of Manchester
Dynamic Task Execution on Shared and Distributed Memory Architectures
University of Tennessee, Knoxville Trace: Tennessee Research and Creative Exchange Doctoral Dissertations Graduate School 12-2012 Dynamic Task Execution on Shared and Distributed Memory Architectures Asim
Hierarchical DAG Scheduling for Hybrid Distributed Systems
June 16, 2015 Hierarchical DAG Scheduling for Hybrid Distributed Systems Wei Wu, Aurelien Bouteiller, George Bosilca, Mathieu Faverge, Jack Dongarra IPDPS 2015 Outline! Introduction & Motivation! Hierarchical
Heterogenous Acceleration for Linear Algebra in Mulit-Coprocessor Environments
Heterogenous Acceleration for Linear Algebra in Mulit-Coprocessor Environments Azzam Haidar 1, Piotr Luszczek 1, Stanimire Tomov 1, and Jack Dongarra 1,2,3 1 University of Tennessee Knoxville, USA 2 Oak
Parallel Computing & Accelerators. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
Parallel Computing Accelerators John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Purpose of this talk This is the 50,000 ft. view of the parallel computing landscape. We want
Parallel Linear Algebra in Julia
Parallel Linear Algebra in Julia Britni Crocker and Donglai Wei 18.337 Parallel Computing 12.17.2012 1 Table of Contents 1. Abstract... 2 2. Introduction... 3 3. Julia Implementation...7 4. Performance...
Best Practice Guide to Hybrid PaRSEC + OpenMP Programming
Best Practice Guide to Hybrid PaRSEC + OpenMP Programming Version 1.0, 30 th September 2018 Jakub Šístek, Reazul Hoque and George Bosilca Table of Contents TABLE OF CONTENTS... 2 1 INTRODUCTION... 3 2
Overview. CS 472 Concurrent & Parallel Programming University of Evansville
Overview CS 472 Concurrent & Parallel Programming University of Evansville Selection of slides from CIS 410/510 Introduction to Parallel Computing Department of Computer and Information Science, University
HPC with Multicore and GPUs
HPC with Multicore and GPUs Stan Tomov Electrical Engineering and Computer Science Department University of Tennessee, Knoxville COSC 594 Lecture Notes March 22, 2017 1/20 Outline Introduction - Hardware
Making a Case for a Green500 List
Making a Case for a Green500 List S. Sharma, C. Hsu, and W. Feng Los Alamos National Laboratory Virginia Tech Outline Introduction What Is Performance? Motivation: The Need for a Green500 List Challenges
Mathematical computations with GPUs
Master Educational Program Information technology in applications Mathematical computations with GPUs Introduction Alexey A. Romanenko arom@ccfit.nsu.ru Novosibirsk State University How to.. Process terabytes
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Waiting for Moore s Law to save your serial code started getting bleak in 2004 Source: published SPECInt
Preparing GPU-Accelerated Applications for the Summit Supercomputer
Preparing GPU-Accelerated Applications for the Summit Supercomputer Fernanda Foertter HPC User Assistance Group Training Lead foertterfs@ornl.gov This research used resources of the Oak Ridge Leadership
Aggregation of Real-Time System Monitoring Data for Analyzing Large-Scale Parallel and Distributed Computing Environments
Aggregation of Real-Time System Monitoring Data for Analyzing Large-Scale Parallel and Distributed Computing Environments Swen Böhm 1,2, Christian Engelmann 2, and Stephen L. Scott 2 1 Department of Computer
Fast and reliable linear system solutions on new parallel architectures
Fast and reliable linear system solutions on new parallel architectures Marc Baboulin Université Paris-Sud Chaire Inria Saclay Île-de-France Séminaire Aristote - Ecole Polytechnique 15 mai 2013 Marc Baboulin
Flexible Linear Algebra Development and Scheduling with Cholesky Factorization
Flexible Linear Algebra Development and Scheduling with Cholesky Factorization Azzam Haidar haidar@eecs.utk.edu Chongxiao Cao ccao1@vols.utk.edu Stanimire Tomov tomov@eecs.utk.edu Asim YarKhan yarkhan@eecs.utk.edu
Towards an efficient tile matrix inversion of symmetric positive definite matrices on multicore architectures
Towards an efficient tile matrix inversion of symmetric positive definite matrices on multicore architectures Emmanuel Agullo, Henricus Bouwmeester, Jack Dongarra, Jakub Kurzak, Julien Langou, Lee Rosenberg
Hybrid Multicore Cholesky Factorization with Multiple GPU Accelerators
Hybrid Multicore Cholesky Factorization with Multiple GPU Accelerators Hatem Ltaief 1, Stanimire Tomov 1, Rajib Nath 1, and Jack Dongarra 1,2,3 1 Department of Electrical Engineering and Computer Science,
MAGMA QR, 1 GPU, All Available Cores. Mitch Horton, Stanimire Tomov, Jack Dongarra Innovative Computing Laboratory
A Class of Hybrid LAPACK Algorithms for Multicore and GPU Architectures MAGMA QR, 1 GPU, All Available Cores Mitch Horton, Stanimire Tomov, Jack Dongarra Innovative Computing Laboratory University of Tennessee
Matrix Computations. Enrique S. Quintana-Ortí. September 11, 2012, Hamburg, Germany
Doing Nothing to Save Energy in Matrix Computations Enrique S. Quintana-Ortí quintana@icc.uji.esuji eeclust Workshop, Energy efficiency Motivation Doing nothing to save energy? Why at Ena-HPC then? Energy
Towards an Efficient Tile Matrix Inversion of Symmetric Positive Definite Matrices on Multicore Architectures
Towards an Efficient Tile Matrix Inversion of Symmetric Positive Definite Matrices on Multicore Architectures Emmanuel Agullo 1, Henricus Bouwmeester 2, Jack Dongarra 1, Jakub Kurzak 1, Julien Langou 2,andLeeRosenberg
Confessions of an Accidental Benchmarker
Confessions of an Accidental Benchmarker http://bit.ly/hpcg-benchmark 1 Appendix B of the Linpack Users Guide Designed to help users extrapolate execution Linpack software package First benchmark report
Enhancing Parallelism of Tile Bidiagonal Transformation on Multicore Architectures using Tree Reduction
Enhancing Parallelism of Tile Bidiagonal Transformation on Multicore Architectures using Tree Reduction Hatem Ltaief 1, Piotr Luszczek 2, and Jack Dongarra 2,3,4 1 KAUST Supercomputing Laboratory Thuwal,
A class of communication-avoiding algorithms for solving general dense linear systems on CPU/GPU parallel machines
Available online at www.sciencedirect.com Procedia Computer Science 9 (2012 ) 17 26 International Conference on Computational Science, ICCS 2012 A class of communication-avoiding algorithms for solving
Runtime Systems and Out-of-Core Cholesky Factorization on the Intel Xeon Phi System
Runtime Systems and Out-of-Core Cholesky Factorization on the Intel Xeon Phi System Allan Richmond R. Morales, Chong Tian, Kwai Wong, Eduardo D Azevedo The George Washington University, The Chinese University
How HPC Hardware and Software are Evolving Towards Exascale
How HPC Hardware and Software are Evolving Towards Exascale Kathy Yelick Associate Laboratory Director and NERSC Director Lawrence Berkeley National Laboratory EECS Professor, UC Berkeley NERSC Overview
Implementing a systolic algorithm for QR factorization on multicore clusters with PaRSEC
Implementing a systolic algorithm for QR factorization on multicore clusters with PaRSEC Guillaume Aupy 1, Mathieu Faverge 2, Yves Robert 1,3, Jakub Kurzak 3, Piotr Luszczek 3, and Jack Dongarra 3 1 Laboratoire
HPC as a Driver for Computing Technology and Education
HPC as a Driver for Computing Technology and Education Tarek El-Ghazawi The George Washington University Washington D.C., USA NOW- July 2015: The TOP 10 Systems Rank Site Computer Cores Rmax [Pflops] %
Petascale Parallel Computing and Beyond. General trends and lessons. Martin Berzins
Petascale Parallel Computing and Beyond General trends and lessons Martin Berzins 1. Technology Trends 2. Towards Exascale 3. Trends in programming large scale systems What kind of machine will you use
Designing Power-Aware Collective Communication Algorithms for InfiniBand Clusters
Designing Power-Aware Collective Communication Algorithms for InfiniBand Clusters Krishna Kandalla, Emilio P. Mancini, Sayantan Sur, and Dhabaleswar. K. Panda Department of Computer Science & Engineering,
Parallel Tiled QR Factorization for Multicore Architectures
Parallel Tiled QR Factorization for Multicore Architectures Alfredo Buttari 1, Julien Langou 2, Jakub Kurzak 1, and Jack Dongarra 1,3,4 1 Computer Science Dept. University of Tennessee Knoxville, USA 2
A Design of Hybrid Operating System for a Parallel Computer with Multi-Core and Many-Core Processors
A Design of Hybrid Operating System for a Parallel Computer with Multi-Core and Many-Core Processors Mikiko Sato 1,5 Go Fukazawa 1 Kiyohiko Nagamine 1 Ryuichi Sakamoto 1 Mitaro Namiki 1,5 Kazumi Yoshinaga
CSE5351: Parallel Procesisng. Part 1B. UTA Copyright (c) Slide No 1
Slide No 1 CSE5351: Parallel Procesisng Part 1B Slide No 2 State of the Art In Supercomputing Several of the next slides (or modified) are the courtesy of Dr. Jack Dongarra, a distinguished professor of
Algorithm and Library Software Design Challenges for Tera, Peta, and Future Exascale Computing
Algorithm and Library Software Design Challenges for Tera, Peta, and Future Exascale Computing Bo Kågström Department of Computing Science and High Performance Computing Center North (HPC2N) Umeå University,
MPI RUNTIMES AT JSC, NOW AND IN THE FUTURE
, NOW AND IN THE FUTURE Which, why and how do they compare in our systems? 08.07.2018 I MUG 18, COLUMBUS (OH) I DAMIAN ALVAREZ Outline FZJ mission JSC s role JSC s vision for Exascale-era computing JSC
QR Factorization on a Multicore Node Enhanced with Multiple GPU Accelerators
QR Factorization on a Multicore Node Enhanced with Multiple GPU Accelerators Emmanuel Agullo, Cédric Augonnet, Jack Dongarra, Mathieu Faverge, Hatem Ltaief, Samuel Thibault and Stanimire Tomov INRIA, LaBRI,
Parallel Computing & Accelerators. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
Parallel Computing Accelerators John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Purpose of this talk This is the 50,000 ft. view of the parallel computing landscape. We want
Unified Development for Mixed Multi-GPU and Multi-Coprocessor Environments using a Lightweight Runtime Environment
Unified Development for Mixed Multi-GPU and Multi-Coprocessor Environments using a Lightweight Runtime Environment Azzam Haidar 1, Chongxiao Cao 1, Asim YarKhan 1, Piotr Luszczek 1, Stanimire Tomov 1,
LU Factorization for Accelerator-based Systems
LU Factorization for Accelerator-based Systems Emmanuel Agullo, Cédric Augonnet, Jack Dongarra, Mathieu Faverge, Julien Langou, Hatem Ltaief, Stanimire Tomov To cite this version: Emmanuel Agullo, Cédric
Why we need Exascale and why we won t get there by 2020 Horst Simon Lawrence Berkeley National Laboratory
Why we need Exascale and why we won t get there by 2020 Horst Simon Lawrence Berkeley National Laboratory 2013 International Workshop on Computational Science and Engineering National University of Taiwan
Technology on Dense Linear Algebra
Impact of Multi core and Many core Technology on Dense Linear Algebra Enrique S. Quintana-Ortí Berlin, September 2011 Berlin, September 2011 1 Multi-core and Many-core The free lunch is over (H. Sutter,
DISP: Optimizations Towards Scalable MPI Startup
DISP: Optimizations Towards Scalable MPI Startup Huansong Fu, Swaroop Pophale*, Manjunath Gorentla Venkata*, Weikuan Yu Florida State University *Oak Ridge National Laboratory Outline Background and motivation
Analysis of Dynamically Scheduled Tile Algorithms for Dense Linear Algebra on Multicore Architectures
Analysis of Dynamically Scheduled Tile Algorithms for Dense Linear Algebra on Multicore Architectures Azzam Haidar, Hatem Ltaief, Asim YarKhan and Jack Dongarra Department of Electrical Engineering and
Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory
Managing HPC Active Archive Storage with HPSS RAIT at Oak Ridge National Laboratory Quinn Mitchell HPC UNIX/LINUX Storage Systems ORNL is managed by UT-Battelle for the US Department of Energy U.S. Department
High Performance Computing
High Performance Computing Oliver Rheinbach oliver.rheinbach@math.tu-freiberg.de http://www.mathe.tu-freiberg.de/nmo/ Vorlesung Introduction to High Performance Computing Hörergruppen Woche Tag Zeit Raum
A unified Energy Footprint for Simulation Software
A unified Energy Footprint for Simulation Software Hartwig Anzt, Armen Beglarian, Suren Chilingaryan, Andrew Ferrone, Vincent Heuveline, Andreas Kopmann Hartwig Anzt September 12, 212 ENGINEERING MATHEMATICS
Parallel Block Hessenberg Reduction using Algorithms-By-Tiles for Multicore Architectures Revisited LAPACK Working Note #208
Parallel Block Hessenberg Reduction using Algorithms-By-Tiles for Multicore Architectures Revisited LAPACK Working Note #208 Hatem Ltaief 1, Jakub Kurzak 1, and Jack Dongarra 1,2,3 1 Department of Electrical
Comparative Study of One-Sided Factorizations with Multiple Software Packages on Multi-Core Hardware
Comparative Study of One-Sided Factorizations with Multiple Software Packages on Multi-Core Hardware Emmanuel Agullo University of Tennessee 1122 Volunteer Blvd Knoxville, TN eagullo@eecs.utk.edu Bilel
HPCG UPDATE: ISC 15 Jack Dongarra Michael Heroux Piotr Luszczek
www.hpcg-benchmark.org 1 HPCG UPDATE: ISC 15 Jack Dongarra Michael Heroux Piotr Luszczek www.hpcg-benchmark.org 2 HPCG Snapshot High Performance Conjugate Gradient (HPCG). Solves Ax=b, A large, sparse,
A Scalable Framework for Heterogeneous GPU-Based Clusters
A Scalable Framework for Heterogeneous GPU-Based Clusters (Regular Submission) Fengguang Song Innovative Computing Laboratory University of Tennessee Knoxville TN 37996-3450 song@eecs.utk.edu Jack Dongarra
Parallel Programming
Parallel Programming Introduction Diego Fabregat-Traver and Prof. Paolo Bientinesi HPAC, RWTH Aachen fabregat@aices.rwth-aachen.de WS15/16 Acknowledgements Prof. Felix Wolf, TU Darmstadt Prof. Matthias
CUDA Accelerated Linpack on Clusters. E. Phillips, NVIDIA Corporation
CUDA Accelerated Linpack on Clusters E. Phillips, NVIDIA Corporation Outline Linpack benchmark CUDA Acceleration Strategy Fermi DGEMM Optimization / Performance Linpack Results Conclusions LINPACK Benchmark
Present and Future Leadership Computers at OLCF
Present and Future Leadership Computers at OLCF Al Geist ORNL Corporate Fellow DOE Data/Viz PI Meeting January 13-15, 2015 Walnut Creek, CA ORNL is managed by UT-Battelle for the US Department of Energy
Hierarchical QR factorization algorithms for multi-core cluster systems
2012 IEEE 26th International Parallel and Distributed Processing Symposium Hierarchical QR factorization algorithms for multi-core cluster systems Jack Dongarra 1,2,3, Mathieu Faverge 1, Thomas Herault
Out Of Memory SVD Solver for Big Data
Out Of Memory SVD Solver for Big Data Azzam Haidar, Khairul Kabir, Diana Fayad, Stanimire Tomov, Jack Dongarra {haidar fayad tomov dongarra}@icl.utk.edu, Innovative Computing Laboratory (ICL), University
A Class of Parallel Tiled Linear Algebra Algorithms for Multicore Architectures
A Class of Parallel Tiled Linear Algebra Algorithms for Multicore Architectures Buttari, Alfredo and Langou, Julien and Kurzak, Jakub and Dongarra, Jack 2007 MIMS EPrint: 2007.122 Manchester Institute
arxiv: v2 [cs.ms] 11 Sep 2007
![arxiv: v2 [cs.ms] 11 Sep 2007](/thumbs/74/71097574.jpg "arxiv: v2 [cs.ms] 11 Sep 2007")
A Class of Parallel Tiled Linear Algebra Algorithms for Multicore Architectures LAPACK Working Note # 191 Alfredo Buttari 1, Julien Langou 4, Jakub Kurzak 1, Jack Dongarra 123 arxiv:0709.1272v2 [cs.ms]
Performance Analysis and Optimization of the Tiled Cholesky Factorization on NUMA Machines
Performance Analysis and Optimization of the Tiled Cholesky Factorization on NUMA Machines Emmanuel Jeannot Inria Bordeaux Sud-Ouest, LaBRI, France Email: emmanuel.jeannot@inria.fr Abstract We discuss
One-sided dense matrix factorizations on a multicore with multiple GPU accelerators in MAGMA 1
Procedia Computer Science Procedia Computer Science 00 1 10 International Conference on Computational Science, ICCS One-sided dense matrix factorizations on a multicore with multiple GPU accelerators in
Overview of research activities Toward portability of performance
Overview of research activities Toward portability of performance Do dynamically what can t be done statically Understand evolution of architectures Enable new programming models Put intelligence into