Opportunities and Challenges in Sparse Linear Algebra on Many-Core Processors with High-Bandwidth Memory

Transcription

1 Opportunities and Challenges in Sparse Linear Algebra on Many-Core Processors with High-Bandwidth Memory Jongsoo Park, Parallel Computing Lab, Intel Corporation with contributions from MKL team 1

2 Algorithm/ Applications Processor Architecture Programming Systems 2

3 Algorithm/ Applications Processor Architecture Programming Systems 3

4 Many-core Processors + HBM Knights Landing Example Up to 72 cores Connected via 2D Mesh 3+TF DP, 6+TF SP ~480 GB/s 90+ GB/s DDR, 100s GB MCDRAM, 16 GB package MCDRAM can be used as the last level cache (cache mode), or can be mapped to a separate address space (flat mode) 4

5 Improvements in KNL Avinash Sodani 5

6 Algorithm/ Applications Processor Architecture Programming Systems 6

7 Sparse Linear Algebra Primitives Sparse-matrix * dense-vector (SpMV) Sparse-matrix k * dense-vector Sparse-matrix * dense-matrix Sparse-matrix * sparse-vector Sparse-matrix * sparse-matrix Sparse factorization (direct solver, ILU, ) Sparse triangular solver Sparse tensor operations (MTTKRP, ) Gray just means we did not spend as much time. Doesn t mean unimportant. 7

8 Challenges in Sparse Linear Algebra Memory-BW bound only a fraction of peak FLOP utilized e.g., SpMV in CSR has at most 1/6 FLOP/Byte arithmetic intensity Fine-grain parallelism e.g., sparse triangular solver, sometimes <100 FLOPs per sync. Indirect access hard to prefetch and vectorize, low cache-line utilization Irregular control flow branch miss prediction, non-flop operations e.g., Short loop count in SpMV. Unknown output sparsity in sparsematrix * sparse-vector/matrix 8

9 Challenges in Sparse Linear Algebra Memory- BW bound Fine-grain Parallelism Indirect Access Irregular control flow Sparse-matrix * dense-vector * dense-matrix * sparse-vector * sparse-matrix Factorization Triangular solver More important Shows overall trend. Doesn t intend to be precise for specific cases. Less important 9

10 Algorithm/ Applications Processor Architecture Programming Systems 10

11 Architecture Algorithm/Applications (Current) Memory BW Fine-grain parallelism Xeon Phis Coherent cores (~100s cycles of latency) HBM GPUs Fast sync. in thread block Limitations among blocks Indirect access SIMD gather/scatter Highly-banked local memory Irregular control flow SMT, branch prediction SIMT 11

12 Gather/scatter History Research: Kumar et al. ISCA 08, our experiments show that GLSC provides an average performance improvement on a set of important Recognition-Mining- Synthesis kernels of 54% for 4-wide SIMD Xeon Phi KNC: first chip with gather/scatter support, needs a loop around it KNL: native gather/scatter support Xeon HSW: gather support BDW: scatter support, improved gather SKL: AVX-512 gather/scatter 12

13 Architecture Algorithm/Applications (Future) Memory BW More control over cache policy [1]? Fine-grain parallelism More control over coherency [1]? Faster atomics? Indirect access Indirect prefetch [2]? [1] Location-aware Cache Management for Many-core Processors with Deep Cache Hierarchy, Park et al., SC13 [2] IMP: Indirect Memory Prefetcher, Yu et al., MICRO15 13

14 Future Architecture Feature Study Example Shaden et al., Sparse tensor Factorization on Many-Core Processors with High- Bandwidth Memory, to appear in IPDPS17 14

15 Algorithm/Applications Architecture Memory BW: matrix-free, RCM reordering, communication avoiding, kernel fusion, Fine-grain parallelism: multi-color reordering, asynchronous algorithm [3], inspector-executor parallelization, remove redundant synchronization [4] Irregular control flow: new sparse matrix format (e.g., SELLPACK, blocked formats, ) Your new ideas (e.g., using high BW of HBM + high capacity of DDR) [3] Fine-grained Parallel Incomplete LU Factorization, Chow and Patel, SIAM J. SISC, 2015 [4] Sparsifying Synchronization for High-performance Shared-memory Sparse Triangular Solver, Park et al.,

16 Algorithm/ Applications Processor Architecture Programming Systems 16

17 Algorithm/Applications Architecture Memory BW: matrix-free, RCM reordering, kernel fusion, communication avoiding Fine-grain parallelism: multi-color reordering, asynchronous algorithm [3], inspector-executor parallelization, remove redundant synchronization [4] Irregular control flow: new sparse matrix format (e.g., SELLPACK) 17

18 Optimizing Widely-Used Libraries HYPRE example High-Performance Algebraic Multigrid Solver Optimized for Multi-Core Based Distributed Parallel Systems, Park et al., SC15 Some optimizations merged in HYPRE

19 Optimizing Primitives in MKL SpGEMM example Parallel Efficient Sparse Matrix-Matrix Multiplication on Multicore Platforms, Patwary at al., ISC15 19

20 Experimental Open Source Libraries SpMP for fast triangular solver and reordering Sparsifying Synchronization for High-Performance Shared-Memory Sparse Triangular Solver, Park et al., ISC14 20

21 New Inspector-executor Sparse BLAS Interface for Many-core Processors mkl_sparse_set_sv_hint(a, ) mkl_sparse_optimize(a); // inspection while (!converged) { } mkl_sparse_d_trsv(, alpha, A, ); // execution Great, but there can be variations of triangular solver and SpMV not covered by MKL. Can t compiler automatically generate inspector/executor? [5] [5] Automating Wavefront Parallelization for Sparse Matrix Codes, Venkat et al., SC16 21

22 Automating RCM Reordering for Memory-BW Saving A = P*A*P^T; // reorder while (!converged) { SpMV(y, A, x); } z = P^T*z; // inverse reorder Reordering often involves non-local changes. What if compiler can automate? [6] [6] Sparso: Context-driven Optimizations of Sparse Linear Algebra, Rong et al. PACT16 22

23 Contributing to Other Domains Graph Analytics Graph analytics folks are used to vertex programming model. But, graph algorithms can be expressed in sparse linear algebra and this is in fact usually the most performant implementation [7,8]. [7] The Combinatorial BLAS: Design, implementation, and applications, Buluç and Gilbert, IJHPCA11 [8] GraphMat: High performance graph analytics made productive, Sundaram et al., VLDB15 23

24 Contributing to Other Domains Deep Learning Recent work on sparse deep learning models [9] Not much sparse support yet in machine libraries like cudnn or MKL-DNN Different characteristics from scientific computing (e.g., smaller matrices) but optimization principles are similar. : a library for small dense and sparse matrixmatrix multiplications for deep learning primitives [9] Learning Both Weights and Connections for Efficient Neural Network, Han et al., NIPS15 24

25 Conclusion Your experience in scientific computing can help shape future processors other domains (graph analytics, ML, ) 25

26 Notice and Disclaimers INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND/OR USE OF INTEL PRODUCTS, INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT, OR OTHER INTELLECTUAL PROPERTY RIGHT. Intel products are not intended for use in medical, life saving, or life sustaining applications. Intel may make changes to specifications, product descriptions, and plans at any time, without notice. All products, dates, and figures are preliminary for planning purposes and are subject to change without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined. Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. The Intel products discussed herein may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling , or by visiting Intel's website at Intel Itanium, Intel Xeon, Xeon Phi, Pentium, Intel SpeedStep and Intel NetBurst, Intel, and VTune are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Copyright 2014, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.. 26

27 Notice and Disclaimers Continued Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to Intel's compilers may or may not optimize to the same degree for non-intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision #

28 28 Q&A