Implementing Strassen-like Fast Matrix Multiplication Algorithms with BLIS

Transcription

1 Implementing Strassen-like Fast Matrix Multiplication Algorithms with BLIS Jianyu Huang, Leslie Rice Joint work with Tyler M. Smith, Greg M. Henry, Robert A. van de Geijn BLIS Retreat 2016

2 *Overlook of the Bay Area. Photo taken in Mission Peak Regional Preserve, Fremont, CA. Summer STRASSEN, from 30,000 feet Volker Strassen (Born in 1936, aged 80) Original Strassen Paper (1969)

3 One-level Strassen s Algorithm (In theory) Direct Computation Strassen s Algorithm 8 multiplications, 8 additions 7 multiplications, 22 additions *Strassen, Volker. "Gaussian elimination is not optimal." Numerische Mathematik 13, no. 4 (1969):

4 Multi-level Strassen s Algorithm (In theory) M 0 := (A 00 +A 11 )(B 00 +B 11 ); M 1 := (A 10 +A 11 )B 00 ; M 2 := A 00 (B 01 B 11 ); M 3 := A 11 (B 10 B 00 ); M 4 := (A 00 +A 01 )B 11 ; M 5 := (A 10 A 00 )(B 00 +B 01 ); M 6 := (A 01 A 11 )(B 10 +B 11 ); C 00 M 0 + M 3 M 4 + M 6 One-level Strassen (1+14.3% speedup) 8 multiplications 7 multiplications ; Two-level Strassen (1+30.6% speedup) 64 multiplications 49 multiplications; d-level Strassen (n 3 /n speedup) 8 d multiplications 7 d multiplications; C 01 M 2 + M 4 C 10 M 1 + M 3 C 11 M 0 M 1 + M 2 + M 5

5 Multi-level Strassen s Algorithm (In theory) M 0 := (A 00 +A 11 )(B 00 +B 11 ); M 1 := (A 10 +A 11 )B 00 ; M 2 := A 00 (B 01 B 11 ); M 3 := A 11 (B 10 B 00 ); M 4 := (A 00 +A 01 )B 11 ; M 5 := (A 10 A 00 )(B 00 +B 01 ); M 6 := (A 01 A 11 )(B 10 +B 11 ); C 00 M 0 + M 3 M 4 + M 6 C 01 M 2 + M 4 C 10 M 1 + M 3 C 11 M 0 M 1 + M 2 + M 5 One-level Strassen (1+14.3% speedup) 8 multiplications 7 multiplications ; Two-level Strassen (1+30.6% speedup) 64 multiplications 49 multiplications; d-level Strassen (n 3 /n speedup) 8 d multiplications 7 d multiplications;

6 Strassen s Algorithm (In practice) M 0 := (A 00 +A 11 )(B 00 +B 11 ); M 1 := (A 10 +A 11 )B 00 ; M 2 := A 00 (B 01 B 11 ); M 3 := A 11 (B 10 B 00 ); M 4 := (A 00 +A 01 )B 11 ; M 5 := (A 10 A 00 )(B 00 +B 01 ); M 6 := (A 01 A 11 )(B 10 +B 11 ); C 00 M 0 + M 3 M 4 + M 6 C 01 M 2 + M 4 C 10 M 1 + M 3 C 11 M 0 M 1 + M 2 + M 5

7 Strassen s Algorithm (In practice) T 0 M 0 := (A 00 +A 11 )(B 00 +B 11 ); T 2 M 1 := (A 10 +A 11 )B 00 ; M 2 := A 00 (B 01 B 11 ); M 3 := A 11 (B 10 B 00 ); T 5 T 3 T 4 M 4 := (A 00 +A 01 )B 11 ; T 1 T 6 T 7 M 5 := (A 10 A 00 )(B 00 +B 01 ); T 8 T 9 M 6 := (A 01 A 11 )(B 10 +B 11 ); C 00 M 0 + M 3 M 4 + M 6 One-level Strassen (1+14.3% speedup) 7 multiplications + 22 additions; Two-level Strassen (1+30.6% speedup) 49 multiplications additions; d-level Strassen (n 3 /n speedup) Numerical unstable; Not achievable C 01 M 2 + M 4 C 10 M 1 + M 3 C 11 M 0 M 1 + M 2 + M 5

8 Strassen s Algorithm (In practice) T 0 M 0 := (A 00 +A 11 )(B 00 +B 11 ); T 2 M 1 := (A 10 +A 11 )B 00 ; M 2 := A 00 (B 01 B 11 ); M 3 := A 11 (B 10 B 00 ); T 5 T 3 T 4 M 4 := (A 00 +A 01 )B 11 ; T 1 T 6 T 7 M 5 := (A 10 A 00 )(B 00 +B 01 ); T 8 T 9 M 6 := (A 01 A 11 )(B 10 +B 11 ); C 00 M 0 + M 3 M 4 + M 6 One-level Strassen (1+14.3% speedup) 7 multiplications + 22 additions; Two-level Strassen (1+30.6% speedup) 49 multiplications additions; d-level Strassen (n 3 /n speedup) Numerical unstable; Not achievable C 01 M 2 + M 4 C 10 M 1 + M 3 C 11 M 0 M 1 + M 2 + M 5

9 To achieve practical high performance of Strassen s algorithm... Conventional Implementations Our Implementations Matrix Size Must be large Matrix Shape Must be square No Additional Workspace Parallelism

10 To achieve practical high performance of Strassen s algorithm... Conventional Implementations Our Implementations Matrix Size Must be large Matrix Shape Must be square No Additional Workspace Parallelism Usually task parallelism

11 To achieve practical high performance of Strassen s algorithm... Conventional Implementations Our Implementations Matrix Size Must be large Matrix Shape Must be square No Additional Workspace Parallelism Usually task parallelism Can be data parallelism

12 Outline Review of State-of-the-art GEMM in BLIS Strassen s Algorithm Reloaded Theoretical Model and Practical Performance Extension to Other BLAS-3 Operation Extension to Other Fast Matrix Multiplication Conclusion

13 Level-3 BLAS Matrix-Matrix Multiplication (GEMM) (General) matrix-matrix multiplication (GEMM) is supported in the level-3 BLAS* interface as Ignoring transa and transb, GEMM computes We consider the simplified version of GEMM *Dongarra, Jack J., et al. "A set of level 3 basic linear algebra subprograms."acm Transactions on Mathematical Software (TOMS) 16.1 (1990): 1-17.

14 State-of-the-art GEMM in BLIS BLAS-like Library Instantiation Software (BLIS) is a portable framework for instantiating BLAS-like dense linear algebra libraries. Field Van Zee, and Robert van de Geijn. BLIS: A Framework for Rapidly Instantiating BLAS Functionality." ACM TOMS 41.3 (2015): 14. BLIS provides a refactoring of GotoBLAS algorithm (best-known approach) to implement GEMM. Kazushige Goto, and Robert van de Geijn. "High-performance implementation of the level-3 BLAS." ACM TOMS 35.1 (2008): 4. Kazushige Goto, and Robert van de Geijn. "Anatomy of high-performance matrix multiplication." ACM TOMS 34.3 (2008): 12. GEMM implementation in BLIS has 6-layers of loops. The outer 5 loops are written i. The inner-most loop (micro-kernel) is written in assembly for high performance. Partition matrices into smaller blocks to fit into the different memory hierarchy. The order of these loops is designed to utilize the cache reuse rate. BLIS opens the black box of GEMM, leading to many applications built on BLIS. Chenhan D. Yu, *Jianyu Huang, Woody Austin, Bo Xiao, and George Biros. "Performance optimization for the k-nearest neighbors kernel on x86 architectures." In SC 15, p. 7. ACM, *Jianyu Huang, Tyler Smith, Greg Henry, and Robert van de Geijn. Strassen s Algorithm Reloaded. In submission to SC 16. Devin Matthews, Field Van Zee, and Robert van de Geijn. High-Performance Tensor Contraction without BLAS. In submission to SC 16

15 State-of-the-art GEMM in BLIS BLAS-like Library Instantiation Software (BLIS) is a portable framework for instantiating BLAS-like dense linear algebra libraries. Field Van Zee, and Robert van de Geijn. BLIS: A Framework for Rapidly Instantiating BLAS Functionality." ACM TOMS 41.3 (2015): 14. BLIS provides a refactoring of GotoBLAS algorithm (best-known approach) to implement GEMM. Kazushige Goto, and Robert van de Geijn. "High-performance implementation of the level-3 BLAS." ACM TOMS 35.1 (2008): 4. Kazushige Goto, and Robert van de Geijn. "Anatomy of high-performance matrix multiplication." ACM TOMS 34.3 (2008): 12. GEMM implementation in BLIS has 6-layers of loops. The outer 5 loops are written i. The inner-most loop (micro-kernel) is written in assembly for high performance. Partition matrices into smaller blocks to fit into the different memory hierarchy. The order of these loops is designed to utilize the cache reuse rate. BLIS opens the black box of GEMM, leading to many applications built on BLIS. Chenhan D. Yu, Jianyu Huang, Woody Austin, Bo Xiao, and George Biros. "Performance Optimization for the k-nearest Neighbors Kernel on x86 Architectures." In SC 15. Jianyu Huang, Tyler Smith, Greg Henry, and Robert van de Geijn. Strassen s Algorithm Reloaded. To appear in SC 16. Devin Matthews. High-Performance Tensor Contraction without BLAS., arxiv: Paul Springer, Paolo Bientinesi. Design of a High-performance GEMM-like Tensor-Tensor Multiplication, arxiv:

16 GotoBLAS algorithm for GEMM in BLIS *Field G. Van Zee, and Tyler M. Smith. Implementing high-performance complex matrix multiplication." In ACM Transactions on Mathematical Software (TOMS), accepted pending modifications.

17 5 th loop around micro-kernel GotoBLAS algorithm for GEMM in BLIS C j A B j main memory *Field G. Van Zee, and Tyler M. Smith. Implementing high-performance complex matrix multiplication." In ACM Transactions on Mathematical Software (TOMS), accepted pending modifications.

18 5 th loop around micro-kernel GotoBLAS algorithm for GEMM in BLIS C j A B j 4 th loop around micro-kernel C j A p B p Pack B p B p B p main memory L3 cache *Field G. Van Zee, and Tyler M. Smith. Implementing high-performance complex matrix multiplication." In ACM Transactions on Mathematical Software (TOMS), accepted pending modifications.

19 5 th loop around micro-kernel GotoBLAS algorithm for GEMM in BLIS C j A B j 4 th loop around micro-kernel C j A p B p 3 rd loop around micro-kernel Pack B p B p C i m C m C A i Pack A i A i B p A i m R main memory L3 cache L2 cache *Field G. Van Zee, and Tyler M. Smith. Implementing high-performance complex matrix multiplication." In ACM Transactions on Mathematical Software (TOMS), accepted pending modifications.

20 5 th loop around micro-kernel GotoBLAS algorithm for GEMM in BLIS C j A B j 4 th loop around micro-kernel C j A p B p 3 rd loop around micro-kernel Pack B p B p C i m C m C A i Pack A i A i B p C i 2 nd loop around micro-kernel n R n R A i B p m R main memory L3 cache L2 cache *Field G. Van Zee, and Tyler M. Smith. Implementing high-performance complex matrix multiplication." In ACM Transactions on Mathematical Software (TOMS), accepted pending modifications.

21 5 th loop around micro-kernel GotoBLAS algorithm for GEMM in BLIS C j A B j 4 th loop around micro-kernel C j A p B p 3 rd loop around micro-kernel Pack B p B p C i m C m C A i Pack A i A i B p C i 2 nd loop around micro-kernel n R n R A i B p m R n R 1 st loop around micro-kernel m R Update C ij main memory L3 cache L2 cache L1 cache registers *Field G. Van Zee, and Tyler M. Smith. Implementing high-performance complex matrix multiplication." In ACM Transactions on Mathematical Software (TOMS), accepted pending modifications.

22 5 th loop around micro-kernel GotoBLAS algorithm for GEMM in BLIS C j A B j 4 th loop around micro-kernel C j A p B p 3 rd loop around micro-kernel Pack B p B p C i m C m C A i Pack A i A i B p C i 2 nd loop around micro-kernel n R n R A i B p m R n R 1 st loop around micro-kernel m R Update C ij main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1 *Field G. Van Zee, and Tyler M. Smith. Implementing high-performance complex matrix multiplication." In ACM Transactions on Mathematical Software (TOMS), accepted pending modifications.

23 5 th loop around micro-kernel GotoBLAS algorithm for GEMM in BLIS C j A B j 4 th loop around micro-kernel C j A p B p 3 rd loop around micro-kernel Pack B p B p C i m C m C A i Pack A i A i B p C i 2 nd loop around micro-kernel n R n R A i B p m R n R 1 st loop around micro-kernel m R Update C ij main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1 *Field G. Van Zee, and Tyler M. Smith. Implementing high-performance complex matrix multiplication." In ACM Transactions on Mathematical Software (TOMS), accepted pending modifications.

24 Outline Review of State-of-the-art GEMM in BLIS Strassen s Algorithm Reloaded Theoretical Model and Practical Performance Extension to Other BLAS-3 Operation Extension to Other Fast Matrix Multiplication Conclusion

25 One-level Strassen s Algorithm Reloaded M 0 := a(a 00 +A 11 )(B 00 +B 11 ); M 1 := a(a 10 +A 11 )B 00 ; M 2 := aa 00 (B 01 B 11 ); M 3 := aa 11 (B 10 B 00 ); M 4 := a(a 00 +A 01 )B 11 ; M 5 := a(a 10 A 00 )(B 00 +B 01 ); M 6 := a(a 01 A 11 )(B 10 +B 11 ); C 00 M 0 + M 3 M 4 + M 6 C 01 M 2 + M 4 C 10 M 1 + M 3 C 11 M 0 M 1 + M 2 + M 5 M 0 := a(a 00 +A 11 )(B 00 +B 11 ); C 00 M 0 ; C 11 M 0 ; M 1 := a(a 10 +A 11 )B 00 ; C 10 M 1 ; C 11 = M 1 ; M 2 := aa 00 (B 01 B 11 ); C 01 M 2 ; C 11 M 2 ; M 3 := aa 11 (B 10 B 00 ); C 00 M 3 ; C 10 M 3 ; M 4 := a(a 00 +A 01 )B 11 ; C 01 M 4 ; C 00 = M 4 ; M 5 := a(a 10 A 00 )(B 00 +B 01 ); C 11 M 5 ; M 6 := a(a 01 A 11 )(B 10 +B 11 ); C 00 M 6 ; M := a(x+y)(v+w); C M; D M; General operation for one-level Strassen: M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}.

26 High-performance implementation of the general operation? M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}. *

27 M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}. *Jianyu Huang, Tyler Smith, Greg Henry, and Robert van de Geijn. Strassen s Algorithm Reloaded. In SC 16.

28 M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}. D j C n j C 5 th loop around micro-kernel Y X W V j main memory *Jianyu Huang, Tyler Smith, Greg Henry, and Robert van de Geijn. Strassen s Algorithm Reloaded. In SC 16.

29 M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}. D j C n j C 5 th loop around micro-kernel Y X W V j 4 th loop around micro-kernel D j C j Y p X p W p V p Pack V p + ew p B p B p main memory L3 cache *Jianyu Huang, Tyler Smith, Greg Henry, and Robert van de Geijn. Strassen s Algorithm Reloaded. In SC 16.

30 M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}. D j C n j C 5 th loop around micro-kernel Y X W V j 4 th loop around micro-kernel D j C j Y p X p W p V p 3 rd loop around micro-kernel Pack V p + ew p B p D i C i m C m C Y i X i B p Pack X i + dy i A i A i m R main memory L3 cache L2 cache *Jianyu Huang, Tyler Smith, Greg Henry, and Robert van de Geijn. Strassen s Algorithm Reloaded. In SC 16.

31 M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}. D j C n j C 5 th loop around micro-kernel Y X W V j 4 th loop around micro-kernel D j C j Y p X p W p V p 3 rd loop around micro-kernel Pack V p + ew p B p D i C i m C m C Y i X i B p Pack X i + dy i A i 2 nd loop around micro-kernel n R n R A i B p D i C i n R m R 1 st loop around micro-kernel Update C ij, D ij m R *Jianyu Huang, Tyler Smith, Greg Henry, and Robert van de Geijn. Strassen s Algorithm Reloaded. In SC 16. main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1

32 M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}. D j C n j C 5 th loop around micro-kernel Y X W V j 4 th loop around micro-kernel D j C j Y p X p W p V p 3 rd loop around micro-kernel Pack V p + ew p B p D i C i m C m C Y i X i B p Pack X i + dy i A i 2 nd loop around micro-kernel n R n R A i B p D i C i n R m R 1 st loop around micro-kernel Update C ij, D ij m R *Jianyu Huang, Tyler Smith, Greg Henry, and Robert van de Geijn. Strassen s Algorithm Reloaded. In SC 16. main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1

33 5 th loop around micro-kernel 5 th loop around micro-kernel C j A B D j C n j C Y X W V j 4 th loop around micro-kernel 4 th loop around micro-kernel C j A p B p D j C j Y p X p W p V p 3 rd loop around micro-kernel Pack B p B p 3 rd loop around micro-kernel Pack V p + ew p B p C i m C m C A i Pack A i A i B p D i C i m C m C Y i X i B p Pack X i + dy i A i C i 2 nd loop around micro-kernel n R n R A i B p m R n R 1 st loop around micro-kernel 2 nd loop around micro-kernel n R n R A i B p D i C i n R m R 1 st loop around micro-kernel m R Update C ij Update C ij, D ij m R main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1 main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1

34 Two-level Strassen s Algorithm Reloaded

35 Two-level Strassen s Algorithm Reloaded (Continue) M := a(x 0 +X 1 +X 2 +X 3 )(V+V 1 +V 2 +V 3 ); C 0 M; C 1 M; C 2 M; C 3 M; General operation for two-level Strassen: M := a(x 0 +d 1 X 1 +d 2 X 2 +d 3 X 3 )(V+e 1 V 1 +e 2 V 2 +e 3 V 3 ); C 0 g 0 M; C 1 g 1 M; C 2 g 2 M; C 3 g 3 M; g i,d i,e i {-1, 0, 1}.

36 Additional Levels of Strassen Reloaded The general operation of one-level Strassen: M := a(x+dy)(v+ew); C g 0 M; D g 1 M; g 0, g 1,d,e {-1, 0, 1}. The general operation of two-level Strassen: M := a(x 0 +d 1 X 1 +d 2 X 2 +d 3 X 3 )(V+e 1 V 1 +e 2 V 2 +e 3 V 3 ); C 0 g 0 M; C 1 g 1 M; C 2 g 2 M; C 3 g 3 M; g i,d i,e i {-1, 0, 1}. The general operation needed to integrate k levels of Strassen is given by

37 Building blocks BLIS framework A routine for packing B p into C/Intel intrinsics Adapted to general operation Integrate the addition of multiple matrices V t into A routine for packing A i into C/Intel intrinsics Integrate the addition of multiple matrices X s into A micro-kernel for updating an m R n R submatrix of C. SIMD assembly (AVX, AVX512, etc) Integrate the update of multiple submatrices of C.

38 Variations on a theme Naïve Strassen A traditional implementation with temporary buffers. AB Strassen Integrate the addition of matrices into and. ABC Strassen Integrate the addition of matrices into and. Integrate the update of multiple submatrices of C in the micro-kernel.

39 5 th loop around micro-kernel D j C n j C Y X W V j Parallelization 3 rd loop (along m C direction) 4 th loop around micro-kernel D j C j Y p X p W p V p D i C i m C 3 rd loop around micro-kernel m C Y i X i Pack V p + ew p B p B p Pack X i + dy i A i 2 nd loop (along n R direction) 2 nd loop around micro-kernel n R n R A i B p D i C i n R m R 1 st loop around micro-kernel both 3 rd and 2 nd loop Update C ij, D ij m R main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1 *Tyler M. Smith, Robert Van De Geijn, Mikhail Smelyanskiy, Jeff R. Hammond, and Field G. Van Zee. "Anatomy of high-performance many-threaded matrix multiplication." In Parallel and Distributed Processing Symposium, 2014 IEEE 28th International, pp IEEE, 2014.

40 Outline Review of State-of-the-art GEMM in BLIS Strassen s Algorithm Reloaded Theoretical Model and Practical Performance Extension to Other BLAS-3 Operation Extension to Other Fast Matrix Multiplication Conclusion

41 Performance Model Performance Metric Total Time Breakdown Arithmetic Operations Memory Operations

42 Arithmetic Operations DGEMM No extra additions Submatrix multiplication Extra additions with submatrices of A, B, C, respectively One-level Strassen (ABC, AB, Naïve) 7 submatrix multiplications 5 extra additions of submatrices of A and B 12 extra additions of submatrices of C M 0 := a(a 00 +A 11 )(B 00 +B 11 ); C 00 M 0 ; C 11 M 0 ; M 1 := a(a 10 +A 11 )B 00 ; C 10 M 1 ; C 11 = M 1 ; M 2 := aa 00 (B 01 B 11 ); C 01 M 2 ; C 11 M 2 ; M 3 := aa 11 (B 10 B 00 ); C 00 M 3 ; C 10 M 3 ; M 4 := a(a 00 +A 01 )B 11 ; C 01 M 4 ; C 00 = M 4 ; M 5 := a(a 10 A 00 )(B 00 +B 01 ); C 11 M 5 ; M 6 := a(a 01 A 11 )(B 10 +B 11 ); C 00 M 6 ; Two-level Strassen (ABC, AB, Naïve) 49 submatrix multiplications 95 extra additions of submatrices of A and B 154 extra additions of submatrices of C

43 Memory Operations DGEMM One-level ABC Strassen AB Strassen Naïve Strassen Two-level ABC Strassen AB Strassen Naïve Strassen

44 Modeled and Actual Performance on Single Core

45 Observation (Square Matrices) Modeled Performance Actual Performance

46 Observation (Square Matrices) Modeled Performance Actual Performance

47 Observation (Square Matrices) Modeled Performance Actual Performance

48 Observation (Square Matrices) Modeled Performance Actual Performance

49 Observation (Square Matrices) Modeled Performance Actual Performance

50 Observation (Square Matrices) Modeled Performance Actual Performance 26.0% 13.1% Theoretical Speedup over DGEMM One-level Strassen (1+14.3% speedup) 8 multiplications 7 multiplications; Two-level Strassen (1+30.6% speedup) 64 multiplications 49 multiplications;

51 Observation (Square Matrices) Both one-level and two-level For small square matrices, ABC Strassen outperforms AB Strassen For larger square matrices, this trend reverses Reason ABC Strassen avoids storing M (M resides in the register) ABC Strassen increases the number of times for updating submatrices of C

52 5 th loop around micro-kernel 5 th loop around micro-kernel C j A B D j C n j C Y X W V j 4 th loop around micro-kernel 4 th loop around micro-kernel C j A p B p D j C j Y p X p W p V p 3 rd loop around micro-kernel Pack B p B p 3 rd loop around micro-kernel Pack V p + ew p B p C i m C m C A i Pack A i A i B p D i C i m C m C Y i X i B p Pack X i + dy i A i C i 2 nd loop around micro-kernel n R n R A i B p m R n R 1 st loop around micro-kernel 2 nd loop around micro-kernel n R n R A i B p D i C i n R m R 1 st loop around micro-kernel m R Update C ij Update C ij, D ij m R main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1 main memory L3 cache L2 cache L1 cache registers 1 micro-kernel 1

53 Observation (Square Matrices) Both one-level and two-level For small square matrices, ABC Strassen outperforms AB Strassen For larger square matrices, this trend reverses Reason ABC Strassen avoids storing M (M resides in the register) ABC Strassen increases the number of times for updating submatrices of C

54 Observation (Rank-k Update) What is Rank-k update? n m m k n

55 Observation (Rank-k Update) Importance of Rank-k update Blocked LU with partial pivoting (getrf) A 12 A 21 A 22

56 Observation (Rank-k Update) Importance of Rank-k update A 12 A 21 A 22

57 Observation (Rank-k Update) Modeled Performance Actual Performance

58 Observation (Rank-k Update) Modeled Performance Actual Performance

59 Observation (Rank-k Update) Modeled Performance Actual Performance

60 Observation (Rank-k Update) Modeled Performance Actual Performance

61 Observation (Rank-k Update) Modeled Performance Actual Performance

62 Observation (Rank-k Update) Modeled Performance Actual Performance Reason: ABC Strassen avoids forming the temporary matrix M explicitly in the memory (M resides in register), especially important when m, n >> k.

63 Single Node Experiment

64 Rank-k Update Square Matrices 1 core 5 core 10 core

65 Many-core Experiment

66 Intel Xeon Phi coprocessor (KNC)

67 Distributed Memory Experiment

68

69

70

71 Outline Review of State-of-the-art GEMM in BLIS Strassen s Algorithm Reloaded Theoretical Model and Practical Performance Extension to Other BLAS-3 Operation Extension to Other Fast Matrix Multiplication Conclusion

72 Level-3 BLAS Symmetric Matrix-Matrix Multiplication (SYMM) Symmetric matrix-matrix multiplication (SYMM) is supported in the level-3 BLAS* interface as SYMM computes *Dongarra, Jack J., et al. "A set of level 3 basic linear algebra subprograms."acm Transactions on Mathematical Software (TOMS) 16.1 (1990): 1-17.

73 Level-3 BLAS Symmetric Matrix-Matrix Multiplication (SYMM) A is a symmetric matrix (square and equal to its transpose); Assume only the lower triangular half is stored; Follow the same algorithm we used for GEMM by modifying the packing routine for matrix A to account for the symmetric nature of A. Example Matrix A is stored as

74 SYMM with One-Level Strassen When partitioning A for one-level Strassen operations, we integrate the symmetric nature of A. A 00 A 01 A 10 A 11

75 SYMM Implementation Platform: BLISlab* What is BLISlab? A Sandbox for Optimizing GEMM that mimics implementation in BLIS A set of exercises that use GEMM to show how high performance can be attained on moderpus with hierarchical memories Used in Prof. Robert van de Geijn s CS 383C Numerical Linear Algebra Contributions Added DSYMM Applied Strassen s algorithm to DGEMM and DSYMM 3 versions: Naive, AB, ABC Strassen improves performance in both DGEMM and DSYMM *

76 DSYMM Results in BLISlab

77 Outline Review of State-of-the-art GEMM in BLIS Strassen s Algorithm Reloaded Theoretical Model and Practical Performance Extension to Other BLAS-3 Operation Extension to Other Fast Matrix Multiplication Conclusion

78 (2,2,2) Strassen Algorithm M 0 := (A 00 +A 11 )(B 00 +B 11 ); C 00 M 0 ; C 11 M 0 ; M 1 := (A 10 +A 11 )B 00 ; C 10 M 1 ; C 11 = M 1 ; M 2 := A 00 (B 01 B 11 ); C 01 M 2 ; C 11 M 2 ; M 3 := A 11 (B 10 B 00 ); C 00 M 3 ; C 10 M 3 ; M 4 := (A 00 +A 01 )B 11 ; C 01 M 4 ; C 00 = M 4 ; M 5 := (A 10 A 00 )(B 00 +B 01 ); C 11 M 5 ; M 6 := (A 01 A 11 )(B 10 +B 11 ); C 00 M 6 ;

79 (3,2,3) Fast Matrix Multiplication M 0 := ( A 01 +A 11 +A 21 )(B 11 +B 12 ); C 02 = M 0 ; M 1 := ( A 00 +A 10 +A 21 )(B 00 B 12 ); C 02 = M 1 ; C 21 = M 1 ; M 2 := ( A 00 +A 10 )( B 01 B 11 ); C 00 =M 2 ;C 01 M 2 ;C 21 M 2 ; M 3 := ( A 10 A 20 +A 21 )( B 02 B 10 +B 12 ); C 02 M 3 ;C 12 M 3 ;C 20 =M 3 ; M 4 := ( A 00 +A 01 +A 10 )( B 00 +B 01 +B 11 ); C 00 =M 4 ;C 01 M 4 ;C 11 M 4 ; M 5 := ( A 10 )( B 00 ); C 00 M 5 ;C 01 =M 5 ;C 10 M 5 ;C 11 =M 5 ;C 20 M 5 ; M 6 := ( A 10 +A 11 +A 20 A 21 )( B 10 B 12 ); C 02 =M 6 ;C 12 =M 6 ; M 7 := ( A 11 )( B 10 ); C 02 M 7 ;C 10 M 7 ;C 12 M 7 ; M 8 := ( A 00 +A 10 +A 20 )( B 01 +B 02 ); C 21 M 8 ; M 9 := ( A 00 +A 01 )( B 00 B 01 ); C 01 M 9 ;C 11 M 9 ; M 10 :=( A 21 )( B 02 +B 12 ); C 02 M 10 ;C 20 M 10 ;C 22 M 10 ; M 11 := ( A 20 A 21 )( B 02 ); C 02 M 11 ;C 12 M 11 ;C 21 =M 11 ;C 22 M 11 ; M 12 := ( A 01 )( B 00 +B 01 +B 10 +B 11 ); C 00 M 12 ; M 13 := ( A 10 A 20 )( B 00 B 02 +B 10 B 12 ); C 20 =M 13 ; M 14 := ( A 00 +A 01 +A 10 A 11 )( B 11 ); C 02 =M 14 ;C 11 =M 14 ; *Austin R. Benson, Grey Ballard. "A framework for practical parallel fast matrix multiplication." PPOPP15.

80

81 Outline Review of State-of-the-art GEMM in BLIS Strassen s Algorithm Reloaded Theoretical Model and Practical Performance Extension to Other BLAS-3 Operation Extension to Other Fast Matrix Multiplication Conclusion

82 To achieve practical high performance of Strassen s algorithm... Conventional Implementations Our Implementations Matrix Size Must be large Matrix Shape Must be square No Additional Workspace Parallelism Usually task parallelism Can be data parallelism

83 Acknowledgement - NSF grants ACI / , CCF Intel Corporation through an Intel Parallel Computing Center (IPCC). - Access to the Maverick and Stampede supercomputers administered by TACC. We thank Field Van Zee, Chenhan Yu, Devin Matthews, and the rest of the SHPC team ( for their supports. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

84 Thank you!