Auto-Vectorization with GCC

Transcription

1 Auto-Vectorization with GCC Hanna Franzen Kevin Neuenfeldt HPAC High Performance and Automatic Computing Seminar on Code-Generation Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 1 / 26

2 Outline 1 Introduction What Is Vectorization What Are The Challenges 2 Structure Of GCC 3 Vectorization 4 Code Generation & Algorithm 5 Summary Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 2 / 26

3 Outline 1 Introduction What Is Vectorization What Are The Challenges 2 Structure Of GCC 3 Vectorization 4 Code Generation & Algorithm 5 Summary Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 3 / 26

4 What Is Vectorization SIMD: Single Instruction Multiple Data Most modern CPUs have SIMD vector registers of mostly 8 to 16 bytes Check support on UNIX (Intel/AMD): cat /proc/cpuinfo grep -E \s(sse mmx)\s Definition Vectorization is the rewriting of loops into vector instructions. Notation: a[i:j] = Elements between index i and j (including). Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 4 / 26

5 Challenges Memory references must (mostly) be consecutive Memory must be aligned to natural vector size boundary Number of iterations must be countable Data Dependences Semantics of the vectorized program must be identical to the original program Example for (int i=1; i<5; i++) { a[i] = a[i-1] + b[i]; // S1 Assume a = [1,2,3,4,5] and b = [5,4,3,2,1]. Sequential: a = [1,5,8,10,11] Vectorized: a = [1,5,5,5,5] Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 5 / 26

6 Challenges Memory references must (mostly) be consecutive Memory must be aligned to natural vector size boundary Number of iterations must be countable Data Dependences Semantics of the vectorized program must be identical to the original program Example for (int i=1; i<5; i++) { a[i] = a[i-1] + b[i]; // S1 Assume a = [1,2,3,4,5] and b = [5,4,3,2,1]. Sequential: a = [1,5,8,10,11] Vectorized: a = [1,5,5,5,5] Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 5 / 26

7 Challenges Memory references must (mostly) be consecutive Memory must be aligned to natural vector size boundary Number of iterations must be countable Data Dependences Semantics of the vectorized program must be identical to the original program Example for (int i=1; i<5; i++) { a[i] = a[i-1] + b[i]; // S1 Assume a = [1,2,3,4,5] and b = [5,4,3,2,1]. Sequential: a = [1,5,8,10,11] Vectorized: a = [1,5,5,5,5] Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 5 / 26

8 Outline 1 Introduction What Is Vectorization What Are The Challenges 2 Structure Of GCC 3 Vectorization 4 Code Generation & Algorithm 5 Summary Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 6 / 26

9 Structure Of GCC vectorization assembly source code Front-end optimizations on trees GIMPLify Middle-end optimizations on GIMPLE expand Back-end optimizations on RTL codegen GIMPLE RTL Vector instructions are platform dependent Different processors have different SIMD instruction sets When built GCC uses machine description file to store which operations are available Intermediate Languages (IL): RTL (Register Transfer Language) IL: GIMPLE Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 7 / 26

10 Structure Of GCC vectorization assembly source code Front-end optimizations on trees GIMPLify Middle-end optimizations on GIMPLE expand Back-end optimizations on RTL codegen GIMPLE RTL Vector instructions are platform dependent Different processors have different SIMD instruction sets When built GCC uses machine description file to store which operations are available Intermediate Languages (IL): RTL (Register Transfer Language) IL: GIMPLE Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 7 / 26

11 Outline 1 Introduction What Is Vectorization What Are The Challenges 2 Structure Of GCC 3 Vectorization 4 Code Generation & Algorithm 5 Summary Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 8 / 26

12 Overview Vectorization Steps Analyze... loop form: exit condition, control-flow,... memory references: compute function that describes memory reference modifications scalar dependence cycles data reference access patterns data reference alignment data reference dependences Determine VF (Vectorization Factor) Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 9 / 26

13 Overview Vectorization Steps Analyze... loop form: exit condition, control-flow,... memory references: compute function that describes memory reference modifications scalar dependence cycles data reference access patterns data reference alignment data reference dependences Determine VF (Vectorization Factor) Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 9 / 26

14 Scalar Dependences Use of scalar variables (no arrays, pointers) Example: Reduction Example for (i=0; i<n; i++) { sum += A[i]; Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 10 / 26

15 Scalar Dependences Use of scalar variables (no arrays, pointers) Example: Reduction Example for (i=0; i<n; i++) { sum += A[i]; Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 10 / 26

16 Scalar Dependences: Reduction Example for (i=0; i<n; i++) { sum += A[i]; Intel Core 2Duo B950 VF = 4 N = 1024, 256 partial sums Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 11 / 26

17 Access Patterns Most SIMD instruction sets require consecutive memory access Otherwise permutations on data is needed Some SIMD instruction sets support certain access patterns (e.g. odd/even for complex numbers) Costs of permutations are critical to the worth of the vectorization Example for (i=0; i<n; i++) { A[i*2] = x; Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 12 / 26

18 Access Patterns Most SIMD instruction sets require consecutive memory access Otherwise permutations on data is needed Some SIMD instruction sets support certain access patterns (e.g. odd/even for complex numbers) Costs of permutations are critical to the worth of the vectorization Example for (i=0; i<n; i++) { A[i*2] = x; Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 12 / 26

19 Alignment Access on an unaligned memory location is needed. Target platform needs at least one capability: memory read/write to unaligned memory location general vector shuffling ability specialized alignment support Most SIMD platforms load VS bytes from closest previous aligned address (e.g. AltiVec, Alpha). Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 13 / 26

20 Data Dependences S 1 δ 0 1 δ 1 S 2 δ 1 1, δ 1 δ δ 2, δ 1, δ 1 1 δ 1 1 δ δ 1 S 3 δ 0 1 δ 0 1 S 4 Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 14 / 26

21 Outline 1 Introduction What Is Vectorization What Are The Challenges 2 Structure Of GCC 3 Vectorization 4 Code Generation & Algorithm 5 Summary Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 15 / 26

22 Vectorization Algorithm I for (i=1; i<=100; i++) { X[i] = Y[i] + 10; // S1 for (j=1; j<=100; j++) { B[j] = A[j][N]; // S2 for (k=1; k<=100; k++) { A[j+1][k] = B[j] + C[j][k] // S3 Y[i+j] = A[j+1][N] // S4 codegen(r,k,d), R region, k min nesting level, D dependence graph Base for gcc s vectorization procedure Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 16 / 26

23 Vectorization Algorithm I for (i=1; i<=100; i++) { X[i] = Y[i] + 10; // S1 for (j=1; j<=100; j++) { B[j] = A[j][N]; // S2 for (k=1; k<=100; k++) { A[j+1][k] = B[j] + C[j][k] // S3 Y[i+j] = A[j+1][N] // S4 codegen(r,k,d), R region, k min nesting level, D dependence graph Base for gcc s vectorization procedure Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 16 / 26

24 Vectorization Algorithm II S 1 δ 0 1 δ 1 S 2 δ 1 1, δ 1 δ δ 2, δ 1, δ 1 1 δ 1 1 δ δ 1 S 3 δ 0 1 δ 0 1 S 4 Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 17 / 26

25 Vectorization Algorithm III S 2 for (i=1; i<=100; i++) { codegen({s2,s3,s4,2,d2) X[1:100] = Y[1:100] + 10; δ δ 2 δ S 3 S 4 Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 18 / 26

26 Vectorization Algorithm IV for (i=1; i<=100; i++) { for (j=1; j<=100; j++) { codegen({s2,s3,3,d3) Y[i+1:i+100] = A[2:101][N]; X[1:100] = Y[1:100] + 10; S 2 δ S 3 Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 19 / 26

27 Vectorization Algorithm V for (i=1; i<=100; i++) { for (j=1; j<=100; j++) { B[j] = A[j][N]; A[j+1][1:100] = B[j] + C[j][1:100]; Y[i+1:i+100] = A[2:101][N]; X[1:100] = Y[1:100] + 10; S 2 δ S 3 Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 20 / 26

28 Code Generation vectorization assembly source code Front-end optimizations on trees GIMPLify Middle-end optimizations on GIMPLE expand Back-end optimizations on RTL codegen GIMPLE RTL Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 21 / 26

29 Outline 1 Introduction What Is Vectorization What Are The Challenges 2 Structure Of GCC 3 Vectorization 4 Code Generation & Algorithm 5 Summary Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 22 / 26

30 Overview scalar dependence cycles data reference dependences data reference access patterns data reference alignment Still in progress: Cost model MIMD support Enhancement of existing techniques Much more... Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 23 / 26

31 Overview scalar dependence cycles data reference dependences data reference access patterns data reference alignment Still in progress: Cost model MIMD support Enhancement of existing techniques Much more... Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 23 / 26

32 Summary: Speed Enhancement Compile code like: gcc example.c -o example_vect -O3 /usr/bin/time -f "%E time, %P CPU"./example 0:00.13 time, 97% CPU /usr/bin/time -f "%E time, %P CPU"./example_vect 0:00.03 time, 93% CPU Speed enhancement of factor 4 VF = 4. Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 24 / 26

33 Summary: Speed Enhancement Compile code like: gcc example.c -o example_vect -O3 /usr/bin/time -f "%E time, %P CPU"./example 0:00.13 time, 97% CPU /usr/bin/time -f "%E time, %P CPU"./example_vect 0:00.03 time, 93% CPU Speed enhancement of factor 4 VF = 4. Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 24 / 26

34 Summary: Speed Enhancement Compile code like: gcc example.c -o example_vect -O3 /usr/bin/time -f "%E time, %P CPU"./example 0:00.13 time, 97% CPU /usr/bin/time -f "%E time, %P CPU"./example_vect 0:00.03 time, 93% CPU Speed enhancement of factor 4 VF = 4. Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 24 / 26

35 GCC Code GCC 4.9 Source gcc/tree-vect* well documented LOC Introduced on Jan 1st, 2004 in the lno branch Initiated by Dorit Nuzman (IBM) Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 25 / 26

36 References Dorit Nuzman, Richard Henderson (2006) Multi-platform Auto-vectorization Proceedings of the International Symposium on Code Generation and Optimization CGO 06, Dorit Naishlos (2004) Autovectorization in GCC Dorit Nuzman, Ayal Zaks (2006) Autovectorization in GCC - Two years later GCC-Summit-Proceedings 2006, Alexandre Eichenberger, Peng Wu, Kevin O Brien (2004) Vectorization for SIMD Architectures with Alignment Constraints Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation, Kennedy, Ken and Allen, John R (2001) Optimizing compilers for modern architectures: a dependence-based approach Hanna Franzen, Kevin Neuenfeldt (RWTH) Auto-Vectorization with GCC Seminar on Code-Generation 26 / 26