Tutorial Outline. 9:00 am 10:00 am Pre-RTL Simulation Framework: Aladdin. 8:30 am 9:00 am! Introduction! 10:00 am 10:30 am! Break!

Transcription

1 Tutorial Outline Time Topic! 8:30 am 9:00 am! Introduction! 9:00 am 10:00 am Pre-RTL Simulation Framework: Aladdin 10:00 am 10:30 am! Break! 10:30 am 11:00 am! Workload Characterization Tool: WIICA! 11:00 am 12:00 pm! CAD & Benchmarks: HLS & MachSuite! 12:00 pm 2:00 pm! Lunch! 2:00 pm 3:00 pm! Embedded Keynote Talk: Mark Horowitz (Stanford)! 3:00 pm 3:30 pm! Accelerator Selection Tool: Sigil! 3:30 pm 4:00 pm! Break! 4:00 pm 5:00 pm! Hands-on Exercise 1

2 A Pre- RTL, Power- Performance Accelerator Simulator Enabling Large Design Space of Customized Architectures Yakun Sophia Shao, Brandon Reagen, Gu- Yeon Wei, David Brooks Harvard University 2

3 Today s SoC CPU CPU GPU/ DSP Acc Acc Buses Acc Acc Acc Acc Mem Inter- face Acc Acc Acc 3

4 Future Accelerator- Centric Architectures Big Cores Small Cores GPU/DSP Shared Resources Sea of Fine- Grained Accelerators Memory Interface How to decompose an to accelerators? How to rapidly design lots of accelerators? How to design and manage the shared resources? Flexibility Design Cost Programmability 4

5 Aladdin: A pre- RTL, Power- Performance Accelerator Simulator Shared Memory/Interconnect Models Unmodified C- Code Aladdin Power/Area Accelerator Design Parameters (e.g., # FU, mem. BW) Accelerator Specific Datapath Private L1/ Scratchpad Performance Accelerator Simulator Design Accelerator- Rich SoC Fabrics and Memory Systems 5

6 Aladdin: A pre- RTL, Power- Performance Accelerator Simulator Shared Memory/Interconnect Models Unmodified C- Code Aladdin Power/Area Accelerator Design Parameters (e.g., # FU, mem. BW) Accelerator Specific Datapath Private L1/ Scratchpad Performance Accelerator Simulator Design Accelerator- Rich SoC Fabrics and Memory Systems Flexibility Programmability 6

7 Aladdin: A pre- RTL, Power- Performance Accelerator Simulator Shared Memory/Interconnect Models Unmodified C- Code Aladdin Power/Area Accelerator Design Parameters (e.g., # FU, mem. BW) Accelerator Specific Datapath Private L1/ Scratchpad Performance Accelerator Simulator Design Accelerator- Rich SoC Fabrics and Memory Systems Flexibility Programmability Design Assistant Understand Algorithmic- HW Design Space before RTL Design Cost 7

8 Future Accelerator- Centric Architecture Big Cores Small Cores GPU/ DSP Shared Resources Sea of Fine- Grained Accelerators Memory Interface HLS 100 Power (mw) Execution Time (us) 8

9 Future Accelerator- Centric Architecture Big Cores Small Cores GPU/ DSP Shared Resources Sea of Fine- Grained Accelerators Memory Interface ALADDIN HLS 100 Aladdin can rapidly evaluate large design space of accelerator- centric architectures. Power (mw) Execution Time (us) 9

10 Aladdin Overview Op>miza>on Phase C Code Acc Design Parameters Op@mis@c IR Ini@al Idealis@c Dynamic Data Dependence Graph () Program Constrained Resource Constrained Power/Area Models Performance Ac>vity Power/Area Realiza>on Phase 10

11 Aladdin Overview Op>miza>on Phase C Code Op@mis@c IR Ini@al Idealis@c Performance Acc Design Parameters Program Constrained Resource Constrained Power/Area Models Ac>vity Power/Area Realiza>on Phase 11

12 Aladdin is NOT An HLS flow: No RTL is generated. High- level es#mates of power and performance; Aladdin uses fully dynamic analysis to expose algorithmic parallelism for unmodified HLL codes; Limit of ILP study: but is constructed to model accelerators. 12

13 From C to Design Space C Code: for(i=0; i<n; ++i) c[i] = a[i] + b[i]; 13

14 Aladdin Overview Op>miza>on Phase C Code Op@mis@c IR Ini@al Idealis@c Performance Acc Design Parameters Program Constrained Resource Constrained Power/Area Models Ac>vity Power/Area Realiza>on Phase 14

15 From C to Design Space IR Dynamic Trace C Code: for(i=0; i<n; ++i) c[i] = a[i] + b[i]; 0. r0=0 //i = 0 1. r4=load (r0 + r1) //load a[i] 2. r5=load (r0 + r2) //load b[i] 3. r6=r4 + r5 4. store(r0 + r3, r6) //store c[i] 5. r0=r0 + 1 //++i 6. r4=load(r0 + r1) //load a[i] 7. r5=load(r0 + r2) //load b[i] 8. r6=r4 + r5 9. store(r0 + r3, r6) //store c[i] 10. r0 = r0 + 1 //++i 15

16 IR LLVM IR High- level IR: Machine- and ISA- independent Features: Unlimited Registers Simple Opcodes: add, mul, sin, sqrt Only load/store access memory Shao, et al., ISA-Independent Workload Characterization and Implications for Specialized Architecture,! ISPASS, 2013! 16

17 Aladdin Overview Op>miza>on Phase C Code Op@mis@c IR Ini@al Idealis@c Performance Acc Design Parameters Program Constrained Resource Constrained Power/Area Models Ac>vity Power/Area Realiza>on Phase 17

18 From C to Design Space Ini@al C Code: for(i=0; i<n; ++i) c[i] = a[i] + b[i]; IR Trace: 0. r0=0 //i = 0 1. r4=load (r0 + r1) //load a[i] 2. r5=load (r0 + r2) //load b[i] 3. r6=r4 + r5 4. store(r0 + r3, r6) //store c[i] 5. r0=r0 + 1 //++i 6. r4=load(r0 + r1) //load a[i] 7. r5=load(r0 + r2) //load b[i] 8. r6=r4 + r5 9. store(r0 + r3, r6) //store c[i] 10.r0 = r0 + 1 //++i 0. i=0 5. i++ 1. ld a 2. ld b 10. i++ 6. ld a 7. ld b ld a 12. ld b st c st c 4. st c 18

19 Aladdin Overview Op>miza>on Phase C Code Op@mis@c IR Ini@al Idealis@c Performance Acc Design Parameters Program Constrained Resource Constrained Power/Area Models Ac>vity Power/Area Realiza>on Phase 19

20 From C to Design Space Idealis@c C Code: for(i=0; i<n; ++i) c[i] = a[i] + b[i]; IR Trace: 0. r0=0 //i = 0 1. r4=load (r0 + r1) //load a[i] 2. r5=load (r0 + r2) //load b[i] 3. r6=r4 + r5 4. store(r0 + r3, r6) //store c[i] 5. r0=r0 + 1 //++i 6. r4=load(r0 + r1) //load a[i] 7. r5=load(r0 + r2) //load b[i] 8. r6=r4 + r5 9. store(r0 + r3, r6) //store c[i] 10.r0 = r0 + 1 //++i 0. i=0 5. i i ld a 6. ld a 12. ld b 1. ld a 7. ld b ld b st c 0. i=0 5. i i++ 1. ld a 2. ld b 6. ld a 7. ld b 11. ld a 12. ld b st c st c st c st c 14. st c 20

21 From C to Design Space Idealis@c Include applica@on- specific customiza@on strategies. Node- Level: Bit- width Analysis Strength Reduc@on Tree- height Reduc@on Loop- Level: Remove dependences between loop index variables Memory Op@miza@on: Memory- to- Register Conversion Store- Load Forwarding Store Buffer Extensible e.g. Model CAM accelerator by matching nodes in 21

22 Aladdin Overview Op>miza>on Phase C Code Op@mis@c IR Ini@al Idealis@c Performance Acc Design Parameters Program Constrained Resource Constrained Power/Area Models Ac>vity Power/Area Realiza>on Phase 22

23 From C to Design Space One Design Idealis@c 0. i=0 5.i i i++ 0. i=0 Resource Ac@vity 1. ld a 2. ld b st c 6. ld a 7. ld b st c 11. ld a 12. ld b st c 16. ld a 17. ld b st c 1. ld a 2. ld b st c 5.i++ MEM MEM + + MEM Acc Design Parameters: ü Memory BW <= 2 ü 1 Adder 6. ld a 7. ld b st c Cycle MEM MEM + MEM 23

24 From C to Design Space Another Design Idealis@c 0. i=0 5.i i i++ 0. i=0 5.i++ Resource Ac@vity + 1. ld a 2. ld b ld a 7. ld b ld a 12. ld b ld a 17. ld b ld a 2. ld b ld a 7. ld b 8. + MEM MEM MEM MEM st c 9. st c Acc Design Parameters: ü Memory BW <= 4 ü 2 Adders 14. st c 19. st c 4. st c 10. i ld a 12. ld b st c 9. st c 15. i ld a 17. ld b st c MEM + + MEM MEM MEM MEM MEM Cycle + + MEM MEM 24

25 From C to Design Space Realiza@on Phase: - >Power- Perf Constrain the with program and user- defined resource constraints Program Constraints Control Dependence Memory Ambigua@on Resource Constraints Loop- level Parallelism Loop Pipelining Memory Ports # of FUs (e.g., adders, mul@pliers) 25

26 Memory Idealistic optimistically removes all false memory dependences! Input-dependent memory accesses cannot be calculated statically.! 26

27 Memory 0.i=0 for(i=0; i<n; ++i) { bucket[ a[i] & 0x11 ]++; } Input: a[0] = 1; a[1] = 1; a[2] = 1; 1.ld a[0] 2.& 3.ld b[1] 4.b[1]++ 5.st b[1] 27

28 Memory for(i=0; i<n; ++i) { bucket[ a[i] & 0x11 ]++; } Input: a[0] = 1; a[1] = 2; a[2] = 1; 0.i=0 1.ld a[0] 2.& 3.ld b[1] 4.b[1]++ 5.st b[1] 6.i++ 7.ld a[1] 8.& 9.ld b[2] 10.b[2]++ 11.st b[2] 28

29 Memory 0.i=0 6.i++ 12.i++ for(i=0; i<n; ++i) { bucket[ a[i] & 0x11 ]++; } 1.ld a[0] 2.& 7.ld a[1] 8.& 13.ld a[2] 14.& Input: a[0] = 1; a[1] = 2; a[2] = 2; 3.ld b[1] 4.b[1]++ 5.st b[1] 9.ld b[2] 10.b[2]++ 11.st b[2] 15.ld b[2] 16.b[2]++ 17.st b[2] 29

30 Memory 0.i=0 6.i++ 12.i++ for(i=0; i<n; ++i) { bucket[ a[i] & 0x11 ]++; } 1.ld a[0] 2.& 7.ld a[1] 8.& 13.ld a[2] 14.& Input: a[0] = 1; a[1] = 2; a[2] = 2; 3.ld b[1] 4.b[1]++ 5.st b[1] 9.ld b[2] 10.b[2]++ 11.st b[2] 15.ld b[2] 16.b[2]++ 17.st b[2] 30

31 Memory 0.i=0 6.i++ 12.i++ for(i=0; i<n; ++i) { bucket[ a[i] & 0x11 ]++; } 1.ld a[0] 2.& 7.ld a[1] 8.& 13.ld a[2] 14.& Input: a[0] = 1; a[1] = 2; a[2] = 2; 3.ld b[1] 4.b[1]++ 5.st b[1] 9.ld b[2] 10.b[2]++ 11.st b[2] 15.ld b[2] 16.b[2]++ 17.st b[2] 31

32 Memory 0.i=0 6.i++ 12.i++ for(i=0; i<n; ++i) { bucket[ a[i] & 0x11 ]++; } 1.ld a[0] 2.& 7.ld a[1] 8.& 13.ld a[2] 14.& Input: a[0] = 1; a[1] = 2; a[2] = 2; 3.ld b[1] 4.b[1]++ 5.st b[1] 9.ld b[2] 10.b[2]++ 11.st b[2] 15.ld b[2] 16.b[2]++ 17.st b[2] 32

33 Memory 0.i=0 6.i++ 12.i++ for(i=0; i<n; ++i) { bucket[ a[i] & 0x11 ]++; } 1.ld a[0] 2.& 7.ld a[1] 8.& 13.ld a[2] 14.& Input: a[0] = 1; a[1] = 2; a[2] = 2; 3.ld b[1] 4.b[1]++ 5.st b[1] 9.ld b[2] 10.b[2]++ 11.st b[2] 15.ld b[2] 16.b[2]++ 17.st b[2] 33

34 From C to Design Space Power- Performance per Design Power Acc Design Parameters: ü Memory BW <= 4 ü 2 Adders Acc Design Parameters: ü Memory BW <= 2 ü 1 Adder Cycle 34

35 From C to Design Space Design Space of an Algorithm Power Cycle 35

36 Cycle- Level 200 Twiddle Active Functional Units Memory Bandwidth Number of Active Functional Units and Bandwidth FFT8 Shuffle FFT8 Twiddle Shuffle FFT Time (Cycles) 36

37 Power Model Units Power Model Microbenchmarks characterize various FUs. Design Compiler with 40nm Standard Cell Power = (activity i * Pi dynamic ) + Pi leakage 1<i<N SRAM Power Model Commercial register file and SRAM memory compilers with the same 40nm standard cell library 37

38 Aladdin Overview Op>miza>on Phase C Code Op@mis@c IR Ini@al Idealis@c Performance Acc Design Parameters Program Constrained Resource Constrained Power/Area Models Ac>vity Power/Area Realiza>on Phase 38

39 Aladdin Aladdin C Code Power/Area Performance Verilog Design Compiler Ac@vity ModelSim 39

40 Aladdin Aladdin C Code Power/Area Performance RTL Designer HLS C Tuning Vivado HLS Verilog Design Compiler Ac@vity ModelSim 40

41 Benchmarks Type! Benchmark! Description! MD! Pairwise calculation of the L-J Potential! STENCIL! Apply 3x3 filter to an image! SHOC! Benchmark Suite! FFT! GEMM! TRIAD! SORT! 1D 512 FFT! Blocked Matrix Multiply! Single Computation in DOALL loop! Radix Sort! Optimized! HLS! Designs! SCAN! Parallel prefix sum! REDUCTION! Return sum of an array! Proposed! Accelerator! Constructs! NPU! Memcached! HARP! An individual neuron in a network [MICRO 12]! GET function in Memcached [ISCA 13]! Data partition accelerator [ISCA 13]! Hand RTL! Designs! 41

42 Aladdin Time (KCycles) % Aladdin RTL Flow 0 FFT Power (mw) % FFT Aladdin RTL Flow % Area ( mm 2 ) Aladdin RTL Flow 0.0 FFT 42

43 Aladdin Time (KCycles) MD STENCIL FFT GEMM TRIAD SORT SCAN REDUCTION 0.9% Aladdin RTL Flow 4.9% Time (KCycles) NPU HASH HARP Power (mw) Aladdin RTL Flow Power (mw) 2 1 Area ( mm 2 ) MD STENCIL FFT GEMM TRIAD SORT SCAN REDUCTION 6.5% Aladdin RTL Flow Area ( mm 2 ) NPU HASH HARP 0.0 MD STENCIL FFT GEMM TRIAD SORT SCAN REDUCTION 0 NPU HASH HARP 43

44 Aladdin enables rapid design space for accelerators. Aladdin C Code Power/Area Performance RTL Designer HLS C Tuning Vivado HLS Verilog Design Compiler Ac@vity ModelSim 44

45 Algorithm Choices Aladdin generates a design space per algorithm Can use Aladdin to quickly compare the design spaces of algorithms Input Dependent Inputs that exercise all paths of the code Input C Code Aladdin can create for any C code. C constructs that require resources outside the accelerator, such as system calls and dynamic memory alloca@on, are not modeled. 45

46 Aladdin enables pre- RTL of accelerators with the rest of the SoC. gem5 Big Cores... gem5 Small Cores GPGPU- GPU Sim Shared CacL/Orion2 Resources Sea of Fine- Grained Accelerators Memory DRAMSim2 Interface 46

47 Accelerator with Memory System using Aladdin Acc! Cache! Memory! 47

48 Acc! Core! Cache! Modeling Accelerators in an SoC- like Environment Memory! Power (mw) block=16 block=32 Without Memory Contention Time (Million Cycles)

49 Acc! Cache! Core! Modeling Accelerators in a SoC- like Environment Memory! block=16 block= block=16 block=32 Power (mw) Without Memory Contention Power (mw) With Memory Contention Time (Million Cycles) Time (Million Cycles) 49

50 Aladdin: A pre- RTL, Power- Performance Accelerator Simulator Architectures with 1000s of accelerators will be radically different; New design tools are needed. Aladdin enables rapid design space explora@on of future accelerator- centric plavorms. You can find Aladdin at hwp://vlsiarch.eecs.harvard.edu/accelerators 50

51 Tutorial References Y.S. Shao and D. Brooks, ISA-Independent Workload Characterization and its Implications for Specialized Architectures, ISPASS 13.! B. Reagen, Y.S. Shao, G.-Y. Wei, D. Brooks, Quantifying Acceleration: Power/ Performance Trade-Offs of Application Kernels in Hardware, ISLPED 13.! Y.S. Shao, B. Reagen, G.-Y. Wei, D. Brooks, Aladdin: A Pre-RTL, Power- Performance Accelerator Simulator Enabling Large Design Space Exploration of Customized Architectures, ISCA 14.! B. Reagen, B. Adolf, Y.S. Shao, G.-Y. Wei, D. Brooks, MachSuite: Benchmarks for Accelerator Design and Customized Architectures, IISWC 14.! 51