VLIW/EPIC: Statically Scheduled ILP

Transcription

1 6.823, L21-1 VLIW/EPIC: Statically Scheduled ILP Computer Science & Artificial Intelligence Laboratory Massachusetts Institute of Technology Based on the material prepared by Krste Asanovic and Arvind

2 Little s Law Parallelism = Throughput * Latency or N = T L 6.823, L21-2 Throughput per Cycle One Operation Latency in Cycles

3 6.823, L21-3 Example Pipelined ILP Machine Max Throughput, Six Instructions per Cycle Latency in Cycles Two Integer Units, Single Cycle Latency Two Load/Store Units, Three Cycle Latency One Pipeline Stage Two Floating-Point Units, Four Cycle Latency How much instruction-level parallelism (ILP) required to keep machine pipelines busy? T = 6 L = (2x1 + 2x3 + 2x4) 2 2 = 2 N = 6 2 =

4 6.823, L21-4 Superscalar Control Logic Scaling Issue Group Issue Width W Previously Issued Instructions Lifetime L Each issued instructions must make interlock checks against W*L instructions, i.e., growth in interlocks W*(W*L) For in-order machines, L is related to pipeline latencies For out-of-order machines, L also includes time spent in instruction buffers (instruction window or ROB) As W increases, larger instruction window is needed to find enough parallelism to keep machine busy => greater L => Out-of-order control logic grows faster than W 2 (~W 3 )

5 6.823, L21-5 Out-of-Order Control Complexity: MIPS R10000 Control Logic [ SGI/MIPS Technologies Inc., 1995 ]

6 Sequential ISA Bottleneck 6.823, L21-6 Sequential source code a = foo(b); for (i=0, i< Superscalar compiler Sequential machine code Find independent operations Schedule operations Superscalar processor Check instruction dependencies Schedule execution

7 6.823, L21-7 VLIW: Very Long Instruction Word Int Op 1 Int Op 2 Mem Op 1 Mem Op 2 FP Op 1 FP Op 2 Two Integer Units, Single Cycle Latency Two Load/Store Units, Three Cycle Latency Two Floating-Point Units, Four Cycle Latency Multiple operations packed into one instruction Each operation slot is for a fixed function Constant operation latencies are specified Architecture requires guarantee of: Parallelism within an instruction => no x-operation RAW check No data use before data ready => no data interlocks

8 6.823, L21-8 VLIW Compiler Responsibilities The compiler: Schedules to maximize parallel execution Guarantees intra-instruction parallelism Schedules to avoid data hazards (no interlocks) Typically separates operations with explicit NOPs

9 6.823, L21-9 Early VLIW Machines FPS AP120B (1976) scientific attached array processor first commercial wide instruction machine hand-coded vector math libraries using software pipelining and loop unrolling Multiflow Trace (1987) commercialization of ideas from Fisher s Yale group including trace scheduling available in configurations with 7, 14, or 28 operations/instruction 28 operations packed into a 1024-bit instruction word Cydrome Cydra-5 (1987) 7 operations encoded in 256-bit instruction word rotating register file

10 Loop Execution 6.823, L21-10 for (i=0; i<n; i++) B[i] = A[i] + C; Compile loop: Int1 Int 2 M1 M2 FP+ add r1 ld FPx loop: ld f1, 0(r1) add r1, 8 fadd f2, f0, f1 Schedule fadd sd f2, 0(r2) add r2, 8 add r2 bne sd bne r1, r3, loop How many FP ops/cycle? 1 fadd / 8 cycles = 0.125

11 Loop Unrolling 6.823, L21-11 for (i=0; i<n; i++) B[i] = A[i] + C; for (i=0; i<n; i+=4) { } B[i] = A[i] + C; Unroll inner loop to perform 4 iterations at once B[i+1] = A[i+1] + C; B[i+2] = A[i+2] + C; B[i+3] = A[i+3] + C; Need to handle values of N that are not multiples of unrolling factor with final cleanup loop

12 Scheduling Loop Unrolled Code 6.823, L21-12 Unroll 4 ways loop: ld f1, 0(r1) Int1 Int 2 M1 M2 FP+ FPx ld f2, 8(r1) ld f3, 16(r1) ld f4, 24(r1) add r1, 32 fadd f5, f0, f1 fadd f6, f0, f2 fadd f7, f0, f3 fadd f8, f0, f4 sd f5, 0(r2) sd f6, 8(r2) sd f7, 16(r2) sd f8, 24(r2) add r2, 32 bne r1, r3, loop loop: Schedule How many FLOPS/cycle? ld f1 ld f2 ld f3 add r1 ld f4 fadd f5 fadd f6 fadd f7 fadd f8 sd f5 sd f6 sd f7 add r2 bne sd f8 4 fadds / 11 cycles = 0.36

13 Unroll 4 ways first loop: ld f1, 0(r1) ld f2, 8(r1) ld f3, 16(r1) ld f4, 24(r1) add r1, 32 fadd f5, f0, f1 fadd f6, f0, f2 fadd f7, f0, f3 fadd f8, f0, f4 sd f5, 0(r2) sd f6, 8(r2) sd f7, 16(r2) add r2, 32 sd f8, -8(r2) bne r1, r3, loop How many FLOPS/cycle? Software Pipelining prolog iterate epilog 4 fadds / 4 cycles = 1 loop: Int1 Int 2 M1 M2 FP+ FPx ld f1 ld f2 ld f3 add r1 ld f4 ld f1 fadd f5 ld f2 fadd f6 ld f3 fadd f7 add r1 ld f4 fadd f8 ld f1 sd f5 fadd f5 ld f2 sd f6 fadd f6 add r2 ld f3 sd f7 fadd f7 add r1 bne ld f4 sd f8 fadd f8 sd f5 fadd f5 sd f6 fadd f6 add r2 sd f7 fadd f7 bne sd f8 fadd f8 sd f , L21-13

14 Software Pipelining vs. Loop Unrolling 6.823, L21-14 performance Loop Unrolled Wind-down overhead Startup overhead performance Loop Iteration Software Pipelined time Loop Iteration time Software pipelining pays startup/wind-down costs only once per loop, not once per iteration

15 What if there are no loops? 6.823, L21-15 Basic block Branches limit basic block size in control-flow intensive irregular code Difficult to find ILP in individual basic blocks

16 Trace Scheduling [ Fisher,Ellis] 6.823, L21-16 Pick string of basic blocks, a trace, that represents most frequent branch path Use profiling feedback or compiler heuristics to find common branch paths Schedule whole trace at once Add fixup code to cope with branches jumping out of trace

17 6.823, L21-17 Problems with Classic VLIW Object-code compatibility have to recompile all code for every machine, even for two machines in same generation Object code size instruction padding wastes instruction memory/cache loop unrolling/software pipelining replicates code Scheduling variable latency memory operations caches and/or memory bank conflicts impose statically unpredictable variability Knowing branch probabilities Profiling requires an significant extra step in build process Scheduling for statically unpredictable branches optimal schedule varies with branch path

18 6.823, L21-18 VLIW Instruction Encoding Group 1 Group 2 Group 3 Schemes to reduce effect of unused fields Compressed format in memory, expand on I-cache refill» used in Multiflow Trace» introduces instruction addressing challenge Mark parallel groups» used in TMS320C6x DSPs, Intel IA-64 Provide a single-op VLIW instruction» Cydra-5 UniOp instructions

19 Rotating Register Files 6.823, L21-19 Problems: Scheduled loops require lots of registers, Lots of duplicated code in prolog, epilog ld r1, () add r2, r1, #1 ld r1, () st r2, () add r2, r1, #1 ld r1, () st r2, () add r2, r1, #1 st r2, () Prolog ld r1, () ld r1, () add r2, r1, #1 Loop ld r1, () add r2, r1, #1 st r2, () Epilog add r2, r1, #1 st r2, () st r2, () Solution: Allocate new set of registers for each loop iteration

20 Rotating Register File 6.823, L21-20 R1 RRB=3 + P7 P6 P5 P4 P3 P2 P1 P0 Rotating Register Base (RRB) register points to base of current register set. Value added on to logical register specifier to give physical register number. Usually, split into rotating and non-rotating registers. Prolog Loop Epilog ld r1, () ld r1, () ld r1, () add r3, r2, #1 add r3, r2, #1 add r2, r1, #1 st r4, () st r4, () st r4, () dec RRB dec RRB bloop dec RRB dec RRB Loop closing branch decrements RRB

21 Rotating Register File (Previous Loop Example) 6.823, L21-21 Three cycle load latency encoded as difference of 3 in register specifier number (f4 - f1 = 3) Four cycle fadd latency encoded as difference of 4 in register specifier number (f9 f5 = 4) ld f1, () fadd f5, f4,... sd f9, () bloop ld P9, () fadd P13, P12, sd P17, () bloop ld P8, () fadd P12, P11, sd P16, () bloop ld P7, () fadd P11, P10, sd P15, () bloop ld P6, () fadd P10, P9, sd P14, () bloop ld P5, () fadd P9, P8, sd P13, () bloop ld P4, () fadd P8, P7, sd P12, () bloop ld P3, () fadd P7, P6, sd P11, () bloop ld P2, () fadd P6, P5, sd P10, () bloop RRB=8 RRB=7 RRB=6 RRB=5 RRB=4 RRB=3 RRB=2 RRB=1

22 Cydra-5: Memory Latency Register (MLR) 6.823, L21-22 Problem: Loads have variable latency Solution: Let software choose desired memory latency Compiler schedules code for maximum load-use distance Software sets MLR to latency that matches code schedule Hardware ensures that loads take exactly MLR cycles to return values into processor pipeline Hardware buffers loads that return early Hardware stalls processor if loads return late

23 Five-minute break to stretch your legs 6.823, L21-23

24 6.823, L21-24 Intel EPIC IA-64 EPIC is the style of architecture (cf. CISC, RISC) Explicitly Parallel Instruction Computing IA-64 is Intel s chosen ISA (cf. x86, MIPS) IA-64 = Intel Architecture 64-bit An object-code compatible VLIW Itanium (aka Merced) is first implementation (cf. 8086) First customer shipment expected 1997 (actually 2001) McKinley, second implementation shipped in 2002

25 IA-64 Instruction Format 6.823, L21-25 Instruction 2 Instruction 1 Instruction 0 Template 128-bit instruction bundle Template bits describe grouping of these instructions with others in adjacent bundles Each group contains instructions that can execute in parallel bundle j-1 bundle j bundle j+1 bundle j+2 group i-1 group i group i+1 group i+2

26 6.823, L21-26 IA-64 Registers 128 General Purpose 64-bit Integer Registers 128 General Purpose 64/80-bit Floating Point Registers 64 1-bit Predicate Registers GPRs rotate to reduce code size for software pipelined loops

27 IA-64 Predicated Execution 6.823, L21-27 Problem: Mispredicted branches limit ILP Solution: Eliminate hard to predict branches with predicated execution Almost all IA-64 instructions can be executed conditionally under predicate Instruction becomes NOP if predicate register false b0: b1: b2: b3: Inst 1 Inst 2 br a==b, b2 Inst 3 Inst 4 br b3 Inst 5 Inst 6 Inst 7 Inst 8 if else then Four basic blocks Predication Inst 1 Inst 2 p1,p2 <- cmp( a==b) (p1) Inst 3 (p2) Inst 5 (p1) Inst 4 (p2) Inst 6 Inst 7 Inst 8 One basic block Mahlke et al, ISCA95: On average >50% branches removed

28 6.823, L21-28 Predicate Software Pipeline Stages Single VLIW Instruction (p1) ld r1 (p2) add r3 (p3) st r4 (p1) bloop Dynamic Execution (p1) ld r1 (p1) bloop (p1) ld r1 (p2) add r3 (p1) bloop (p1) ld r1 (p2) add r3 (p3) st r4 (p1) bloop (p1) ld r1 (p2) add r3 (p3) st r4 (p1) bloop (p1) ld r1 (p2) add r3 (p3) st r4 (p1) bloop (p2) add r3 (p3) st r4 (p1) bloop (p3) st r4 (p1) bloop Software pipeline stages turned on by rotating predicate registers Î Much denser encoding of loops

29 Fully Bypassed Datapath 6.823, L21-29 stall PC for JAL,... 0x4 Add nop ASrc IR E M W IR 31 IR PC addr inst Inst Memory IR D we rs1 rs2 rd1 ws wd rd2 GPRs Imm Ext BSrc A B MD1 ALU Y MD2 we addr rdata Data Memory wdata wdata R Where does predication fit in?

30 IA-64 Speculative Execution 6.823, L21-30 Problem: Branches restrict compiler code motion Solution: Speculative operations that don t cause exceptions Inst 1 Inst 2 br a==b, b2 Load.s r1 Inst 1 Inst 2 br a==b, b2 Speculative load never causes exception, but sets poison bit on destination register Load r1 Use r1 Inst 3 Can t move load above branch because might cause spurious exception Chk.s r1 Use r1 Inst 3 Check for exception in original home block jumps to fixup code if exception detected Particularly useful for scheduling long latency loads early

31 IA-64 Data Speculation 6.823, L21-31 Problem: Possible memory hazards limit code scheduling Solution: Hardware to check pointer hazards Inst 1 Inst 2 Store Load r1 Use r1 Inst 3 Can t move load above store because store might be to same address Load.a r1 Inst 1 Inst 2 Store Load.c Use r1 Inst 3 Data speculative load adds address to address check table Store invalidates any matching loads in address check table Check if load invalid (or missing), jump to fixup code if so Requires associative hardware in address check table

32 Clustered VLIW 6.823, L21-32 Cluster Interconnect Local Regfile Local Regfile Memory Interconnect Cache/Memory Banks Cluster Divide machine into clusters of local register files and local functional units Lower bandwidth/higher latency interconnect between clusters Software responsible for mapping computations to minimize communication overhead Exists in some superscalar processors,.e.g., Alpha Common in commercial embedded processors, examples include TI C6x series DSPs, and HP Lx processor

33 6.823, L21-33 Limits of Static Scheduling Unpredictable branches Variable memory latency (unpredictable cache misses) Code size explosion Compiler complexity Question: How applicable are the VLIW-inspired techniques to traditional RISC/CISC processor architectures?

34 Thank you! 6.823, L21-34