COSC3330 Computer Architecture Lecture 7. Datapath and Performance

Size: px

Start display at page:

Download "COSC3330 Computer Architecture Lecture 7. Datapath and Performance"

Lucas York
6 years ago
Views:

1 COSC3330 Computer Architecture Lecture 7. Datapath and Performance Instructor: Weidong Shi (Larry), PhD Computer Science Department University of Houston

2 Datapath Performance 2

Datapath Control Signals Clock 5 5 5 Zwa Xra Yra Zdi Logical Flag 0001: AND 0011: A 0101: B 0110: XOR 0111: OR RegFile we Ydo Xdo Sign-extended immediate Shift Type (ST) 00: No Shift 01: Logical 10:

3 Datapath Control Signals Clock Zwa Xra Yra Zdi Logical Flag 0001: AND 0011: A 0101: B 0110: XOR 0111: OR RegFile we Ydo Xdo Sign-extended immediate Shift Type (ST) 00: No Shift 01: Logical 10: Arithmetic 11: Rotate imm enable 1 0 Memory A B ā/s 00: AU 2 LF 01: LU ALS ALU ST SD 10: SU 11: Disable ALU Shift Direction (SD) 0: Left 1: Right st enable 4 2 ld enable msel Address Data r/w load $Z, ($X) store $Y, ($X)

4 A Simple Processor Instruction Register Program Counter Next PC gen Microcode Memory (Single-Cycle Implementation) X 5 Y 5 Z 5 imm 16 imm_en we ALS 2 ā/s LF 4 ST 2 SD ld_en st_en ṝ/w msel Single Cycle Datapath Memory

5 A Simple Processor add $4, $3, $ opcode rs rt rd shamt funct Microcode Memory X=00011 Y=00010 Z=00100 imm=0 imm_en=0 we=1 ALS=00 ā/s=0 LF=0000 ST=00 SD=0 ld_en=0 st_en=0 ṝ/w=0 msel=0 Single Cycle Datapath Memory

6 Microcode Control (1) Clear memory location 100, 104 (r0 hardwired to 0) Datapath Control Signals instruction sequence X (5) Y (5) Z (5) w e Imm_ en Imm_val ALS ā/s LF ST SD ld_e n st_e n ṝ/w msel li r1,100 sw r0, (r1) addi r1,r1,4 sw r0, (r1) ALS 00: AU 01: LU 10: SU 11: Disable ALU Logical Flag (LF) 0001: AND 0011: A 0101: B 0110: XOR 0111: OR Shift Type (ST) 00: No Shift 01: Logical 10: Arithmetic 11: Rotate Shift Direction (SD) 0: Left 1: Right

7 Microcode Control (1) Clear memory location 100, 104 (r0 hardwired to 0) instruct ion sequenc e X (5) Y (5) Z (5) we Im m_ en Imm_val ALS ā/s LF ST SD ld_e n st_e n ṝ/w msel li r1,100 sw r0, (r1) addi r1,r1,4 sw r0, (r1) x x x x 0101 x x 0 0 x x 0 0 x 11 x x x x x x x x x 0 0 X x 0 0 x 11 x x x x ALS 00: AU 01: LU 10: SU 11: Disable ALU Logical Flag (LF) 0001: AND 0011: A 0101: B 0110: XOR 0111: OR Shift Type (ST) 00: No Shift 01: Logical 10: Arithmetic 11: Rotate Shift Direction (SD) 0: Left 1: Right

8 Microcode Control (2) copy 4-byte data from 0xF000 to 0xA100 clear data at 0xF000 instruction sequence X (5) Y (5) Z (5) w e Imm_ en Imm_val ALS ā/s LF ST SD ld_e n st_e n ṝ/w msel li r5, 0xF000 lw r6, (r5) li r7, 0xA100 sw r6, (r7) sw r0, (r5) ALS 00: AU 01: LU 10: SU 11: Disable ALU Logical Flag (LF) 0001: AND 0011: A 0101: B 0110: XOR 0111: OR Shift Type (ST) 00: No Shift 01: Logical 10: Arithmetic 11: Rotate Shift Direction (SD) 0: Left 1: Right

9 Microcode Control (2) instru ction sequen ce li r5, 0xF000 lw r6, (r5) li r7, 0xA10 0 sw r6, (r7) sw r0, (r5) copy 4-byte data from 0xF000 to 0xA100 clear data at 0xF000 X (5) Y (5) Z (5) w e Imm _en Imm_v al AL S ā/s LF ST SD ld_ en X X xF X 0101 X X 0 0 X X X 11 X X X X X X xA X 0101 X X 0 0 X X 0 0 X 11 X X X X X 0 0 X 11 X X X X st_ en ṝ/ w msel ALS 00: AU 01: LU 10: SU 11: Disable ALU Logical Flag (LF) 0001: AND 0011: A 0101: B 0110: XOR 0111: OR Shift Type (ST) 00: No Shift 01: Logical 10: Arithmetic 11: Rotate Shift Direction (SD) 0: Left 1: Right

10 Clock Zwa Xra Yra Zdi Logical Flag 0001: AND 0011: A 0101: B 0110: XOR 0111: OR Datapath Control Signals RegFile we Ydo Xdo Sign-extended immediate Shift Type (ST) 00: No Shift 01: Logical 10: Arithmetic 11: Rotate imm enable 1 0 Memory A B ā/s 00: AU 2 LF 01: LU ALS ALU ST SD 10: SU 11: Disable ALU Shift Direction (SD) 0: Left 1: Right st enable 4 2 ld enable msel Address Data r/w load $Z, 16 ($X) store $Y, 4 ($X)

11 A Simple Processor Instruction Register Program Counter Next PC gen Microcode Memory (Single-Cycle Implementation) X 5 Y 5 Z 5 imm 16 imm_en we ALS 2 ā/s LF 4 ST 2 SD ld_en st_en ṝ/w msel Single Cycle Datapath Memory

12 Instruction Fetching (PC Update) Next PC generation Program Counter Instruction Register addr Memory data x RegFile Datapath Microcode ROM

13 Absolute vs. Relative Instruction Addressing The next PC address is given as an absolute value PC address = <given address> Jump class examples J LABEL JR $r An offset relative to the current PC address is given instead of an absolute address PC address = <current PC address> + <offset> Branch class examples bne $src, $dest, LABEL beq $src, $dest, LABEL

14 Sequential Instruction Fetch 4 + Program Counter Instruction Register addr Memory data x RegFile Datapath Microcode ROM

15 Branch Support Offset (from ROM) ext 4 beq 1 mux bne (if true) 0 + Program Counter Instruction Register addr Memory data x RegFile Datapath Microcode ROM

(if true) 1 4 mux 0 + 0 mux Program Counter

16 Branch and Jump Support rs Offset (from ROM) Target addr (from ROM) jr/j ext 1 mux 0 jr j ext beq 1 bne (if true) 1 4 mux mux Program Counter Instruction Register addr Memory data x RegFile Datapath Microcode ROM

17 Performance Measure, Report, and Summarize Make intelligent choices See through the marketing hype Key to understanding underlying organizational motivation Why is some hardware better than others for different programs? What factors of system performance are hardware related? (e.g., Do we need a new machine, or a new operating system?)

18 Which of These Airplanes Has the Best Performance? Airplane Passengers Range (mi) Speed (mph) Boeing Boeing BAC/Sud Concorde Douglas DC How much faster is the Concorde compared to the 747? How much bigger is the 747 than the Douglas DC-8?

19 Computer Performance: TIME, TIME, TIME Response Time (latency) How long does it take for my job to run? How long does it take to execute a job? How long must I wait for the database query? Throughput How many jobs can the machine run at once? What is the average execution rate? How much work is getting done?

20 Clock Cycles Instead of reporting program execution time in seconds, we often use cycles seconds program cycles program seconds cycle Clock ticks indicate when to start activities (one abstraction): cycle time = time between ticks = seconds per cycle clock rate = cycles per second (1 Hz = 1 cycle/sec)

21 How to Improve Performance seconds program cycles program seconds cycle So, to improve performance (everything else being equal) you can either the # of required cycles for a program, or the clock cycle time or, said another way, the clock rate.

22 How Many Cycles Are Required for a Program? 1st instruction 2nd instruction 3rd instruction 4th 5th 6th... Could assume that # of cycles = # of instructions? time This assumption is incorrect, different instructions take different amounts of time on different machines.

23 Different Numbers of Cycles for Different Instructions time Multiplication takes more time than addition Floating point operations take longer than integer ones Accessing memory takes (in general) more time than accessing registers Important point: changing the cycle time often changes the number of cycles required for various instructions (more later)

24 Performance Performance is determined by execution time Do any of the other variables equal performance? # of cycles to execute program? # of instructions in program? # of cycles per second? (frequency) average # of cycles per instruction (CPI)? average # of instructions per second? Common pitfall: thinking one of the variables is indicative of performance when it really isn t.

25 CPI Example (CPI - average # of cycles per instruction) Suppose we have two implementations of the same instruction set architecture (ISA). For some program, Machine A has a clock cycle time of 10 ns. and a CPI of ns * 2.0 * # of instructions Machine B has a clock cycle time of 20 ns. and a CPI of ns * 1.2 * # of instructions What machine is faster for this program?

26 # of Instructions Example A compiler designer is trying to decide between two code sequences for a particular machine. Class A Instruction Class B Instruction Class C Instruction One Cycle Two Cycles Three Cycles Class A Instructions Class B Instructions Class C Instructions 2 of A 1 of B 2 of C 4 of A 1 of B 1 of C Total Instructions 5 6 Total Cycles 2+1*2+2*3 = *2+1*3 = 9 Which sequence will be faster? What is the CPI for each sequence?

27 Amdahl's Law Execution Time After Improvement = Execution Time Unaffected +( Execution Time Affected / Amount of Improvement ) Time before Improvement Time after Improvement

$mode is limited by the fraction the faster mode can be applied.$

28 Amdahl s Law Speed-up = Perf new / Perf old =Exec_time old / Exec_time new = Performance improvement from using faster mode is limited by the fraction the faster mode can be applied. (1 - f) T old f Gene Amdahl 1 (1 f ) f P (1 - f)t new f / P

Amdahl s Law Analogy Spring Break Driving from

you can save compared against driving all the

29 Amdahl s Law Analogy Spring Break Driving from Houston to South Padre Island 60 miles/hr from Houston to Kingsville 120 miles/hr from Kingsville to South Padre Island How much time you can save compared against driving all the way at 60 miles/hr from Houston to Padre Island? about 5hr 10min vs. 6hr 10min Key is to speed up the biggie portion, i.e. speed up frequently executed blocks

30 Example Suppose a program runs in 100 seconds on a machine, with multiply responsible for 80 seconds of this time. How much do we have to improve the speed of multiplication if we want the program to run 4 times faster? 20S 80S 5S Principle: Make the common case fast

to use GPU for enhancing performance Part of the calculation (0.

31 Example A Finance Company developed an in-house option pricing SW based on Monte Carlo method (computing intensive) Take 1 second to finish Monte Carlo Consider to use GPU for enhancing performance Part of the calculation (0.6s out of the 1s) can be offloaded to GPU (6x speedup) 0.4S 0.6s what will the total speedup be?

32 Amdahl s Law Accelerate HD encoding using GPU Assume 50% time spent on ME Overall performance speedup ME offloaded to GPU, F = 50% The rest on CPU

33 Remember Performance is specific to a particular program Total execution time is a consistent summary of performance For a given architecture performance increases come from: increases in clock rate (without adverse CPI affects) improvements in processor organization that lower CPI compiler enhancements that lower CPI and/or instruction count Pitfall: expecting improvement in one aspect of a machine s performance to affect the total performance

34 Speed-up Parallelism vs. Speedup 100 P=1 P=2 P=4 P=8 P=16 P= P=64 Amdahl's Law speed-up as a function of parallelism x 1.11x 1.33x Code portion in Faster mode (f) 34

Gustafson s Law Amdahl s Law killed massive parallel processing (MPP) Gustafson came to rescue John Gustafson Seq T new Parallel Seq T old P * Parallel

35 Gustafson s Law Amdahl s Law killed massive parallel processing (MPP) Gustafson came to rescue John Gustafson Seq T new Parallel Seq T old P * Parallel Time Assume: Seq + Parallel = 1 (T new ) Speedup = Seq + p * (1 Seq) where p=parallel factor If Seq diminishes with increased problem size, Speedup p 35

36 36

37 50mph 100 miles 50mph 100 miles Gustafson s Law 100mph 2nd 100 miles 100mph 2nd 100 miles 66.67mph 100mph 3rd 100 miles 75mph 50mph 100 miles 100mph 2nd 100 miles 100mph 3rd 100 miles 100mph 4th 100 miles 80mph Suppose a car has already been travelling for some time at speed of less than 50km/h, and when given enough time and distance to travel, the car s average speed can reach 100km/h as long as it drives faster than 100 km/h for some time. And also the average speed can reach 120km/h and even 150km/h as long as it drives fast enough in the following part 37

38 Amdahl versus Gustafson Who is right? 38

39 Amdahl versus Gustafson Amdahl s presumption of fixed data size Both laws are in fact different perspective over the same truth one sees data size as fixed and the other sees the relation as a function of data size 39

40 Additional Example of Performance Evaluation Operation Frequency Clock cycle count ALU Ops (regreg) 43% 1 Loads 21% 2 Stores 12% 2 Branches 24% 2 Assume 25% of the ALU ops directly use a loaded operand that is not used again. We propose adding ALU instructions that have one src operand in memory. These new reg-mem instructions spend 2 clock cycles. Also assume that the extended instruction set increase branch s clock by 1, but no impact to cycle time. Would this change improve performance? 40

41 Additional Example of Performance Evaluation Operation Frequency Clock cycle count ALU Ops (regreg) 43% 1 Loads 21% 2 Stores 12% 2 Branches 24% 2 Assume 25% of the ALU ops directly use a loaded operand that is not used again. We propose adding ALU instructions that have one src operand in memory. These new reg-mem instructions spend 2 clock cycles. Also assume that the extended instruction set increase branch s clock by 1, but no impact to cycle time. Would this change improve performance? Cycles old * Cycles new ( ) 1 ( *0.43)*2 0.12*2 0.24*

42 Additional Example of Performance Evaluation FP instructions = 25% Average CPI of FP instructions = 4.0 Average CPI of other instructions = 1.33 FPSQRT = 2% of all instructions, CPI of FPSQRT = 20 Design Option 1: decrease the CPI of FQSQRT to 2 Design Option 2: decease the average CPI of all FP instructions to

43 Additional Example of Performance Evaluation FP instructions = 25% Average CPI of FP instructions = 4.0 Average CPI of other instructions = 1.33 FPSQRT = 2% of all instructions, CPI of FPSQRT = 20 Design Option 1: decrease the CPI of FPSQRT to 2 Design Option 2: decease the average CPI of all FP instructions to 2.5 Original CPI = 0.25* *(1-0.25) = 2.0 Option 1 CPI = 2.0 2%*(20-2) = 1.64 Option 2 CPI = 0.25* *(1-0.25) = Speedup of Option 1 = 2/1.64 = Speedup of Option 2 = 2/1.625 =

44 Additional Example of Performance Evaluation Clock freq = 1.4 GHz FP insturctionss = 25% Average CPI of FP instructions = 4.0 Average CPI of other instructions = 1.33 FPSQRT = 2%, CPI of FPSQRT = 20 Design Option 1: decrease the CPI of FPSQRT to 2, clock freq = 1.2GHz Design Option 2: decease the average CPI of all FP instructions to 2.5, clock freq = 1.1 GHz 44

45 Additional Example of Performance Evaluation Clock freq = 1.4 GHz FP insturctionss = 25% Average CPI of FP instructions = 4.0 Average CPI of other instructions = 1.33 FPSQRT = 2%, CPI of FPSQRT = 20 Design Option 1: decrease the CPI of FPSQRT to 2, clock freq = 1.2GHz Design Option 2: decease the average CPI of all FP instructions to 2.5, clock freq = 1.1 GHz Original CPI = 2.0, IPC = 1/2, Inst/Sec = ½*1.4G = 0.7G inst/s Option 1 CPI = 1.64, IPC = 1/1.64, Inst/Sec = 1/1.64*1.2G = 0.73G inst/s Option 2 CPI = 1.625, IPC = 1/1.625, Inst/Sec = 1/1.625*1.1G = 0.68G inst/s 45

Defining Performance. Performance. Which airplane has the best performance? Boeing 777. Boeing 777. Boeing 747. Boeing 747

Defining Performance. Performance. Which airplane has the best performance? Boeing 777. Boeing 777. Boeing 747. Boeing 747 Defining Which airplane has the best performance? 1 Boeing 777 Boeing 777 Boeing 747 BAC/Sud Concorde Douglas DC-8-50 Boeing 747 BAC/Sud Concorde Douglas DC- 8-50 0 100 200 300 400 500 Passenger Capacity