Introduction To Computer Architecture

Transcription

1 Introduction To Computer Architecture. Virendra Singh Computer Design and Test Lab. Supercomputer Education and Research Centre Indian Institute of Science Bangalore

2 Computer Architecture Instruction Set Architecture (IBM 360) the attributes of a [computing] system as seen by the programmer. I.e. the conceptual structure and functional behavior, as distinct from the organization of the data flows and controls, the logic design, and the physical implementation. -- Amdahl, Blaauw, & Brooks, 1964 Machine Organization (microarchitecture) ALUS, Buses, Caches, Memories, etc. Machine Implementation (realization) Gates, cells, transistors, wires

3 Computer Architecture Exercise in engineering tradeoff analysis Find the fastest/cheapest/power-efficient/etc. solution Optimization problem with 100s of variables All the variables are changing At non-uniform rates With inflection points Only one guarantee: Today s right answer will be wrong tomorrow Two high-level effects: Technology push Application Pull

4 Technology Push What do these two intervals have in common? (224 years) (2 years) Answer: Equal progress in processor speed! The power of exponential growth! Driven by Moore s Law Device per chips doubles every months Computer architects work to turn the additional resources into speed/power savings/functionality!

5 Some History Date Event Comments 1939 First digital computer John Atanasoff (UW PhD 30) st transistor Bell Labs st IC Jack Kilby (MSEE Winner of 2000 Nobel prize st microprocessor Intel 1974 Intel transistors 1978 Intel K transistors 1989 Intel M transistors, pipelined 1995 Intel Pentium Pro 5.5M transistors 2005 Intel Montecito 1B transistors

6 Technology Push Technology advances at varying rates E.g. DRAM capacity increases at 60%/year But DRAM speed only improves 10%/year Creates gap with processor frequency! Inflection points Crossover causes rapid change E.g. enough devices for multicore processor (2001) Current issues causing an inflection point Power consumption Reliability Variability

7 Application Pull Corollary to Moore s Law: Cost halves every two years In a decade you can buy a computer for less than its sales tax today. Jim Gray Computers cost-effective for National security weapons design Enterprise computing banking Departmental computing computer-aided design Personal computer spreadsheets, , web Pervasive computing prescription drug labels

8 Application Pull What about the future? Must dream up applications that are not costeffective today Virtual reality Telepresence Mobile applications Sensing, analyzing, actuating in real-world environments This is your job!

9 Building Computer Chips Complex multi-step process Slice silicon ingots into wafers Process wafers into patterned wafers Dice patterned wafers into dies Test dies, select good dies Bond to package Test parts Ship to customers and make money

10 Building Computer Chips

11 Performance vs. Design Time Time to market is critically important E.g., a new design may take 3 years It will be 3 times faster But if technology improves 50%/year In 3 years = 3.38 So the new design is worse! (unless it also employs new technology)

12 Bottom Line Designers must know BOTH software and hardware Both contribute to layers of abstraction IC costs and performance Compilers and Operating Systems

13 Performance and Cost Which of the following airplanes has the best performance? Airplane Passengers Range (mi) Speed (mph) Boeing Boeing BAC/Sud Concorde Douglas DC How much faster is the Concorde vs. the 747 How much bigger is the 747 vs. DC-8?

14 Performance and Cost Which computer is fastest? Not so simple Scientific simulation FP performance Program development Integer performance Database workload Memory, I/O

15 Performance of Computers Want to buy the fastest computer for what you want to do? Workload is all-important Correct measurement and analysis Want to design the fastest computer for what the customer wants to pay? Cost is an important criterion

16 Forecast Time and performance Iron Law MIPS and MFLOPS Which programs and how to average Amdahl s law

17 Defining Performance What is important to whom? Computer system user Minimize elapsed time for program = time_end time_start Called response time Computer center manager Maximize completion rate = #jobs/second Called throughput

18 Response Time vs. Throughput Is throughput = 1/av. response time? Only if NO overlap Otherwise, throughput > 1/av. response time E.g. a lunch buffet assume 5 entrees Each person takes 2 minutes/entrée Throughput is 1 person every 2 minutes BUT time to fill up tray is 10 minutes Why and what would the throughput be otherwise? 5 people simultaneously filling tray (overlap) Without overlap, throughput = 1/10

19 What is Performance for us? For computer architects CPU time = time spent running a program Intuitively, bigger should be faster, so: Performance = 1/X time, where X is response, CPU execution, etc. Elapsed time = CPU time + I/O wait We will concentrate on CPU time

20 Improve Performance Improve (a) response time or (b) throughput? Faster CPU Helps both (a) and (b) Add more CPUs Helps (b) and perhaps (a) due to less queueing

21 Performance Comparison Machine A is n times faster than machine B iff perf(a)/perf(b) = time(b)/time(a) = n Machine A is x% faster than machine B iff perf(a)/perf(b) = time(b)/time(a) = 1 + x/100 E.g. time(a) = 10s, time(b) = 15s 15/10 = 1.5 => A is 1.5 times faster than B 15/10 = 1.5 => A is 50% faster than B

22 Breaking Down Performance A program is broken into instructions H/W is aware of instructions, not programs At lower level, H/W breaks instructions into cycles Lower level state machines change state every cycle For example: 1GHz Snapdragon runs 1000M cycles/sec, 1 cycle = 1ns 2.5GHz Core i7 runs 2.5G cycles/sec, 1 cycle = 0.25ns

23 Iron Law Time Processor Performance = Program Instructions Cycles = X X Program Instruction (code size) (CPI) Time Cycle (cycle time) Architecture --> Implementation --> Realization Compiler Designer Processor Designer Chip Designer

24 Iron Law Instructions/Program Instructions executed, not static code size Determined by algorithm, compiler, ISA Cycles/Instruction Determined by ISA and CPU organization Overlap among instructions reduces this term Time/cycle Determined by technology, organization, clever circuit design

25 Our Goal Minimize time which is the product, NOT isolated terms Common error to miss terms while devising optimizations E.g. ISA change to decrease instruction count BUT leads to CPU organization which makes clock slower Bottom line: terms are inter-related

26 Other Metrics MIPS and MFLOPS MIPS = instruction count/(execution time x 10 6 ) = clock rate/(cpi x 10 6 ) But MIPS has serious shortcomings

27 Problems with MIPS For instance, without FP hardware, an FP op may take 50 single-cycle instructions With FP hardware, only one 2-cycle instruction Thus, adding FP hardware: CPI increases (why?) Instructions/program decreases (why?) Total execution time decreases BUT, MIPS gets worse! 50/50 => 2/1 50 => 1 50 => 2 50 MIPS => 2 MIPS

28 Problems with MIPS Ignores program Usually used to quote peak performance Ideal conditions => guaranteed not to exceed! When is MIPS ok? Same compiler, same ISA E.g. same binary running on AMD Phenom, Intel Core i7 Why? Instr/program is constant and can be ignored

29 Other Metrics MFLOPS = FP ops in program/(execution time x 10 6 ) Assuming FP ops independent of compiler and ISA Often safe for numeric codes: matrix size determines # of FP ops/program However, not always safe: Missing instructions (e.g. FP divide) Optimizing compilers Relative MIPS and normalized MFLOPS Adds to confusion

30 Rules Use ONLY Time Beware when reading, especially if details are omitted Beware of Peak Guaranteed not to exceed

31 Iron Law Example Machine A: clock 1ns, CPI 2.0, for program x Machine B: clock 2ns, CPI 1.2, for program x Which is faster and how much? Time/Program = instr/program x cycles/instr x sec/cycle Time(A) = N x 2.0 x 1 = 2N Time(B) = N x 1.2 x 2 = 2.4N Compare: Time(B)/Time(A) = 2.4N/2N = 1.2 So, Machine A is 20% faster than Machine B for this program

32 Iron Law Example Keep 1ns and For equal performance, if CPI(B)=1.2, what is CPI(A)? Time(B)/Time(A) = 1 = (Nx2x1.2)/(Nx1xCPI(A)) CPI(A) = 2.4

33 Iron Law Example Keep CPI(A)=2.0 and CPI(B)=1.2 For equal performance, if clock(b)=2ns, what is clock(a)? Time(B)/Time(A) = 1 = (N x 2.0 x clock(a))/(n x 1.2 x 2) clock(a) = 1.2ns

34 Which Programs Execution time of what program? Best case your always run the same set of programs Port them and time the whole workload In reality, use benchmarks Programs chosen to measure performance Predict performance of actual workload Saves effort and money Representative? Honest? Benchmarketing

35 How to Average Machine A Machine B Program Program Total One answer: for total execution time, how much faster is B? 9.1x

36 How to Average Another: arithmetic mean (same result) Arithmetic mean of times: n 1 time( i) AM(A) = 1001/2 = i 1 n AM(B) = 110/2 = /55 = 9.1x Valid only if programs run equally often, so use weighted arithmetic mean: n 1 weight( i) time( i) n i 1

37 Other Averages E.g., 30 mph for first 10 miles, then 90 mph for next 10 miles, what is average speed? Average speed = (30+90)/2 WRONG Average speed = total distance / total time = (20 / (10/ /90)) = 45 mph

38 Harmonic Mean Harmonic mean of rates = i n n 1 1 rate( n) Use HM if forced to start and end with rates (e.g. reporting MIPS or MFLOPS) Why? Rate has time in denominator Mean should be proportional to inverse of sums of time (not sum of inverses) See: J.E. Smith, Characterizing computer performance with a single number, CACM Volume 31, Issue 10 (October 1988), pp

39 Dealing with Ratios Machine A Machine B Program Program Total If we take ratios with respect to machine A Machine A Machine B Program Program

40 Dealing with Ratios Average for machine A is 1, average for machine B is 5.05 If we take ratios with respect to machine B Machine A Machine B Program Program Average Can t both be true!!! Don t use arithmetic mean on ratios!

41 Geometric Mean Use geometric mean for ratios Geometric mean of ratios = n n ratio( i) i 1 Independent of reference machine In the example, GM for machine a is 1, for machine B is also 1 Normalized with respect to either machine

42 But GM of ratios is not proportional to total time AM in example says machine B is 9.1 times faster GM says they are equal If we took total execution time, A and B are equal only if Program 1 is run 100 times more often than program 2 Generally, GM will mispredict for three or more machines

43 Summary Use AM for times Use HM if forced to use rates Use GM if forced to use ratios Best of all, use unnormalized numbers to compute time

44 Benchmarks: SPEC2000 System Performance Evaluation Cooperative Formed in 80s to combat benchmarketing SPEC89, SPEC92, SPEC95, SPEC integer and 14 floating-point programs Sun Ultra-5 300MHz reference machine has score of 100 Report GM of ratios to reference machine

45 Benchmarks: SPEC CINT2000 Benchmark Description 164.gzip 175.vpr 176.gcc 181.mcf 186.crafty 197.parser 252.eon 253.perlbmk 254.gap 255.vortex 256.bzip2 300.twolf Compression FPGA place and route C compiler Combinatorial optimization Chess Word processing, grammatical analysis Visualization (ray tracing) PERL script execution Group theory interpreter Object-oriented database Compression Place and route simulator

46 Benchmarks: SPEC CFP2000 Benchmark Description 168.wupwise 171.swim 172.mgrid 173.applu 177.mesa 178.galgel 179.art 183.equake 187.facerec 188.ammp 189.lucas 191.fma3d 200.sixtrack 301.apsi Physics/Quantum Chromodynamics Shallow water modeling Multi-grid solver: 3D potential field Parabolic/elliptic PDE 3-D graphics library Computational Fluid Dynamics Image Recognition/Neural Networks Seismic Wave Propagation Simulation Image processing: face recognition Computational chemistry Number theory/primality testing Finite-element Crash Simulation High energy nuclear physics accelerator design Meteorology: Pollutant distribution

47 Benchmark Pitfalls Benchmark not representative Your workload is I/O bound, SPEC is useless Benchmark is too old Benchmarks age poorly; benchmarketing pressure causes vendors to optimize compiler/hardware/software to benchmarks Need to be periodically refreshed

48 Amdahl s Law Motivation for optimizing common case Speedup = old time / new time = new rate / old rate Let an optimization speed fraction f of time by a factor of s Speedup 1 1 f f 1 1 f s f f oldtime oldtime f oldtime s

49 Amdahl s Law Example Your boss asks you to improve performance by: Improve the ALU used 95% of time by 10% Improve memory pipeline used 5% of time by 10x Let f=fraction sped up and s = speedup on that fraction New_time = (1-f) x old_time + (f/s) x old_time Speedup = old_time / new_time Speedup = old_time / ((1-f) x old_time + (f/s) x old_time) Amdahl s Law: Speedup 1 1 f f s

50 Amdahl s Law Example, cont d f s Speedup 95% % % 1.052

51 Speedup Amdahl s Law: Limit Make common case fast: lim f f f s s f

52 Amdahl s Law: Limit Consider uncommon case! If (1-f) is nontrivial Speedup is limited! lim Particularly true for exploiting parallelism in the large, where large s is not cheap GPU with e.g processors (shader cores) Parallel portion speeds up by s (1024x) Serial portion of code (1-f) limits speedup E.g. 10% serial limits to 10x speedup! f f s s f

53 Instructions Instructions are the words of a computer Instruction set architecture (ISA) is its vocabulary This defines most of the interface to the processor (not quite everything) Implementations can and do vary Intel 486->Pentium->P6->Core Duo->Core i7

54 Instructions cont d MIPS ISA: Simple, sensible, regular, widely used Most common: x86 (IA-32) Intel Pentium/Core i7, AMD Athlon, etc. Others: PowerPC (Mac, IBM servers) SPARC (Sun) ARM (cell phones, embedded systems)

55 Forecast Basics Registers and ALU ops Memory and load/store Branches and jumps Etc.

56 Basics C statement f = (g + h) (i + j) MIPS instructions add t0, g, h add t1, i, j sub f, t0, t1 Opcode/mnemonic, operands, source/destination

57 Basics Opcode: specifies the kind of operation (mnemonic) Operands: input and output data (source/destination) Operands t0 & t1 are temporaries One operation, two inputs, one output Multiple instructions for one C statement

58 Why not bigger instructions? Why not f = (g + h) (i + j) as one instruction? Church s thesis: A very primitive computer can compute anything that a fancy computer can compute you need only logical functions, read and write memory, and data-dependent decisions Therefore, ISA selected for practical reasons: Performance and cost, not computability Regularity tends to improve both E.g. H/W to handle arbitrary number of operands is complex and slow and UNNECESSARY

59 Registers and ALU ops Operands must be registers, not variables add $8, $17, $18 add $9, $19, $20 sub $16, $8, $9 MIPS has 32 registers $0-$31 $8 and $9 are temps, $16 is f, $17 is g, $18 is h, $19 is i and $20 is j MIPS also allows one constant called immediate Later we will see immediate is restricted to 16 bits

60 Registers Registers and ALU Processor $0 $31 ALU

61 ALU ops Some ALU ops: add, addi, addu, addiu (immediate, unsigned) sub mul, div wider result 32b x 32b = 64b product 32b / 32b = 32b quotient and 32b remainder and, andi or, ori sll, srl Why registers? Short name fits in instruction word: log 2 (32) = 5 bits But are registers enough?

62 Memory and Load/Store Need more than 32 words of storage An array of locations M[j] indexed by j Data movement (on words or integers) Load word for register <= memory lw $17, 1002 # get input g Store word for register => memory sw $16, 1001 # save output f

63 Registers Memory and load/store Processor $ Memory $31 ALU f g maxmem

64 Memory and load/store Important for arrays A[i] = A[i] + h # $8 is temp, $18 is h, $21 is (i x 4) # Astart is &A[0] is 0x8000 lw $8, Astart($21) # or 8000($21) add $8, $18, $8 sw $8, Astart($21) MIPS has other load/store for bytes and halfwords

65 Registers Memory and load/store Processor $0 0 Memory $31 ALU f g 8000 A[0] 8004 A[1] 8008 A[2] maxmem

66 Aside on Endian Big endian: MSB at address xxxxxx00 E.g. IBM, SPARC Little endian: MSB at address xxxxxx11 E.g. Intel x86 Mode selectable E.g. PowerPC, MIPS

67 Branches and Jumps While ( i!= j) { j= j + i; i= i + 1; } # $8 is i, $9 is j # $10 is k Loop: beq $8, $9, Exit add $9, $9, $8 addi $8, $8, 1 j Loop Exit:

68 Branches and Jumps # better: beq $8, $9, Exit # not!= Loop: add $9, $9, $8 addi $8, $8, 1 bne $8, $9, Loop Exit: Best to let compilers worry about such optimizations

69 Branches and Jumps What does bne do really? read $, read $9, compare Set PC = PC + 4 or PC = Target To do compares other than = or!= E.g. blt $8, $9, Target # pseudoinstruction Expands to: slt $1, $8, $9 # $1==($8<$9)==($8-$9)<0 bne $1, $0, Target # $0 is always 0

70 Branches and Jumps Other MIPS branches/jumps beq $8, $9, imm # if ($8==$9) PC = PC + imm<< 2 else PC += 4; bne slt, sle sgt, sge With immediate, unsigned j addr # PC = addr jr $12 # PC = $12 jal addr # $31 = PC + 4; PC = addr; used for???

71 MIPS Machine Language All instructions are 32 bits wide Assembly: add $1, $2, $3 Machine language: alu-rr zero add/signed

72 Instruction Format R-format Opc rs rt rd shamt function Digression: How do you store the number 4,392,976? Same as add $1, $2, $3 Stored program: instructions are represented as numbers Programs can be read/written in memory like numbers

73 Instruction Format Other R-format: addu, sub, subi, etc. Assembly: lw $1, 100($2) Machine: I-format lw (in binary) Opc rs rt address/immediate

74 Instruction Format I-format also used for ALU ops with immediates addi $1, $2, What about constants larger than 16 bits Outside range: [-32768, 32767]? ? lui $4, 12 # $4 == ori $4, $4, 15 # $4 == All loads and stores use I-format

75 Instruction Format beq $1, $2, PC = PC + ( << 2) # word offset Finally, J-format J address Opcode addr 6 26 Addr is weird in MIPS: addr = 4 MSB of PC // addr // 00

76 Summary: Instruction Formats R: opcode rs rt rd shamt function I: opcode rs rt address/immediate J: opcode addr 6 26 Instruction decode: Read instruction bits Activate control signals

77 Procedure Calls Caller Save registers Set up parameters Call procedure Get results Restore registers Callee Save more registers Do some work, set up result Restore registers Return Jal is special, otherwise just software convention

78 Procedure Calls Stack is all-important Stack grows from larger to smaller addresses (arbitrary) $29 is stack pointer; points just beyond valid data Push $2: addi $29, $29, -4 sw $2, 4($29) Pop $2: lw $2, 4($29) addi $29, $29, 4 Cannot change order. Why? Interrupts.

79 Procedure Example Swap(int v[], int k) { int temp = v[k]; v[k] = v[k+1]; v[k+1] = temp; } # $4 is v[] & $5 is k -- 1st & 2nd incoming argument # $8, $9 & $10 are temporaries that callee can use w/o saving swap: add $9,$5,$5 # $9 = k+k add $9,$9,$9 # $9 = k*4 add $9,$4,$9 # $9 = v + k*4 = &(v[k]) lw $8,0($9) # $8 = temp = v[k] lw $10,4($9) # $10 = v[k+1] sw $10,0($9) # v[k] = v[k+1] sw $8,4($9) # v[k+1] = temp jr $31 # return

80 Addressing Modes There are many ways of accessing operands Register addressing: add $1, $2, $3 op rs rt rd... funct register

81 Addressing Modes Base addressing (aka displacement) lw $1, 100($2) # $2 == 400, M[500] == 42 op rs rt Offset/displacement register Memory Effective address 42

82 Addressing Modes Immediate addressing addi $1, $2, 100 op rs rt immediate

83 Addressing Modes PC relative addressing beq $1, $2, 100 # if ($1==$2) PC = PC op rs rt address PC Memory Effective address

84 Addressing Modes Not found in MIPS: Indexed: add two registers base + index Indirect: M[M[addr]] two memory references Autoincrement/decrement: add operand size Autoupdate found in PowerPC, PA-RISC Like displacement, but update base register

85 Addressing Modes Autoupdate lwupdate $1,24($2) # $1 = M[$2+24]; $2 = $ op rs rt address register Delay Effective address Memory

86 Addressing Modes for(i=0; i < N, i += 1) sum += A[i]; # $7 is sum, $8 is &a[i], $9 is N,$2 is tmp, $3 is i*4 Inner loop: Or: lw $2, 0($8) lwupdate $2, 4($8) addi $8, $8, 4 add $7, $7, $2 add $7, $7, $2 Where s the bug? Before loop: sub $8, $8, 4

87 How to Choose ISA Minimize what? Instrs/prog x cycles/instr x sec/cycle!!! In technology, simple modes like MIPS were great As technology changes, computer design options change If memory is limited, dense instructions are important For high speed, pipelining and ease of pipelining is important

88 Some Intel x86 (IA-32) History Year CPU Comment bit with 8-bit bus from 8080; selected for IBM PC Floating Point Unit bit addresses, memory-map, protection bit registers, flat memory addressing, paging Pipelining 1992 Pentium Superscalar 1995 Pentium Out-of-order execution, 1997 MMX Pro 1999 P-III SSE streaming SIMD

89 Intel 386 Registers & Memory Registers 8 32b registers (but backward 16b & 8b: EAX, AX, AH, AL) 4 special registers: stack (ESP) & frame (EBP) Condition codes: overflow, sign, zero, parity, carry Floating point uses 8-element stack Memory Flat 32b or segmented (rarely used) Effective address = (base_reg + (index_reg x scaling_factor) + displacement)

90 Intel 386 ISA Two register instructions: src1/dst, src2 reg/reg, reg/immed, reg/mem, mem/reg, mem/imm Examples mov EAX, 23 # 32b 2 s C imm 23 in EAX neg [EAX+4] # M[EAX+4] = -M[EAX+4] faddp ST(7), ST # ST = ST + ST(7) jle label # PC = label if sign or zero flag set

91 Intel 386 ISA cont d Decoding nightmare Instructions 1 to 17 bytes Optional prefixes, postfixes alter semantics AMD64 64-bit extension: 64b prefix byte Crazy formats E.g. register specifiers move around But key 32b 386 instructions not terrible Yet entire ISA has to correctly implemented

92 Current Approach Current technique used by Intel and AMD Decode logic translates to RISC uops Execution units run RISC uops Backward compatible Very complex decoder Execution unit has simpler (manageable) control logic, data paths We use MIPS/DLX to keep it simple and clean

93 Complex Instructions More powerful instructions not faster E.g. string copy Option 1: move with repeat prefix for memory-to-memory move Special-purpose Option 2: use loads/stores to/from registers Generic instructions Option 2 faster on same machine! (but which code is denser?)

94 Conclusions Simple and regular Constant length instructions, fields in same place Small and fast Small number of operands in registers Compromises inevitable Pipelining should not be hindered Make common case fast! Backwards compatibility!