Advanced Computer Architecture Week 1: Introduction. ECE 154B Dmitri Strukov

Transcription

1 Advanced Computer Architecture Week 1: Introduction ECE 154B Dmitri Strukov 1

2 Outline Course information Trends (in technology, cost, performance) and issues 2

3 Course organization Class website: er2015/home.htm Instructor office hours: Wed, 2:00 pm 4:00 pm Teacher Assistant: David McCarthy office hours: Friday 2:00 pm 4:00 pm 3

4 Textbook Computer Architecture: A Quantitative Approach, John L. Hennessy and David A. Patterson, Fifth Edition, Morgan Kaufmann, 2012, ISBN: Modern Processor Design: Fundamentals of Superscalar Processors, John Paul Shen and Mikko H. Lipasti, Waveland Press, 2013, ISBN:

5 Class topics and tentative schedule Computer fundamentals (historical trends, performance metrics) 1 week Memory hierarchy design - 2 weeks Instruction level parallelism (static and dynamic scheduling, speculation) 2 weeks Data level parallelism (vector, SIMD and GPUs) 2 weeks Thread level parallelism (shared-memory architectures, synchronization and cache coherence) 2 weeks Warehouse-scale computers or Detailed analysis of some specific up (1 week) 5

6 Ultimate goal of the class To get intuition behind main techniques for improving performance To understand advanced microprocessors such as - ARM Cortex A8 - Intel Core i7 - Tesla GPU This is what you supposed to know! 5-STAGE MIPS PIPELINE 6

7 This is what we learn in this class! 7

8 Grading Projects: 70 % Final: 30 % Project course work will involve program performance analysis and architectural optimizations for superscalar processors using SimpleScalar simulation tools Number of problems will be assigned before final (but not graded) 8

9 Course prerequisites ECE 154A or equivalent 9

10 A bit of history: ENIAC - Electronic Numerical Integrator And Computer,

11 VLSI Developments 1946: ENIAC electronic numerical integrator and computer Floor area 140 m 2 Performance multiplication of two 10-digit numbers in 2 ms 2011: High Performance microprocessor Chip area mm 2 (for multi-core) Board area 200 cm 2 ; improvement of 10 4 Performance: 64 bit multiply in few ns; improvement of

12 Computer trends: Performance of a (single) processor The next series of question is centered around understanding that important graph 12

13 Question Q1: what is performance shown on the figure and how do we define it? - A1a: Performance is typically related to how fast a certain task can be executed, i.e. reciprocal of execution time Performance = 1/ ExecTime Wall clock time: includes all system overheads CPU time: only computation time ExecTime = IC * CCT * <CPI> - A1b: Many different metrics of performance today because of different application of ups - What kind of metrics? 13

14 Measuring Performance Typical performance metrics: Execution Time (or latency) Throughput Q2: How is throughput related to latency? A2: In general these are two different concepts. Throughput can be improved by providing more parallelism, but also be improved by reducing latency. For example, with no parallelism throughput is reversely proportional to latency Energy Q3: Is energy metric the same as power consumption one? A3: Power = energy / time, so in general, it is the same metric only when execution time is the same. Response time Typical way to measure performance is to run benchmark (i.e. collection of representative for the tested hardware application) Kernels (e.g. matrix multiply) Toy programs (e.g. sorting) Synthetic benchmarks (e.g. Dhrystone) Benchmark suites (e.g. SPEC06fp, TPC-C) Speedup of X relative to Y Execution time Y / Execution time X 14

15 Bandwidth vs. Latency Bandwidth or throughput Total work done in a given time 10,000-25,000X improvement for processors X improvement for memory and disks Latency or response time Time between start and completion of an event 30-80X improvement for processors 6-8X improvement for memory and disks 15

16 Computer trends: Performance of a (single) processor 16

17 Questions: Reasons behind performance improvement? Q4: Why it was improving originally (from ~1978-~1984 on the figure)? A4: Moore s law and the resulting increase in clock frequency 17

18 CMOS improvements: Transistor density: 4x / 3 yrs Die size: 10-25% / yr 18

19 Scaling with Feature Size (for short channel devices before running into leakage problems) Let s 1) scale all the dimensions of the transistors and wires down by factor of s and 2) supply voltage V down by factor of s (together with threshold voltage Vth) Then Density: ~ s 2 Logic gate capacitance Cgate (traditionally dominating parasitics): ~ 1/s Saturation current I ON : ~ 1/s Gate delay Tgate: ~ CgateV/I ON = 1/s Clock frequency: s, i.e. it is reversely proportional to gate delay. Clock cycle time is typically around ten or more of logic gate delays See, e.g. page 124 of Digital Integrated Circuits by Jan Rabaey et al, 2 nd edition 19

20 Frequency Scaling with Feature Size If s is scaling factor, then density scale as s 2 Voltage V: 1/s Logic gate capacitance C (traditionally dominating): ~ 1/s Saturation current I ON : ~ 1/s Gate delay: ~ CV/I ON = 1/s 20

21 Computer trends: Performance of a (single) processor 21

22 Question: Q5: Reasons behind further performance improvement? What happened in 1986? A5: CISC to RISC which enabled additional architectural improvements (see next slide) Review: Dimensions of ISA (1) Class of ISA: register-memory vs load-store (2) Memory addressing: byte addressabile (3) Addressing modes (what are operands and addressing modes of memory): registers, immediate, displacement, indirect, indexed, absolute) (4) Types and sizes of operands: byte, half-word, word (5) Operations: data transfer, arithmetic logical, control and fp (6) Control flow instructions: conditional branches, unconditional jumps, returns (7) Encoding an ISA: variable versus fixed length 22

23 Question: Reasons behind performance improvement? What happened in 1986? CISC to RISC ExecTime = IC * CCT * <CPI> Q6: How are these terms affected by this move and in particular how are the terms in performance equation are affected by pipelining? -A6: Design Inst count CPI CCT Single Cycle (SC) Multi cycle (MC) 1 N CPI > 1 (closer to N than 1) > 1/N Multi cycle pipelined (MCP) 1 > 1 >1/N 23

24 Question: Pipelining improve performance (instruction per cycle with respect to multi-cycle processor with non pipelining, by overlapping instructions) One kind of instruction level parallelism (ILP) Q7: Problems with improving ILP? What are the problems in pipelines? A7: Clock cycle is determined by slowest component» What is typically the slowest component? memory A7: Data and control hazards (pipeline stalls and flushes) Further improvement in ILP? A7: Limited parallelism in ILP 24

25 Memory Wall problem DRAM access (main memory) could take hundreds of cycles Memory hierarchy to rescue to alleviate the problem Will spend much time later in class reviewing advanced techniques for reducing effective access time to main memory 25

26 CPU high, Memory low ( Memory Wall ) Bandwidth and Latency Performance Milestones Processor: 286, 386, 486, Pentium, Pentium Pro, Pentium 4 (21x,2250x) Ethernet: 10Mb, 100Mb, 1000Mb, Mb/s (16x,1000x) Memory Module: 16bit plain DRAM, Page Mode DRAM, 32b, 64b, SDRAM, DDR SDRAM (4x,120x) Disk : 3600, 5400, 7200, 10000, RPM (8x, 143x) Log-log plot of bandwidth and latency milestones Bandwidth is much easier to improve why?

27 Question: Pipelining improve performance (instruction per cycle with respect to multi-cycle processor with non pipelining, by overlapping instructions) One kind of instruction level parallelism (ILP) Q7: Problems with improving ILP? What are the problems in pipelines? A7: Clock cycle is determined by slowest component» What is typically the slowest component? memory A7: Data and control hazards (pipeline stalls and flushes) Further improvement in ILP? A7: Limited parallelism in ILP 27

28 ILP techniques

29 Summary of Trends in Technology Integrated circuit technology Transistor density: 35%/year Die size: 10-20%/year Integration overall: 40-55%/year DRAM capacity: 25-40%/year (slowing) Flash capacity: 50-60%/year 15-20X cheaper/bit than DRAM Magnetic disk technology: 40%/year 15-25X cheaper/bit then Flash X cheaper/bit than DRAM 29

30 Computer trends: Performance of a (single) processor The area of high performance chip has been close to ~ cm^2, why? 30

31 Question: Q8: Why did the die size only grew by 10% / year? Performance of single processor could be improved by using more hardware (larger cache, more sophisticated branch prediction etc.) Drawing single-crystal Si ingot from furnace. Then, slice into wafers and pattern it 8 MIPS64 R20K wafer (564 dies) 31

32 Trends in Cost Cost driven down by learning curve Yield DRAM: price closely tracks cost Microprocessors: price depends on volume 10% less for each doubling of volume 32

33 Integrated Circuits Costs IC cost = Die cost + Testing cost + Packaging cost Final test yield Die cost = Wafer cost Dies per Wafer * Die yield Final test yield: fraction of packaged dies which pass the final testing state Die yield: fraction of good dies on a wafer Defects per unit area = defects per square cm (2010) N = process-complexity factor = (40 nm, 2010) 33

34 Die yield / wafer yield Die cost (arbitrary units) N = 11.5 Defects per unit area = Defects per unit area = Die area (cm^2) Answer to Q Die area (cm^2) 34

35 Total cost $ ASIC vs. up this is just an example of NRE cost. It may vary by much in in general total cost for up > that of ASIC 10 6 $1 M NRE (non recurrent engineering cost) Total cost = NRE/volume + IC cost IC cost = $100 IC cost = $ up ASIC Volume Q9: - What is typically denser ASIC or up for the same task? ASIC - What is typically more energy efficient and faster? ASIC - What cost less to produce ASIC or up? depends on volume (see graph above) 35

36 Major computing platforms Application Specific Integrated Circuit Field Programmable Gate Array Microprocessor Density, speed Flexibility In this class, the focus is on the microprocessors only

37 Computer trends: Performance of a (single) processor 37

38 Questions: Reasons behind performance improvement? Q10: What happened after > 2002 on the performance figure? A10: Power wall A:10 End of ILP Limits to pipelining Limits to superscalar 38

39 Power consumption Problem: Get power in, get power out Thermal Design Power (TDP) - Characterizes sustained power consumption, used as target for power supply and cooling system, Lower than peak power, higher than average power consumption Intel consumed ~ 2 W 3.3 GHz Intel Core i7 consumes 130 W Typical max temperatures: ~70 C Maximum power density for fan-based cooling: 200W/cm^2 water based cooling: 1000W/cm^2 39

40 Ambient temperature (Tlow) Fourier Law in 1D : Similar to Ohms law when replacing - thermal conductance with electrical conductance - heat source (total generated power) with current source - temperature with voltage Heat flux (Q) Thermal conductance K Chip temperature (Thigh) Tlow K Q Vlow I 1/R Thigh Vhigh Thigh = Tlow + Q/K Vhigh = Vlow + IR Temperature is roughly (in 1D lumped model) linearly proportional to the Q or total dissipated power 40

41 Scaling with Feature Size (for short channel devices before running into leakage problems) Let s 1) scale all the dimensions of the transistors and wires down by factor of s and 2) supply voltage V down by factor of s (together with threshold voltage Vth) Then Density: ~ s 2 Logic gate capacitance Cgate (traditionally dominating parasitics): ~ 1/s Saturation current I ON : ~ 1/s Gate delay Tgate: ~ CgateV/I ON = 1/s Clock frequency f : s, i.e. it is reversely proportional to gate delay. Clock cycle time is typically around ten or more of logic gate delays Power (dynamic component only): ~1/2 Ctotal*V 2 *f ~ 1 (if chip area remain the same power scaling is the same as power density) No issue with power (or temperature) scaling but the problem is that supply voltage is no longer scaled down by factor of s, why? see next slides 41

42 Static vs. dynamic power Static power Ron Dynamic power Roff Static power is permanent Dynamic power only when switching Leakage (static power) increases exponentially when lowering V! Cannot be neglected anymore Leakage power ~ V^2/Roff Roff/Ron ~ Exp(V)

43 Technique for Reducing Power Consumption Do nothing well Low power state for DRAM, disks Energy proportionality concept (don t consume energy when no work is done) very important for data center for which power is huge portion of running cost Power gating to reduce static component Dynamic Voltage-Frequency Scaling Since saturation current I ON ~ V 2 f ~ 1/Tgate I ON / (Cgate V )~ V Lowering voltage reduces the dynamic power consumption and energy per operation but decrease performance because of increased CCT Q11: Any benefits for multiprocessors? A11: If task is easily parallelizable, then running this task on m processors in parallel at lower V (say V/m) and slower f (say f/m) can lead to the same execution time but much lower dynamic power CtotalV^2f ~ 1/m^3 (not accounting for static power) Overclocking, turning off cores Race-to-halt Thermal capacitance/ turbo mode 43

44 Reducing energy consumption: Choice of optimal voltage supply

45 Other problems with scaling: Transistors and Wires Feature size Minimum size of transistor or wire in x or y dimension 10 microns in 1971 to.032 microns in 2011 Transistor performance scales linearly Wire delay does not improve with feature size! There is always need in long wires Problem related to Rent Rule (number of pins versus number of gates) 45

46 Questions: Reasons behind performance improvement? Q10: What happened after > 2002 on the performance figure? A10: Power wall A10: End of ILP Limits to pipelining Limits to superscalar» Will discuss it in detail after covering advanced ILP topics 46

47 What is next: Current Trends in Architecture Cannot continue to leverage Instruction-Level parallelism (ILP) Single processor performance improvement ended in 2003 New ways of improving performance: Data-level parallelism (DLP) Thread-level parallelism (TLP) Request-level parallelism (RLP) These require explicit restructuring of the application 47

48 New applications appears: Classes of computers now Personal Mobile Device (PMD) e.g. start phones, tablet computers Emphasis on energy efficiency and real-time Desktop Computing Emphasis on price-performance Servers Emphasis on availability, scalability, throughput Clusters / Warehouse Scale Computers Used for Software as a Service (SaaS) Emphasis on availability and price-performance Sub-class: Supercomputers, emphasis: floating-point performance and fast internal networks Embedded Computers Emphasis: price 48

49 Dark silicon Only some parts of a chip are active at a time Q12: Specialized cores make sense now in general purpose microprocessor Qualcomm Zeroth chip 49

50 Summary of trends in up 50

51 Not covered in class 51

52 Quantitative Principles of Design Take Advantage of Parallelism Principle of Locality Focus on the Common Case Amdahl s Law E.g. common case supported by special hardware; uncommon cases in software The Performance Equation 52

53 1. Parallelism How to improve performance? (Super)-pipelining Powerful instructions MD-technique multiple data operands per operation MO-technique multiple operations per instruction Multiple instruction issue single instruction-program stream multiple streams (or programs, or tasks) 53

54 Flynn s Taxonomy Single instruction stream, single data stream (SISD) Single instruction stream, multiple data streams (SIMD) Vector architectures Multimedia extensions Graphics processor units Multiple instruction streams, single data stream (MISD) No commercial implementation Multiple instruction streams, multiple data streams (MIMD) Tightly-coupled MIMD Loosely-coupled MIMD 54

55 MIPS Pipeline Five stages, one step per stage 1. IF: Instruction fetch from memory 2. ID: Instruction decode & register read 3. EX: Execute operation or calculate address 4. MEM: Access memory operand 5. WB: Write result back to register lw Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 IFetch Dec Exec Mem WB

56 f = Fetch f r a d w 5 r = Reg read a = ALU op 6 d = Data access w = Writeback 7 f r f a r f d a r Instruction (a) Task-time diagram Review from Last Lecture f multi cycle pipelined r f a r f d a r w d a w d w w d a w Cycle d w f Clock f f f f f f Time Pipeline stage (b) Space-time diagram Cycle Time r r r r r r r Drainage needed region a a a a a a a Time allotted Instr 1 Instr 2 Instr 3 Instr 4 Start-up d d d d d d d region Clock w w w w w w w needed Time allotted 3 cycles 5 cycles 3 cycles 4 cycles Instr 1 Instr 2 Instr 3 Instr 4 Time saved Single cycle multi cycle Execution time = 1/ Performance = Inst count x CPI x CCT N = # of stages for pipeline design or ~ maximum number of steps for MC CPI ideal MCP =N /InstCount + 1 1/InstCount large N and/or small InstCount result in worse CPI Performance to run one instruction is the same as of CP (i.e. latency for single instruction is not reduced) Design Inst count CPI Single Cycle (SC) Multi cycle (MC) 1 N CPI > 1 (closer to N than 1) Multi cycle pipelined (MCP) CCT > 1/N 1 > 1 >1/N What are the other issues affecting CCT and CPI for MC and MCP?

57 ALU ALU ALU ALU Pipelined Instruction Execution Time (clock cycles) I n s t r. Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Ifetch Reg Ifetch Reg DMem Reg DMem Reg O r d e r Ifetch Reg Ifetch Reg DMem Reg DMem Reg 57

58 ALU ALU ALU ALU Limits to pipelining Hazards prevent next instruction from executing during its designated clock cycle Structural hazards: attempt to use the same hardware to do two different things at once Data hazards: Instruction depends on result of prior instruction still in the pipeline Control hazards: Caused by delay between the fetching of instructions and decisions about changes in control flow (branches and jumps). Time (clock cycles) I n s t r. Ifetch Reg Ifetch Reg Ifetch DMem Reg Reg DMem Reg DMem Reg O r d e r Ifetch Reg DMem Reg 58

59 2. The Principle of Locality Programs access a relatively small portion of the address space at any instant of time. Two Different Types of Locality: Temporal Locality (Locality in Time): If an item is referenced, it will tend to be referenced again soon (e.g., loops, reuse) Spatial Locality (Locality in Space): If an item is referenced, items whose addresses are close by tend to be referenced soon (e.g., straight-line code, array access) Last 30 years, HW relied on locality for memory perf. P $ MEM 59

60 Capacity Access Time Cost CPU Registers 100s Bytes ps ( ns) L1 and L2 Cache 10s-100s K Bytes ~1 ns - ~10 ns ~ $100s/ GByte Main Memory G Bytes 80ns- 200ns ~ $10/ GByte Disk 10s T Bytes, 10 ms (10,000,000 ns) ~ $0.1 / GByte Tape infinite sec-min ~$0.1 / GByte Memory Hierarchy Levels Registers L1 Cache Memory Disk Tape Instr. Operands Blocks L2 Cache Blocks Pages Files Staging Xfer Unit prog./compiler 1-8 bytes cache cntl bytes cache cntl bytes OS 4K-8K bytes user/operator Gbytes Upper Level faster Larger Lower Level still needed? 60

61 3. Focus on the Common Case Favor the frequent case over the infrequent case E.g., Instruction fetch and decode unit used more frequently than multiplier, so optimize it first E.g., If database server has 50 disks / processor, storage dependability dominates system dependability, so optimize it first Frequent case is often simpler and can be done faster than the infrequent case E.g., overflow is rare when adding 2 numbers, so improve performance by optimizing more common case of no overflow May slow down overflow, but overall performance improved by optimizing for the normal case What is frequent case? How much performance improved by making case faster? => Amdahl s Law 61

62 serial part parallel part serial part Amdahl s Law Speedup overall = T exec,old T exec,new = 1 (1 - Fraction enhanced ) + Fraction enhanced Speedup enhanced 62

63 Amdahl s Law Floating point instructions improved to run 2 times faster, but only 10% of actual instructions are FP T exec,new = Speedup overall = 63

64 Amdahl s Law Floating point instructions improved to run 2X; but only 10% of actual instructions are FP T exec,new = T exec,old x ( /2) = 0.95 x T exec,old Speedup overall = =

65 Amdahl's law 65

66 Principles of Computer Design The Processor Performance Equation 66

67 Principles of Computer Design Different instruction types having different CPIs 67

68 Acknowledgements Some of the slides contain material developed and copyrighted by Henk Corporaal (TU/e) and instructor material for the textbook 68