Advanced Computer Architecture (CS620)

Transcription

1 Advanced Computer Architecture (CS620) Background: Good understanding of computer organization (eg.cs220), basic computer architecture (eg.cs221) and knowledge of probability, statistics and modeling (eg.cs433). Text for CS620: Hennessy and Patterson s Computer Architecture, A Quantitative Approach, 4 th Edition Topics Measuring performance/cost/power Instruction level parallelism, dynamic and static Memory hierarchy Multiprocessors Storage systems and networks 1

2 Organizational Issues Course materials adapted from several existing advanced computer architecture courses at various universities (primarily Utah, UCBerkeley, UPenn). Class syllabus, note and announcements to be posted to Blackboard. Currently available on shared folder: Grades: Paper Reviews + Participation 25%. Homework. 25%. Midterm and Final. 50% No tolerance for cheating. 2

3 Lecture 1: Computing Trends, Metrics Topics: (Sections , ) Technology trends Performance summaries Performance equations 3

4 Historical Microprocessor Performance Source: H&P textbook 4

5 Points to Note The 52% growth per year is because of faster clock speeds and architectural innovations (led to 25x higher speed) Clock speed increases have dropped to 1% per year in recent years The 22% growth includes the parallelization from multiple cores Moore s Law: transistors on a chip double every months 5

6 Clock Speed Increases Source: H&P textbook 6

7 Processor Technology Trends Transistor density increases by 35% per year and die size increases by 10-20% per year more cores! Transistor speed improves linearly with size (complex equation involving voltages, resistances, capacitances) can lead to clock speed improvements! The power wall: it is not possible to consistently run at higher frequencies without hitting power/thermal limits (Turbo Mode can cause occasional frequency boosts) Wire delays do not scale down at the same rate as logic delays 7

8 Recent Microprocessor Trends Transistors: 1.43x / year Cores: x Performance: 1.15x Frequency: 1.05x Power: 1.04x Source: Micron University Symp. 8

9 What Helps Performance? Note: no increase in clock speed In a clock cycle, can do more work -- since transistors are faster, transistors are more energy-efficient, and there s more of them Better architectures: finding more parallelism in one thread, better branch prediction, better cache policies, better memory organizations, more thread-level parallelism, etc. Core design is undergoing little change, but more cores available per chip; most future innovations will likely be in multi-threaded prog models and memory hierarchies 9

10 Where Are We Headed? Modern trends: Clock speed improvements are slowing power constraints Difficult to further optimize a single core for performance Multi-cores: each new processor generation will accommodate more cores Need better programming models and efficient execution for multi-threaded applications Need better memory hierarchies Need greater energy efficiency 10

11 Modern Processor Today Intel Core i7 Clock frequency: GHz 45nm and 32nm products Cores: 4 6 Power: W Two threads per core 3-level cache, 12 MB L3 cache Price: $300 - $

12 Power Consumption Trends Dyn power a activity x capacitance x voltage 2 x frequency Capacitance per transistor and voltage are decreasing, but number of transistors is increasing at a faster rate; hence clock frequency must be kept steady Leakage power is also rising; is a function of transistor count, leakage current, and supply voltage Power consumption is already between W in high-performance processors today Energy = power x time = (dynpower + lkgpower) x time 12

13 Power Vs. Energy Energy is the ultimate metric: it tells us the true cost of performing a fixed task Power (energy/time) poses constraints; can only work fast enough to max out the power delivery or cooling solution If processor A consumes 1.2x the power of processor B, but finishes the task in 30% less time, its relative energy is 1.2 X 0.7 = 0.84; Proc-A is better, assuming that 1.2x power can be supported by the system 13

14 Reducing Power and Energy Can gate off transistors that are inactive (reduces leakage) Design for typical case and throttle down when activity exceeds a threshold DFS: Dynamic frequency scaling -- only reduces frequency and dynamic power, but hurts energy DVFS: Dynamic voltage and frequency scaling can reduce voltage and frequency by (say) 10%; can slow a program by (say) 8%, but reduce dynamic power by 27%, reduce total power by (say) 23%, reduce total energy by 17% (Note: voltage drop slow transistor freq drop) 14

15 Other Technology Trends DRAM density increases by 40-60% per year, latency has reduced by 33% in 10 years (the memory wall!), bandwidth improves twice as fast as latency decreases Disk density improves by 100% every year, latency improvement similar to DRAM Emergence of NVRAM technologies that can provide a bridge between DRAM and hard disk drives 15

16 Measuring Performance Two primary metrics: wall clock time (response time for a program) and throughput (jobs performed in unit time) To optimize throughput, must ensure that there is minimal waste of resources Performance is measured with benchmark suites: a collection of programs that are likely relevant to the user SPEC CPU 2006: cpu-oriented programs (for desktops) SPECweb, TPC: throughput-oriented (for servers) EEMBC: for embedded processors/workloads 16

17 Summarizing Performance Consider 25 programs from a benchmark set how do we capture the behavior of all 25 programs with a single number? P1 P2 P3 Sys-A Sys-B Sys-C Total (average) execution time Total (average) weighted execution time or Average of normalized execution times Geometric mean of normalized execution times 17

18 AM Example We fixed a reference machine X and ran 4 programs A, B, C, D on it such that each program ran for 1 second The exact same workload (the four programs execute the same number of instructions that they did on machine X) is run on a new machine Y and the execution times for each program are 0.8, 1.1, 0.5, 2 With AM of normalized execution times, we can conclude that Y is 1.1 times slower than X perhaps, not for all workloads, but definitely for one specific workload (where all programs run on the ref-machine for an equal #cycles) With GM, you may find inconsistencies 18

19 GM Example Computer-A Computer-B Computer-C P1 1 sec 10 secs 20 secs P secs 100 secs 20 secs Conclusion with GMs: (i) A=B (ii) C is ~1.6 times faster For (i) to be true, P1 must occur 100 times for every occurrence of P2 With the above assumption, (ii) is no longer true Hence, GM can lead to inconsistencies 19

20 Summarizing Performance GM: does not require a reference machine, but does not predict performance very well So we multiplied execution times and determined that sys-a is 1.2x faster but on what workload? AM: does predict performance for a specific workload, but that workload was determined by executing programs on a reference machine Every year or so, the reference machine will have to be updated 20

21 Normalized Execution Times Advantage of GM: no reference machine required Disadvantage of GM: does not represent any real entity and may not accurately predict performance Disadvantage of AM of normalized: need weights (which may change over time) Advantage: can represent a real workload 21

22 CPU Performance Equation Clock cycle time = 1 / clock speed CPU time = clock cycle time x cycles per instruction x number of instructions Influencing factors for each: clock cycle time: technology and pipeline CPI: architecture and instruction set design instruction count: instruction set design and compiler CPI (cycles per instruction) or IPC (instructions per cycle) can not be accurately estimated analytically 22

23 Lecture 1: System Metrics and Pipelining Today s topics: (Sections ) Power/Energy examples Performance summaries Measuring cost and dependability 23

24 Reducing Power and Energy Can gate off transistors that are inactive (reduces leakage) Design for typical case and throttle down when activity exceeds a threshold DFS: Dynamic frequency scaling -- only reduces frequency and dynamic power, but hurts energy DVFS: Dynamic voltage and frequency scaling can reduce voltage and frequency by (say) 10%; can slow a program by (say) 8%, but reduce dynamic power by 27%, reduce total power by (say) 23%, reduce total energy by 17% (Note: voltage drop slow transistor freq drop) 24

25 DVFS Example 25

26 Other Technology Trends DRAM density increases by 40-60% per year, latency has reduced by 33% in 10 years (the memory wall!), bandwidth improves twice as fast as latency decreases Disk density improves by 100% every year, latency improvement similar to DRAM Emergence of NVRAM technologies that can provide a bridge between DRAM and hard disk drives 26

27 Measuring Performance Two primary metrics: wall clock time (response time for a program) and throughput (jobs performed in unit time) To optimize throughput, must ensure that there is minimal waste of resources Performance is measured with benchmark suites: a collection of programs that are likely relevant to the user SPEC CPU 2006: cpu-oriented programs (for desktops) SPECweb, TPC: throughput-oriented (for servers) EEMBC: for embedded processors/workloads 27

28 Summarizing Performance Consider 25 programs from a benchmark set how do we capture the behavior of all 25 programs with a single number? P1 P2 P3 Sys-A Sys-B Sys-C Total (average) execution time Total (average) weighted execution time or Average of normalized execution times Geometric mean of normalized execution times 28

29 AM Example 29

30 AM Example We fixed a reference machine X and ran 4 programs A, B, C, D on it such that each program ran for 1 second The exact same workload (the four programs execute the same number of instructions that they did on machine X) is run on a new machine Y and the execution times for each program are 0.8, 1.1, 0.5, 2 With AM of normalized execution times, we can conclude that Y is 1.1 times slower than X perhaps, not for all workloads, but definitely for one specific workload (where all programs run on the ref-machine for an equal #cycles) With GM, you may find inconsistencies 30

31 GM Example Computer-A Computer-B Computer-C P1 1 sec 10 secs 20 secs P secs 100 secs 20 secs Conclusion with GMs: (i) A=B (ii) C is ~1.6 times faster For (i) to be true, P1 must occur 100 times for every occurrence of P2 With the above assumption, (ii) is no longer true Hence, GM can lead to inconsistencies 31

32 Summarizing Performance GM: does not require a reference machine, but does not predict performance very well So we multiplied execution times and determined that sys-a is 1.2x faster but on what workload? AM: does predict performance for a specific workload, but that workload was determined by executing programs on a reference machine Every year or so, the reference machine will have to be updated 32

33 Normalized Execution Times Advantage of GM: no reference machine required Disadvantage of GM: does not represent any real entity and may not accurately predict performance Disadvantage of AM of normalized: need weights (which may change over time) Advantage: can represent a real workload 33

34 CPU Performance Equation Clock cycle time = 1 / clock speed CPU time = clock cycle time x cycles per instruction x number of instructions Influencing factors for each: clock cycle time: technology and pipeline CPI: architecture and instruction set design instruction count: instruction set design and compiler CPI (cycles per instruction) or IPC (instructions per cycle) can not be accurately estimated analytically 34

35 Measuring System CPI Assume that an architectural innovation only affects CPI For 3 programs, base CPIs: 1.2, 1.8, 2.5 CPIs for proposed model: 1.4, 1.9, 2.3 What is the best way to summarize performance with a single number? AM, HM, or GM of CPIs? 35

36 Example AM of CPI for base case = 1.2 cyc cyc cyc /3 instr instr instr 5.5 cycles is execution time if each program ran for one instruction therefore, AM of CPI defines a workload where every program runs for an equal #instrs HM of CPI = 1 / AM of IPC ; defines a workload where every program runs for an equal number of cycles GM of CPI: warm fuzzy number, not necessarily representing any workload 36

37 Speedup Vs. Percentage Speedup is a ratio Improvement, Increase, Decrease usually refer to percentage relative to the baseline A program ran in 100 seconds on my old laptop and in 70 seconds on my new laptop What is the speedup? What is the percentage increase in performance? What is the reduction in execution time? 37

38 Wafers and Dies An entire wafer is produced and chopped into dies that undergo testing and packaging 38

39 Integrated Circuit Cost Cost of an integrated circuit = (cost of die + cost of packaging and testing) / final test yield Cost of die = cost of wafer / (dies per wafer x die yield) Dies/wafer = wafer area / die area - p wafer diam / die diag Die yield = wafer yield x (1 + (defect rate x die area) / a) -a Thus, die yield depends on die area and complexity arising from multiple manufacturing steps (a ~ 4.0) 39

40 Integrated Circuit Cost Examples Bottomline: cost decreases dramatically if the chip area is smaller, if the chip has fewer manufacturing steps (less complex), if the chip is produced in high volume (10% lower cost if volume doubles) A 30 cm diameter wafer cost $5-6K in 2001 Such a wafer yields about 366 good 1 cm 2 dies and 1014 good 0.49 cm 2 dies (note the effect of area and yield) Die sizes: Alpha cm 2, Itanium 3.0 cm 2, embedded processors are between cm 2 40

41 Contribution of IC Costs to Total System Cost Subsystem Cabinet: sheet metal, plastic, power supply, fans, cables, nuts, bolts, manuals, shipping box Fraction of total cost 6% Processor 22% DRAM (128 MB) 5% Video card 5% Motherboard 5% Processor board subtotal 37% Keyboard and mouse 3% Monitor 19% Hard disk (20 GB) 9% DVD drive 6% I/O devices subtotal 37% Software (OS + Office) 20% 41

42 Defining Fault, Error, and Failure A fault produces a latent error; it becomes effective when activated; it leads to failure when the observed actual behavior deviates from the ideal specified behavior Example I : a programming mistake is a fault; the buggy code is the latent error; when the code runs, it is effective; if the buggy code influences program output/behavior, a failure occurs Example II : an alpha particle strikes DRAM (fault); if it changes the memory bit, it produces a latent error; when the value is read, the error becomes effective; if program output deviates, failure occurs 42

43 Defining Reliability and Availability A system toggles between Service accomplishment: service matches specifications Service interruption: services deviates from specs The toggle is caused by failures and restorations Reliability measures continuous service accomplishment and is usually expressed as mean time to failure (MTTF) Availability measures fraction of time that service matches specifications, expressed as MTTF / (MTTF + MTTR) 43