Design-Induced Latency Variation in Modern DRAM Chips:

Transcription

1 Design-Induced Latency Variation in Modern DRAM Chips: Characterization, Analysis, and Latency Reduction Mechanisms Donghyuk Lee 1,2 Samira Khan 3 Lavanya Subramanian 2 Saugata Ghose 2 Rachata Ausavarungnirun 2 Gennady Pekhimenko 4 Vivek Seshadri 4 Onur Mutlu 5,2 1 NVIDIA 2 Carnegie Mellon University 3 University of Virginia 4 Microsoft Research 5 ETH Zürich June 7, 2017

2 What Is Design-Induced Variation? across column distance from wordline driver Inherently fast wordline drivers fast slow sense amplifiers slow fast inherently slow across row distance from sense amplifier Systematic variation in cell access times caused by the physical organization of DRAM 2

3 Executive Summary Design-Induced Variation Inherently slow regions due to DRAM cell array organization Goal: Use design-induced variation to reduce DRAM latency Analysis: Characterization of 96 real DRAM modules Three types of systematic variation due to design Great potential to reduce DRAM latency at low cost Our Approach: DIVA-DRAM DIVA Profiling: Reliably reduce DRAM latency Profile only cells in inherently slow regions Use error correction (ECC) for slow cells that are not profiled DIVA Shuffling: Exploit variation to improve ECC reliability 15.1%/14.2% higher performance for 2-/8-core workloads 3

4 Presentation Outline DRAM Background Experimental Study of Design-Induced Variation Goal: Understand, identify inherently slower regions Methodology: Profile 96 real DRAM modules by using FPGA-based DRAM test infrastructure Exploiting Design-Induced Variation DIVA Profiling: Using low-cost slow region profiling to reliably and dynamically reduce DRAM latency DIVA Shuffling: Using variation across data bursts to reduce uncorrectable errors 4

5 High-Level DRAM Organization memory channel: 64-bit data bus 8-bit data bus per chip DRAM chip Processor DIMM dual in-line memory module 5

6 High-Level DRAM Organization memory channel: 64-bit data bus 8-bit data bus per chip Processor data burst Read request DIMM dual in-line memory module 6

7 High-Level DRAM Organization memory channel: 64-bit data bus 8-bit data bus per chip Processor 8 chips X 8 bits X 8 bursts = 64 bytes Read request DIMM dual in-line memory module 7

8 Organization Inside a DRAM Chip DRAM mat DRAM cell wordline drivers sense amplifiers 8

9 Organization Inside a DRAM Chip global wordline row group row decoder row decoder 9

10 DRAM Operations Memory controller sends commands to DRAM row decoder 10

11 DRAM Operations Memory controller sends commands to DRAM row decoder Activation: Open one row in each mat Row decoder, wordline drivers select a row of cells 11

12 DRAM Operations Memory controller sends commands to DRAM row decoder Activation: Open one row in each mat Row decoder, wordline drivers select a row of cells Each cell in the row shares charge with a sense amplifier 12

13 DRAM Operations Memory controller sends commands to DRAM row decoder column column column column Activation: Open one row in each mat Row decoder, wordline drivers select a row of cells Each cell in the row shares charge with a sense amplifier Read: Send one column of data from sense amplifiers to CPU 13

14 DRAM Operations Memory controller sends commands to DRAM row decoder Activation: Open one row in each mat Row decoder, wordline drivers select a row of cells Each cell in the row shares charge with a sense amplifier Read: Send one column of data from sense amplifiers to CPU Precharge: Prepare mats for next row 14

15 DRAM Operations Memory controller sends commands to DRAM row decoder Activation: Open one row in each mat Row decoder, wordline drivers select a row of cells Each cell in the row shares charge with a sense amplifier Read: Send one column of data from sense amplifiers to CPU Precharge: Prepare mats for next row Timing parameters: how long to wait for each step to finish 15

16 DRAM Timing Parameters Standard timing parameters are dictated by the worst case Must ensure reliable operation for: The smallest cell with the smallest charge in all DRAM modules (process variation) The highest operating temperature allowed (charge leakage) Large timing margin for the common case Goal: Lower common-case latency 16

17 DRAM Timing Parameters Standard timing parameters are dictated by the worst case Must ensure reliable operation for: The smallest cell with the smallest charge in all DRAM modules (process variation) The highest operating temperature allowed (charge leakage) Large timing margin for the common case Can we use design-induced variation to find and Goal: use Lower common-case common-case latency latency at low cost? 17

18 Presentation Outline DRAM Background Experimental Study of Design-Induced Variation Goal: Understand, identify inherently slower regions Methodology: Profile 96 real DRAM modules by using FPGA-based DRAM test infrastructure Exploiting Design-Induced Variation DIVA Profiling: Using low-cost slow region profiling to reliably and dynamically reduce DRAM latency DIVA Shuffling: Using variation across data bursts to reduce uncorrectable errors 18

19 Expected DRAM Characteristics Variation Some regions are slower than others If we reduce DRAM latency, slower regions are more vulnerable to errors Repeatability Latency (error) characteristics repeat periodically, if the same component (e.g., mat) is duplicated Similarity Characteristics repeat across different organizations (e.g., chip/dimm) that share same design 19

20 Characterization Methodology We use error behavior when we reduce latency to infer DRAM organization and characteristics FPGA-based memory controller infrastructure 20

21 DRAM Testing Infrastructure Temperature Controller FPGAs Heater FPGAs PC Tested 96 DIMMs from three vendors 21

22 Characterization Methodology We use error behavior when we reduce latency to infer DRAM organization and characteristics FPGA-based memory controller infrastructure We reverse engineer row address mapping Lower trp (precharge timing parameter) to 7.5 ns We characterize three types of variation Variation across columns Variation across rows Variation across data bursts Data and circuit model will be available on GitHub: 22

23 row decoder 1. Variation Across Rows darker cells are faster global wordline 512 rows row group Latency characteristics vary across 512 rows 23

24 row decoder row decoder 1. Variation Across Rows darker cells are faster global wordline 512 rows 512 rows row group Latency characteristics vary across 512 rows Same organization repeats every 512 rows 24

25 1. Variation Across Rows darker cells are faster global wordline row group 512 rows 512 rows row decoder row decoder sweep across rows Latency characteristics vary across 512 rows Latency Same characteristics organization repeats repeat every every rows rows 25

26 1.2. Periodic Row Variation Behavior Sorting with Discovered Row Mapping Erroneous Request Count sorted row address Row error (latency) characteristics periodically repeat every 512 rows 26

27 2. Variation Across Columns row decoder darker cells are faster global wordline Column latency depends on distance from row decoder, wordline driver 27

28 2. Variation Across Columns row decoder darker cells are faster global wordline Column latency depends on distance from row decoder, wordline driver 28

29 2. Variation Across Columns Erroneous Request Count Column error (latency) characteristics have specific patterns that repeat across row groups 29

30 3. Variation Across Data Bursts Processor Read request 64-bit data bus in memory channel A B8-bit Cdata D bus Eper FchipG H A B C D E F G H A B C D E F G H A B C D E F G H A B C D E F G H A A A B B B C C C D D D E E E F F F G G G H H H 64-bit data from different locations in the same row in the same chip DIMM Chip 1 Chip 2 Chip 3 Chip 4 Chip 5 Chip 6 Chip 7 Chip 8 30

31 3. Variation Across Data Bursts Error Count chip 1 chip 2 chip 3 chip 4 chip 5 chip 6 chip 7 chip 8 A B C D E F G H data bits in 8 data bursts Specific bits in a request induce more errors 31

32 Summary: Design-Induced Variation Systematic variation across rows Slow cells further from sense amplifier Systematic variation across columns Slow cells further from row decoder Slow cells further from wordline driver Systematic variation across data bursts Slow cells at certain bits in a burst Clustering of errors Can we use design-induced variation to find and use common-case latency at low cost? 32

33 Presentation Outline DRAM Background Experimental Study of Design-Induced Variation Goal: Understand, identify inherently slower regions Methodology: Profile 96 real DRAM modules by using FPGA-based DRAM test infrastructure Exploiting Design-Induced Variation DIVA Profiling: Using low-cost slow region profiling to reliably and dynamically reduce DRAM latency DIVA Shuffling: Using variation across data bursts to reduce uncorrectable errors 33

34 Challenges of Lowering Latency Static DRAM latency (e.g., AL-DRAM [HPCA 2015]) DRAM vendors need to provide fixed timings, increasing testing costs Doesn t account for latency changes over time (e.g., aging and wear out) Conventional online profiling Takes long time (high cost) to profile all DRAM cells Our Goal: Use design-induced variation to minimize profiling 34

35 1. DIVA Profiling Design-Induced-Variation-Aware wordline driver inherently slow sense amplifier Profile only slow regions to determine latency 35

36 What About Process Variation? Design-Induced-Variation-Aware slow cells process variation random error wordline driver inherently slow architectural variation localized error sense amplifier 36

37 What About Process Variation? Design-Induced-Variation-Aware slow cells process variation random error error-correcting codes (ECC) wordline driver sense amplifier inherently slow architectural variation localized error online profiling Combine ECC & online profiling Reliably reduce DRAM latency at low cost 37

38 Correction with Conventional ECC Processor Read request 64-bit data bus in memory channel 8-bit data bus per chip DIMM Error-Correcting Code (ECC) 38

39 Challenge of Conventional ECC Processor error 8-bit data bus per chip DIMM uncorrectable by ECC uncorrectable by ECC 39

40 Challenge of Conventional ECC Processor error 8-bit data bus per chip DIMM uncorrectable by ECC uncorrectable by ECC Clusters of slow cells due to design-induced variation lead to more uncorrectable errors 40

41 2. DIVA Shuffling Design-Induced-Variation-Aware Processor error DIMM 8-bit data bus per chip Shuffle data bursts 41

42 2. DIVA Shuffling Design-Induced-Variation-Aware Processor error DIMM 8-bit data bus per chip Shuffle data bursts Reduce uncorrectable errors 42

43 2. DIVA Shuffling Design-Induced-Variation-Aware Processor error DIMM 8-bit data bus per chip How do DIVA Profiling and DIVA Shuffling perform? Shuffle data bursts Reduce uncorrectable errors 43

44 DIVA Shuffling Improves ECC Fraction of Errors Removed 100% 80% 60% 40% 20% 0% ECC w/o DIVA AVA Suffling ECC with AVA DIVA Suffling Uncorrectable 33 DIMMs average % 16.1% DIVA Shuffling uses architectural variation to improve error correction using the same codeword 44

45 DIVA-DRAM Reduces Latency 50% Read 50% Write Latency Reduction 40% 30% 20% 10% 0% 35.1%34.6% 36.6% 35.8% 31.2% 25.5% 55 C 85 C 55 C 85 C 55 C 85 C 40% 30% 20% 10% 0% 39.4%38.7% 41.3% 40.3% 36.6% 27.5% 55 C 85 C 55 C 85 C 55 C 85 C AL-DRAM DIVA AVA Profiling DIVA AVA Profiling + Shuffling AL-DRAM DIVA AVA Profiling DIVA AVA Profiling + Shuffling 45

46 Latency Reduction DIVA-DRAM Reduces Latency 50% 40% 30% 20% 10% 0% Read 35.1%34.6% 36.6% 35.8% 31.2% 25.5% 55 C 85 C 55 C 85 C 55 C 85 C AL-DRAM DIVA AVA Profiling DIVA AVA Profiling + Shuffling 50% 40% 30% 20% 10% 0% 36.6% 27.5% Write 39.4%38.7% 41.3% 40.3% DIVA-DRAM reduces latency more aggressively and uses ECC to correct random slow cells 55 C 85 C 55 C 85 C 55 C 85 C AL-DRAM DIVA AVA Profiling DIVA AVA Profiling + Shuffling How do DRAM latency reductions translate to system performance? 46

47 DIVA-DRAM Improves Performance 32 single-core benchmarks: SPEC, Stream, TPC, GUPS 96 multicore workloads constructed with benchmarks 20% AL-DRAM DIVA AVA Profiling DIVA AVA Profiling + Shuffling System Performance Improvement 15% 10% 5% 7.0% 9.2% 9.5% 11.7% 14.7% 15.1% 13.7% 14.2% 13.8% 14.1% 11.0% 11.5% 0% 1-core 2-core 4-core 8-core 47

48 DIVA-DRAM Improves Performance 32 single-core benchmarks: SPEC, Stream, TPC, GUPS 96 multicore workloads constructed with benchmarks System Performance Improvement 20% 15% 10% 5% AL-DRAM DIVA AVA Profiling DIVA AVA Profiling + Shuffling 7.0% 9.2% 9.5% 11.7% 14.7% 15.1% 13.7% 14.2% 13.8% 14.1% 11.0% 11.5% DIVA-DRAM 0% outperforms the best prior work 1-core 2-core 4-core 8-core and can adapt to dynamic latency changes 48

49 Conclusion Design-Induced Variation: Inherently slow regions due to DRAM cell array organization Analysis: Characterization of 96 real DRAM modules Three types of systematic variation due to design Great potential to reduce DRAM latency at low cost Our Approach: DIVA-DRAM DIVA Profiling: Reliably reduce DRAM latency Profile only cells in inherently slow regions Use error correction (ECC) for slow cells that are not profiled DIVA Shuffling: Exploit variation to improve ECC reliability 15.1%/14.2% higher performance for 2-/8-core workloads 49

50 Design-Induced Latency Variation in Modern DRAM Chips: Characterization, Analysis, and Latency Reduction Mechanisms Donghyuk Lee 1,2 Samira Khan 3 Lavanya Subramanian 2 Saugata Ghose 2 Rachata Ausavarungnirun 2 Gennady Pekhimenko 4 Vivek Seshadri 4 Onur Mutlu 5,2 1 NVIDIA 2 Carnegie Mellon University 3 University of Virginia 4 Microsoft Research 5 ETH Zürich Data, Circuit Model Will Be Available at

51 Backup Slides 51

52 Hierarchical Organization of DRAM 52

53 Sending Data From a DRAM Chip row decoder global sense amplifiers 64 bits IO interface 8 bits X 8 bursts Data in a request transferred as multiple data bursts 53

54 DRAM Stores Data as Charge DRAM cell Three steps of charge movement Sense amplifier 54

55 DRAM Stores Data as Charge DRAM cell Three steps of charge movement 1. Sensing Sense amplifier 55

56 DRAM Stores Data as Charge DRAM cell Three steps of charge movement 1. Sensing 2. Restore Sense amplifier 56

57 DRAM Stores Data as Charge DRAM cell Three steps of charge movement 1. Sensing 2. Restore 3. Precharge Sense amplifier 57

58 DRAM Charge over Time cell Sense amplifier charge cell Sense amplifier Data 1 Data 0 time 58

59 DRAM Charge over Time cell Sense amplifier charge cell Sense amplifier Sensing Data 1 Data 0 time 59

60 DRAM Charge over Time cell cell Data 1 charge Sense amplifier Sense amplifier Timing Parameters Sensing Restore In theory In practice margin Data 0 time Why does DRAM need the extra timing margin? 60

61 Two Reasons for Timing Margin 1. Process Variation DRAM cells are not equal Leads to extra timing margin for cells that can store large small amount of charge; 2. Temperature Dependence DRAM leaks more charge at higher temperature Leads to extra timing margin when operating at low temperature 61

62 DRAM Cells are Not Equal Ideal Real Same size Same charge Same latency Different size Different charge Different latency 62

63 DRAM Cells are Not Equal Ideal Real Smallest cell Large variation in cell size Largest cell Large variation in charge Large variation in access latency 63

64 Two Reasons for Timing Margin 1. Process Variation DRAM cells are not equal Leads to extra timing margin for cells that can store large amount of charge 2. Temperature Dependence DRAM leaks more charge at higher temperature Leads to extra timing margin when operating at low temperature 64

65 Charge Leakage Room Temp. Temperature Hot Temp. (85 C) Small leakage Large leakage 65

66 Charge Leakage Room Temp. Temperature Hot Temp. (85 C) Cells store small charge at high temperature and large charge at low temperature Large variation in access latency 66

67 DRAM Timing Parameters DRAM timing parameters are dictated by the worst case The smallest cell with the smallest charge in all DRAM products Operating at the highest temperature Large timing margin for the common case Can lower latency for the common case 67

68 DRAM Timing Parameters Command ACTIVATE Data Duration 43 68

69 DRAM Timing Parameters 1 Activation latency: trcd (13ns / 50 cycles) Command Data Duration ACTIVATE READ Cache line (64B) 43 69

70 DRAM Timing Parameters 1 2 Activation latency: trcd (13ns / 50 cycles) Precharge latency: trp (13ns / 50 cycles) Command Data Duration ACTIVATE READ PRECHARGE Cache line (64B) Next ACT 43 70

71 Design-Induced Variation in Open Bitline DRAM 71

72 72 Challenge: External Internal DRAM chip Internal address Address mapping IO interface External address External address Internal address

73 DRAM-Internal vs. DRAM-External IntMSB IntMID IntLSB near distance from s/a far ExtMID ExtMSB ExtLSB low error count high Estimated Mapping External Internal ExtMSB IntMID 4/4 = 100% ExtMID IntMSB 4/4 = 100% ExtLSB IntLSB 3/4 = 75% Estimated Mapping (External Internal) Based on Error Counts for the External Address 73

74 Row Address Mapping Confidence Confidence 120% 100% 80% 60% 40% 20% 0% IntMSB IntLSB

75 1.1. Measuring Row Variation Lower trp (precharge timing parameter) to 7.5 ns Erroneous Request Count row address (mod. 512) Periodic Errors 75

76 1.1. Measuring Row Variation Lower trp (precharge timing parameter) to 7.5 ns Erroneous Request Count row address (mod. 512) Need to reverse engineer Periodic row address mapping details in Errors our paper 76

77 1.2. Periodic Row Variation Behavior Sorting with Discovered Row Mapping Erroneous Request Count sorted row address Row error (latency) characteristics periodically repeat every 512 rows 77

78 Erroneous Request Count Variation in Rows trp (precharge timing parameter) 10.0 ns 7.5 ns 5.0 ns row addr. (mod. 512) row addr. (mod. 512) row addr. (mod. 512) Random Errors Periodic Errors Mostly Errors 78

79 1.2. Periodic Row Variation Behavior Aggregated & Sorted Apply sorted order to each 512-row group Erroneous Request Count row address (mod. 512) row address Error (latency) characteristics periodically repeat every 512 rows 79

80 2. Variation Across Columns row decoder 80

81 2. Variation Across Columns row decoder column column column global sense amplifier 64 bits IO interface global wordline 8 bits X 8 bursts column Different columns data from different locations different characteristics 81

82 3. Variation in Data Bits row decoder global sense amplifier 64 bits IO interface 8 bits X 8 bursts Data in a request transferred as multiple data bursts 82

83 DIVA-DRAM Evaluation Methodology Modified version of Ramulator (cycle accurate DRAM simulator) 83

84 Erroneous Request Count ms 128ms 64ms Row Address (modulo 512)

85 Erroneous Request Count C 65 C 45 C Row Address (modulo 512)

86 K K Error Count Ratio K vendor A vendor B Company A Company B Company C Tested DIMMs vendor C Error Count Ratio K vendor A vendor B Company A Company B Company C Tested DIMMs vendor C

87 global wordline driver local wordline driver cell capacitor access transistor wordline bitline parasitic resistance & capacitance rows (cells per bitline) local sense amplifier local sense amplifier mat 512 columns (cells per wordline) mat

88

89