CS 152 Computer Architecture and Engineering

Transcription

1 CS 152 Computer Architecture and Engineering Lecture Cache I John Lazzaro (not a prof - John is always OK) TA: Eric Love www-inst.eecs.berkeley.edu/~cs152/ Play: CS 152 L10: Cache I UC Regents Spring 2014 UCB 1

2 Today: Caches and the Memory System Static Memory: Used in cache designs. Short Break Memory Hierarchy: Technology motivation for caching. Processor Input Control Memory Datapath Output CS 250 L10: Memory UC Regents Fall 2013 UCB 2

3 Static Memory Circuits Dynamic Memory: Circuit remembers for a fraction of a second. Static Memory: Circuit remembers as long as the power is on. Non-volatile Memory: Circuit remembers for many years, even if power is off. CS L8: Cache UC Regents Fall 2008 UCB 3

4 Preliminaries CS 152 L11: VLSI UC Regents Fall 2006 UCB 4

5 Inverters: Building block for SRAM Vdd symbol Vin Vout Vin Vout CS 152 L11: VLSI UC Regents Fall 2006 UCB 5

6 Inverter: Die Cross Section Vout Vin oxide n+ n+ p- Vin oxide p+ p+ n+ n-well Vin Vout CS 152 L11: VLSI UC Regents Fall 2006 UCB 6

7 Recall: Our simple inverter model... In Inverter Out Out = In Correctly predicts logic output for simple static CMOS circuits. In 0 1 Out Circuit In 1 0 Vdd PMOS Out NMOS Extensions to model subtler circuit families, or to predict timing, have not worked well... pfet. A switch. On if gate is grounded. nfet. A switch. On if gate is at Vdd. CS 250 L3: Timing UC Regents Fall 2013 UCB 7

8 When the 0/1 model is too simple... I sd Vth We wire the output of the inverter to drive its input. What happens? Vin I ds Vout Logic simulators based on our too-simple model predict this circuit will oscillate! This prediction is incorrect. In reality, V in = V out settles to a stable value, defined as Vth, where nfet and pfet current match. Can we figure out Vth, without solving tedious equations? CS 152 L11: VLSI UC Regents Fall 2006 UCB 8

9 Graphical equation solving... Vth nfet I ds Intersection defines Vth I sd pfet I sd Vin Vout I ds CS 152 L11: VLSI Note: Ignores second-order effects. Vin = Vout Recall: Graphs from power and energy lecture... UC Regents Fall 2006 UCB 9

10 Recall: Transistors as water valves If electrons are water molecules, transistor strengths (W/L) are pipe diameters, and capacitors are buckets... Vdd 1 A on p-fet fills up the capacitor with charge. Open Charge 0 Water level Time Vdd Vdd 1 A on n-fet empties the bucket. n Open Out Discharge 0 Water level Time CS 250 L3: Timing UC Regents Fall 2013 UCB 10

11 What happens when we break tie wire? Small amounts of noise on Vin causes Ids > Isd or Isd > Ids... and output bucket randomly fills and empties. Result: Vout randomly flips between logic 0 and logic 1. Tie wire broken I ds I sd I sd Vin Vout I ds Vth CS 152 L11: VLSI Vin left free to float. UC Regents Fall 2006 UCB 11

12 SRAM 1971 state of the art. Intel 2102, a 1kb, 1 MHz static RAM chip with 6000 nfets transistors in a 10 μm process. CS 250 L1: Fab/Design Interface UC Regents Fall 2013 UCB 12

13 Recall DRAM cell: 1 T + 1 C Word Line Row Column Bit Line Column Row Word Line Vdd CS L8: Cache Bit Line UC Regents Fall 2008 UCB 13

14 Idea: Store each bit with its complement x x Row Why? y Gnd Vdd Vdd Gnd We can use the redundant representation to compensate for noise and leakage. y CS L8: Cache UC Regents Fall 2008 UCB 14

15 Case #1: y = Gnd, y = Vdd... x x Row I sd y y Gnd Vdd I ds CS L8: Cache UC Regents Fall 2008 UCB 15

16 Case #2: y = Vdd, y = Gnd... x x Row I sd y Vdd y Gnd I ds CS L8: Cache UC Regents Fall 2008 UCB 16

17 Combine both cases to complete circuit Gnd Vdd Vth Vth Vdd Gnd Crosscoupled inverters noise noise y y x CS L8: Cache x UC Regents Fall 2008 UCB 17

18 SRAM Challenge #1: It s so big! SRAM area is 6-10X DRAM area, same generation... Cell has both transistor types Vdd AND Gnd Capacitors are usually parasitic capacitance of wires and transistors. CS L8: Cache More contacts, more devices, two bit lines... UC Regents Fall 2008 UCB 18

19 Intel SRAM core cell (45 nm) Bit Lines Word Lines 19

20 Challenge #2: Writing is a fight When word line goes high, bitlines fight with cell inverters to flip the bit -- must win quickly! Solution: tune W/L of cell & driver transistors Initial state Vdd Initial state Gnd CS L8: Cache Bitline drives Gnd Bitline drives Vdd UC Regents Fall 2008 UCB 20

21 Challenge #3: Preserving state on read When word line goes high on read, cell inverters must drive large bitline capacitance quickly, to preserve state on its small cell capacitances Cell state Vdd Cell state Gnd Bitline a big capacitor Bitline a big capacitor CS L8: Cache UC Regents Fall 2008 UCB 21

22 SRAM array: like DRAM, but non-destructive Architects specify number of rows and columns. Word and bit lines slow down as array grows larger! Din 3 Din 2 Din 1 Din 0 Precharge WrEn Parallel Data I/O Lines CS 250 L10: Memory WrWrite Driver & WrWrite Driver & WrWrite Driver & WrWrite Driver & - Precharger Driver + - Precharger Driver + - Precharger Driver + - Precharger Driver + SRAM Cell SRAM Cell SRAM Cell SRAM Cell SRAM Cell SRAM Cell SRAM Cell SRAM Cell SRAM Cell SRAM Cell SRAM Cell : : : : SRAM Cell - Sense Amp + - Sense Amp + - Sense Amp + - Sense Amp + Dout 3 Dout 2 Dout 1 Dout 0 Word 0 Word 1 Word 15 Address Decoder A0 A1 A2 A3 Add muxes to select subset of bits Q: Which is longer: word line or bit line? For large SRAMs: Tile a small array, connect with muxes, decoders. UC Regents Fall 2013 UCB 22

23 SRAM vs DRAM, pros and cons Big win for DRAM DRAM has a 6-10X density advantage at the same technology generation. SRAM advantages SRAM has deterministic latency: its cells do not need to be refreshed. SRAM is much faster: transistors drive bitlines on reads. SRAM easy to design in logic fabrication process (and premium logic processes have SRAM add-ons) CS L8: Cache UC Regents Fall 2008 UCB 23

24 RAM Compilers Fig degree image of 22 nm tri-gate LVC SRAM bitcell. On average, 30% of a modern logic chip is SRAM, which is generated by RAM compilers. Compile-time parameters set number of bits, aspect ratio, ports, etc. Fig nm HDC and LVC SRAM bitcells. Figure : 22nm HDC CS 250 L1: Fab/Design Interface low voltage, achieving low SRAM minimum operati and LVC Tri-gate SRAM bitcells. is desirable to avoid integration, routing, and control of multiple supply domains. In the 22 nm tri-gate technology, fin quantization the fine-grained width tuning conventionally used to read stability and write margin and presents a ch designing minimum-area SRAM bitcells constrain pitch. The 22 nm process technology includes bo density m 6T SRAM bitcell (HDC) and a lo m 6T SRAM bitcell (LVC) to support tradeo performance, and minimum operating voltage acro of application requirements. In Fig. 1, a 45-degree im LVC tri-gate SRAM is pictured showing the UCB thin s UC Regents Fall 2013 wrapped on three sides by a polysilicon gate. The 24

25 Flip Flops Revisited CS 250 L10: Memory UC Regents Fall 2013 UCB 25

26 Recall: Static RAM cell (6 Transistors) Gnd Vdd Vth Vth Vdd Gnd Crosscoupled inverters noise noise x CS 250 L10: Memory x! UC Regents Fall 2013 UCB 26

27 Recall: Positive edge-triggered flip-flop D Q A flip-flop samples right before the edge, and then holds value. clk Sampling circuit Holds value clk clk Clock to Q delay results fr 16 Transistors: Makes an SRAM look compact! What do we get for the 10 extra transistors? Clocked logic semantics. clk CS 250 L10: Memory UC Regents Fall 2013 UCB 27

28 Sensing: When clock is low D Q clk A flip-flop samples right before the edge, and then holds value. Sampling circuit Holds value clk clk clk = 0 = 1 clk Clock to Q delay results fr clk clk clk CS 250 L10: Memory Will capture new value on posedge. Clock to Q delay results fr Outputs clk last value captured. UC Regents Fall 2013 UCB 28

29 Capture: When clock goes high D Q clk A flip-flop samples right before the edge, and then holds value. Sampling circuit Holds value clk clk clk = 1 = 0 Clock to clk Q delay results fr clk clk clk CS 250 L10: Memory Remembers value just captured. Clock to Q delay results fr Outputs value clk just captured. UC Regents Fall 2013 UCB 29

30 Flip Flop delays: clk-to-q? setup? hold? clk D Q CLK clk clk Clock to Q delay results fr CLK == 0 Sense D, but Q outputs old value. setup clk CLK 0->1 Capture D, pass value to Q hold clk-to-q CS 250 L10: Memory UC Regents Fall 2013 UCB 30

31 From flip-flops to latches... D Q clk Sampling circuit Holds value clk clk Latch-based design: Break up the flip-flop circuit into two latch state elements. Then, add combinational logic between the latches. CS 250 L3: Timing Clock to Q delay results fr D CLK D Q clk D Q Latches are good for making small memories. Saves half the area over using D flip-flops. Q UC Regents Fall 2013 UCB 31

32 Break CS 152 L10: Cache I Play: UC Regents Spring 2014 UCB 32

33 The Memory Hierarchy CS L8: Cache UC Regents Fall 2008 UCB 33

34 60% of the area of this CPU is devoted to SRAM cache. But the role of cache in computer design has varied widely over time. CS 152 L14: Cache I UC Regents Spring 2005 UCB 34

35 1977: DRAM faster than microprocessors Apple ][ (1977) CPU: 1000 ns DRAM: 400 ns Steve Jobs Steve Wozniak CS L8: Cache UC Regents Fall 2008 UCB 35

36 Since then: Technology scaling... Circuit in 250 nm technology (introduced in 2000) Same circuit in 180 nm technology (introduced in 2003) Each dimension 30% smaller. Area is 50% smaller 0.7 x L nm L nanometers long Logic circuits use smaller C s, lower Vdd, and higher kn and kp to speed up clock rates. CS L8: Cache UC Regents Fall 2008 UCB 36

37 DRAM scaled for more bits, not more MHz CS L8: Cache Assume Ccell = 1 ff Bit line may have 2000 nfet drains, assume bit line C of 100 ff, or 100*Ccell. Ccell holds Q = Ccell*(Vdd-Vth) When we dump this charge onto the bit line, what voltage do we see? dv = [Ccell*(Vdd-Vth)] / [100*Ccell] dv = (Vdd-Vth) / 100 tens of millivolts! In practice, scale array to get a 60mV signal. UC Regents Fall 2008 UCB 37

38 , CPU speed outpaced DRAM... Performance (1/latency) Q. How do architects address this gap? A. Put smaller, faster cache memories between CPU and DRAM. Create a memory hierarchy. CPU 60% per yr 2X in 1.5 yrs CPU Gap grew 50% per year DRAM 9% per yr 2X in 10 yrs The power wall DRAM Year CS L8: Cache UC Regents Fall 2008 UCB 38

39 Caches: Variable-latency memory ports Data in upper memory returned with lower latency. Data in lower level returned with higher latency. Data To Processor Address From Processor Upper Level Memory Small, fast Blk X Lower Level Memory Large, slow Blk Y From CPU To CPU CS L8: Cache UC Regents Fall 2008 UCB 39

40 Queues as a building block for memory systems Avoid blocking by using a queue (a First-In, First-Out buffer, or FIFO) to communicate between two sub-systems. 40

41 Variable-latency port that doesn t stall on a miss CPU makes a request by placing the following items in Queue 1: CMD: Read, write, etc... From CPU To CPU Queue 1 Queue 2 MTYPE: 8-bit, 16-bit, 32-bit, or 64-bit. TAG: 9-bit number identifying the request. MADDR: Memory address of first byte. STORE-DATA: For stores, the data to store. 41

42 This cache is used in an ASPIRE CPU (Rocket) When request is ready, cache places the following items in Queue 2: From CPU To CPU Queue 1 Queue 2 TAG: Identity of the completed command. LOAD-DATA: For loads, the requested data. CPU saves info about requests, indexed by TAG. Why use TAG approach? Multiple misses can proceed in parallel. Loads can return out of order. 42

43 Cache replaces data, instruction memory IF (Fetch) ID (Decode) EX (ALU) MEM WB Replace with Instruction Cache and Data Cache of DRAM main memory + 0x4 IR Mux,Logic RegFile rs1 rs2 rd1 IR A op A L U 32 IR Y Data Memory Addr Dout Din WE MemToReg IR R ws wd rd2 WE M M PC Instr Mem D Q Addr Data Ext B CS L8: Cache UC Regents Fall 2008 UCB 43

44 Recall: Intel ARM XScale CPU (PocketPC) 32 KB Instruction Cache 32 KB Data Cache 180 nm process (introduced 2003) CS L8: Cache UC Regents Fall 2008 UCB 44

45 ARM CPU 32 KB instruction cache uses 3 million transistors Typical miss rate: 1.5% DRAM interface uses 61 pins that toggle at 100 MHz 45

46 2005 Memory Hierarchy: Apple imac G5 Managed by compiler Managed by hardware Managed by OS, hardware, application Reg L1 Inst L1 Data L2 DRAM Disk Size 1K 64K 32K 512K 256M 80G Latency (cycles) M Goal: Illusion of large, fast, cheap memory Let programs address a memory space that scales to the disk size, at a speed that is usually as fast as register access imac G5 1.6 GHz $ CS L8: Cache UC Regents Fall 2008 UCB 46

47 90 nm, 58 M transistors L1 (64K Instruction) 512K L2 R e g i s t e r s (1K) CS 152 L14: Cache I L1 (32K Data) PowerPC UC Regents Spring UCB FX 47

48 CS L8: Cache Latency: A closer look Read latency: Time to return first byte of a random access Reg L1 Inst L1 Data L2 DRAM Disk Size 1K 64K 32K 512K 256M 80G Latency (cycles) E+07 Latency (sec) 0.6n 1.9n 1.9n 6.9n 100n 12.5m Hz 1.6G 533M 533M 145M 10M 80 Architect s latency toolkit: (1) Parallelism. Request data from N 1-bit-wide memories at the same time. Overlaps latency cost for all N bits. Provides N times the bandwidth. Requests to N memory banks (interleaving) have potential of N times the bandwidth. (2) Pipeline memory. If memory has N cycles of latency, issue a request each cycle, receive it N cycles later. UC Regents Fall 2008 UCB 48

49 Recall: Adding pipeline stages to memory Before we pipelined, slow! Only read behavior shown. A7-A0: 8-bit read address 3 A7 A6 A5 A4 A3 { { A2 3 Can we add two pipeline stages? 1 D E M U X... OE OE OE OE --> Tri-state Q outputs! Byte 0-31 Byte Byte Q Q Q M U X 3 Data output is 32 bits D0-D31 32 i.e. 4 bytes Each register holds 32 bytes (256 bits) CS 152: L6: Superpipelining + Branch Prediction UC Regents Spring 2014 UCB 49

50 Recall: Reading an entire row for later use Thus, push to faster DRAM interfaces 13-bit row address input 1 o f d e c o d e r CS 152 L9: Memory What if we want all of the bits? In row access time (55 ns) we can do 22 transfers at 400 MT/s. 16-bit chip bus -> 22 x 16 = 352 bits << Now the row access time looks fast! 8192 rows columns usable bits (tester found good bits in bigger array) bits delivered by sense amps Select requested bits, send off the chip UC Regents Spring 2014 UCB 50

51 Recall: Interleaved access to 4 banks CK# T0 T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 CK Command ACT READ ACT READ ACT READ ACT READ NOP NOP ACT Address Row Col Row Col Row Col Row Col Row Bank address Bank a Bank a Bank b Bank b Bank c Bank c Bank d Bank d Bank e t RRD (MIN) t FAW (MIN) Interleaving: Design the right interface to the 4 memory banks on the chip, so several row requests run in parallel. Bank a Bank b Bank c Bank d Can also do other commands on banks concurrently. CS 152 L9: Memory UC Regents Spring 2014 UCB 51

52 Recall: Leveraging banks and row reads (A) Without access scheduling (56 DRAM Cycles) Time (Cycles) References (Bank, Row, Column) (0,0,0) (0,1,0) (0,0,1) (0,1,3) (1,0,0) (1,1,1) (1,0,1) (1,1,2) P A C P A C P A C P A C P A C P A C P A C P A C (B) With access scheduling (19 DRAM Cycles) Time (Cycles) DRAM Operations: References (Bank, Row, Column) (0,0,0) (0,1,0) (0,0,1) (0,1,3) (1,0,0) (1,1,1) (1,0,1) (1,1,2) P A C P A 8 C P A C C C P A C C C P: bank precharge (3 cycle occupancy) A: row activation (3 cycle occupancy) C: column access (1 cycle occupancy) From: Memory Access Scheduling CS 152 L9: Memory Scott Rixner 1, William J. Dally, Ujval J. Kapasi, Peter Mattson, and John D. Owens UC Regents Spring 2014 UCB 52

53 Set scheduling algorithms in gates... V L/S Row Col Data State Memory Access Scheduler Logic Precharge 0 Bank 0 Pending References Row Arbiter 0 Memory References Column Arbiter Address Arbiter DRAM Operations V L/S Row Col Data State Row Bank N Pending References Arbiter N Precharge N Figure 4. Memory access scheduler architecture. From: Memory Access Scheduling CS 250 L11: DRAM Scott Rixner 1, William J. Dally, Ujval J. Kapasi, Peter Mattson, and John D. Owens UC Regents Fall 2009 UCB 53

54 On Tuesday Caches, part two... Have a good weekend! 54