Page 1. Why Care About the Memory Hierarchy? Memory. DRAMs over Time. Virtual Memory!

Transcription

1 Why Care About the Memory Hierarchy? Memory Virtual Memory -DRAM Memory Gap (latecy) Reasos: Multi process systems (abstractio & memory protectio) Solutio: Tables (holdig per process traslatios) Fast traslatio by Traslatio Lookaside Buffer (TLB) Demad pagig (OS support Performace 100 -Memory Performace Gap: (grows 50% / year) 10 DRAM" 1 Memory Hierarchy µproc 60%/yr. (2X/1.5yr) CPU Moore s Law " DRAM 9%/yr. (2X/10 yrs) Reaso: Mai memory too slow Solutio: Memory Time MicroComputer Egieerig MemoryAcceleratio slide 1 MicroComputer Egieerig MemoryAcceleratio slide 2 MicroComputer Egieerig MemoryAcceleratio slide 3 MicroComputer Egieerig MemoryAcceleratio slide 4 DRAMs over Time DRAM Geeratio Mb 4 Mb Die Sie (mm2) Memory Area (mm2) st Ge. Sample Memory Sie Mb 1 Gb Mb 64 Mb Memory Cell Area (µm2) (from Kauhiro Sakashita, Mitsubishi) MicroComputer Egieerig MemoryAcceleratio slide 5 MicroComputer Egieerig MemoryAcceleratio slide 6 1 1

2 Observatios: Two Differet Types of Locality: Temporal Locality (Locality i Time): If a item is refereced, it will ted to be refereced agai soo. Spatial Locality (Locality i Space): If a item is refereced, items whose addresses are close by ted to be refereced soo. By takig advatage of the priciple of locality: Preset the user with as much memory as is available i the cheapest techology. Provide access at the speed offered by the fastest techology. DRAM (mai memory) is slow but cheap ad dese: Good choice for presetig the user with a BIG memory system SRAM (cache) is fast but expesive ad ot very dese: Good choice for providig the user FAST access time. Memory Hierarchy of a Moder Computer By takig advatage of the priciple of locality: Preset the user with as much memory as is available i the cheapest techology. Provide access at the speed offered by the fastest techology. Datapath Cotrol Registers O-Chip Secod Level (SRAM) Mai Memory (DRAM) Speed : 1s Xs 10s 100s Sie (bytes): K K..M G T Secodary Storage (Disk) 10 ms MicroComputer Egieerig MemoryAcceleratio slide 7 MicroComputer Egieerig MemoryAcceleratio slide 8 Levels of the Memory Hierarchy The Art of Memory System Desig Reg:s Istr. Operads" Stagig Xfer Uit Prog 4/8 bytes faster" Workload or Bechmark programs Optimie the memory system orgaiatio to miimie the average memory access time for typical workloads Blocks" Memory s" cache ctl bytes OS 512-4K bytes referece stream <op,addr>, <op,addr>,<op,addr>,<op,addr>,... op: i-fetch, read, write Disk Files" Tape (backup) user/operator Mbytes Larger" Memory $ MEM MicroComputer Egieerig MemoryAcceleratio slide 9 MicroComputer Egieerig MemoryAcceleratio slide 10 Istr MIPS PIPELINE -bit Virtual Address Address Mappig Data User process 2 ruig 24-bit (out of ) Physical Address Here we eed page table 2 for address mappig User Memory Kerel Memory Traslatio Lookaside Buffer (TLB) MIPS PIPELINE Virtual Address O TLB hit, the -bit virtual address is traslated ito a 24-bit physical address by hardware We ever call the Kerel D R Physical Addr [:10] 24 User Memory Kerel Memory MicroComputer Egieerig MemoryAcceleratio slide 11 MicroComputer Egieerig MemoryAcceleratio slide

3 So Far, NO GOOD IM DE EX DM STALL 60 s, RAM IM DE EX DM Let s put i a 60 s, RAM Critical path 20 s TLB 24-bit Physical Address Critical path 20 s TLB MIPS pipe is clocked at 50 MH But RAM eeds 3 cycles to read/write STALLS the pipe 5s Kerel Memory MIPS pipe is clocked at 50 MH 5s A cache Hit ever STALLS the pipe 15s Kerel Memory MicroComputer Egieerig MemoryAcceleratio slide 13 MicroComputer Egieerig MemoryAcceleratio slide bit PA Fully Associative Tag PA[:2] Check all lies Hit if PA[:2]=TAG Data Word PA[1:0] Fully Associative Very good hit ratio (r hits/r accesses) But Too expesive checkig all 2 lies cocurretly A comparator for each lie A lot of hardware all 2 lies 2 * 4=256kb MicroComputer Egieerig MemoryAcceleratio slide 15 MicroComputer Egieerig MemoryAcceleratio slide 24-bit PA 1 lie Direct Mapped Tag PA[:18] Selects ONE cache lie Hit if PA[:18]=TAG Data Word PA[1:0] Direct Mapped Not so good hit ratio Each lie ca hold oly certai addresses, less freedom But Much cheaper to implemet, oly oe lie checked Oly oe comparator 2 * 4=256kb MicroComputer Egieerig MemoryAcceleratio slide 17 MicroComputer Egieerig MemoryAcceleratio slide

4 24-bit PA 2 lies Set Associative Tag PA[:18-] Selects ONE set of lies, sie 2 Hit if PA[:18-]=TAG i the set Data Word PA[1:0] Set Associative Quite good hit ratio The umber (set) of differet addresses for each lie is greater tha that of a directly mapped cache The larger Z the better hit ratio, but more expesive 2 comparators Cost-performace tradeoff 2 -way set associative 2 * 4=256kb MicroComputer Egieerig MemoryAcceleratio slide 19 MicroComputer Egieerig MemoryAcceleratio slide 20 Miss A Miss should be hadled by the hardware If hadled by the OS it would be very slow (>>60 s) O a Miss Stall the pipe Read i ew data to cache Release the pipe, ow we get a Hit Extreme Example: sigle big lie Tag Data Byte 3 Byte 2 Byte 1 Byte 0 0 Sie = 4 bytes Block Sie = 4 bytes Oly ONE etry i the cache If a item is accessed, likely that it will be accessed agai soo But it is ulikely that it will be accessed agai immediately The ext access will likely to be a miss agai Cotiually loadig data ito the cache but discard (force out) them before they are used agai Worst ightmare of a cache desiger: Pig Pog Effect Coflict Misses are misses caused by: Differet memory locatios mapped to the same cache idex Solutio 1: make the cache sie bigger Solutio 2: Multiple etries for the same Idex MicroComputer Egieerig MemoryAcceleratio slide 21 MicroComputer Egieerig MemoryAcceleratio slide 22 A Summary o Sources of Misses Compulsory (cold start or process migratio, first referece): first access to a block Cold fact of life: ot a whole lot you ca do about it Note: If you are goig to ru billios of istructios, Compulsory Misses are isigificat Coflict (collisio): Multiple memory locatios mapped to the same cache locatio Solutio 1: icrease cache sie Solutio 2: icrease associativity Capacity: caot cotai all blocks access by the program Solutio: icrease cache sie Ivalidatio: other process (e.g., I/O) updates memory Block Sie Tradeoff I geeral, larger block sie take advatage of spatial locality BUT: Larger block sie meas larger miss pealty: Takes loger time to fill up the block If block sie is too big relative to cache sie, miss rate will go up: Too few cache blocks I gereral, Average Access Time: Time Av = Hit Time x (1 - Miss Rate) + Miss Pealty x Miss Rate Miss Pealty Miss Rate Exploits Spatial Locality" Fewer blocks: " compromises" temporal locality" Average Access Time Icreased Miss Pealty" & Miss Rate" Block Sie Block Sie Block Sie MicroComputer Egieerig MemoryAcceleratio slide MicroComputer Egieerig MemoryAcceleratio slide

5 Hierarchy Small, fast ad expesive VS Slow big ad iexpesive Cotais copies What if copies are chaged? INCONSISTENCY 256kb RAM Mb I D HD 2 Gb Miss, Write Through/Back To avoid INCONSISTENCY we ca Write Through Always write data to RAM Not so good performace (write 60s) Therefore, WT always combied with write buffers so that do t wait for lower level memory Write Back Write data to memory oly whe cache lie is replaced We eed a Dirty bit (D) for each cache lie D-bit set by hardware o write operatio Much better performace, but more complex hardware MicroComputer Egieerig MemoryAcceleratio slide 25 MicroComputer Egieerig MemoryAcceleratio slide 26 Write Buffer for Write Through Write Buffer Saturatio Write Buffer DRAM A Write Buffer is eeded betwee the ad Memory : writes data ito the cache ad the write buffer Memory cotroller: write cotets of the buffer to memory Write buffer is just a FIFO: Typical umber of etries: 4 Works fie if: Store frequecy (w.r.t. time) << 1 / DRAM write cycle Memory system desiger s ightmare: Store frequecy (w.r.t. time) -> 1 / DRAM write cycle Write buffer saturatio Store frequecy (w.r.t. time) -> 1 / DRAM write cycle If this coditio exist for a log period of time (CPU cycle time too quick ad/or too may store istructios i a row): Store buffer will overflow o matter how big you make it The CPU Cycle Time <= DRAM Write Cycle Time Solutio for write buffer saturatio: Use a write back cache Istall a secod level (L2) cache: Write Buffer L2 DRAM MicroComputer Egieerig MemoryAcceleratio slide 27 MicroComputer Egieerig MemoryAcceleratio slide 28 Replacemet Strategy i Hardware Sequetial RAM Access A Direct mapped cache selects ONE cache lie No replacemet strategy Set/Fully Associative selects a set of lies. Strategy to select oe lie Radom, Roud Robi Not so good, spoils the idea with Associative Least Recetly Used, (move to top strategy) Good, but complex ad costly for large Z We could use a approximatio (heuristic) Not Recetly Used, (replace if ot used for a certai time) MicroComputer Egieerig MemoryAcceleratio slide 29 Accessig sequetial words from RAM is faster tha accessig RAM radomly Oly lower address bits will chage How could we exploit this? Let each Lie hold a Array of Data words Give the Base address ad array sie Burst Read the array from RAM to Burst Write the array from to RAM We might deploy early restart, i.e. Fill the cache lie startig with the word causig the miss. Oce that word is filled, the pipe ca be released ad we ca fill the rest of the lie i the backgroud. MicroComputer Egieerig MemoryAcceleratio slide

6 System Startup, RESET Radom Cotets We might read icorrect values from the We eed to kow if the cotets is Valid, a V- bit for each cache lie Let the hardware clear all V-bits o RESET Set the V-bit ad clear the D-bit for the lie copied from RAM to 24-bit PA V D Fial Model Tag PA[:18-] Data Word PA[1+j:0] 2+j 1+j 0 Selects ONE set of lies, sie 2 Hit if (PA[:18-]=TAG) ad V i set Set D bit if Write 2 lies... MicroComputer Egieerig MemoryAcceleratio slide 31 MicroComputer Egieerig MemoryAcceleratio slide Traslatio Lookaside Buffer (TLB) MIPS PIPELINE Virtual Address O TLB hit, the -bit virtual address is traslated ito a 24-bit physical address by hardware We ever call the Kerel D R Physical Addr [:10] 24 User Memory Kerel Memory Physical Address It takes a extra memory access to traslate VA to PA This makes cache access very expesive, ad this is the "iermost loop" that you wat to go as fast as possible data CPU Tras- latio hit VA PA miss Mai Memory MicroComputer Egieerig MemoryAcceleratio slide 33 MicroComputer Egieerig MemoryAcceleratio slide 34 Virtual Address Why access cache with PA at all? VA caches have a problem syoym / alias problem: two differet virtual addresses map to same physical address => two differet cache etries holdig data for the same physical address for update: must update all cache etries with same physical address or memory becomes icosistet determiig this requires sigificat hardware, essetially a associative lookup o the physical address tags to see if you have multiple hits; or software eforced alias boudary: same lsb of VA &PA withi cache sie, data CPU hit miss Tras- latio VA VA PA Mai Memory Traslatio Look-Aside Buffers Just like ay other cache, the TLB ca be orgaied as fully associative, set associative, or direct mapped TLBs are usually small, typically ot more tha 256 etries eve o high ed machies. This permits fully associative lookup o these machies. Most mid-rage machies use small -way set associative orgaiatios. Traslatio with a TLB CPU hit VA PA miss TLB Lookup miss OS table hit data Mai Memory MicroComputer Egieerig MemoryAcceleratio slide 35 MicroComputer Egieerig MemoryAcceleratio slide

7 Reducig Traslatio Time Overlapped & TLB Access Machies with TLBs go oe step further to reduce cycles/cache access They overlap the cache access with the TLB access Works because high order bits of the VA are used to look i the TLB while low order bits are used as idex ito cache PA TLB Hit/ Miss assoc lookup idex page # disp = 4 bytes 1 K PA Data Hit/ Miss IF cache hit AND (cache tag = PA) the deliver data to CPU ELSE IF [cache miss OR (cache tag <> PA)] ad TLB hit THEN access memory with the PA from the TLB ELSE do stadard VA traslatio MicroComputer Egieerig MemoryAcceleratio slide 37 MicroComputer Egieerig MemoryAcceleratio slide 38 Problems With Overlapped TLB Access Overlapped access oly works as log as the address bits used to idex ito the cache do ot chage as the result of VA traslatio This usually limits thigs to small caches, large page sies, or high -way set associative caches if you wat a large cache Example: suppose everythig the same except that the cache is icreased to 8 K bytes istead of 4 K: 11 2 cache idex virt page # disp This bit is chaged by VA traslatio, but is eeded for cache lookup Summary:, TLB, Virtual Memory s, TLBs, Virtual Memory all uderstood by examiig how they deal with 4 questios: Where ca a data be placed? How is data foud? What data is replaced o miss? How are writes hadled (cocistecy problem)? s speed up average access time Solutios: go to 8K byte page sies; go to 2 way set 10 associative cache; or SW guaratee VA[13]=PA[13] 4 4 1K 2 way set assoc cache tables map virtual address to physical address TLBs are importat for fast traslatio TLB misses are sigificat i processor performace: (some systems ca t access all of 2d level cache without TLB misses) MicroComputer Egieerig MemoryAcceleratio slide 39 MicroComputer Egieerig MemoryAcceleratio slide 40 Summary: Memory Hierachy Virtual memory was cotroversial at the time: ca SW automatically maage 64KB across may programs? 1000X DRAM growth removed the cotroversy Today VM allows may processes to share sigle memory without havig to swap all processes to disk; VM protectio is more importat tha memory space icrease Today CPU time is a fuctio of (ops, cache misses) vs. just of(ops): What does this mea to Compilers, Data structures, Algorithms? Vtue performace aalyer, cache misses. MicroComputer Egieerig MemoryAcceleratio slide