Course Site: Copyright 2012, Elsevier Inc. All rights reserved.

Transcription

1 Course Site: 1

2 Computer Architecture A Quatitative Approach, Fifth Editio Chapter 2 Memory Hierarchy Desig 2

3 Outlie Memory Hierarchy Cache Desig Basic Cache Optimizatios Advaced Cache Optimizatios Memory ad its Optimizatios Flash Memory Protectio ad Isolatio Itroductio 3

4 Itroductio Programmers wat ulimited amouts of memory with low latecy Fast memory techology is more expesive per bit tha slower memory Solutio: orgaize memory system ito a hierarchy Etire addressable memory space available i largest, slowest memory Icremetally smaller ad faster memories, each cotaiig a subset of the memory below it, proceed i steps up toward the processor Temporal ad spatial locality isures that early all refereces ca be foud i smaller memories Gives the allusio of a large, fast memory beig preseted to the processor Itroductio 4

5 Memory Hierarchy Itroductio 5

6 Memory Performace Gap Itroductio 6

7 Memory Hierarchy Desig Memory hierarchy desig becomes more crucial with recet multi-core processors: Aggregate peak badwidth grows with # cores: Itel Core i7 ca geerate two refereces per core per clock Four cores ad 3.2 GHz clock 25.6 billio 64-bit data refereces/secod billio 128-bit istructio refereces = GB/s! DRAM badwidth is oly 6% of this (25 GB/s) Requires: Multi-port, pipelied caches Two levels of cache per core Shared third-level cache o chip Itroductio 7

8 High-ed microprocessors have >10 MB o-chip cache Itroductio Performace ad Power Cosumes large amout of area ad power budget 8

9 Memory Hierarchy Basics Whe a word is ot foud i the cache, a miss occurs: Fetch word from lower level i hierarchy, requirig a higher latecy referece Lower level may be aother cache or the mai memory Also fetch the other words cotaied withi the block Takes advatage of spatial locality Place block ito cache i ay locatio withi its set, determied by address block address MOD umber of sets Itroductio 9

10 Memory Hierarchy Basics sets => -way set associative Direct-mapped cache => oe block per set Fully associative => oe set Itroductio Writig to cache: two strategies Write-through Immediately update lower levels of hierarchy Write-back Oly update lower levels of hierarchy whe a updated block is replaced Both strategies use write buffer to make writes asychroous 10

11 Memory Hierarchy Basics Miss rate Fractio of cache access that result i a miss Itroductio Causes of misses Compulsory First referece to a block Capacity Blocks discarded ad later retrieved Coflict Program makes repeated refereces to multiple addresses from differet blocks that map to the same locatio i the cache 11

12 Memory Hierarchy Basics Itroductio Note that speculative ad multithreaded processors may execute other istructios durig a miss Reduces performace impact of misses 12

13 Memory Hierarchy Basics Six basic cache optimizatios: Larger block size Reduces compulsory misses Icreases capacity ad coflict misses, icreases miss pealty Larger total cache capacity to reduce miss rate Icreases hit time, icreases power cosumptio Higher associativity Reduces coflict misses Icreases hit time, icreases power cosumptio Higher umber of cache levels Reduces overall memory access time Itroductio Givig priority to read misses over writes Reduces miss pealty Avoidig address traslatio i cache idexig Reduces hit time 13

14 Te Advaced Optimizatios Advaced Cache Optimizatios Reducig the hit time Icreasig cache badwidth Reducig the miss pealty Reducig the miss rate Reducig the miss pealty or miss rate via parallelism Advaced Optimizatios 14

15 1. Small ad simple L1 caches Small ad simple first level caches Critical timig path: addressig tag memory, the comparig tags, the selectig correct set Direct-mapped caches ca overlap tag compare ad trasmissio of data Lower associativity reduces power because fewer cache lies are accessed Advaced Optimizatios 15

16 L1 Size ad Associativity Advaced Optimizatios Access time vs. size ad associativity 16

17 L1 Size ad Associativity Advaced Optimizatios Eergy per read vs. size ad associativity 17

18 2. Way Predictio To improve hit time, predict the way to pre-set mux Mis-predictio gives loger hit time Predictio accuracy > 90% for two-way > 80% for four-way I-cache has better accuracy tha D-cache First used o MIPS R10000 i mid-90s Used o ARM Cortex-A8 Exted to predict block as well Way selectio Icreases mis-predictio pealty Advaced Optimizatios 18

19 3. Pipeliig Cache Pipelie cache access to improve badwidth Examples: Petium: 1 cycle Petium Pro Petium III: 2 cycles Petium 4 Core i7: 4 cycles Advaced Optimizatios Icreases brach mis-predictio pealty Makes it easier to icrease associativity 19

20 4. Noblockig Caches Allow hits before previous misses complete Hit uder miss Hit uder multiple miss L2 must support this techique Advaced Optimizatios I geeral, processors ca hide L1 miss pealty but ot L2 miss pealty 20

21 Noblockig Caches Advaced Optimizatios Effectiveess of a oblockig cache 21

22 5. Multibaked Caches Orgaize cache as idepedet baks to support simultaeous access ARM Cortex-A8 supports 1-4 baks for L2 Itel i7 supports 4 baks for L1 ad 8 baks for L2 Advaced Optimizatios Iterleave baks accordig to block address 22

23 6. Critical Word First, Early Restart Critical word first Request missed word from memory first Sed it to the processor as soo as it arrives Early restart Request words i ormal order Sed missed work to the processor as soo as it arrives Advaced Optimizatios Effectiveess of these strategies depeds o block size ad likelihood of aother access to the portio of the block that has ot yet bee fetched 23

24 7. Mergig Write Buffer Whe storig to a block that is already pedig i the write buffer, update write buffer Reduces stalls due to full write buffer Do ot apply to I/O addresses Advaced Optimizatios No write bufferig Write bufferig 24

25 8. Compiler Optimizatios Loop Iterchage Swap ested loops to access memory i sequetial order Blockig Istead of accessig etire rows or colums, subdivide matrices ito blocks Requires more memory accesses but improves locality of accesses Advaced Optimizatios 25

26 9. Hardware Prefetchig Fetch two blocks o miss (iclude ext sequetial block) Advaced Optimizatios Petium 4 Pre-fetchig 26

27 10. Compiler Prefetchig Isert prefetch istructios before data is eeded No-faultig: prefetch does t cause exceptios Advaced Optimizatios Register prefetch Loads data ito register Cache prefetch Loads data ito cache Combie with loop urollig ad software pipeliig 27

28 Summary Advaced Optimizatios 28

29 Memory Techology Performace metrics Latecy is cocer of cache Badwidth is cocer of multiprocessors ad I/O Access time Time betwee read request ad whe desired word arrives Cycle time Miimum time betwee urelated requests to memory Memory Techology DRAM used for mai memory, SRAM used for cache 29

30 Memory Techology SRAM Requires low power to retai bit Requires 6 trasistors/bit Memory Techology DRAM Must be re-writte after beig read Must also be periodically refeshed Every ~ 8 ms Each row ca be refreshed simultaeously Oe trasistor/bit Address lies are multiplexed: Upper half of address: row access strobe (RAS) Lower half of address: colum access strobe (CAS) 30

31 Memory Techology Orgaizatio of a DRAM 31

32 Memory Techology Amdahl: Memory capacity should grow liearly with processor speed Ufortuately, memory capacity ad speed has ot kept pace with processors Memory Techology Some optimizatios: Multiple accesses to same row Sychroous DRAM Added clock to DRAM iterface Burst mode with critical word first Wider iterfaces Double data rate (DDR) Multiple baks o each DRAM device 32

33 Memory Optimizatios Memory Techology 33

34 Memory Optimizatios Memory Techology 34

35 Memory Optimizatios DDR: DDR2 Lower power (2.5 V -> 1.8 V) Higher clock rates (266 MHz, 333 MHz, 400 MHz) DDR3 1.5 V 800 MHz DDR V 1600 MHz Memory Techology GDDR5 is graphics memory based o DDR3 35

36 Memory Optimizatios Graphics memory: Achieve 2-5 X badwidth per DRAM vs. DDR3 Wider iterfaces (32 vs. 16 bit) Higher clock rate Possible because they are attached via solderig istead of socketted DIMM modules Memory Techology Reducig power i SDRAMs: Lower voltage Low power mode (igores clock, cotiues to refresh) 36

37 Memory Power Cosumptio Memory Techology Power cosumptio for a DDR3 SDRAM 37

38 Flash Memory Type of EEPROM Must be erased (i blocks) before beig overwritte No volatile Limited umber of write cycles Cheaper tha SDRAM, more expesive tha disk Slower tha SRAM, faster tha disk Memory Techology 38

39 Memory Depedability Memory is susceptible to cosmic rays Soft errors: dyamic errors Detected ad fixed by error correctig codes (ECC) Hard errors: permaet errors Use sparse rows to replace defective rows Memory Techology Chipkill: a RAID-like error recovery techique 39

40 Virtual Memory Protectio via virtual memory Keeps processes i their ow memory space Role of architecture: Provide user mode ad supervisor mode Protect certai aspects of CPU state Provide mechaisms for switchig betwee user mode ad supervisor mode Provide mechaisms to limit memory accesses Provide TLB to traslate addresses Virtual Memory ad Virtual Machies 40

41 Virtual Machies Supports isolatio ad security Sharig a computer amog may urelated users Eabled by raw speed of processors, makig the overhead more acceptable Allows differet ISAs ad operatig systems to be preseted to user programs System Virtual Machies SVM software is called virtual machie moitor or hypervisor Idividual virtual machies ru uder the moitor are called guest VMs Virtual Memory ad Virtual Machies 41

42 Impact of VMs o Virtual Memory Each guest OS maitais its ow set of page tables VMM adds a level of memory betwee physical ad virtual memory called real memory VMM maitais shadow page table that maps guest virtual addresses to physical addresses Requires VMM to detect guest s chages to its ow page table Occurs aturally if accessig the page table poiter is a privileged operatio Virtual Memory ad Virtual Machies 42