Computer Architecture ELEC3441

Size: px

Start display at page:

Download "Computer Architecture ELEC3441"

Toby Powell
5 years ago
Views:

CPU-Memory Bottleeck Computer Architecture ELEC44 CPU Memory Lecture 8 Cache Dr.

$Badwidth (umber of accesses per uit time) if fractio m of istructios access memory +m memory refereces /$ istructio To achieve CPI =, we eed +m memory refs / cycle (assumig ISC-V ISA) d semester, '7-8 Badwidth vs

istructio To achieve CPI =, we eed +m memory refs / cycle (assumig ISC-V ISA) d semester, '7-8 Badwidth vs

5- GT/s 0-0 GB/s CPU Cache Memory Moder processors: I the rage of - GHz clock rate Multiple istructio issues (-4

die Small, ast (SAM) Large, Slow (DAM) High speed memory that holds temporary copy of frequetly used data from

1 CPU-Memory Bottleeck Computer Architecture ELEC44 CPU Memory Lecture 8 Cache Dr. Hayde Kwok-Hay So Departmet of Electrical ad Electroic Egieerig Performace of high-speed computers is usually limited by memory badwidth & latecy Latecy (time for a sigle access) Memory access time >> Processor cycle time Badwidth (umber of accesses per uit time) if fractio m of istructios access memory +m memory refereces / istructio To achieve CPI =, we eed +m memory refs / cycle (assumig ISC-V ISA) d semester, '7-8 Badwidth vs Latecy Cache Memory Example: DD SDAM Latecy i the rage of 0-50 s Badwidth i the rage of.5- GT/s 0-0 GB/s CPU Cache Memory Moder processors: I the rage of - GHz clock rate Multiple istructio issues (-4 memory istructios at the same time) Multiple -8 cores Gap: Memory badwidth is -6x slower Latecy > 0x slower Same die Small, ast (SAM) Large, Slow (DAM) High speed memory that holds temporary copy of frequetly used data from mai memory Usually o the same die as the CPU Low latecy Typical: processor cycles Limited capacity (compared to mai memory) Typical: k to 0 Mbytes L cache d semester, '7-8 d semester, '7-8 4

Cache Operatio Overview CPU Cache Memory To access a memory

foud, look ito memory Low latecy access statistically Need

semester, '7-8 L$ L$ L$ Memory Hard Disk egister << SAM <<

disk Badwidth o-chip Cost ($/bit) egister off-chip >> SAM >>

illusio of fast ad large memory spas from register file to

Capacity >> Memory Address (oe dot per access) CPU d

large + fast memory statistically Doald J.

2 Cache Operatio Overview CPU Cache Memory To access a memory locatio: Look up memory cotet from cache If foud, retur If ot foud, look ito memory Low latecy access statistically Need ways to make sure data that will be eeded are i cache d semester, '7-8 L$ L$ L$ Memory Hard Disk egister << SAM << DAM << Magetic disk Latecy egister << SAM << DAM << Magetic disk Badwidth o-chip Cost ($/bit) egister off-chip >> SAM >> >> DAM I/O bus >> Magetic disk The same cocept of creatig the illusio of fast ad large memory spas from register file to hard disk d semester, '7-8 6 eal Memory eferece Patters Capacity >> Memory Address (oe dot per access) CPU d semester, '7-8 5 Memory Hierarchy regfile Give illusio of a large + fast memory statistically Doald J. Hatfield, Jeaette Gerald: Program estructurig for Virtual Memory. IBM Systems Joural 0(): 68-9 (97) 7 Time 8

Typical Memory eferece Patters Two predictable properties of

Temporal Locality: If a locatio is refereced it is likely to

Stack accesses subroutie call argumet access subroutie retur

that locatios ear it will be refereced i the ear future.

Caches exploit both types of predictability: Memory Address

Exploit temporal locality by rememberig the cotets of

Exploit spatial locality by fetchig blocks of data aroud

3 Typical Memory eferece Patters Two predictable properties of memory refereces: Istructio fetches Address loop iteratios Temporal Locality: If a locatio is refereced it is likely to be refereced agai i the ear future. Stack accesses subroutie call argumet access subroutie retur Spatial Locality: If a locatio is refereced it is likely that locatios ear it will be refereced i the ear future. accesses scalar accesses Time 9 0 Memory eferece Patters Caches exploit both types of predictability: Memory Address (oe dot per access) Spatial Locality Temporal Locality Exploit temporal locality by rememberig the cotets of recetly accessed locatios. Exploit spatial locality by fetchig blocks of data aroud recetly accessed locatios. Time Doald J. Hatfield, Jeaette Gerald: Program estructurig for Virtual Memory. IBM Systems Joural 0(): 68-9 (97)

Address Processor copy of mai memory locatio 00 00 04 6848 46

mai memory locatio 0 Mai Memory Lie Block Cache Algorithm

MISS ead block of data from Mai Memory Wait etur data to

whe desigig cache How big is the cache How much data to fetch

4 Address Processor copy of mai memory locatio Iside a Cache Address Byte Byte Byte CACHE Address copy of mai memory locatio 0 Mai Memory Lie Block Cache Algorithm (ead) Look at Processor Address, search cache s to fid match. The either oud i cache a.k.a. HIT etur copy of data from cache Not i cache a.k.a. MISS ead block of data from Mai Memory Wait etur data to processor ad update cache 4 Desigig Cache actors to cosider whe desigig cache How big is the cache How much data to fetch from memory every time Where to put a data i the cache whe it is fetched? How to deal with coflict? Sychroizatio with memory Capacity Lie Size Cache orgaizatio eplacemet Policy read/write policies d semester, '7-8 5 d semester, '7-8 6

5 Split CPU address Lie Size ad Spatial Locality A lie is uit of trasfer betwee the cache ad memory Word0 Word Word Lie Address Word Offset 4 word lie, b= Cache Cofiguratios ully Associative Direct Map Set Associative -b bits b = lie size a.k.a lie size (i bytes) b + bits Larger lie size has distict hardware advaes less overhead exploit fast burst trasfers from DAM exploit fast burst trasfers over wide busses word => bit What are the disadvaes of icreasig lie size? ewer lies => more coflicts. Ca waste badwidth. 7 d semester, '7-8 8 ully Associative Cache Example: ully Associative ully Associative with 8 etries, lie size = 4 words (b=) Cache lies ca be stored i ay locatio of the cache Offset -bit address Offset Evict (overwrite) cache lie oly whe out of space Work similar to a ideal cache except with realistic capacity limitatio Valid Cotet T ABC00E D0D0D0D0 T E0009D E0E0E0E0 CAA000E Offset Size: Size: 4 bits 8 bits Loc Valid Cotet d semester, '7-8 9 d semester, '7-8 0

6 Example: ully Associative ully Associative with 8 etries, lie size = 4 words (b=) 0xBEE6 B offset Example: ully Associative ully Associative with 8 etries, lie size = 4 words (b=) 0xBEE6 B offset Loc Valid Cotet 000 T DDDD DDDD DDDD DDDD Loc Valid Cotet 000 T DDDD DDDD DDDD DDDD d semester, '7-8 d semester, '7-8 Example: ully Associative ully Associative with 8 etries, lie size = 4 words (b=) 0xBEE6 B 0 00 offset Example: ully Associative ully Associative with 8 etries, lie size = 4 words (b=) 0xBEE6 B offset Loc Valid Cotet 000 T 0000A00 DDDD DDDD DDDD DDDD T BEE EEEE EEEE EEEE EEEE Loc Valid Cotet 000 T 0000A00 DDDD DDDD DDDD DDDD T BEE EEEE EEEE EEEE EEEE T 088A d semester, '7-8 d semester, '7-8 4

Example: ully Associative ully Associative with 8 etries, lie size = 4 words (b=) V ully Associative Cache 0x0000A000 0x0000A004 0xBEE6 0x088A80 0x0000A00C B 0000 00 0000 0000 00 Loc Valid Cotet 000

realistic implemetatio Each memory address may be stored at oly oe possible locatio i the cache Usually by takig k bits of address for a k lie $ More tha memory addresses may be mapped to the same

7 Example: ully Associative ully Associative with 8 etries, lie size = 4 words (b=) V ully Associative Cache 0x0000A000 0x0000A004 0xBEE6 0x088A80 0x0000A00C B Loc Valid Cotet 000 T 0000A00 DDDD DDDD DDDD DDDD T BEE EEEE EEEE EEEE EEEE T 088A offset Offset b t t = = = Word or Byte HIT d semester, ' Direct Map Cache Simple ad realistic implemetatio Each memory address may be stored at oly oe possible locatio i the cache Usually by takig k bits of address for a k lie $ More tha memory addresses may be mapped to the same cache locatio è Collisio Evict old cotet before ew cotet is stored Simple ad fast Ofte used i L cache Example: Direct Map Direct map with 8 etries, lie size = 4 words (b=) Idex Size: bits Offset Size: 4 bits Size: 5 bits Idex Offset Loc Valid Cotet d semester, '7-8 7 d semester, '7-8 8

8 Example: Direct Map Direct map with 8 etries, lie size = 4 words (b=) 0xBEE Example: Direct Map Direct map with 8 etries, lie size = 4 words (b=) 0xBEE Loc Valid Cotet 000 T DDDD DDDD DDDD DDDD Loc Valid Cotet 000 T DDDD DDDD DDDD DDDD d semester, '7-8 9 d semester, '7-8 0 Example: Direct Map Direct map with 8 etries, lie size = 4 words (b=) 0x0000A000 0x0000A004 0xBEE6 0x088A80 0x0000A00C B 0 00 Example: Direct Map Direct map with 8 etries, lie size = 4 words (b=) 0x0000A000 0x0000A004 0xBEE6 0x088A80 0x0000A00C B Loc Valid Cotet 000 T DDDD DDDD DDDD DDDD T 0 EEEE EEEE EEEE EEEE0000 Loc Valid Cotet 000 T DDDD 000 DDDD 000 DDDD DDDD T 0 EEEE EEEE EEEE EEEE0000 d semester, '7-8 d semester, '7-8

Example: Direct Map Direct-Mapped Cache Direct map with 8 etries, lie size = 4 words (b=) 0x0000A000 0x0000A004 0xBEE6 0x088A80 0x0000A00C B 0000 00 0000 0000 00 t V Idex k Offset b Loc Valid Cotet

a cache is to icrease the umber of possible locatios to store a data block A N-way set associative cache has N locatios to store each data block A data block ca be placed i ay of the N locatios The N

9 Example: Direct Map Direct-Mapped Cache Direct map with 8 etries, lie size = 4 words (b=) 0x0000A000 0x0000A004 0xBEE6 0x088A80 0x0000A00C B t V Idex k Offset b Loc Valid Cotet 000 T DDDD 000 DDDD DDDD 000 DDDD T 0 EEEE EEEE EEEE EEEE0000 HIT = t k lies Word or Byte d semester, '7-8 4 Set Associative Cache Oe way to reduce miss o a cache is to icrease the umber of possible locatios to store a data block A N-way set associative cache has N locatios to store each data block A data block ca be placed i ay of the N locatios The N locatios form a set Each set may hold data with the same idex Allows N differet data blocks with the same idex be stored i the cache Need replacemet policy to determie which of the N data blocks i the set to be evicted whe the N+ data is writte Example: -Way Set Associative -way Set Associative with 4 etries, lie size = 4 words (b=) 0x0000A000 0x0000A004 0xBEE6 0x088A80 0x0000A00C V Cotet 00 T A0_00 D D D D V Cotet d semester, '7-8 5 d semester, '7-8 6

10 Example: -Way Set Associative -way Set Associative with 4 etries, lie size = 4 words (b=) 0xBEE Example: -Way Set Associative -way Set Associative with 4 etries, lie size = 4 words (b=) 0x0000A000 0x0000A004 0xBEE6 0x088A80 0x0000A00C V Cotet 00 T 0000A0_00 D D D D0 0 0 T BEE_ E E E E0 V Cotet T 088A_0 0 V Cotet 00 T 0000A0_00 D D D D0 0 0 T BEE_ E E E E0 V Cotet T 088A_0 0 What happes if the ext access is 0xC000A8C? d semester, '7-8 7 d semester, '7-8 8 t V -Way Set-Associative Cache Idex k V t = = Offset b Word or Byte Cache Orgaizatios data Same size, differet orgaizatios data data 0 -way Set Associative Direct Map data data data data data data HIT ully Associative 9 d semester, '7-8 40

11 eplacemet Policy I a associative cache, which lie from a set should be evicted whe the set becomes full? adom Least-ecetly Used (LU) LU cache state must be updated o every access True implemetatio oly feasible for small sets (-way) Pseudo-LU biary tree ofte used for 4-8 way irst-i, irst-out (IO) a.k.a. oud-obi Used i highly associative caches Not-Most-ecetly Used (NMU) IO with exceptio for most-recetly used lie or lies Least ecetly Used O replacemet, select the lie that was accessed the least recetly (oldest lie) Need to memorize the access time of each lie 0 Access: Example: 4-way set assoc. lie size = word; arrays a[], b[], c[], d[], e[] all map to row 0 at data at data at data at data 0 a a0 b b0 c c0 d d0 time access a0 b0 c0 d0 c0 d0 e0 a0 e0 b0 c0 d0 H/M M M M M 4 d semester, '7-8 4 Least ecetly Used at data at data at data at data 0 0 a a0 b b0 4 c c0 5 d d0 Access: time access a0 b0 c0 d0 c0 d0 e0 a0 e0 b0 c0 d0 H/M M M M M H H Least ecetly Used at data at data at data at data 0 06 e a a0 e0 b b0 4 c c0 5 d d0 Access: time access a0 b0 c0 d0 c0 d0 e0 a0 e0 b0 c0 d0 H/M M M M M H H M d semester, '7-8 4 d semester, '7-8 44

12 Least ecetly Used at data at data at data at data 0 6 e e0 7 a b a0 b0 4 c c0 5 d d0 Access: time access a0 b0 c0 d0 c0 d0 e0 a0 e0 b0 c0 d0 H/M M M M M H H M M Least ecetly Used at data at data at data at data 0 68 e e0 7 a a0 4 c c0 5 d d0 Access: time access a0 b0 c0 d0 c0 d0 e0 a0 e0 b0 c0 d0 H/M M M M M H H M M H d semester, ' d semester, ' Least ecetly Used at data at data at data at data 0 8 e e0 d d0 9 b b0 0 c c0 Access: time access a0 b0 c0 d0 c0 d0 e0 a0 e0 b0 c0 d0 H/M M M M M H H M M H M M M Pseudo LU Implemetatio challeges for true LU: equires storage for access time o every lie Eormous amout of storage couter wrap aroud equires compariso of all access time withi a set Compariso is slow i hardware Pseudo LU relaxes the requiremet to fid the absolutely oldest piece of data i a set adomly pick ay oe of the older data i the set Oe simple implemetatio: Set bit for each lie of cache whe accessed To replace, evict ay oe of the cache lies with 0 flag Periodically reset all flags to 0 Advaced versio: Evict oly lies that are ot dirty whe there s a draw d semester, ' d semester, '7-8 48

13 Ackowledgemets These slides cotai material developed ad copyright by: Arvid (MIT) Krste Asaovic (MIT/UCB) Joel Emer (Itel/MIT) James Hoe (CMU) Joh Kubiatowicz (UCB) David Patterso (UCB) Joh Lazzaro (UCB) MIT material derived from course 6.8 UCB material derived from course CS5, CS5 d semester, '7-8 49

Computer Architecture ELEC3441

Computer Architecture ELEC3441 CPU-Memory Bottleneck Computer Architecture ELEC44 CPU Memory Lecture 9 Cache Dr. Hayden Kwok-Hay So Department of Electrical and Electronic Engineering Performance of high-speed computers is usually limited