Least Recently Frequently Used Caching Algorithm with Filtering Policies

Transcription

1 VLSI Project Least Recently Frequently Used Caching Algorithm with Filtering Policies Alexander Zlotnik Marcel Apfelbaum Supervised by: Michael Behar, Winter 2005/2006 VLSI Project Winter 2005/2006 1

2 Introduction (cont.) Cache definition Memory chip part of the Processor Same technology Speed: same order of magnitude as accessing Registers Relatively small and expensive Acts like an HASH function : holds part of the address spaces. VLSI Project Winter 2005/2006 2

3 Introduction (cont.) Cache memories Main idea When processor needs instruction or data it first looks for it in i the cache. If that fails, it brings the data from the main memory to the cache and uses it from there. Address space is partitioned into blocks Cache holds lines, each line holds a block A block may not exist in the cache -> > cache miss If we miss the Cache Entire block is fetched into a line buffer, and then put into the e cache Before putting the new block in the cache, another block may need to be evicted from the cache (to make room for the new block) VLSI Project Winter 2005/2006 3

4 Introduction (cont.) Cache aim Fast access time Fast search mechanism High Hit-Ratio Highly effective replacement mechanism High Adaptability - fast replacement of not need lines Long sighted - estimation if a block will be used in future VLSI Project Winter 2005/2006 4

5 Project Objective Develop an LRFU caching mechanism Implementation of a cache entrance filtering technique Compare and analyze against LRU Researching various configurations of LRFU, on order to achieve maximum hit rate VLSI Project Winter 2005/2006 5

6 Project Requirements Develop for SimpleScalar platform to simulate processor caches Run developed caching & filtering mechanisms on accepted benchmarks C language No hardware components equivalence needed, software implementation only VLSI Project Winter 2005/2006 6

7 Background and Theory Cache Replacement options: FIFO, LRU, Random, Pseudo LRU, LFU Currently used algorithms: LRU(2 ways requires 1 bit per set to mark latest accessed) Pseudo LRU (4 ways and more, Fully associative) Pseudo LRU (4-way example) Bit 0 specify if way is (0,1) or (2,3) Bit 1 specify who was between 0 and 1 Bit 2 specify who was between 2 and 3 Bit 0 Bit 1 Bit 2 VLSI Project Winter 2005/2006 7

8 Background and Theory (cont) LRU Advantages High Adaptability 1 cycle algorithm Low memory usage Disadvantage Short sighted LFU Advantage Long sighted Smarter Disadvantages Cache pollution Requires many cycles More memory needed VLSI Project Winter 2005/2006 8

9 Background and Theory (cont) Observation Both recency and frequency affect the likelihood of future references Goal A replacement algorithm that allows a flexible trade-off between recency and frequency The idea: LRFU (Least Recently/Frequently Used) Subsumes both LRU and LFU algorithms Overcome the cycles used by LFU by filtering Cache entrances Yields better performance than them VLSI Project Winter 2005/2006 9

10 Development Stages 1. Studying the background 2. Learning SimpleScalar sim-cache platform 3. Develop LRFU caching algorithm for SimpleScalar 4. Develop filtering policy 5. Benchmarking (smart environment) 6. Analyzing various LRFU configurations and comparison with LRU algorithm VLSI Project Winter 2005/

11 Principles The LRFU policy associates a value with each block. This value quantifies the likelihood that the block will be referenced in the near future. Each reference to a block in the past adds a contribution to this value and its contribution is determined by a weighing function F. time t1 δ1 t2 δ2 δ3 t3 Current time tc Ctc(block) = F(δ1) + F(δ2) + F(δ3) tc -t1 tc -t2 tc -t3 VLSI Project Winter 2005/

12 Principles (cont) 1/2) λx Weighing function F(x) = (1/2) Monotonically decreasing Subsume LRU and LFU When λ = 0, (i.e. F(x) = 1), then it becomes LFU When λ = 1, (i.e. F(x) = (1/2) x ), then it becomes LRU When 0 < λ < 1, it is between LFU and LRU F(x) F(x) = 1 (LFU extreme) 1 Spectrum (LRU/LFU) 0 F(x) = (1/2) x (LRU extreme) VLSI Project Winter 2005/ current time - reference time X

13 Principles (cont) Update of C(block) over time Only two counters for each block are needed to calculate C(block) Proof: time t1 δ 1 δ 2 t2 δ 3 t3 t1 δ = (t2 -t1) t2 C t2(b) = F (δ1+δ) + F (δ2+δ) + F (δ3+δ) = (1/2) λ(δ 1+ δ) + (1/2) λ (δ 2+ δ) + (1/2) λ (δ 3+ δ) = ((1/2) λδ 1 + (1/2) λδ 2 + (1/2) λδ 3 ) (1/2) λδ = C t1(b) x F (δ) VLSI Project Winter 2005/

14 Design and Implementation Filtering Data Address In cache Not in cache END In Victims cache? In cache Not in cache END Filter Filter out Insert Data into Victims Cache Insert into cache Insert Data removed from cache by LRFU VLSI Project Winter 2005/

15 Design and Implementation (cont) Data structure LRFU uses for each block two BOUNDED counters VLSI Project Winter 2005/

16 Hardware budget Counters Each block in cache requires two bounded counters Previous C(t) Time that passed from previous access Victims cache The size will be based on empirical analysis VLSI Project Winter 2005/

17 Algorithms Filtering We implemented a very simple filtering algorithm, whose single task is to cause less changes in cache. After a cache miss, the brought block is entered in cache with a probability 0<p<1, p configurable. If the block is not entered in i cache, is entered automatically in victims cache. Replacement After a cache miss, C(t) is calculated for each block in set and the one with the smallest C(t) is selected for replacement. VLSI Project Winter 2005/

18 Results Hit Rate Cache Size (# of blocks) VLSI Project Winter 2005/

19 Results (cont) Hit rate λ VLSI Project Winter 2005/

20 Special Problems Software simulation of hardware Utilizing existing data structures of SimpleScalar Finding the perfect C(t) Applying mathematical theory into practice VLSI Project Winter 2005/

21 Conclusions We implemented a different cache replacement mechanism and received exciting results Hardware implementation of the mechanism is hard, but possible The Implementation achieved the goals Subsumes both the LRU and LFU algorithms Yields better performance than them (up to 30%!!!) VLSI Project Winter 2005/

22 Future Research Implementation of better filtering techniques Dynamic version of the LRFU algorithm Adjust λ periodically depending on the evolution of workload Research of hardware needed for LRFU VLSI Project Winter 2005/