Operating Systems. Memory: replacement policies

Transcription

1 Operating Systems Memory: replacement policies

2 Last time caching speeds up data lookups if the data is likely to be re-requested again data structures for O(1)-lookup data source set-associative (hardware) hash tables (software) cache coherence with 1 cache: buffer writes back to memory slow with multiple caches: things get complicated today: replacement policy data user fast cache

3 THE PROBLEM input: a sequence of requests for objects the cache? GOAL minimize number of cache misses

4 clairvoyant LRU LFU FIFO BASIC ALGORITHMS

5 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

6 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

7 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

8 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

9 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

10 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

11 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

12 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

13 clairvoyant BASIC ALGORITHMS LRU LFU FIFO

14 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

15 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

16 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

17 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

18 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

19 BASIC ALGORITHMS LRU clairvoyant LFU FIFO cache hit!

20 BASIC ALGORITHMS LRU clairvoyant LFU FIFO cache hit!

21 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

22 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

23 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

24 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

25 BASIC ALGORITHMS LRU clairvoyant LFU FIFO

26 clairvoyant BASIC ALGORITHMS LRU LFU FIFO x frequency(x) = number of requests evict object with smallest frequency

27 Model of virtual memory p p p disk: slow memory slow input: request sequence p0, p1, p2, p3,., pn fast RAM cache of size k k much smaller than # of pages in disk

28 Model of virtual memory p p p goal minimize total cost of slow bringing pages from disk input: request sequence p0, p1, p2, p3,., pn fast RAM

29 Paging problem p p p input s = p0, p1, p2, p3, pn output slow for each pi, which p to evict from cache goal input: request sequence p0, p1, p2, p3,., pn fast RAM minimize # misses

30 LFD is optimal proposed by Belady LFD: longest forward distance synonyms: OPT, MIN, clairvoyant, Belady s optimal alg proof given in class

31 LFD proof of optimality let ALG be any paging algorithm, s input seq. Will construct a seq of algorithms: ALG = ALG0, ALG1, ALG2, ALGn = LFD st cost(alg0, s) >= cost(alg1, s) >= cost(alg2, s) >= cost(algn) = LFD, n = len(s) how? by induction: suppose ALGi is determined construct ALG(i+1) as follows: for the first i-1 requests, do the same eviction decisions as ALGi. On request i, ALGi evicts the page currently in its cache that will be request furthest into the future, call this page v. ALG(i) may or may not evict this same page. If it does, let ALG(i+1) = ALG(i) for the future too, so then done. Otherwise ALG(i) evicts page u!= v. So after processing i, both caches are the same, but ALG(i) has v and ALG(i+1) has u, where u is requested before v. On future requests, ALG(i+1) mirrors the same decisions as ALG(i) except if ALG(i) evicts v, then ALG(i+1) evicts u. Since u will be requested first, ALG(i) will incur a miss when u is requested, at which point both may or may not be identical (off by 1). If v is requested in the future, then the costs will be identical.

32 Belady s anomaly increasing the cache size may result in more misses hw: FIFO suffers from this phenomenon hw: LRU doesn t

33 Competitive analysis Which replacement policy is better? competitive ratio of ALG = max sequences s misses(alg, s) misses(opt, s) ALG is said to be k-competitive if its competitive ratio is k

34 LRU is k-competitive for any input sequence s: misses(lru, s) k misses(opt, s) Proof sketch: divide s into k-phases, where each phase consists of references to exactly k distinct pages then misses(lru, s) k and misses(opt, s) 1

35 FIFO is k-competitive for any input sequence s: misses(fifo, s) k misses(opt, s) Proof: homework

36 LFU is not competitive there exists a sequence of input sequences s1, s2, s3, misses(lfu, si) lim i!1 = 1 misses(opt, si) Proof: let si = p0 (x i+1), p1 (x i+1), p(k-1) (x i+1), [pk, p(k+1)] (x i) then misses(lfu, si) = k-1 + 2i and misses(opt, si) = k

37 LIFO is not competitive there exists a sequence of input sequences s1, s2, s3, misses(lifo, si) lim i!1 = 1 misses(opt, si) Proof: homework

38 A timeline of paging analysis Belady s paper shows that MIN is optimal competitive analysis, Sleator and Tarjan 1995: 1st analysis of paging with locality of reference, Borodin et al. using access graphs 1996: strongly competitiveness, Irani et al. 1999: LRU is better than FIFO 2000: markov paging, Karlin et al. 2007: relative worst order, Boyar et al over the years: alternative performance measures proposed various frameworks for locality of reference proposed 2009: relative dominance, Dorrigiv et al 2010: parameterized analysis, Dorrigiv et al 2012: access graph model under worst order analysis, Boyar et al 2013: access graph model under relative interval analysis, Boyar et al. 2015: competitiveness in terms of locality parameters, Albers et al.

39 LRU LFU FIFO STRENGTHS AND WEAKNESSES

40 good if popularity changes quickly unpopular objects stay too long LRU LFU FIFO STRENGTHS AND WEAKNESSES

41 good if popularity changes quickly unpopular objects stay too long LRU LFU FIFO good if popularity is static suffers from cache pollution over time STRENGTHS AND WEAKNESSES

42 good if popularity changes quickly unpopular objects stay too long good when fragmentation is an issue LRU LFU FIFO good if popularity is static suffers from cache pollution over time STRENGTHS AND WEAKNESSES

43 LRU FIXING IT admission control find a middle ground between LRU and LFU

44 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

45 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

46 Q ARC LRU-K priority = k-th most recent reference LRU common LRFU FIFO k LRU-LFU HYBRID EVICTION ALGORITHMS

47 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

48 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

49 Q ARC enter here LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

50 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

51 Q ARC evict LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

52 Q ARC evict LRU-K cache hit! LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

53 Q ARC LRU-K cache hit! LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

54 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

55 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

56 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

57 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

58 Q ARC aka Segmented LRU 80% 20% LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

59 Q ARC LRU-K S4LRU LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

60 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

61 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

62 Q ARC LRU-K second hit and more enter here LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

63 Q ARC dynamically adjust ghost ghost LRU-K second hit and more enter here LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

64 Q ARC LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

65 combined recency and frequency CRF(x) = P i t(x,i) Q ARC time since last i-th request to x LRU-K LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

66 combined recency and frequency CRF(x) = P i t(x,i) Q ARC time since last i-th request to x LRU-K 1 2 LFU LRU LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

67 combined recency and frequency CRF(x) = P i t(x,i) Q ARC time since last i-th request to x LRU-K 1 2 LFU in practice LRU LRFU LRU-LFU HYBRID EVICTION ALGORITHMS

68 Q ARC W-LFU geometric fading LRU-K LRFU HYBRID SOLUTIONS TEND TO DO WELL Evaluation of Caching Strategies Based on Access Statistics of Past Requests, Hasslinger, Ntougias, MMB & DFT 2014

69 LRU CACHING POLICIES LFU exact TOWARDS PRACTICALITY approximate UNDERSTANDING THE WORKLOAD

70 LRU handles bursts in popularity well the colder can push out the warmer LFU suffers from cache pollution optimal if popularity is static

71 2nd hit caching score-based LRU SIZE LRU handles bursts in popularity well the colder can push out the warmer LFU suffers from cache pollution optimal if popularity is static Q ARC LRU-K LRFU

72 x e f d t m z a... evict this guy first what a caching algorithm boils down to

73 x e f d t m z a... evict this guy first r v l o m r i a... priority e 2000 what a caching algorithm boils down to

74 x e f d t m z a... evict this guy first r v l o m r i a... priority e 2000 evict this guy what a caching algorithm boils down to

75 caching algorithm template r v l o m r i a e 2000 handle-request(object o): if o is in cache: update priority(o) (for all p in cache: update priority(p)) else: while there is not enough room in cache for o: find and evict object in cache with smallest priority bring o in cache compute priority(o) (for all p in cache: update priority(p))

76 caching algorithm template r v l o m r i a e 2000 handle-request(object o): if o is in cache: update priority(o) (for all p in cache: update priority(p)) else: while there is not enough room in cache for o: find and evict object in cache with smallest priority bring o in cache compute priority(o) (for all p in cache: update priority(p))

77 caching algorithm template r v l o m r i a e 2000 handle-request(object o): if o is in cache: update priority(o) (for all p in cache: update priority(p)) else: while there is not enough room in cache for o: find and evict object in cache with smallest priority bring o in cache compute priority(o) (for all p in cache: update priority(p))

78 caching algorithm template r v l o m r i a... LRU e 2000 handle-request(object o): if o is in cache: update priority(o) (for all p in cache: update priority(p)) O(1) else: while there is not enough room in cache for o: find and evict object in cache with smallest priority O(1) bring o in cache compute priority(o) (for all p in cache: update priority(p)) O(1)

79 clock: an approximation of LRU a 0 g 0 b 0 f 0 c 0 e 0 d 0 e d a e b h

80 clock: an approximation of LRU a 0 g 0 b 0 f 0 c 0 e 1 d 0 e d a e b h

81 clock: an approximation of LRU a 0 g 0 b 0 f 0 c 0 e 1 d 1 e d a e b h

82 clock: an approximation of LRU a 1 g 0 b 0 f 0 c 0 e 1 d 1 e d a e b h

83 clock: an approximation of LRU a 1 g 0 b 0 f 0 c 0 e 1 d 1 e d a e b h

84 clock: an approximation of LRU a 1 g 0 b 1 f 0 c 0 e 1 d 1 e d a e b h

85 clock: an approximation of LRU a 1 g 0 b 1 f 0 c 0 e 1 d 1 e d a e b h

86 clock: an approximation of LRU a 0 g 0 b 1 f 0 c 0 e 1 d 1 e d a e b h

87 clock: an approximation of LRU a 0 g 0 b 0 f 0 c 0 e 1 d 1 e d a e b h

88 clock: an approximation of LRU a 0 g 0 b 0 f 0 ch 0 e 1 d 1 e d a e b h

89 clock: an approximation of LRU a 0 g 0 b 0 f 0 ch 0 e 1 d 1 e d a e b h

90 handle-request(object o): if o is in cache: update priority(o) else: while there is not enough room in cache for o: find and evict object in cache with smallest priority bring o in cache compute priority(o)

91 LRU handle-request(object o): if o is in cache: update priority(o) O(1) else: while there is not enough room in cache for o: find and evict object in cache with smallest priority O(1) bring o in cache compute priority(o) O(1)

92 LRU clock handle-request(object o): if o is in cache: update priority(o) O(1) move to front bit =1 else: while there is not enough room in cache for o: find and evict object in cache with smallest priority O(1) remove from tail move hand bring o in cache set bits to 0 compute priority(o) O(1) place in front bit = 0

93 handle-request(object o): if o is in cache: update priority(o) else: while there is not enough room in cache for o: find and evict object in cache with smallest priority bring o in cache compute priority(o) what about in general

94 using a priority queue (heap) handle-request(object o): if o is in cache: update priority(o) O(log n) else: while there is not enough room in cache for o: find and evict object in cache with smallest priority O(log n) bring o in cache compute priority(o) O(log n) n = nb objects in cache

95 exact algorithm approximate algorithm LRU clock

96 exact algorithm approximate algorithm LRU clock your favorite algorithm pre-sorting as much as possible coarse graining/bucketing/rounding priorities good enough algorithm

97 The popularity of objects follows Zipf s law. # objects heavy tail of unpopular objects popular popularity UNDERSTANDING THE WORKLOAD

98 The object size follows a log-normal distribution # objects # objects size 1K 2K 3K 4K 5K 6K size UNDERSTANDING THE WORKLOAD

99 FIGURES OF MERIT origin server edge server client reduce load maximize byte hit ratio sum of sizes of objects that were request hits reduce perceived delay maximize hit ratio # request hits

100 opularity popularity size size prefer to keep small objects large objects prefer to keep small objects large objects better hit ratio worse worse hit ratio better same byte hit ratio same worse byte hit ratio better fferent popularity-size correlations will yield different performances

101 Recap LFD, LRU, FIFO, LFU provable results: LFD is optimal LRU and FIFO are k-competitive, LFU is not practice: LRU-LFU hybrids are preferred because workloads tend to exhibit locality of reference practice: clock approximates LRU with low overhead practice: best algorithm will depend on the workload

102 CACHE REPLACEMENT CONSIDERED LESS IMPORTANT large caches: decreasing storage cost reduction of cacheable traffic and rate of change (changes to objects over time reduce value of having a large cache to store them longer) plenty of good enough algorithms

103 Extra

104 Locality of reference An input sequence of references exhibits locality of reference if a page that is referenced is likely to be referenced again in the near future.

105 Modeling locality of reference Characteristic vector of an input sequence s C =(c 0,...,c p 1 ) p: number of distinct pages referenced c_i: number of distance-i requests distance-i request: a request to a page such that the number of distinct pages requested since the last time this page was requested is i Ref: Quantifying Competitiveness in Paging with Locality of Reference, Albers, Frascaria, 2015

106 LRU beats FIFO in the presence of locality of reference Characteristic vector of an input sequence s C =(c 0,...,c p 1 ) What does the characteristic vector of a sequence with high locality of reference look like? Ref: Quantifying Competitiveness in Paging with Locality of Reference, Albers, Frascaria, 2015