Dynacache: Dynamic Cloud Caching

Transcription

1 Dynacache: Dynamic Cloud Caching Asaf Cidon, Assaf Eisenman, Mohammad Alizadeh and Sachin Ka? Stanford University MIT 1

2 Driving Web Applica4on Performance Web- scale applica4ons heavily reliant on memory caches Facebook: 1000s servers, 5 billion requests per second

3 Hit- rate Impacts Performance +1% cache hit- rate à 35% speedup Old latency: 374 µs New latency: 278 µs Facebook study: A4koglu et al [Sigmetrics 12] End- to- end speedup even greater User queries wait on hundreds of requests

4 How Does Work? Get(key) Set(key, value) Client

5 How Does Work? Get(key) Set(key, value) Response(value) ACK Client

6 : Sounds Familiar? CPU Cache Memory Management Fixed- sized entries First Come First Serve Evic4on Policy LRU LRU Fixed- sized slabs First Come First Serve Resource Alloca4on Across Applica4ons First Come First Serve First Come First Serve

7 Not the Same! CPU Cache Request size Even Highly variable Available resources and complexity Constrained Unconstrained Workload predictability Predictable Unpredictable

8 Dynacache: Dynamic Cache Controller Hit-rate curve estimation Controller Request distribution Client

9 Dynacache: Dynamic Cache Controller Change Memory Allocation Controller Change Eviction Policy Client

10 Opportuni4es for Dynacache 1. Dynamic memory alloca4on 2. Dynamic evic4on policy 3. Dynamic resource alloca4on across applica4ons 4.

11 Opportuni4es for Dynacache 1. Dynamic memory allocaeon 2. Dynamic evic4on policy 3. Dynamic resource alloca4on across applica4ons 4.

12 Memory Alloca4on in Key 1 35 bytes Empty Page Empty Page

13 Memory Alloca4on in Key 1 35 bytes 64 bytes slab class Key 1 Empty Page

14 Memory Alloca4on in Key bytes 64 byte slab class Key 1 Key bytes 256 byte slab class Key 2 Key 3

15 Memory Alloca4on in Key 4 33 bytes Key 5 31 bytes Key bytes 64 byte slab class Key 1 Key 7 Key 4 Key 8 Key 5 Key 9 Key 6 Key byte slab class Key 2 Key 3

16 Separate Evic4on Queue for Each Slab Class Key bytes Key bytes Eviction Eviction 64 byte slab class Key 1 Key 4 Key 5 Key 6 Key 7 Key 8 Key 9 Key byte slab class Key 2 Key 3

17 Problems with Memory Alloca4on Key bytes Key bytes Eviction Eviction 64 byte slab class Key 1 Key 4 Key 5 Key 6 Key 7 Key 8 1. Greedy slab class alloca4on favors large slab classes Key 9 Key byte slab class Key 2 Key 3

18 Problems with Memory Alloca4on Key bytes Key bytes Eviction Eviction 64 byte slab class Key 1 Key 4 Key 5 Key 6 Key 7 Key 8 1. Greedy slab class alloca4on favors large slab classes 2. Slab calcifica4on when request sizes change over 4me Key 9 Key byte slab class Key 2 Key 3

19 MemCachier Dataset Weeklong trace taken from MemCachier 30 servers 490 applica4ons Each applica4on has its own pages Each slab class has its own evic4on queue

20 Applica4on 3: Greedy Alloca4on Favors Large Slabs Classes 80% Slab Class AllocaEon: ApplicaEon 3, Memcachier 70% 60% 50% 40% 30% 20% % Memory Allocated by Default Slab Alloca4on % Memory Allocated by Op4mal Slab Alloca4on 10% 0% K 2K 4K 8K 16K 32K 64K 128K Slab Class Size (bytes)

21 Hit- rate Curve Profiling

22 Dynamically Op4mize Memory Alloca4on Using Hit- rate curves ƒ frequency of requests h hit- rate of requests m memory allocated to slab class e evic4on policy M memory allocated to applica4on

23 Poten4al of Dynamic Memory Alloca4on in Memcachier 45% Speedup 40% 35% 30% 25% 20% 15% 10% 5% 0% ApplicaEon ID

24 Related Work Op4mizing memory alloca4on across applica4ons based on hitrate curves Mimir: Saemundsson et al [SOCC 14] client McRouter: Likhtarov et al [Facebook blog 14] Rebalancing slabs to reduce slab calcifica4on Twimer: Rajashekhar et al [Twimer blog 12] Facebook: Nishtala et al [NSDI 13]

25 Summary Web applica4ons heavily reliant on s policies are sta4c and greedy Resource alloca4on Evic4on policy Web applica4ons are dynamic and variable Dynacache is dynamic cache controller that can dynamically adapt resource alloca4on and evic4on policy

26 Dynacache Prototype Implementa4on Low Entropy Low Classifier Entropy High Entropy Classifier App 7 Classifie Hitrate r Curve App 7 Op4mal Slab Alloc. App 1, Slab 0 App 3, Slab 6 App 7, Slab 3 LRU Queues Client Requests Responses Profiler Optimize r Dynacache Controller

27 Evalua4on Throughput) Overhead) 8.00%$ 6.00%$ 4.00%$ 2.00%$ 0.00%$ 4$ 8$ 12$ 16$ 20$ 24$ 28$ 32$ 36$ 40$ Requests)per)Second)(1000s))