羅仲成 ROY LOU 17MEDIA 分散式緩存服務實踐 DISTRIBUTED CACHING SERVICE

Transcription

1 羅仲成 ROY LOU 17MEDIA 分散式緩存服務實踐 DISTRIBUTED CACHING SERVICE

2 ABOUT ME 17media architect Past: HTC, Google, NVIDIA 2-year-old monster s dad Jogging, basketball, snowboarding

3 There are only two hard things in Computer Science: cache invalidation and naming things. Phil Karlton

4 database cache Persistent storage In-memory storage Huge size Small size Slow (>10ms) Fast (<5ms) Low QPS High QPS (50x)

5 AGENDA Distributed cache models Distributed cache implementation (handle consistency)

6 DISTRIBUTED CACHE MODELS

7 CASE #1: BIG COMPLEX LIST Ex. Regional popular live streams Calculated by complex algorithm (>2s) One list per region (about 10 regions) Very high QPS

8 TW:Hot:List TTL = 2 minute

9 CACHE WITHOUT INVALIDATION - USE TTL (TIME TO LIVE) If cache hit { } return cache value expire after 2 min cache Read from database; Write database value to cache with ttl; database

10 AND IT COMES WITH COST No good timing to evict cache Cache too old Thundering herd problem

11 Cache miss Cache hit Request Response time 0:00 2:00 4:00 cache ttl = 2min have another key to update with shorter ttl

12 CASE #2: KEY-VALUE PAIR Ex. user profile, live stream You know when to evict cache Need cache updated immediately More memory-efficient way than TTL

13

14 db/cache consistent when server crashes vs Which one is better?

15 CASE #3: RELATION STYLE Ex. follow, block, membership Usage 1: Do X follow Y? Usage 2: List X s followers.

16 CACHE ONE BY ONE roy:follow#1 roy:follow#2 roy:follow#3 roy:follow#4 roy:follow#5 roy:follow#6 roy:follow#7 roy:follow#8 roy:follow#9 roy:follow#10 roy:follow#11

17 OR CACHE AS BLOB roy:follow

18 Facebook TAO

19 type from to Roy follows A B C..

20 type from to redis string redis zset

21 ZADD roy:follow ZREM (timestamp) ZRANGE Redis Sorted Set (ZSET)

22 roy:follow 17:00 16:30 16:25 16:23...

23

24 KV / RELATION DISTRIBUTED CACHE IMPLEMENTATION

25 1. Read from cache 2. If miss, read from database 3. Write back to cache cache server database

26 Read Write 1. Read from cache (miss) 2. Read from DB (get X) Time 3. Invalidate cache 5. Write back X to cache 4. Write DB with Y Cache holds X while DB holds Y

27 Attempt #1: Distributed Lock

28 Read 1. Read from cache (miss) Write 2. lock.acquire() 3. Invalidate cache 6. lock.acquire() Time 4. Write DB with Y 5. lock.release() 7. Read from DB (get Y) 8. Write back Y to cache 9. lock.release() Adding a lock

29 distributed lock client 50ms etcd etcd etcd cache library 5ms cache 10ms client database cache library

30 DISTRIBUTED LOCK? Redis latency: 5ms MondoDB latency: 10ms ETCD latency: 50ms - Read from cache (miss) - lock.acquire() - Read from DB (get Y) - Write back Y to cache - lock.release() 5ms 50ms 10ms 5ms 70ms

31 Attempt #2: Sharding

32 client client get user of id 120 set user of id 120 server #1 (%3 = 0) local lock server #2 (%3 = 1) local lock server #3 (%3 = 2) local lock Sharding + Local Lock

33 etcd etcd etcd shard info - server #1: server #2: server #3: client agent - server #1: server #2: server #3: server #1 cache server #2 database client agent - server #1: server #2: server #3: server #3

34 SHARDING WITH CLIENT SIDE AGENT Language dependent Sharding inconsistency during deployment

35 shard info etcd etcd etcd - server #1: server #3: client agent - server #1: server #2: server #3: server #1 cache server #2 database client agent - server #1: server #3: server #3

36 Attempt #3: Built-in strong consistency

37 UBER RINGPOP LIBRARY github.com/uber/ringpop-go Server side library Application strong consistency (not data) Consistency via SWIM Gossip Protocol

38 client Load Balancer server#2 rp lib server#3 rp lib cache server#1 rp lib server#4 rp lib database server#5 rp lib

39 UBER RINGPOP LIBRARY No dependency of client side language Short disconnection (about 100ms) when unstable Network jitter Deployment Autoscale Higher volume of inter-server network (but okay)

40 TEXT SUMMARY Caching models With TTL, no eviction Key-value pair Relational Handle consistency Distributed lock Sharding at client + server Sharding at server (ringpop)

41