Fluxo. Improving the Responsiveness of Internet Services with Automa7c Cache Placement

Transcription

1 Fluxo Improving the Responsiveness of Internet Services with Automac Cache Placement Alexander Rasmussen UCSD (Presenng) Emre Kiciman MSR Redmond Benjamin Livshits MSR Redmond Madanlal Musuvathi MSR Redmond 1

2 Caching in Internet Services Sasfying user request involves calling many external components, aggregang data Want to cache computaon performed by some components to improve performance Disk intensive operaons, DB queries, etc. What you cache and when depends on a number of factors Workload, architecture, SLAs,

3 Caching in Internet Services Choice of what, where, how much to cache is usually very ad hoc Programmer intuion Localized profiling Best choice can change rapidly over me; too quickly for humans to respond manually Need an automac soluon! 3 3

4 Fluxo Automac Cache Opmizaon A E B C +... < , B, C, x> < , C, D, x > < , A, B, x > < , B, E, y> < , E, C, y >... Fluxo <{A, B}, 50 MB> <{B, E, C}, 256 KB> D Describe Internet service as dataflow graph Gather runme request traces Simulate and opmize to converge on reasonably good cache placement policy 4 4

5 Fluxo Dataflow Graphs node produces request as tuple node consumes response as tuple All other nodes are components which may call external services 5 5

6 Sample Service Weather Report 6 6

7 Weather Service Sample Input

8 Weather Service Sample Input

9 Weather Service Sample Input

10 Weather Service Sample Input

11 Weather Service Sample Input

12 Weather Service Sample Input La Jolla, CA

13 Weather Service Sample Input La Jolla, CA

14 Weather Service Sample Input 2 F, Sunny

15 Weather Service Sample Input 2 F, Sunny

16 Weather Service Sample Input 2 F, Sunny

17 Weather Service Sample Input <p>2 F, Sunny</p>

18 Weather Service Sample Input <p>2 F, Sunny</p>

19 Weather Service Sample Input

20 Weather Service Sample Input 2 F, Sunny

21 Weather Service Sample Input 2 F, Sunny

22 Weather Service Sample Input

23 Caching {, } $

24 Caching {, } $

25 Caching {, } $

26 Caching {, } $

27 Caching {, } $

28 Caching {, } $

29 Caching {, } $ La Jolla, CA

30 Caching {, } $ La Jolla, CA

31 Caching {, } $ 2 F, Sunny

32 Caching {, } $ 2 F, Sunny

33 Caching {, } 2 F, Sunny : 2 F, Sunny $

34 Caching {, } 2 F, Sunny : 2 F, Sunny $

35 Caching {, } <p>2 F, Sunny</p> : 2 F, Sunny $

36 Caching {, } <p>2 F, Sunny</p> : 2 F, Sunny $

37 Caching {, } : 2 F, Sunny $

38 Caching {, } 2 F, Sunny : 2 F, Sunny $

39 Caching {, } 2 F, Sunny : 2 F, Sunny $

40 Caching {, } : 2 F, Sunny $

41 Caching {, } : 2 F, Sunny $ 9 9

42 Caching {, } : 2 F, Sunny $ 9 9

43 Caching {, } : 2 F, Sunny $ 9 9

44 Caching {, } : 2 F, Sunny $ 9 9

45 Caching {, } : 2 F, Sunny $ 9 9

46 Caching {, } : 2 F, Sunny $ 2 F, Sunny 9 9

47 Caching {, } 2 F, Sunny : 2 F, Sunny $ 9 9

48 Caching {, } 2 F, Sunny : 2 F, Sunny $ 9 9

49 Caching {, } <p>2 F, Sunny</p> : 2 F, Sunny $ 9 9

50 Caching {, } <p>2 F, Sunny</p> : 2 F, Sunny $ 9 9

51 Caching {, } : 2 F, Sunny $ 9 9

52 Caching {, } 2 F, Sunny : 2 F, Sunny $ 9 9

53 Caching {, } 2 F, Sunny : 2 F, Sunny $ 9 9

54 Caching {, } : 2 F, Sunny $ 9 9

55 Simplifying Assumpons C Cache C (B Bytes Total) C Data center, single administrave domain Caching provided by cluster of caching servers Service runs on single machine, makes calls to external services during execuon Goal: allocate B total bytes from cache servers to a service 10 10

56 Fluxo Runme Fluxo Components Provides tracing and simulaon funconality Produces ordered stream of events as service runs Fluxo Opmizer Takes stream of events and service graph, produces a caching policy: {<service subgraph, cache size> pairs} Evaluates N random cache policies, hill climbs from the top K policies In our experiments, N=20,000, K=200 To evaluate a policy, simulate its performance on recorded event stream 11 11

57 Evaluaon Reference Policies 64% 2% 9% 25% Random (sample) 100% Uniform (subset) All Encompassing 12 12

58 Results Frequency of Occurrence 31% 4% 65% Input Value Median Latency Improvement: vs. Random: vs. Uniform: vs. All Encompassing: +5% +6% +5% 13 13

59 Results Frequency of Occurrence Input Value Median Latency Improvement: 52% vs. Random: vs. Uniform: vs. All Encompassing: 13% +12% +15% +3% 13% 13% 14 14

60 Results 62% Frequency of Occurrence 22% 9% Input Value Median Latency Improvement: vs. Random: vs. Uniform: vs. All Encompassing: +12% +1% +1% 15 15

61 Future Work Evaluaon on real service with real workload Scaling opmizer s analysis Considering parallelized analysis, more aggressive result memoizaon, more sophiscated ML Seems hard to beat all encompassing cache Might be an arfact of test service Imperave programs? 16 16

62 Conclusion Fluxo: Dataflow model of Internet services Runme tracing + model = caching policy Simulaon and search to converge on good policy Thanks to John Wilkes for shepherding this work, and to MSR for travel funding 1 1