Efficient QoS for Multi-Tiered Storage Systems

Transcription

1 Efficient QoS for Multi-Tiered Storage Systems Ahmed Elnably Hui Wang Peter Varman Rice University Ajay Gulati VMware Inc

2 Tiered Storage Architecture Client Multi-Tiered Array Client 2 Scheduler SSDs... Client n HDs Problem: How to provide QoS in this environment? 2

3 Fair Scheduling Example Allocate IOs in proportion of client weights w = Client Multi-Tiered Array SSD ms FS w 2 = Client 2 HD 0ms How will Fair Scheduling behave in a multi-tiered storage architecture? 3

4 Fair Scheduling Example Time = 0 ms Client Multi-Tiered Array w = SSD ms FS w 2 = Client HD 0ms 4

5 Fair Scheduling Example Time = ms Client Multi-Tiered Array w = SSD ms FS 2 w 2 = Client HD 0ms 5

6 Fair Scheduling Example Time = 2 ms Client Multi-Tiered Array w = SSD ms FS 2 w 2 = Client HD 0ms Issue: Low utilization SSD will idle for 9 out of 0 ms 6

7 Fair Scheduling in Multi-Tier Is the behavior of FS reasonable? The answer is simply NO! 7

8 Time Slice Schedulers Allocate time quantum in proportion of client weights w = Client Multi-Tiered Array SSD Time Slice w 2 = Client 2 HD 8

9 Time Slice Schedulers Allocate time quantum in proportion of client weights w = Client Multi-Tiered Array SSD Time Slice w 2 = Client 2 HD Issue: Low utilization Both devices will idle for large intervals 9

10 Queue Based Schedulers Allocate queue slots in proportion of client weights w = 2 Client Multi-Tiered Array SSD Queue Based w 2 = Client 2 HD 0

11 Queue Based Schedulers Allocate queue slots in proportion of client weights w = 2 Client Multi-Tiered Array SSD Queue Based w 2 = Client 2 HD Issue: Static queues can cause low utilization Empty slots are not getting re-used

12 Agenda Problem Motivation Reward Scheduling Preliminary Evaluation Open Issues & Conclusion 2

13 Reward Allocation Policy Goals Provide QoS (fairness) Higher system throughput / device utilization 3

14 Reward Scheduling Key Ideas: Reward clients with cheaper IOs Track IO latency of each client separately Try to allocate in the ratio of clients throughput if running alone (entitlement) 4

15 Algorithm Details Tag based scheduling Each client queue has a tag Tag represents scheduling priority Scheduler selects queue with smallest tag Monitor service time of client requests Increment tags using Measured service time of client Weight of the client 5

16 Algorithm Details Φ j : Measured service time of completing request of client j UpdateServiceTime (j, Φ j ) Average of the service time of client j over window 6

17 Reward Scheduling Time = 0 ms Client Multi-Tiered Array w = w 2 = Client RS 2 SSD ms HD 0ms 7

18 Reward Scheduling Time = ms Client Multi-Tiered Array w = w 2 = Client RS SSD ms HD 0ms Higher utilization Client with lower latency gets higher effective weight 8

19 Agenda Problem Motivation Reward Scheduling Preliminary Evaluation Open Issues & Conclusion 9

20 Two implementations Simulation-based Experimental Setup Linux-based evaluation (Interposing a reward scheduler in user space) Three different cases Variable hit ratios Variable read-write ratio Variable weights (In paper) 20

21 Reward Scheduling vs. CFQ 2

22 Reward Scheduling vs. CFQ 22

23 Reward Scheduling vs. CFQ Isolation between workloads Higher system throughput 23

24 Variable Hit Rate Client A Multi-Tiered Array SSD 0.2ms RS Client B HD 0ms 24

25 Variable Hit Rate Simulation system Entitlement of A = 5000 IOPS Entitlement of B = 5000 IOPS Ratio A:B = 5000:5000 : 25

26 Variable Hit Rate Simulation system Entitlement of A = 5000 IOPS Entitlement of B = 200 IOPS Ratio A:B = 5000 : : 26

27 Linux-based evaluation Variable Hit Rate Simulation system 27

28 Variable Read-Write Ratio 28

29 Variable Read-Write Ratio Queue = 29

30 Variable Read-Write Ratio Queue = 8 Differentiation became less due to queuing delay 30

31 Agenda Problem Motivation Reward Scheduling Preliminary Evaluation Open Issues & Conclusion 3

32 Open Issues Better measurement of response time Isolating queuing delay Can we make queue-slot based technique work? Interaction between cache management and scheduling 32

33 Conclusion Existing approaches are insufficient Reward based allocation policy is an alternative Other approaches can potentially work We hope to encourage future work in this area Questions 33