Preserving I/O Prioritization in Virtualized OSes

Transcription

1 Preserving I/O Prioritization in Virtualized OSes Kun Suo 1, Yong Zhao 1, Jia Rao 1, Luwei Cheng 2, Xiaobo Zhou 3, Francis C. M. Lau 4 The University of Texas at Arlington 1, Facebook 2, University of Colorado, Colorado Springs 3, The University of Hong Kong 4

2 Virtualization A powerful abstraction that transforms a physical resource into a more general, easy-to-use virtual form ü Workload consolidation ü Fault isolation ü Service migration Ubiquitous in modern IT management ü Java virtual machine (JVM), container, and full-fledged virtual machine (VM) A program running in virtualized environments should exhibit a behavior essentially identical to that in physical environments. -- [Popek and Goldberg 74]

3 An Inherent Semantic Gap Virtualization presents the illusion of dedicated, continuous hardware, but operates on shared, discontinuous resources Time is discontinuous in multi-tenant systems ü VMs with capped CPU capacity account for 40% Amazon EC2 usage ü CPU multiplexing in public and private clouds VM running Clock ticking VM descheduled VM clock stops VM running Sync with host clock and restart ticking Resource management designed for physical systems can be ineffective in virtualized OSes with discontinuous time VM s perception of time passage: discontinuous; clock reading jumps with intermittent gaps between each run

4 I/O prioritization An important OS design to improve system responsiveness without compromising throughput. I/O prioritization relies on two mechanisms ü The identification of I/O-bound tasks ü The preemption of CPU-bound tasks User-configured priority -- static priority ü Real-time vs. normal priorities, e.g., SCHED_RR vs. SCHED_OTHER Linux completely fair scheduler (CFS) -- dynamic priority ü uses virtual runtime (vruntime) to track how much time a task has run on CPU ü prioritizes tasks with smaller vruntimes ( I/O-bound ) over those with larger vruntimes ( CPU-bound )

5 Violation of I/O Prioritization I/O app I/O app + Compute app Throughput (Gbps) Continuous time 0 KVM Xen Docker 100% CPU 100% CPU I/O only I/O + CPU I/O task always has a higher priority

6 Violation of I/O Prioritization I/O app I/O app + Compute app Throughput (Gbps) Continuous time Throughput (Gbps) Discontinuous time 65% loss 0 KVM Xen Docker 0 KVM Xen Docker 50% CPU 50% CPU I/O only I/O + CPU I/O task always has a higher priority I/O only I/O + CPU

7 Violation of I/O Prioritization I/O app I/O app + Compute app 99% Latency (us) Continuous time KVM Xen Docker 99% Latency (ms) Discontinuous time 100x increase 27% variation KVM Xen Docker 50% CPU 50% CPU I/O only I/O + CPU I/O only I/O + CPU Why I/O prioritization is not effective?

8 Short-term Priority Inversion Time discontinuity may cause inaccurate CPU accounting in the guest OS Process starts running Process stops running VM is running VM is not running VM is running vruntime dilation only affects I/O-bound tasks Correct vruntime Time charged to process Dilated vruntime

9 Long-term Priority Inversion 30% 70% 100% capacity I/O task Compute task I/O task s 30% CPU demand is fully satisfied. Thus, I/O performance is not degraded

10 Long-term Priority Inversion 15% 35% 50% 50% capacity I/O task Compute task Unavailable The CPU allocation to the I/O task is significantly reduced

11 Long-term Priority Inversion 100% capacity 50% capacity 50% I/O requests Compute task Unavailable Work-conserving scheduling in the guest OS allows compute-bound tasks to use the CPU that can be used by I/O-bound tasks in the future under constrained resource allocation

12 Preserving Long-term Priority Priority violation 15% 35% 50% Preserving static priority 30% 20% 50% Preserving dynamic priority 25% 25% 50% I/O task Compute task

13 xballoon Two clocks A CPU balloon process A semi-work-conserving (SWC) scheduling algorithm

14 Two Clocks Maintain two clocks in the guest OS üglobal clock - synchronized with wall-clock time üvirtual clock - only ticks when the VM is running. In-guest scheduling uses this clock [KVM] Guest OS is made aware of time discontinuity to address short-term priority inversion

15 The CPU Balloon Process Inspired by memory ballooning üthe size of the memory balloon represents the memory taken away from the VM The runtime of the CPU balloon process represents the amount of time guest OS reserves for the future Whenever the balloon process runs, it suspends the VM. The VM is woken up upon receiving an I/O request

16 Semi-Work-Conserving Scheduling 1 Accounting starts 2 More than enough reservation, balloon sleeps 3 New accounting starts and balloon wakes up I/O task Balloon Non-work-conserving (NWC) Work-conserving (WC) Compute task NWC: balloon is running, reserving CPU for future WC: balloon is suspended, all tasks are free to run

17 Improving I/O performance Throughput(Mb/s) Netperf TCP w/ static priority Throughput(Mb/s) Netperf TCP w/ dynamic priority CPU cap CPU cap I/O only I/O + CPU I/O + CPU + stealclock I/O + CPU + xballoon xbaloon achieves performance close to the I/O only case Most performance gain is due to the prevention of longterm priority inversion

18 Results on Amazon EC2 A brute-force implementation of xballoon on an m3.medium instance ü SCHEDOP_block to suspend VM ü No semi-work-conserving mode due to no access to the hypervisor Normalized Latency Sockperf Ping Wiser scheduling decisions in the guest OS can greatly improve I/O latency in public clouds I/O only I/O+CPU I/O+CPU+xBalloon+fair I/O+CPU+xBalloon+RT

19 Conclusion Motivation Time discontinuity due to CPU multiplexing or capping can render I/O prioritization in the guest ineffective, leading to I/O performance loss and unpredictability. xballoon A lightweight approach to preserving static and dynamic priorities between I/O- and compute-bound tasks under discontinuity. Evaluation Results in a local environment and Amazon EC2 show that xballoon improves I/O performance in both network throughput and tail latency.

20 Thank you! Questions?

21 Backup Slides

22 How SWC Scheduling Works 100% capacity 50% capacity 50% SWC + + = 50% I/O requests Compute task Balloon Unavailable

23 Q: What if workload is CPU-intensive and I/O-intensive at the same time? A: If the I/O workload contains non-negligible CPU processing, it may not be wise to place it with another CPU-bound workload as the aggregate CPU demand of the two workloads could be well above the CPU capacity. The CPU utilization of some workloads, such as memcached, will increase as more user requests sent to the server side. However, they can still benefit from xballoon if user activities drop and become less CPU-bound and more I/O-bound.

24 Q: The overhead of xballoon? A: xballoon s overhead includes the time spent in user space, in the guest kernel, and inside Xen. As the balloon is woken up by I/O events, the frequency of xballoon invocation depends on the intensity of the I/O workload. Our results show that in most cases xballoon s overhead is negligible.

25 The Balloon Process void balloon (void) { int ret; while(1) {ret = syscall( NR_sched_balloon); if (ret)sched_yield(); } }

26 Priority Inversions in Containers As long as the I/O task and the compute task share limited resource allocations, e.g., belonging to the same cgroup, long-term priority inversion could happen

27 Putting It Together wait queue Hypervisor Guest OS sleeps SWC daemon VM Balloon Simplicity: balloon easy to disable Minimal intrusiveness: one tweak in CFS and one new mechanism in Xen Hardware vm_fast_pause vm_unpase VIRQ NIC Autonomy: Guest OS has total control, no optimization at Xen

28 Hardware Evaluation DELL PowerEdge T420 servers, two six-core Intel Xeon E GHz processors, 32GB memory, one Gigabit Network card and a 1TB 7200RPM SATA hard disk. Software Linux as the guest and dom0 OS, and Xen as the hypervisor. Benchmark: netperf, sockperf, memcached, Nginx, MySQL, etc. Configuration VMs were configured with one vcpu (four vcpus) and 4GB memory. Simulate the time discontinuity through CPU capping and CPU sharing.

29 Impact on Compute tasks xballoon deprives the compute task of CPU when the cap is lower than the I/O demand 400 xballoon allows the compute task to use slack CPU resources when CPU cap increases When CPU cap is 20%, I/O task : Compute task = 10:10 When CPU cap is 80%, I/O task : Compute task = 30: Execution time (s) CPU cap I/O+CPU I/O+CPU+xBalloon-fair I/O+CPU+xBalloon-RT

30 SPI only hurts I/O workload I/O-intensive app stops running for a long time when its vruntime dilates

31 SPI only hurts I/O workload CPU-intensive app keeps running for a long time when its vruntime dilates

32 The throughput keeps the same when number of CPUintensive apps increases. Improve the I/O performance The throughput decreases as the number of CPU-intensive apps increases. Sockperf throughput under static priority Sockperf throughput under dynamic priority vb 300 Throughput(Mb/s) Throughput(Mb/s) I/O+CPU I/O+CPU+xBalloon I/O+CPU I/O+CPU+xBalloon Number of CPU-intensive workload Number of CPU-intensive workload

33 Stealclock is not effective to I/O task in static priority while xballoon is Improve the I/O performance Sockperf TCP in static priority xballoon improves the 99% tail latency by 71%. Sockperf TCP in dynamic priority 99% tail latency (us) % tail latency (us) CPU cap CPU cap I/O only I/O+CPU I/O only I/O+CPU I/O+CPU+ stealclock I/O+CPU+stealclock+xBalloon I/O+CPU+ stealclock I/O+CPU+stealclock+xBalloon

34 Violation of I/O Prioritization Continuous time Discontinuous time I/O app I/O app + Compute app 99% Latency (us) KVM Xen Docker 99% Latency (ms) KVM Xen Docker I/O only I/O + CPU I/O only I/O + CPU 50% CPU 50% CPU Virtualization and multi-tenancy do not preserve the property of I/O prioritization I/O tasks are affected by compute-bound tasks with significantly degraded and wildly unpredictable performance