Block I/O bandwidth Control

Transcription

1 Block I/O bandwidth Control Hirokazu Takahashi Hiroya Inakoshi 20-21th November 2008 Copyright 2006, VA Linux Systems Japan K.K. All rights reserved.

2 This work was partly funded by Ministry of Economy, Trade and Industry (METI) of Japan as the Secure Platform project of Association of Super-Advanced Electronics Technologies (ASET).

3 Goal Allow several virtual machines be able to share the same storage without adversely affecting each other, reserving per-virtual machine bandwidth.

4 What I have done Dom0 can now control the flow of I/O requests from domus. domu domu domu I/O request dom0

5 Excellent Result Create three groups with a weight of 10, 20 and 40 on various types of storages. SATA Just an ordinary SATA disk. (WD2500JS-55NCB1) High-end Storage One of high-end storage that was under development. SSD Some prototype of high-speed SSD that was also under development.

6 SATA Read (Direct I/O)

7 SATA Write (Direct I/O)

8 SATA Read (Buffered I/O)

9 SATA Write (Buffered I/O)

10 FC-SAN Read (Direct I/O)

11 FC-SAN Write (Direct I/O)

12 SSD Read (Direct I/O)

13 SSD Write (Direct I/O)

14 Where is the best place to implement the controller? Should be in the block I/O layer or in the device mapper layer to make it generic. Can be used from processes on dom0. The I/O scheduler should only focus on I/O performance. Now started implementing it as a device mapper module since this layer is independent from the others. Backend driver Block I/O layer Device mapper layer I/O scheduler Device driver

15 Design: dm-ioband (1) Create an I/O group and put a virtual machine or several processes in it. Give some weight to each I/O group. Give tokens to each group according to its weight. Every I/O request consumes one or more tokens. An I/O group is blocked once it used up its own tokens. Recharge tokens when all active groups used up their tokens.

16 Design: dm-ioband (2) domu domu processes IO request Blocked token pool dom0

17 dm-ioband: some little ingenuities Active I/O groups will share the unused bandwidth for inactive groups. Support two types of controlling policy; one is based on the number of I/O requests and the other is based on how much date transferred. To gain throughput, dm-ioband will recharge new tokens even if there are some tokens left. Don t block emergency I/O requests such as page-out requests even when the owner group has used up its tokens.

18 bio-cgroup: I/O request tracking Support a method of determining the owner from an I/O request. Implemented on the cgroup memory controller framework. Cgroup : A group of processes associated with some amount of resources. Memory Controller : Controlling the amount of memory allocated to cgroups. The owner of a page is the group whose I/O request starts I/O against the page.

19 The data structure of bio-cgroup memory cgroup bio cgroup struct page bio page I/O Disk

20 The Current Status dm-ioband has been tuned to the point where I/O throughput is as good as in a native environment. Re-implementing bio-cgroup The implementation of the memory controller has been heavily changed. Will help to reduce the I/O tracking overhead.

21 ToDo Yet to merge dm-ioband with Linux kernel The maintainer of device mapper has hardly responded to us. Alternative is to implement the dm-ioband algorithm in the block I/O layer. Improve bio-cgroup Support tracking I/O requests generated inside Linux kernel. The problem is that it also requires to modify the device mapper modules. Add a new I/O controlling policy Add policies for SSD e.g. guarantee response-time guarantee throughput.

22 Any Questions?