Acceleration of Virtual Machine Live Migration on QEMU/KVM by Reusing VM Memory

Transcription

1 Acceleration of Virtual Machine Live Migration on QEMU/KVM by Reusing VM Memory Soramichi Akiyama Department of Creative Informatics Graduate School of Information Science and Technology The University of Tokyo, Japan 1

2 Resource provisioning in datacenters with virtualization technologies consolidation / distribution of Virtual Machines using Live Migration technique VM3 VM1 VM2 VM1 VM4 VM2 VM4 VM3 VM4 migrates for consolidation VM3 migrates to guarantee performance 2

3 time How Live Migration works CPU is running! initial copy our target CPU is running! iterative copy CPU is running! iterative copy Existing work 3

4 The idea: Memory Reusing Dynamic consolidation 1. VMs migrate many times 2. a VM will come back later on Keep the memory image on hosts and reuse it! 4

5 # of pages updated until the time Memory Access Pattern of TPC-C (= 1GB) simulated DB access of a net-shop 17% Suppose: migrate out at 0 sec and migrate back at 600 sec 17% of pages can be reused! time (sec) 5

6 The algorithm 1. Each memory page has a generation 0 in the boot-time Migration: +1 generations of updated pages 2. Keep the memory image of the VM on the source host in a migration 3. Compare latest generations and ones of the kept memory image when migrating back generations Pages with the same generation: reused memory image [0, 0, 0, 0, 0, 0, 0] compare the generations [0, 1, 0, 1, 0, 1, 0] 6

7 Implementation QEMU KVM on Linux original: transfer the whole memory modified: transfer updated pages only A central server manages the generations gathered generations can be used to optimize VM placements 7

8 Modification to QEMU 1. mmap the VM memory with a ram-file in /dev/shm of the host (for later reuse) mmap VM dirty bitmap ram-file host tmpfs (/dev/shm) 8

9 Modification to QEMU 2. use dirty page tracking to detect updates mmap VM dirty bitmap ram-file host tmpfs (/dev/shm) 9

10 Modification to QEMU 2. use dirty page tracking to detect updates mmap VM dirty bitmap ram-file host tmpfs (/dev/shm) 10

11 Modification to QEMU 2. use dirty page tracking to detect updates dirty page tracking dirty bitmap mmap ram-file host tmpfs (/dev/shm) VM 11

12 0. VM is running How the system works: the first migration Host A Host B (VM, A) = [0, 0, 0, 0, 0, 0] (VM, B) = [*, *, *, *, *, *] Host A Host B 12

13 How the system works: the first migration 1. Stop the VM Host A Host B (VM, A) = [0, 0, 0, 0, 0, 0] (VM, B) = [*, *, *, *, *, *] Host A Host B 13

14 How the system works: the first migration 1. Stop the VM Host A Host B 2. Increment the generations (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [*, *, *, *, *, *] Host A Host B 14

15 How the system works: the first migration 1. Stop the VM 3. Migration Host A Host B 2. Increment the generations (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [*, *, *, *, *, *] Host A Host B 15

16 How the system works: when migrating back 0. VM is running memory image Host A Host B (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [0, C 1, 1, 1, 0, 0] Host B Host A 16

17 How the system works: when migrating back 0. memory updates memory image Host A Host B (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [0, C 1, 1, 1, 0, 0] Host B Host A 17

18 How the system works: when migrating back 1. Stop memory image Host A Host B (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [0, C 1, 1, 1, 0, 0] Host B Host A 18

19 How the system works: when migrating back 1. Stop the VM memory image Host A Host B 2. Increment the generations (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [0, C 1, 2, 2, 0, 0] Host B Host A 19

20 How the system works: when migrating back 1. Stop the VM memory image Host A Host B 2. Increment the generations (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [0, C 1, 2, 2, 0, 0] 3. Compare (VM, B) and (VM, A) Host B Host A 20

21 How the system works: when migrating back reusable pages 1. Stop the VM Host A Host B 4. Send reusable page numbers 2. Increment the generations (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [0, C 3. Compare (VM, B) and (VM, A) 1, 2, 2, 0, 0] Host B Host A 21

22 How the system works: when migrating back 5. Migration 1. Stop the VM Host A transfer updated pages only Host B 4. Send reusable page numbers 2. Increment the generations (VM, A) = [0, 1, 1, 1, 0, 0] (VM, B) = [0, C 3. Compare (VM, B) and (VM, A) 1, 2, 2, 0, 0] Host B Host A 22

23 Evaluation: Setting Machine Environment 2 hosts, 1 Intel Xeon X5460 (4 cores), 8GB RAM Debian squeeze (64 bit version) QEMU with our modification, KVM Gigabit Ethernet 1 VM 1 vcpu, 1GB RAM Ubuntu server (64 bit version) Benchmarks 1. Busy Loop CPU-intensive (100%) 2. Apache Static web service, Network-intensive (100Mbps) 3. Video Video transcoding, IO-intensive (read, write) 4. TPC-C DB simulation of a net shop (CPU, memory, IO) 23

24 Evaluation: Procedure 1. Exec a benchmark on host A for N mins 2. Migration from host A to host B 5, 10, Exec the bench on host B for another N mins 4. Migration back from host B to host A host A host B 24

25 Evaluation: Procedure 1. Exec a benchmark on host A for N mins 2. Migration from host A to host B 5, 10, Exec the bench on host B for another N mins 4. Migration back from host B to host A host A host B 25

26 Evaluation: Procedure 1. Exec a benchmark on host A for N mins 2. Migration from host A to host B 5, 10, Exec the bench on host B for another N mins 4. Migration back from host B to host A host A host B 26

27 Evaluation: Procedure 1. Exec a benchmark on host A for N mins 2. Migration from host A to host B 5, 10, Exec the bench on host B for another N mins 4. Migration back from host B to host A host A host B 27

28 Evaluation: Procedure 1. Exec a benchmark on host A for N mins 2. Migration from host A to host B 5, 10, Exec the bench on host B for another N mins 4. Migration back from host B to host A host A host B 28

29 Evaluation: Procedure 1. Exec a benchmark on host A for N mins 2. Migration from host A to host B 5, 10, Exec the bench on host B for another N mins 4. Migration back from host B to host A host A host B 29

30 Evaluation: Results Busy Loop Apache Busy Loop Ideal case, reused more than 90% Apache Uses much memory for file cache, but I/O with static data does not update the memory 30

31 Evaluation: Results Video TPC-C Video Big converted video dirties much memory TPC-C Memory access pattern is complex, but memory reusing can be utilized Out proposal is beneficial for practical transaction systems 31

32 Related work Researches on improved live migration compress diff between older version of the page in iterative copying [1] Transfer execution logs, instead of the memory [2] Uniqueness of our proposal we focus of the initial copy of the memory our work and related ones can be utilized together [1] Evaluation of Delta Compression Techniques for Efficient Live Migration of Large Virtual Machines, VEE2011, Svärd et al. [2] Live Migration of Virtual Machine Based on Full System Trace and Replay, HPDC2009, Liu et al. 32

33 Future work Memory cache management currently all the memory pages must be cached better cache management is needed Granularity of reusing Reuse ratio in TPC-C: 50% (page-wise) more than 80% (byte-wise) updated page updated bytes (small fraction of the page) 33