CloudNet: Dynamic Pooling of Cloud Resources by Live WAN Migration of Virtual Machines

Transcription

1 CloudNet: Dynamic Pooling of Cloud Resources by Live WAN Migration of Virtual Machines Timothy Wood, Prashant Shenoy University of Massachusetts Amherst K.K. Ramakrishnan, and Jacobus Van der Merwe AT&T Labs - Research VEE 2011

2 Cloud Isolation Cloud data centers are isolated from one another and the enterprise Complicates Deployment Security Resource management Need a way to flexibly manage IT resources across data centers Enterprise View 2

3 Vision: Virtual Cloud Pools Flexible cross data center resource pools A secure collection of server, storage, and network resources Seamlessly connected across cloud and enterprise data centers Supports dynamic application placement across sites Enterprise Sites Cloud Sites 3

4 Cloud Pool Use Cases Enterprise Consolidation Simplify deployment into the cloud Minimize downtime and reconfiguration Cloud Bursting Many applications cannot be easily replicated WAN Migration enables dynamic placement Minimize performance impact Follow the Sun Application moves to be closer to clients or data Minimize planning time and migration bandwidth cost 4

5 Goals: Dynamic Cloud Pools Seamlessly and securely connect enterprise and cloud data centers Enable efficient migration of resources between data centers Challenges Networking: security, transparency, flexibility WAN Migration: Efficiency, application performance impact Downtime Pause Time (sec) SpecJBB Kernel Compile TPC-W Bandwidth (Mbps) 5

6 Outline Introduction Seamless Connections with VPNs Optimizing WAN Migration of VMs Implementation & Evaluation Conclusions 6

7 Connectivity Challenges Current approaches lack... Security Firewalls too fine grain, difficult to manage dynamically Transparency Cloud resources have own public IP range separate from enterprise Flexibility Complex reconfiguration required to add resources or move them between sites 7

8 Seamless Data Center Connections CloudNet: Use Virtual Private Networks (VPNs) Creates secure end-to-end network paths Simpler configuration than firewalls Managed by network provider with no end host configuration Layer 2 Virtual Private LAN Service (VPLS) Bridges the local networks at multiple sites Makes cloud resources look as if directly attached to enterprise LAN Allows existing VM migration techniques to work over the WAN! 8

9 Dynamic VPN Endpoints Manipulating VPN endpoints can be slow Manual process, can take days... must reduce this to seconds CloudNet automates VPN endpoint reconfiguration VPN Controller Centralized VPN Controller Acts as route reflector between sites Can adjust ruleset to modify VPN topology Route updates propagated via BGP 9

10 Outline Introduction Seamless Connections with VPNs Optimizing WAN Migration of VMs Implementation & Evaluation Conclusions 10

11 WAN Migration Challenges Existing approaches not well optimized for WAN Requires high bandwidth, low latency links (e.g. 622Mbps / 5msec) [VMware/Cisco 09, Travostino 06] Focus only on storage or ignore it completely [Bradford 07, Ruth 06] Need to support moving full VM state Disk storage Memory data Processor state All with minimal impact on application performance 11

12 VM Migration Procedure Disk Mem Net VPN Setup Live Mem Transfer Asynchronous Copy Synchronous Copy Time ARP Pause VM VM Memory VM Memory West Coast East Coast 12

13 Optimizing WAN Migration Redundancy Elimination: detect identical regions in memory or disk and only send once Cache Cache VM Memory Kernel Compile Source Zeroes Non-0 Duplicates Destination VM Memory TPC-W SPECjbb Redundancy (% of RAM) 13

14 Page Deltas: only send delta for partially changed data blocks during the migration Deltas Cache To send 2 Destination Page Diff Size Frequency 0 20K 40K Kernel compile Frequency 0 20K 40K TPCW Delta Size (B) Delta Size (B) 14

15 Smart Stop 242,987 When to stop iterating? Xen: when very few pages left or after 30 iterations Goals: Minimize total migration time Minimize pause time Number of Pages ,791 Sent Dirtied Iterate until Sent < Dirtied Iteration Reduce total time Then, find local minimum for Dirtied Reduce pause time 15

16 Outline Introduction Seamless Connections with VPNs Optimizing WAN Migration of VMs Implementation & Evaluation Conclusions 16

17 Implementation Uses Xen for memory migration Added in-memory cache for redundancy and deltas Modified control algorithm for smart stop Disk migration based on DRBD Has sync and async modes Evaluated benefits of redundancy elimination with traces VPN controller manages Juniper M7i routers Can be remotely configured Migration wrapper coordinates network, storage, and memory operations 17

18 CloudNet Testbed Testbed for exploring cloud services 3 sites spread across the US Illinois, Texas, and California Small cluster of servers at each site Can create multiple VCPs with resources at each site Migrations performed over active AT&T network links 18

19 Eval: Cloud Burst From Richardson, TX to Chicago, IL 465 Mbps link 27 msec RTT >1200 KM distance Simultaneous migration of four VMs 10GB disk 500 mi + 1.7GB RAM per VM Total BW consumption lowered from 37GB to 18GB Memory migration time reduced from 245 to 87 sec Downtime halved from 6 to 3 sec GB Transferred Memory Disk Total Default Optimized 19

20 Eval: Application Performance CloudNet reduces migration time Shorter period with lower application performance Resp Time (msec) Disk Transfer: 40 min Default Xen Memory Mig: 210s CloudNet: 115s Xen CloudNet Time (min) Synchronous disk replication reduces application performance if latency is high 20

21 Eval: Network Impact Bandwidth: length of migration and pause time Total Time (sec) Xen CloudNet Bandwidth (Mbps) Pause Time (sec) Xen CloudNet Bandwidth (Mbps) Latency: application performance during migration Total Time (sec) Xen CloudNet Latency (msec) Pause Time (sec) Xen CloudNet Latency (msec) 21

22 Related Work Private Clouds & Virtual Networks VIOLIN [Ruth-ICAC 06] and VIRTUOSO [Sundararaj-VM 04] Amazon VPC Optimizing Migration Compression [Jin-Cluster 09] Model based [Breitgand-HotICE 11] Deltas [Svard-previous talk] Storage transfer [Zheng-next talk] 22

23 Conclusions CloudNet: end-to-end support for WAN migration Network reconfiguration Optimized memory and storage transfer Minimizes migration cost Bandwidth: Eliminate redundant data Time: Reduce unnecessary iterations Reduces application impact Asynchronous bulk disk transfer Minimize pause time Questions? 23

24 Where to optimize? Can do redundancy elimination within the network Riverbed, [SmartRE]...but migration data is often encrypted Our end-host based cache can be used for both RE and page deltas Dom-0 Dom-0 Virtual Machine Memory Cache Storage Cache Memory Cache Storage Cache Virtual Machine Xen Hypervisor Xen Hypervisor 24