Flauncher and DVMS Deploying and Scheduling Thousands of Virtual Machines on Hundreds of Nodes Distributed Geographically

Transcription

1 Flauncher and DVMS Deploying and Scheduling Thousands of Virtual Machines on Hundreds of Nodes Distributed Geographically Daniel Balouek 1, Adrien Lèbre 2, Flavien Quesnel 2 1 AVALON, Ecole Normale Supérieure de Lyon / INRIA / LIP 2 ASCOLA, Ecole des Mines de Nantes / INRIA / LINA {daniel.balouek, adrien.lebre, flavien.quesnel}@inria.fr May 2013

2

3

4

5

6

7

8

9

10

11 Outline DVMS - Context & Related Work DVMS Overview Experiments with Flauncher Lessons Learnt 11

12 Context E!cient use of resources in large scale virtual infrastructures 12

13 Most Solutions... Only provide static placement... Resources may not be fully used And/or are centralized Scalability concerns 13

14 DVMS: Proposal Overview Multi-agent system, leveraging a ring topology No service node Virtual infrastructure dynamic partitioning (time & space) Dynamic scheduling Event-driven scheduling 14

15 Main Algorithm

16 Main Algorithm P i P: partition i: initiator

17 Main Algorithm P i l P: partition i: initiator l: leader

18 Main Algorithm P i 2 7 l 3 P: partition i: initiator l: leader Small partitions => fast scheduling 18

19 Main Algorithm

20 Main Algorithm P 1 P i l 2 7 i 3 P: partition i: initiator l: leader l

21 Experiments with Flauncher DVMS prototype Language: Java Scheduling policy: workload o"oading/reallocation Events activated: node overload Comparison between Entropy and DVMS, on Grid Generate several non-viable configurations 2. Start VMs with Flauncher 3. Inject synthetic CPU workload 4. Observe how Entropy and DVMS solve problems for each configuration 21

22 Nodes Reserved on Grid 5000 Nancy Sophia Nancy Sophia Rennes Nancy Sophia Rennes Toulouse VMs, 251 nodes 3325 VMs, 309 nodes 4754 VMs, 467 nodes 22

23 Results 23

24 Results Entropy DVMS Duration of an iteration (s) Mean and standard deviation 251 nodes 2000 VMs 309 nodes 3325 VMs 467 nodes 4754 VMs 23

25 Results Entropy DVMS Duration of an iteration (s) Mean and standard deviation DVMS Time to solve an event (s) Mean and standard deviation 251 nodes 2000 VMs 309 nodes 3325 VMs 467 nodes 4754 VMs 251 nodes 2000 VMs 309 nodes 3325 VMs 467 nodes 4754 VMs 23

26 Results Entropy DVMS Duration of an iteration (s) Mean and standard deviation DVMS Time to solve an event (s) Mean and standard deviation 251 nodes 2000 VMs 309 nodes 3325 VMs 467 nodes 4754 VMs 251 nodes 2000 VMs 309 nodes 3325 VMs 467 nodes 4754 VMs Time to apply a reconfiguration (s) Mean and standard deviation nodes 2000 VMs Entropy DVMS 309 nodes 3325 VMs 467 nodes 4754 VMs 23

27 Demo Live Screencast 1285 VMs 147 nodes / 4 sites 24

28 Lessons Learnt Nodes not correctly deployed Redeploy Nodes that did not reboot correctly Force reboot Routing communications between sites Use VLANs Communications with more than 1024 machines Increase the size of ARP table Migration errors with libvirt Ensure that each node has a truly unique libvirt id 25

29 Conclusion Flauncher A tool to start, stress and migrate a huge number of VMs distributed across several sites of Grid 5000 A tool that can be leveraged to study virtualization and live migration in large scale infrastructures, and to validate other tools DVMS A distributed solution to schedule VMs dynamically in large scale infrastructures A building block of DISCOVERY, a proposal aiming at designing a distributed Cloud OS 26

30 Tutorials Flauncher Booting_and_Using_Virtual_Machines_on_Grid'5000 DVMS id=internet:members:fquesnel:deploysimulatorg5kv3 27

31 Thank you

32 Backup slides 29

33 References (1/2) [Cornabas 10] Jonathan Rouzaud Cornabas. A distributed and collaborative dynamic load balancer for virtual machine. In Euro-Par 2010 workshops, Ischia, Naples Italy, August Springer. [Feller et al. 11]" Eugen Feller, Louis Rilling, and Christine Morin. Snooze: A Scalable and Autonomic Virtual Machine Management Framework for Private Clouds. Research report, INRIA Rennes, Rennes, France, December [Hermenier et al. 09] Fabien Hermenier, Xavier Lorca, Jean M. Menaud, Gilles Muller, and Julia Lawall. Entropy: a consolidation manager for clusters. In VEE ʼ09: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, pages 41 50, New York, NY, USA, March ACM. [Hoffa et al. 08] On the Use of Cloud Computing for Scientific Workflows, Hoffa, C., G. Mehta, T. Freeman, E. Deelman, K. Keahey, B. Berriman, J. Good. SWBES 2008, Indianapolis, IN. December 2008 [Lèbre et al. 11] DISCOVERY, Beyond the Clouds, Lebre, A. and Anedda, P. and Gaggero, M. and Quesnel, F. In Euro-Par 2011 workshops, Bordeaux, France, August Springer. 30

34 References (2/2) [Lowe 09] Scott Lowe. Introducing VMware vsphere 4. Wiley Publishing Inc., Indianapolis, Indiana, first edition, September [Mastroiani et al. 11] Carlo Mastroianni, Michela Meo and Giuseppe Papuzzo. Self-economy in cloud data centers: statistical assignment and migration of virtual machines. In Euro-Par'11: Proceedings of the 17th international conference on Parallel processing, pages , Bordeaux, France, August Springer. [Nurmi et al. 09] Daniel Nurmi, Rich Wolski, Chris Grzegorczyk, Graziano Obertelli, Sunil Soman, Lamia Youseff, and Dmitrii Zagorodnov. The eucalyptus open-source cloud-computing system. In CCGRID ʼ09: Proceedings of the th IEEE/ACM International Symposium on Cluster Computing and the Grid, volume 0, pages , Washington, DC, USA, May IEEE Computer Society. [Sotomayor et al. 09] Borja Sotomayor, Rubén S. Montero, Ignacio M. Llorente, and Ian Foster. Virtual infrastructure management in private and hybrid clouds. IEEE Internet Computing, 13(5):14 22, September [Yagiz et al. 10] Yagiz O. Yazir, Chris Matthews, Roozbeh Farahbod, Stephen Neville, Adel Guitouni, Sudhakar Ganti, and Yvonne Coady. Dynamic resource allocation in computing clouds using distributed multiple criteria decision analysis. In Cloud ʼ10: IEEE 3rd International Conference on Cloud Computing, pages 91 98, Los Alamitos, CA, USA, July IEEE Computer Society. 31

35 Related work More distributed Event-based approaches ( periodic) Still partially centralized (most of them) 32

36 More on DVMS Advanced mechanisms Accelerating the ring traversal By maintaining shortcuts Guaranteeing to find a solution (if it exists) By merging partitions Limitation Worst case: centralized case (not reached in practice) 33