Adaptive VM Management with Two Phase Power Consumption Cost Models in Cloud Datacenter

Transcription

1 Mobile New Appl (2016) 21: DOI /s z Adapive VM Managemen wih Two Phase Power Consumpion Cos Models in Cloud Daacener Dong-Ki Kang 1 & Fawaz Al-Hazemi 1 & Seong-Hwan Kim 1 & Min Chen 2 & Limei Peng 3 & Chan-Hyun Youn 1 Published online: 31 March 2016 # Springer Science+Business Media New York 2016 Absrac As cloud compuing models have evolved from clusers o large-scale daa ceners, reducing he energy consumpion, which is a large par of he overall operaing expense of daa ceners, has received much aenion laely. From a cluser-level viewpoin, he mos popular mehod for an energy efficien cloud is Dynamic Righ Sizing (DRS), which urns off idle servers ha do no have any virual resources running. To maximize he energy efficiency wih DRS, one of he primary adapive resource managemen sraegies is a Virual Machine (VM) migraion which consolidaes VM insances ino as few servers as possible. In his paper, we propose a Two Phase based Adapive Resource Managemen (TP-ARM) scheme ha migraes VM insances from underuilized servers ha are supposed o be urned off o * Chan-Hyun Youn chyoun@kais.ac.kr Dong-Ki Kang dkkang@kais.ac.kr Fawaz Al-Hazemi fawaz@kais.ac.kr Seong-Hwan Kim s.h_kim@kais.ac.kr Min Chen Minchen2012@hus.edu.cn Limei Peng auroraplm@ajou.ac.kr School of Elecrical Engineering, KAIST, Daejeon, Souh Korea School of Compuer Science and Technology, Huazhong Universiy of Science and Technology, Wuhan , China Deparmen of Indusrial Engineering, Ajou Universiy, Suwon, Souh Korea susainable ones based on heir moniored resource uilizaions in real ime. In addiion, we designed a Self-Adjusing Workload Predicion (SAWP) mehod o improve he forecasing accuracy of resource uilizaion even under irregular demand paerns. From he experimenal resuls using real cloud servers, we show ha our proposed schemes provide he superior performance of energy consumpion, resource uilizaion and job compleion ime over exising resource allocaion schemes. Keywords Cloud compuing. Virual machine migraion. Dynamic righ sizing. Energy saving 1 Inroducion In modern cloud daa ceners, resource allocaion wih high energy efficiency has been a key problem because he operaing cos from energy consumpion has increased in recen years. According o an esimaion from [1], he cos of power and cooling has increased 400 % over 10 years, and 59 % of daa ceners idenify hem as key facors limiing server deploymens. Consequenially, challenges from energy consumpion and cooling have led o a growing push o achieve an energy efficien design for daa ceners. A he daa cener level, a promising echnology called Dynamic Righ Sizing (DRS) o save energy dynamically adjuss he number of acive servers (i.e., servers whose power is swiched on) in proporion o he measured user demands [2]. In DRS, energy saving can be achieved by enabling idle servers ha do no have any running VM insances o go ino low-power mode (i.e., sleep or shu down). In order o maximize he energy efficiency via DRS, one of he primary adapive resource managemen sraegies is VM consolidaion in which running VM insances can be dynamically inegraed ino he minimal

2 794 Mobile New Appl (2016) 21: number of cloud servers based on heir resource uilizaion colleced by a hypervisor monioring module [3]. Tha is, running VM insances on under-uilized servers, which are supposed o be urned off, could be migraed o powersusainable servers. However, i is difficul o efficienly manage cloud resources because cloud users ofen have heerogeneous resource demands underlying muliple service applicaions which experience highly variable workloads. Therefore, inconsiderae VM consolidaion using live migraion migh lead o undesirable performance degradaion due primarily o swiching overheads caused by he migraion and by urning off servers on [4]. Running service applicaions wih reckless VM migraion and DRS execuion could encouner serious execuion ime delays, increased laency or failures [5]. As a resul, a careful resource managemen scheme ha considers swiching overheads is necessary o reduce efficienly he energy consumpion of cloud servers while ensuring accepable Qualiy of Service (QoS) based on Service Level Agreemens o cloud users [6, 7]. In his paper, we propose a Two Phase based Adapive Resource Managemen (TP-ARM) scheme o address he above challenges for cloud daaceners. The energy consumpion model based on TP-ARM was formulaed wih a performance cos (repuaion loss) caused by an increased delay from downsizing acive servers, by an energy cosfrom keeping paricular servers acive, and by a cos incurred from swiching off servers on. Subsequenly, we designed an auomaed cloud resource managemen sysem called he Adapive Cloud Resource Broker (ACRB) sysem wih he TP-ARM scheme. Moreover, we inroduced our novel predicion mehod called Self-Adjusing Workload Predicion (SAWP) o increase he predicion accuracy of users fuure demands even under unsaic and irregular workload paerns. The proposed SAWP mehod adapively scales he hisory window size up or down according o he exraced workload s auocorrelaions and sample enropies which measure he periodiciy and bursiness of he workloads [8]. To invesigae he performance characerisics of he proposed approaches, we conduced various experimens o evaluae energy consumpion, resource uilizaion and compleion ime delay by live migraion and DRS execuion on a real esbed based on Opensack which is a well-known cloud plaform using KVM hypervisor [9]. Through meaningful experimenal resuls, we found ha our proposed TP-ARM scheme and SAWP mehod provide significan energy savings while guaraneeing accepable performance required by users in pracice. 2 Proposed adapive cloud resource brokering sysem In his secion, we discuss he archiecure of he auomaed Adapive Cloud Resource Brokering (ACRB) sysem. Our considered cloud environmen including ACRB, which suppors deploying a resource managemen scheme in order o migrae VM requess and adjuss he number of acive servers according o he workload level, is depiced in Fig. 1. There are m physical hoss and n VM requess in he cloud daacener. In he cloud daacener, he informaion on resource uilizaion is colleced hrough KVM hypervisor based monioring modules ino he Resource Monioring DB module, and repored o he ACRB which is responsible for solving he migraion of allocaed VM requess and sizing he daacener. The ACRB has wo modules: he TP-ARM module and he SAWP Manager. The TP-ARM module includes he VM Migraion Manager and he DRS Manager. In he firs phase, he VM Migraion Manager is responsible for selecing he appropriae VM requess o be migraed based on he measured resource uilizaion level. We describe he merics for deermining he VM reques migraion in deail in secion 4.In he second phase, he DRS Manager is responsible for finding he opimal number of acive servers in he cloud daacener. The DRS plan derived by he DRS Manager based on he amoun of submied VM requess and he measured resource uilizaion from each VM reques is delivered o he Daacener Manager, and he deermined percenage of idle servers are powered off. The SAWP Manager is responsible for adjusing he window size of hisorical daa adapively in order o predic he fuure demand of VM requess. The SAWP Manager is able o achieve he exac predicion of fuure demands even under varied workload levels by considering he periodiciy and he flucuaion of he hisorical daa. The owner of he cloud daacener has o minimize he coss for resource operaions while boosing he benefis which can increase based on he good repuaion of he observed QoS of he cloud services. In his paper, our ACRB ries o find a resource managemen soluion o minimize he oal cos of resource operaions including wo sub cos models: he energy consumpion cos and he performance repuaion cos. From he perspecive of he energy consumpion cos, he VM Migraion Manager ries o maximize he resource uilizaion of each physical hos by consolidaing running VM requess based on he whole offered load measured hrough he Libvir VM Monior module aached o each hos. The Libvir VM Monior module gauges he uilizaion of resources such as CPU, memory, and I/ O bandwidh in order o check wheher whole hoss are overloaded [10, 11]. VM requess on overloaded hoss are preferably migraed o oher hoss which have enough exra resource capaciy o accommodae newly allocaed VM requess. The DRS Manager sends he shudown messages o servers which have o be powered off, while i sends magic packes o he ones which have o be powered on again. Obviously, he ransiion of servers from sleeping mode o acive mode (awake ransiion) requires addiional energy consumpion (noe ha he ransiion overhead from acive o sleep (asleep ransiion)

3 Mobile New Appl (2016) 21: Fig. 1 Adapive Cloud Resource Brokering (ACRB) sysem including TP-ARM module and SAWP manager for green cloud daaceners can be negligible because i requires only a shor ime o be carried ou compared o he awake ransiion). Therefore, i is clear ha frequen awake ransiions have o be discouraged in order o minimize unnecessary energy consumpion. To simplify our model, we assume ha he asleep ransiion causes no addiional energy consumpion (in fac, i requires non-zero energy consumpion). In erms of he performance repuaion cos, he VM Migraion Manager ries o decenralize running VM requess over muliple physical hoss in order o avoid QoS deerioraion caused by VM inerference. In cloud daaceners, he VM co-locaion inerference is he key facor ha makes servers undergo severe performance degradaion [12, 13]. VM co-locaion inerference is caused by resource conenion which is refleced mainly by he number of co-locaed VM insances and he resource uilizaion of hem. Briefly, VM co-locaion inerference becomes larger as more VM insances are co-locaed on a common server, and subsequenly, higher resource uilizaion occurs. Therefore, VM requess have o be scaered in order o avoid performance degradaion by VM co-locaion inerference as bes as possible. Because of he complexiy of he opimizaion for resource managemen in large-scale cloud daaceners, our TP-ARM scheme adops a meaheurisic based on GA o obain a near opimal soluion for VM migraion o achieve energy savings and QoS assurance in a cloud daacener. 3 Proposed model for cos opimizaion of TP-ARM in he cloud In his secion, we inroduce an energy cos model and performance repuaion cos model based on our proposed TP-ARM scheme including VM live migraion and DRS execuion in he ACRM. 3.1 Workload cos model for phase 1: VM migraion In general, VM requess have heerogeneous workloads in cloud daaceners. There are hree ypes of VM reques workloads: CPU, block I/O, and nework I/O inensive workloads. The CPU inensive workloads such as scienific applicaions, high-definiion video compression or big daa analysis should be processed wihin an accepable compleion ime deermined by he cloud service users. The block I/O inensive workloads such as huge engineering simulaions for criical areas include asrophysics, climae, and high energy physics which are highly daa inensive [14]. These applicaions conain a large number of I/O accesses where large amouns of daa are sored o and rerieved from disks. Nework I/O inensive workloads such as Inerne web services and mulimedia sreaming services have o be processed wihin a desirable response ime according o heir applicaion ype [10, 15]. Each

4 796 Mobile New Appl (2016) 21: VM reques requires differen resource uilizaions according o heir running applicaions. We caegorized he resource uilizaion of running VM requess on a physical hos in a cloud daacener ino wo pars in his paper, he Flavor Uilizaion (FU) and he Virual Uilizaion (VU). FU represens he raio of he resource flavor (i.e., he specificaion of he resource requiremen) of a VM reques o he resource capaciy of he physical hos. VU represens he resource uilizaion of an k assigned virual resource. The Flavor Uilizaion FU j, i of a VM reques i on a physical hos j in he resource componen k k and he Virual Uilizaion VU j, i of he VM reques i on he physical hos j in he resource componen k are given by FU k j; i ¼ flvk i rcp k j RU k j; i ¼ FUk j; i VUk j; i ð1þ ð2þ where flv k i is he flavor of he VM reques i in he resource componen k; rcp k j is he resource capaciy of he physical hos k j in he resource componen k,andru j, i is he acual resource uilizaion of he VM reques i on he physical hos j in he resource componen k; herefore, i describes he pracical workload in he resource componen k. Noe ha he FU of he VM reques can be deermined hrough is aached service descripion o he ACRM in advance, while he VU can only be measured by he inernal monioring module in he cloud server during he running duraion of he VM reques. In he period, he VM migraion plan is designed according o he FU of he submied VM requess a period and he measured RU of each hos in he pas hisory 1. The VM migraion plan should saisfy he following consrains: flv k i S kðþ; i i; k; ð3þ flv k i 0; i; k ð4þ S saeðþ¼1; i i; ð5þ where S k (i) denoes he remain capaciy of resource k in he physical server S which is assigned o he VM reques i a ime period,ands sae (i) represens he sae of he physical server S which is assigned o he VM reques i a ime period. Ifhe server S is in he sleep mode, hen S sae (i) = 0, oherwise, S sae (i) = 1 (i.e., i is in he acive mode). Consrain (3) represens ha he oal requiremens of he resource capaciy of he newly allocaed VM requess and migraed VM requess a ime period canno exceed he resource capaciy provided by heir assigned physical server S. Nex, we consider a VM performance repuaion which is deermined based on he RU of each physical server. The VM co-locaion inerference implies ha he virualizaion of he cloud suppors resource isolaion explicily when muliple VM requess are running simulaneously on a common PM however, i does no mean he assurance of performance isolaion beween VM requess inernally. There is a srong relaionship beween VM co-locaion inerference and boh of he number of co-locaed VMs and heir resource uilizaion in he PM. As he number of colocaed VM requess increase and he RU of he VM requess becomes larger, VM co-locaion inerference becomes more severe. The VM Migraion Manager selecs server candidaes o be migraed based on he amoun of RU for each physical server. To achieve his, he RU size of each server is measured hrough an inernal monioring module, and he VM Migraion Manager checks wheher hey exceed he predeermined RU hreshold value RU hr. In servers which have an RU size over he RU hr, hey are considered as candidaes for migraion servers during he nex period. Afer he migraion servers are chosen, he VM requess o be migraed and heir desinaion servers are deermined based on he performance repuaion cos model. We propose he objecive funcion of he performance repuaion cos C repu a ime C repu given by 0 1 X PM þ j jj C repu ¼ ρ X X inf ω k PM RU k i¼1 idx j j; i 1C A ð6þ j k þρ mig T mig X j PM 1 j RU k hr PM j where (x) + = max(x, 0), and ρ inf and ρ mig are he price consans of he VM inerference cos and VM migraion cos, respecively. We use T mig o denoe he processing ime for he VM migraion. The firs erm in he righ hand of Eq. (6) represens he following: as he number of concurren running VM requess on a physical server increases, he number of users experiencing undesirable performance degradaion also increases. The second erm represens he following: as he number of migraed VM requess increases, he migraion overhead is also increases. Therefore, a migraion plan needs o be found ha saisfies boh avoiding unnecessary migraion overhead and minimizing performance repuaion degradaion. 3.2 Energy consumpion cos model for phase 2: DRS procedure To achieve a power-proporional cloud daacener which consumes power only in proporion o he workload, we considered a DRS procedure which adjuss he number of acive servers by urning hem on or off dynamically [2]. Obviously, here is no need o urn all he servers in a cloud daacener on when he oal workload is low. In he DRS procedure, he sae of servers which have no running applicaions can be ransied o he power saving mode (e.g., sleep or hibernaion) in order o avoid wasing energy. A each ime, he number of acive servers in he cloud daacener is deermined by a DRS procedure plan according o he workload

5 Mobile New Appl (2016) 21: of he allocaed VM requess. In order o successfully deploy he DRS procedure ono our sysem, we considered he swiching overhead for adjusing he number of acive servers (i.e., for urning servers in sleep mode on again). The swiching overhead includes he following: 1) addiional energy consumpion from he ransiion from a sleep o acive sae (i.e., awaken ransiion); 2) wear-and-ear cos of he server; 3) faul occurrence by urning servers in sleep mode [2]. We considered he energy consumpion as he overhead from DRS execuion. Therefore, we define he consan P awake o denoe he amoun of energy consumpion for he awake ransiion of he servers. Then, he oal energy consumpion C energy of he cloud daacener a ime is given by C energy ¼ ρ X pp acive PM j j X X þ þ ρ p P awake T swich j PM j j PM j ð7þ where (x) =min(x, 1). We use ρ p o denoe he consan of he power consumpion price, and P acive and P awake are he amoun of power consumpion for he acive mode and he awake ransiion of he server, respecively. T swich is he ime requiremen for he awake ransiion. The firs erm on he righ side of Eq. (7) represens he energy consumpion for using servers o serve VM requess allocaed o all he physical servers in he cloud daacener a ime, and he second erm represens he energy consumpion for he awaken ransiion of sleeping servers. Especially, he second erm implies ha a frequen change in he number of acive servers could increase an undesirable wase of energy. Noe ha he overhead by he ransiion from he acive o he sleep sae (i.e., asleep ransiion) is ignored in our model because he ime required for he asleep ransiion is relaively shor compared o he one for he awaken ransiion. The purpose of our algorihm is o achieve a desirable VM migraion and DRS procedure plan ha are energy efficien as well as a QoS aware resource managemen a each period ieraively. A period 1, he proposed TP-ARM approach aims o minimize he cos funcion in Eq. (8) byfinding he soluion PM as follows, minimize PM C oal ¼ C repu þ C energy subjec o: Eq ð3þ; ðþand 4 ð5þ ð8þ To solve he objecive cos funcion in Eq. (8), we prefer a well-known evoluionary meaheurisic called he Geneic Algorihm (GA) in order o approximae he opimal plan P M a each period because Eqs. (6) and(7) havenon-linear characerisics. In nex secion, our proposed TP-ARM wih he VM migraion and DRS procedure is inroduced in deail. 4 Heurisic algorihms for he proposed TP-ARM scheme In his secion, we describe heurisic algorihms for he proposed TP-ARM scheme discussed in Algorihms 1, 2, and 3 shown in Figs. 2, 3, and4. Firs, he process of VM migraion in he TP- ARM approach includes he following four seps: 1) monior and collec he resource uilizaion daa on VM requess for each physical server hrough he aached Libvir based monioring ools; 2) go sep 3 if he average uilizaion of all he acive servers are significanly low (i.e., below he predefined hreshold), oherwise go o sep 4; 3) choose acive servers which are supposed o be urned off, migrae all he VM insances on hem o oher servers and rigger DRS execuion; 4) deermine he number of sleeping servers which are supposed o be urned on and send magic packes o hem for wake-up if he average uilizaion of all he acive servers are significanly high (i.e., above he predefined hreshold), oherwise mainain he curren number of acive servers. Algorihm 1 shows he procedure for Phase 1:VM migraion in he TP-ARM scheme. From line 00 o 06, he resource uilizaion RU of all he VM requess on he physical servers in he cloud daacener is measured by he Libvir API monioring module. The average resource uilizaion RU k a all resource componens k is calculaed in line 07. If he RU k is below he predeermined RU k a all he resource componens, hen he hr low opimal VM migraion plan PM is derived, and he algorihm is finished. Oherwise, if he RU k exceeds he predeermined RU k, hen Algorihm 2 for he DRS procedure is riggered. hr high Algorihm 2 shows he procedure for Phase 2: DRS in he TP-ARM scheme. In line 00, he hisory window size ϖ is deermined by he ACRM hrough Algorihm 4: he SAWP scheme ha is described in nex secion. In line 02, we calculae he sign φ and he angle ξ of he slope of he hisorical resource uilizaion curve from he curren ime 1oheime ϖ.if φ 0 (i.e., he resource uilizaion is increased), and hen, we ge he coefficien α based on α large ξ (α large > α small ) o adapively reac o he large workload level in he cloud daacener. Oherwise, if φ < 0 (i.e., he resource uilizaion is decreased), hen we ge α based on α small ξ o maximize he energy saving of he cloud daacener. In line 08, we deermine he number of servers o be in acive mode in he nex period. Algorihm 3 describes he GA for he TP-ARM scheme in deail. pop h is a populaion wih size P (even number) a he h h generaion. h max is he maximum of he GA ieraion coun. In line 03, f( ) is a finess funcion of he oal cos wih wo parameers, candidae soluion PM h; i and he previous soluion P M 1 a ime 1. From line 04 o 06, he wo candidae soluions are ieraively chosen randomly from pop o generae offsprings by crossover unil here are no remaining unseleced soluions in pop. From line 07 o 08, he finess funcion values of each offspring are calculaed similar o line 03. In line 09, all

6 798 Mobile New Appl (2016) 21: Algorihm 1. Phase 1: VM migraion in TP-ARM Inpu : he VM requess allocaion of each server M, Oupu : he VM migraion plan M, 00: for each, 01: for each 02: for each resource componen 03: measure resource uilizaion of he VM reques wih 04: end for 05: end for 06: end for 07: calculae he average resource uilizaion a all resource componens 08: if hen 09: derive he VM migraion plan based on Eq.(8) hrough Algorihm 3. 10: else if hen 11: go o Algorihm 2. 12: end if Fig. 2 Phase 1: VM migraion procedure consolidaes running VM insances from he low-uilized servers o ohers in order o maximize he energy saving of he cloud daacener he paren soluions in pop and generaed offsprings are sored in ascending order for heir corresponding finess funcion values. In line 10, he nex populaion including only P soluions ha achieve good performance from he union of he original pop and derived offsprings is generaed o improve he qualiy of he final soluion. From line 11 o 12, o reduce he ime complexiy of he GA procedures, when we encouner he firs soluion ha has a finess funcion value below he predeermined finess hreshold value fv hr, i couns as a final soluion for he nex period, and he algorihm is finished. In line 14, if we canno find a soluion ha saisfies fv hr when he ieraion coun reaches h max,henweselec a soluion ha has a minimum finess funcion value in he populaion which saisfies he condiions in Eqs. (3), (4) and(5) as a final soluion for he nex period. If here are no soluions o saisfy all he consrains, hen we jus preserve he curren resource allocaion vecor shown from line 15 o 16. In he populaion of a GA, muaions are ofen applied in order o include new characerisics from he offsprings ha are no inheried rais from he parens [16]. We did no consider muaions in our GA in his paper; however, i can be used o improve he qualiy of he GA for he TP-DRM in fuure work. 5 Self-adjusing workload predicion scheme Figure 5 describes he procedure of he proposed SAWP algorihm in ACRB. The irregulariy funcion g(ε, δ) represens a level of unpredicabiliy for fuure resource Algorihm 2. Phase 2: DRS procedure in TP-ARM Inpu : he average resource uilizaion he VM requess allocaion of each server he hisorical daa of resource uilizaion in Oupu : he se of powered-off servers o be urned on a ime. 00: deermine he boundary size of he hisory window, ( ) hrough Algorihm 4. 01: esimae he sign and angle ξ of an slope of he hisorical resource uilizaion curve from o. 02: if hen 03: α = α ξ 04: end if 05: if hen 06: α = α ξ 07: end if 08: deermine he number of sleeping servers supposed o be urned on according o 09: derive he se of powered-off servers o be urned on a ime Fig. 3 Phase 2: DRS procedure urns sleeping servers on in order o reac he increased workload level

7 Mobile New Appl (2016) 21: Algorihm 3. GA for searching VM migraion plan in TP-ARM Inpu : he se of VM requess allocaion of whole servers M a ime Oupu : he se of VM requess migraion plan, M a ime 00: iniialize =, M,, M,, and even number 01: while h h 02: for each 03: 04: while 05: selec parens randomly from 06: = crossover 07: for each 08: 09: sor in ascending order of soluions' corresponding 10: 11: if hen 12: and exi 13: h++ 14: =argmin (3), (4), (5) 15: if M = hen 16: Fig. 4 GA for VM migraion deermines he appropriae VM requess supposed o be migraed in order o maximize he energy saving of he cloud daacener uilizaion where ε is is flucuaion value (i.e., levels of insabiliy and aperiodiciy), and δ is is bursiness value (i.e., levels of sudden surge and decline), and boh values are calculaed based on [8]. high According o he predeermined hreshold values g hr and g low hr wih g(ε, δ), he hisory window size ϖ 'isadapively updaed a each predicion process. A relaively long hisory window size is no suiable o reac o recen changes in he workload bu is oleran o varied workload paerns over a shor ime while a shor hisory window size is favorable o efficienly respond o he laes workload paerns bu is no good for widely varying workloads. Consequenly, he SAWP algorihm generally ouperforms radiional predicion schemes during drasic uilizaion changes from various cloud applicaions because i is able o cope wih emporary resource uilizaions (i.e., does no reflec overall rends) by adjusing he hisory window size ϖ '. Algorihm 4. Self Adjusing Workload Predicion scheme Inpu : he hisorical daa of resource uilizaion in Oupu : he boundary size of hisory window 00: 01: analyze he hisorical daa of resource uilizaion from o 02: exrac he flucuaion size and he bursness value based on [6] 03: if ( ) hen 04: increase based on ( ) 05: end if 06: if ( ) hen 07: decrease based on ( ) 08: end if 09: predic he fuure demand based on from o Fig. 5 SAWP scheme is able o increase he accuracy of he predicion even under he dynamic workload

8 800 Mobile New Appl (2016) 21: Experimenal resuls and discussion In our experimens, we measured various merics which affec he parameer decision for our proposed algorihms. To his end, we esablished five cluser servers as a cloud plaform, one server for ACRB wih a Mysql DB sysem, a power measuring device from Yoco-Wa [17] and a lapop machine called he VirualHub for collecing and reporing informaion on he measured power consumpion shown in Fig. 6. The hardware specificaion of each server for he cloud compue hos is as follows: an Inel i (8-cores, 3.4Ghz), 16 GB RAM memory, and wo 1 Gbps NICs (Nework Inerface Cards). In order o measure efficienly he power consumpion of he cloud cluser server, we used a power measuring device model called YWATTMK1 by Yoco-Wa. This model has a measuremen uni of 0.2 W for AC power wih an error raio of 3 % and W for he DC power wih an error raio of 1.5 %. The VirualHub collecs informaion on power consumpion from YWATTMK1 hrough he Yoco-Wa Java API and repors i o he power monioring able of he Mysql DB sysem in he ACRB periodically. The dynamic resource uilizaions by each VM insance are measured via our developed VM monioring modules based on he Libvir API and are sen o he resource monioring able of he Mysql DB sysem periodically. In addiion, a SATA 3 TB hard disk called he G-drive was deployed as a NFS server in our esbed for live migraion [18]. We adoped Opensack kilo version which is a well-known open source soluion based on KVM Hypervisor as a cloud plaform in our esbed. Finally, we used Powerwake package [19] o urn remoely off servers on via Wake on Lan (WOL) echnology for DRS execuion. In Table 1, we show he average power consumpion and resource uilizaion of wo running applicaions: Monage m projecion and fp ransfer. Monage projec is an open source based scienific applicaion, and i has been invoked by NASA/IPAC Infrared Science Archive as a oolki for assembling Flexible Image Transpor Sysem (FITS) images ino cusom mosaics [20, 21]. The m projecion in Monage is a cpu-inensive applicaion while he fp ransfer is a nework-inensive one. Therefore, he running of m causes a power consumpion of abou 75 Wh and a cpu uilizaion of abou 15 % whereas he power consumpion by fp ransfer for a 1.5 GB es.avi file is abou 60 Wh, and he nework bandwidh usage is abou 3.7Mbps. Tha is, he cpu usage is he main par ha affecs he power consumpion of server. In erms of DRS execuion, he power consumpion by an off server is abou 2.5 Wh (noe ha his value is no zero because he NIC and some is peripheral componens are sill powered on o mainain he sandby mode o receive he magic packes from he Powerwake conroller) while he asleep and awaken ransiion procedures, which cause he swiching overhead for DRS, require a power consumpion of abou 80 Wh o urn he acive servers off or o urn off servers on, respecively. Fig. 6 Experimenal Environmen

9 Mobile New Appl (2016) 21: Table 1 Average power consumpion and resource uilizaion by Monage and fp ransfer Hos sae Power Consumpion (Wh) CPU uilizaion Mem uilizaion Ne bandwidh Idle 55 Wh 1.5 % 3 % 20 Kbps (byes) Acive Monage m (projecion) 75 Wh 15 % 3.7 % 20 Kbps (byes) es.avi downloading (fp, 1.5GB) 60 Wh 2 % 3.7 % 3.7 Mbps (byes) asleep (o Power Off) Wh (5 7 s) 1.8%(5 7 s) 3%(5 7 s) 20 Kbps (byes) Power Off (hibernaing) 2.5 Wh awake (from Power Off) 78 Wh (50s-1min) The asleep ransiion procedure is rivial because i requires a shor ime (i.e., 5 ~ 7 s) o complee even hough is power consumpion is considerable whereas he awaken ransiion procedure requires a relaively long execuion ime (i.e., more han 1 min) and should be considered carefully. The overhead for he awaken ransiion would be a more serious problem in pracice because is required execuion ime is generally far longer (i.e., above 10 min) for muliple servers of racks in a daacener. Therefore, i is essenial o consider he swiching overhead for awaken ransiion o reduce efficienly he resource usage cos of he daacener. Figure 7 shows he addiional energy consumpion overhead of a single physical server from he DRS execuion wih respec o he asleep ransiion (i.e., from he acive mode o he sleep mode) and he awaken ransiion (i.e., from he sleep mode o he acive mode). Noe ha he asleep ransiion requires a relaively shor processing ime of abou 7 or 8 s and causes an addiional energy consumpion of abou 20 % compared o he idle sae of he server, while he awaken ransiion needs 60 or 90 s o be carried ou. Alhough no included in his paper, we also verified ha some of he HP physical servers in our laboraory required more han 10 min o be urned on. We expec ha physical servers in a real cloud daacener require ens of minues or hundreds of minues o ge back from he sleep mode o he acive mode. Moreover, he awake ransiion causes an addiional energy consumpion of abou 40 % on average compared o he idle sae of a server. These resuls imply ha he frequen DRS execuion migh cause he degradaion of he energy saving performance in a cloud daacener. Fig. 7 Power Consumpion Overhead from DRS execuion of he es server wih respec o he asleep and awaken ransiion Our proposed TP-ARM scheme is able o avoid he unnecessary energy consumpion overhead by he awaken ransiion hrough he cos model by considering he DRS ransiion overhead shown in Eq. (7). Figure 8 shows he resource uilizaion and he power consumpion measured by he Libvir API monioring module and Yoco-Wa power measuring device. VM reques insance-10 runs m mproj, insance- 0f runs m , and insance-0c runs he sreaming server wih he 180 MB movie file. Figure 8 (a) ~ (c) shows ha heir resource uilizaion is consisen wih he resource inensive characerisic of heir running workload. Figure 8d shows he differen energy consumpions according o each workload ype. The energy consumpion of he physical server is 60 Wh in he idle sae; i is abou 82 Wh when running m and 95 Wh when running boh m and m , while he sreaming server jus causes an addiional energy consumpion of abou 2 Wh. As menioned in secion 3, he main par of he resource componens affecing he power consumpion in a physical server is he CPU resource, and he effecs of oher componens such as he memory, sorage, and nework inerface cards are negligible in general. These resuls are consisen wih he ones in Table 1. Therefore, hese resuls imply ha he energy consumpion can be differen according o which resource componen has high uilizaion. Our proposed workload cos model of he TP-ARM scheme considers differen weigh values for each resource componen in order o derive he pracical cos of he resource managemen. Figure 9 shows he performance of he CPU uilizaion for he VM live migraionbeween he source machine (kdkcluser2) and he desinaion machine (kdkcluser1). There are VM reques insances, such as running insance-33 and 34, which execue he compression of video files 5 GB and 4 GB in size, respecively. Insance-33 execued he process of m mproj a 12:40:14 local ime and sared migraion o he kdkcluser1 a 12:44:00 local ime. The migraion of insance-33 was compleed by 13:24:30 local ime, and is execuion of m mproj ended a 13:47:16. Namely, he compleion ime of m mproj a insance-33 was 67 min in oal, and is migraion imes were over 30 min. If we consider he ime o complee m mproj in he case of no VM live migraion, i is forecased o be around 10 min. We can see ha

10 802 Mobile New Appl (2016) 21: Fig. 8 Libir API monioring module shows he resuls of resource uilizaion of running VM requess in he kdkcluser1 in (a), (b) and(c). Yoco-Wa module [17] was used o measure he power consumpion of he kdkcluser1 as shown in (d) here exis quie big overheads in VM live migraion. In addiion, VM migraion shows some problems in power consumpion. We considered cos models o idenify an efficien migraion scheme whichwasdefinedineq.(6). Figure 10 shows he es environmens and he operaion of he VM migraion when he proposed TP-ARM schemes are applied o he es cloud sysems wih heerogeneous applicaions and es programsincluding m , pbzip2, and neperf. From he experimens, we obained ineresing resuls in he comparison of he uilizaion performance, which examined he CPU uilizaion monioring resuls and he measured power consumpion, respecively, shown in Fig. 11. From he resuls, we found rapidly changing insans of uilizaion when he cloud brokering sysem considered he power consumpions for each running applicaion under cloud daa cener environmens. Addiionally, he es se neperf, a nework inensive workload of a es cloud sysem (e.g., kdkcluser4 wih insance-35) in Fig. 11d, showed very low performance in CPU uilizaion. I was enough o saisfy he hreshold value o run he VM migraion efficienly. And hen, he proposed TP-ARM algorihm adjuss he VM migraion procedure o reduce he power consumpion sufficienly. Basically, he TP-ARM is riggered o migrae insance-35 o anoher sysem kdkcluser2; hereafer, i changes he kdkcluser s sleeping mode in advance. Nex, because insance-34 of he kdkcluser Fig. 9 Resuls of CPU uilizaion of he migraed VM reques from (a) a source server:kdkcluser2 o (b) a desinaion server:kdkcluser1

11 Mobile New Appl (2016) 21: Fig. 10 Tes environmens and operaion of he VM migraion when he proposed TP-ARM schemes are applied for es cloud servers (e.g., kdkcluser1 ~ 5 in lab es environmens) wih heerogeneous applicaions and es programs, such as m , pbzip2, and neperf. (a) Anexample of es schedule under cloud esbed environmens (b) Illusraion of he VM migraion using wo-phase power consumpion cos models (a) The schedule of he use case (b) The illusraion of he use case sysem was running neperf, which generaed a very low performance of uilizaion, he TP-ARM also did a migraion of he insance-34 o he kdkcluser2 and ransied o he sleeping mode as well. Therefore, we could expec efficien power consumpion hrough he change in he sleeping mode of kdkcluser3 and kdkcluser4, respecively, as well as keeping acive modes for boh kdkcluser1 and kdkcluser2. Figure 11 shows he performance comparison of he power consumpion for he VM migraion and DRS process based on he TP-ARM algorihm, e.g., seen in Fig. 10. We can see a performance improvemen of 60 Wh each in power consumpion afer he change in he sleeping mode in he case of he VM migraion requess. Through he experimens, we verified ha he proposed TP- ARM scheme, which carries ou adapive migraion and asleep ransiion hrough real-ime monioring of resource uilizaion, has good performance in reducing he power consumpion effecively. This sudy provides good resuls for achieving an energy efficien cloud service for users by no increasing QoS degradaion as much. 7 Conclusion In his paper, we inroduced an Adapive Cloud Resource Broker (ACRB) wih Two Phases based he Adapive Resource Managemen (TP-ARM) scheme for energy efficien resource managemen by real ime based VM monioring in a cloud daa cener. Our proposed approach is able o reduce efficienly he energy consumpion of he servers wihou a significan performance degradaion by live migraion and Dynamic Righ Sizing (DRS) execuion hrough a considerae model ha considers swiching overheads. The various experimenal resuls based on he Opensack plaform sugges ha our proposed algorihms can be deployed o prevalen cloud daa ceners. The novel predicion

12 804 Mobile New Appl (2016) 21: Fig. 11 Performance comparison of CPU uilizaion using 4 es sysems in lab environmens. (a), (b), and (c) show he resuls of he measured uilizaion during he VM live migraion in es use case as shown in Fig. 9. (d) shows uilizaion of he proposed TP-ARM scheme ha was used in es cloud environmens. (e) Measured power consumpion of es cloud sysems (e.g., kdkcluser1 ~ 4) in lab cloud es environmens mehod called Self Adjusing Workload Predicion (SAWP) is proposed in order o improve he accuracy of forecasing fuure demands even under drasic workload changes. Especially, we evaluaed he performance of our proposed TP-ARM scheme hrough various applicaions such as Monage, pbzip2, neperf, and he sreaming server which

13 Mobile New Appl (2016) 21: have heerogeneous workload demands. Our TP-ARM scheme could maximize he energy saving performance of he DRS procedure by Phase 1: VM migraion achieves consolidaed resource allocaion under a low workload level. Moreover, i could ensure he QoS of cloud service users by Phase 2: he DRS procedure increases adapively he number of acive servers under a high workload level. Through experimens based on a pracical use case, our proposed scheme is no only feasible from a heoreical poin of view bu also pracical in a real cloud environmen. In fuure work, we will demonsrae ha our proposed algorihm ouperforms exising approaches for energy efficien resource managemen hrough various experimens based on he implemened sysem in pracice. Acknowledgmens This work was suppored by BThe Cross-Minisry Giga KOREA Projec^ of he Minisry of Science, ICT and Fuure Planning, Korea [GK13P0100, Developmen of Tele-Experience Service SW Plaform based on Giga Media]. References 1. Inernaional Daa cener Corporaion, hp:// 2. Lin M, Wierman A, Andrew LLH, Thereska E (2013) Dynamic righ-sizing for power-proporional daa ceners. IEEE/ACM Trans Neworking 21(5): Xiao Z, Song W, Chen Q (2013) Dynamic resource allocaion using virual machines for cloud compuing environmen. IEEE Trans Parallel Disrib Sys 24(6): Beloglazov A, Buyya R (2012) Opimal online deerminisic algorihms and adapive heurisics for energy and performance efficien dynamic consolidaion of virual machines in cloud daa ceners. Concurr Compu: Prac Experience 24: doi: / cpe Kang DK, Hazemi FA, Kim SH, Youn CH (2015) Dynamic virual machine consolidaion for energy efficien cloud daa ceners. In: Proc. EAI In. Conf on Cloud Compuing, Oc 6. Kim SH, Kang DK, Kim WJ, Chen M, Youn CH () A science gaeway cloud wih cos adapive VM managemen for compuaional science and applicaions. o be appeared in IEEE Sys J, Chen M, Hao Y, Li Y, Lai CF, Wu D (2015) On he compuaion offloading Ad Hoc Cloudle: archiecure and service models. IEEE Commun Mag 53(6): A-Eldin A, Tordsson J, Elmroh E, Kihl M (2013) Workload classficaion for efficien auo-scaling of cloud resources. Umea Universiy, Sweden 9. Opensack, hp:// 10. Chen M, Zhana Y, Hu L, Taleb T, Shena Z (2015) Cloud-based wireless nework: virualized, reconfigurable, smar wireless nework o enable 5G echnologies. ACM/Springer Mob New Appl 20(6): Chen M, Wen Y, Jin H, Leuna V (2013) Enaling echnologies for fuure daa cener neworking: a primer. IEEE New 27(4): Xu F, Liu F, Liu L, Jin H, Li B, Li B (2014) iaware: making live migraion of virual machines inerference-aware in he cloud. IEEE Trans Compu 63(12): Gupa D, Cherkasove L, Gardner R, Vahdaa A (2006) Enforcing performance isolaion across virual machines in Xen. In: Proc. ACM/IFIP/USENIX 2006 In. Conf. Middleware, Nov 14. Nisar A, Liao WK, Choudhary A (2008) Scaling Parallel I/O Peformance hrough I/O delegae and caching sysem. In: Proc. ACM/IEEE conf on Supercompuing, Nov 15. Chen M, Zhang Y, Li Y, Mao S, Leung VCM (2015) EMC: emoion-aware mobile cloud compuing in 5G. IEEE New 29(2): Deb K, Praap A, Agarwal S, Meyarivan T (2002) A fas and eliis muliobjecive geneic algorihm:nsga-ii. IEEE Trans Evol Compu 6(2): YOCTO-WATT, hp:// 18. G-Technology, hp:// 19. PowerWake, hp://manpages.ubunu.com/manpages/uopic/man1/ powerwake.1.hml 20. Monage, hp://monage.ipac.calech.edu/ 21. Kim WJ, Kang DK, Kim SH, Youn CH (2015) Cos adapive VM managemen for scienific workflow applicaion in mobile cloud. J Mob New Appl, Springer 20(3):