University of Bristol - Explore Bristol Research. Peer reviewed version. Link to published version (if available): /JLT.2016.

Transcription

1 Anastasopoulos M. P. zanakak A. & Smeonou D. (206). Scalable montorng an optmzaton technues for mega-scale ata centers. Journal of Lghtwave echnology 34(8) DOI: 0.09/JL Peer revewe verson Lnk to publshe verson (f avalable): 0.09/JL Lnk to publcaton recor n Explore Brstol Research PDF-ocument hs s the author accepte manuscrpt (AAM). he fnal publshe verson (verson of recor) s avalable onlne va Insttute of Electrcal an Electroncs Engneers at Please refer to any applcable terms of use of the publsher. Unversty of Brstol - Explore Brstol Research General rghts hs ocument s mae avalable n accorance wth publsher polces. Please cte only the publshe verson usng the reference above. Full terms of use are avalable:

2 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < Scalable montorng an optmzaton technues for mega-scale ata centers (Invte) Markos P. Anastasopoulos Anna zanakak an Dmtra Smeonou Abstract hs paper focuses on the esgn of servce provsonng schemes sutable for mega ata center (DC) nfrastructures. A major ssue lnke wth the operaton of these nfrastructures s scalablty cause by the ncrease number of resources avalable n mega-sze hghly-ense DCs an the assocate reurements for control an management nformaton. o aress ths scalablty ssues we propose for the frst tme to montor an optmze the operaton of mega DCs aoptng graph factorzaton combne wth compressve sensng theores. hs approach takes avantage of the spatal an temporal correlaton of compute an network resource reuests to montor an optmze metrcs such as elay an energy wth reuce control an management nformaton. Our moellng results ncate rastcally reuce volume of traffc transferre from the ata to control plane an number of optmzaton process varables. Inex erms compressve sensng graph factorzaton mega ata centres optcal packet swtchng network optmzaton. B I. INRODUCION g ata Clou an Content Delvery are rvng the ncrease of global nternet traffc expecte to excee.6 zettabytes by 208. hese reure to store an process massve amounts of ata an rve the nee for mega-sze Data Canters (DCs) scalng up to hunres of thousans of server an storage moules nterconnecte wth hgh spee communcatons lnks. he man challenge n mega-dcs nvolves scalng compute processng storage an nterconnecton capacty. In ths context two relevant archtectural approaches are consere: the scale-up an scale-out []. Accorng to the scale-up approach computatonal ntensve tasks are supporte by large scale computng platforms (eployng hgh prce servers an routers) offerng very hgh computng power levels n a gven system. hs approach offers the reure hgh computng power an storage levels n a gven relatvely smple system but sufferng lmtatons nclung ncrease cost lmte scalablty flexblty ensty avalablty an lack of moularty. he scale-out concept on the other han accommoates the ncreasng nees for computatonal an Manuscrpt receve: 26 Oct. 205 Revse 6 Jan M. P. Anastasopoulos an D. Smeonou are wth the Electrcal & Electronc Engneerng Department Unversty of Brstol Clfton BS8 UB UK (e-mal: m.anastasopoulos mtra.smeonou@brstol.ac.uk ). A. zanakak s wth the Electrcal & Electronc Engneerng Department Unversty of Brstol Clfton BS8 UB an the Unversty of Athens Department of Physcs Greece (e-mal: anna.tzanakak@brstol.ac.uk). A prelmnary verson of ths paper was presente at the European Conference on Optcal Communcaton (ECOC) 205 [9]. storage resources n a much more flexble an effcent manner. Accorng to ths approach nstea of relyng on large scale monolthc evces powerful computng systems are forme eployng a large number of low energy consumng an low cost evces. Connectvty between computng an storage evces s prove through a flat nterconnecton network collapsng together the op-of-the-rack (or) an aggregaton swtches nstea of beng supporte through a or swtch n a mult-layer network. hese swtches are confgure n fferent topologes (e.g. hypercubes 2D/3D meshes XD-torus etc.) that enable lnear scalablty to meet the ncreasng volume of emans an overcome the herarchcal tree-type network archtecture lmtatons. However supportng scalablty can be a challenge ue to the ncrease number of components an the assocate control an management reurements. Software Defne Networkng (SDN) ecouplng the control from the ata plane an movng t to a logcally centralze controller wth a holstc vew of the network has been propose as a key enablng technology [2]. o successfully apply SDN n these envronments novel solutons are neee to measure prect an optmally respon to ynamcally changng traffc workloas n a tmely manner an overcome scalablty constrants assocate wth SDN s centralze nature. Beyon a specfc volume of collecte nformaton network controllers are lmte by nsuffcent capacty to hanle ncomng ata an processng power to cope wth a large number of ecson varables an measurements neee for the network management optmzaton processes. In response to ths the new tren n network scence s to transform ths type of optmzaton problems sufferng hgh computatonal complexty to a practcally solvable problem usng correlaton nferre from ata rather than causalty [3]. A typcal example of such a process s presente n [4] where the DC placement problem s aresse by ntally analyzng bg ata to entfy possble correlatons. hen network coornate technues are apple to reuce the sze of the problems an entfy the optmal matchng between clents an servers. In ths stuy scalablty n mega-dcs assocate wth montorng an optmzaton of management ata s aresse by aoptng an combnng for the frst tme graph factorzaton (GF) theory [5] wth compressve sensng (CS) technues [6]-[8] extenng our prevous work presente n [9]. hrough GF a mega-dc network graph s ecompose nto a small number of smple graphs (factors). On top of these smple graphs an takng avantage of the spatal an temporal correlaton of nter- an ntra-dc traffc

3 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 2 a) b) Fgure : a) ratonal herarchcal DCN soluton b) Lnear scale out approach wth moular racks characterstcs [0]-[] CS s apple to montor varous metrcs e.g. resource utlzaton usng reuce control an management nformaton (low sample number). Once ths nformaton s avalable at the system controller the optmal resource allocaton problem s solve n the compresse space where the varables nvolve are sgnfcantly reuce (reucng computatonal complexty) usng Integer Lnear Programmng (ILP). o the best of the authors knowlege ths s the frst tme that CS an GF s aopte n clou computng envronments wth the am to analyze the optmal servce provsonng problem. Moelng results ncate that applyng the propose approach the volume of nformaton that reaches the controllers together wth the number of varables that are nvolve n the optmzaton process can be rastcally reuce. he rest of the paper s organze as follows. In Sec. II a bref escrpton of the relate work s prove. he problem escrpton s gven n Sec. III whle the propose hybr GF/CS jont network montorng an optmzaton scheme s presente n Sec. IV. he performance of the propose scheme s terms of scalablty an accuracy s examne n Sec. V. Fnally Sec VI conclues the paper. II. RELAED WORK A. State of the art n DC network archtectures DCs have become a key element n supportng the new an emergng ubutous Internet-base applcatons an clou servces. Hunres of thousans of servers are hoste n largescale DCs where huge amounts of ata (erabytes/petabytes [2]) are mantane an processe. DC provers have observe an over 70% annual ncrease n the DC traffc volume [3] an ths ever-growng traffc eman s expecte to stretch the DC nfrastructure reurements. Moreover n Europe the electrcty consumpton of DCs s approachng 60Wh at present an s projecte to reach 04Wh by 2020 [4]. herefore the esgn an evelopment of future DC nfrastructures has attracte sgnfcant attenton both from acaema an nustry. Base on the type of servces supporte an the avalable eupment nformaton exchange varous ntra-dc communcaton archtectures have been propose to ate. hese archtectures are organze nto three major classes As scusse n [0] Web servces emal veo an messagng present correlaton patterns n terms of the nterplay of ata between fferent servces. hese nclue the use of a common ata set or exchange of nformaton prouce by the nteracton wth the user. Furthermore DC traffc exhbts urnal an clear weeken/weekay varaton []. base prmarly on network topology. hese nclue rect networks (also known as server-only) nrect networks (swtch-only) an hybr networks (hybr server an swtch DC) archtectures [5]. Drect network archtectures comprse a set of noes (e.g. servers) each one beng rectly connecte to other noes. In these archtectures each server apart from executng regular applcatons t also partcpates n packet relayng [6]. Although sgnfcant work has focuse on analyzng the performance of varous server-only nterconnecton archtectures only a lmte subset of these have been actually mplemente. Most of the mplemente networks use an orthogonal topology n whch the servers are arrange n an n-mensonal space. Orthogonal topologes are further classfe nto strctly orthogonal an weakly orthogonal [7]. he man avantage of the rect archtectures s that they scale very well to a large number of servers. However they suffer the followng lmtatons: a) they reure sgnfcant processng resources for packet forwarng an b) servers are nterconnecte usng a large number of lnks an network nterface cars. In nrect or swtch-base networks on the other han connectvty between any two noes s carre out through swtches. Multple layers of swtches are then nterconnecte formng a herarchcal networkng moel. Swtches may be organze ether usng smple tree topologes [8] (usually two-ter or three-ter [9]) or nterconnecte n a more sophstcate manner e.g. usng fat trees [20] [2]. he herarchcal moel conssts of the core the aggregaton an the access layers. ypcally the access layer conssts of servers per rack each connecte to a or swtch through a or 0Gbps lnk. Other swtchng solutons for server networkng toay nclue en-of-row (EoR) as well as ntegrate swtchng. Connectvty between layers s acheve usng the IEEE 802.Q famly of Ethernet protocols that enables synchronzaton of physcal an vrtual network confguratons. It s reporte n the lterature [2] that ths type of DC archtectures suffers: a) lmte DC-to-DC capacty b) fragmentaton of resources c) poor relablty an utlzaton an ) hgh latency. hese lmtatons coul be overcome by the use a sngle large scale N N swtch however the cost of such swtch s stll prohbtve for large DCs. Fgure llustrates the herarchcal scale-up an the flat strbute scale-out DC archtectures. Assessng the benefts an lmtatons of these solutons n the present stuy the hybr swtch-server approach ([22] [23]) s aopte n a flexble an ynamc fashon. As such the propose DC network reles on nterconnectng compute an

4 m-axs -axs k-axs > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 3 DC Server (r) Level 0 Level DC box: (jk) Contaner (mn ) or Swtch Output λ λ 2 λ R r-axs λ -λ R Optcal Packet Swtch Input Contaner Inter-contaner lnks -axs Contaner (mn+ ) Contaner Local (jk) λ- (-jk) -axs (+jk) (j-k) j-axs (j+k) (jk+) k-axs (jk-) Input traffc x x x x x x x Optcal Moule Optcal Moule 2 Optcal Moule 3 Optcal Moule 4 Optcal Moule 5 Optcal Moule 6 Optcal Moule 7 Optcal Packet Swtch λ λ λ λ λ λ λ AWG AWG AWG AWG AWG AWG AWG (jk) (-jk) (+jk) (j-k) (j+k) (jk+) (jk-) Output traffc storage moules through a combnaton of server-to-server an strbute swtch type of connectvty base on optcal packets swtches (OPS) [24]. hs soluton can prove sgnfcant benefts n terms of scalablty resource an energy effcency an effectvely mprove system performance n terms of metrcs such as latency. B. State of the art n Compressve Sensng So far CS has been successfully apple to solve a varety of problems rangng from ata gatherng n mult-hop wreless sensor networks (WSNs) (see e.g. [25]-[27]) an network traffc estmaton ([28]-[29]) to network tomography [30]. For example n [25] the authors nvestgate the performance n terms of capacty an elay of ata aggregaton employng CS for a scenaro where sensor noes are ranomly eploye n a regon. In [26] the authors apple CS n ata collecton to effcently reuce communcaton cost an prolong network lfetme for large scale montorng sensor networks. [27] aresse the ata aggregaton problem n WSNs by jontly conserng routng an CS to transport ranom projectons of the montore ata whereas n [28] [29] the authors propose optmzaton approaches to estmate the normal an anomalous traffc usng a small subset of measurements. Fnally n [30] the authors formulate the mnmum path selecton problem that ams at estmatng lnk elays usng a small number of en-to-en elay measurements. Despte ts great potental effcent mplementaton of CS n mega-dc envronments can be ute challengng as for large number of components the storage space reurements for the measurement matrx an the computatonal cost reure to recover the orgnal nformaton s hgh. o aress these ssues we aopt GF an combne t wth CS n orer to ecompose the orgnal problem nto a set of separable subproblems wth reuce computatonal complexty. o the best of the authors knowlege ths s the frst tme that CS s combne wth GF to aress scalablty ssues n mega-dc nfrastructures. III. PROBLEM DESCRIPION System n-axs System Level 2 Fgure 2: Example of a herarchcal mega DC network archtecture wth 2/3 mesh connectvty. Level 0: Small scale servers are combne to form DC boxes. Connectvty between DC boxes s acheve through an 7x7 OPS. Level : DC boxes are groupe to form contaners an fnally several contaners are combne to form the mega-dc (level 2). A mult-ter mega-dc network where computng moules are nterconnecte base on a hybr swtch-server approach s consere (Fgure 2). At the lower level (level 0) a DC box system comprsng servers an network swtches s use. DC boxes are also euppe wth local controllers. he varous moules of the DC box system are nterconnecte formng a 2 mesh topology combnng server-to-server an strbute swtchng connectvty. o acheve low latency hgh banwth an energy effcent connectvty between compute moules the DC boxes eploy an OPS soluton base on [24] [36]. Each non-blockng optcal packet swtch comprses 7 nput an 7 output ports an supports R wavelengths per port. In the next level (level ) the varous DC boxes are groupe n a 3 mesh topology to form contaners. Connectvty between neghborng DC boxes s acheve through the OPSs. Across each menson two ports of the OPS are use for the ngress an two ports for the egress traffc respectvely whereas one port s use to prove local connectvty between

5 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 4 servers. herefore epenng on ts poston every DC box connects wth a number of neghborng DC boxes that vares between 3 an 6. It s clear that DC boxes locate at the ses of each contaner wll be connecte wth up to 5 neghborng noes thus leavng some ports of the OPS swtches unuse. However these ports can be use n the fnal stage to nterconnect the contaners n a 3 mesh manner an form the mega-dc system (level 2). In mega-dcs optmal resource allocaton ams at etermnng n a tmely manner the network an computatonal resources reure to satsfy a set of emans wth volume. ratonally resource allocaton D h problems n DCs are solve by centralze controllers applyng an overall optmzaton crteron through ILP an Mxe ILP technues. Although ILP-base optmzaton schemes can be easly formulate an mplemente they suffer savantages such as: ) reurement of full an accurate nformaton of all parameters nvolve ) exponental scalng of computatonal complexty wth the network sze makng t unsutable for mega-dcs. o cope wth the ncreasng computatonal complexty nherent n ILP formulatons mensonalty reucton base on Lagrangan Relaxaton [30] clusterng [3] an heurstc technues [32] have been propose. In the present stuy a fferent approach s aopte an a hybr GF/CS scheme s employe to reuce the global amount of traffc transferre from the ata to the control plane an the number of varables nvolve n the optmzaton process. IV. MAHEMAICAL MODELING PRELIMINARIES Let x be an N mensonal sgnal vector wth elements x 2... N. Any sgnal x can be represente usng as a bass a N N matrx wth elements j as follows [34]: x Ν Ν s x=ψs x s where DC Box (jk) θ Samples from colu of server (:) Server (2) Server () Phase 0 N Ν ΝΝ N () s s... sn s vector wth weghtng coeffcents s. Sgnal x s sa to be K DC server server colu server colu (2:) (R-:) colu (R:) -sparse n oman f n () there are K non-zero elements n vector s. CS theory states that the K -sparse vector x can be effcently reconstructe base on a set of M measurement capture through the vector y y... y wth M N usng an M N M from box (jk) θ Samples from colu of DCs (j:) DC (jk+3) DC (jk+2) Contaner (mn ) DC colu (+j:) DC colu ranom measurement matrx φ. Mathematcally ths process can be wrtten n the followng form: y Ν Ν x y=φx y x M M MΝ N (2) At ths pont t shoul be note that the mnmum number of measurements M reure to reconstruct the K -sparse sgnal s gven by [35]: x where c 2 M cμ Κ log N (3) s a postve constant number an known as mutual coherence s the largest correlaton between any two elements of an. he mutual coherence s boun by [35]. Once the set of measurements x y has been collecte the orgnal sgnal can be recovere solvng the followng - mnmzaton problem: mn N s subject to y=φx x=ψs (4) s ŝ he output of (4) namely s then use as nput to () n orer to reconstruct an approxmaton of the orgnal sgnal. At ths pont t shoul be mentone that a necessary conton for (4) to effcently reconstruct the orgnal sgnal s the matrx θ = φψ to satsfy the restrcte sometry property (RIP) [35]. As scusse n [35] ths can be acheve wth hgh probablty smply by selectng the elements of φ at ranom. Contaner DC row (:j+:) from contaner (mnl) contaner colu (m:l) System from DC from contaner (jk+) DC colu DC row (:j:) (+2j:) (+Ιj:) (mn+l) Phase 02 Phase 2 Phase 22 Level 0 Phase Level Phase 3 Phase 2 Level 2 Fgure 3: CS-base network montorng wth graph factorzaton V. HYBRID GF/CS-BASED SERVICE PROVISIONING he hybr GF/CS scheme can acheve mprove scalablty f spato-temporal correlate performance DC-relate metrcs are transporte to the system controller over the factorze graphs an are processe jontly. he jont montorng an network optmzaton framework comprses the followng steps: ) Network topology ecomposton: he mega-dc network topology s ecompose nto multple smple graphs base on GF theory. Assumng that a DC box s moelle as an unrecte 2D mesh graph wth sze Q R Box can be rewrtten n a ecompose form as Box r where r are smple lnearly connecte ˆx Phase 23 Box System

6 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 5 a) b) c) Fgure 4: Numercal example: a) Actual average utlzaton per contaner Reconstructon of the orgnal ata wth 8% samples usng b) Compressve Sensng (CS) wth error % c) Least Suares Error (LSE) analyss (System wth 20^3 contaners) wth error 25% R Con graphs comprsng Q an server noes respectvely an enotes the Cartesan prouct operator. Each contaner wth connectvty graph can be ecompose nto a set of subgraphs. Assumng that Con follows a 3D mesh pattern can be wrtten as Con k j where an are lnear graphs wth K an noes respectvely (Fgure 3). he same ratonale can be extene at the system level where the 3D mesh graph of the system can be expresse as: sys m n where graphs wth M N an I J m n sys k an contaners respectvely. j are smple lne 2) Network parameters compresson he etals of the DC are then abstracte an transmtte to the system controller. Startng from level 0 (phase 0 n Fgure r 3) each server wth coornates r R Q multples the parameters of 2 u r r nterest say wth a ranom coeffcent e.g. an transmts the prouct to ts ajacent noe. In Fgure 3 server transmts the proucts u up to u contanng a set of measurements to ts ajacent server. Once server has receve these messages t calculates the ranom proucts 2 u an sens the weghte averages of the measurements generate at server an 2 u 2u 2 to server 3. Each ntermeate server as to the ncomng messages ts prouct u an forwars the weghte average to the next server. hrough ths process the top servers n each graph wll receve packets contanng the weghte 2 averages of the ranom measurements performe by all servers n each. hs s gven by: y u u r ) Q r : (5 r r r r R where :... r r rq u... r ur u rq an s the transpose operator. Now let be the ranom measurement matrx over the factorze graphs efne through: : R: Φ : R: Base on (6) euaton (5) can be wrtten n compact form as follows: y Φ u (7) where u u u2 u R stacks the orgnal set of (6) y y y s a colu measurement nto a vector an vector wth elements y y y r. In phase 02 the ata collecte from all top servers are multple wth the ranom coeffcents r r R '( ' ) an relaye across the ecompose graph. he last server n graph (server RQ ) of DC box j k ) wll get packets contanng weghte averages of all ranom measurements for all servers wthn each DC box that s: ' k j ' : ' r ' z y y ' (8)... R ' ' :... ' :. where ' : ' ' Assumng that Φ s the ranom measurement matrx over the factorze graphs r efne through: : : Φ (9) ' : ' : then (8) can be wrtten as follows: where jk zjk y u (0) z s an ' colu vector efne as z jk z... z jk ' jk. Packets jk z wll be then use as nput to level contaners. Wthn each contaner these r

7 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 6 a) b). c) Fgure 5: Snapshot of the average utlzaton per contaner (system wth 0^3 contaners) a) before system optmzaton b) after optmzng for maxmum performance per watt per space b) after optmzng for loa balancng (the axes represent the coornates of the contaners) packets wll be relaye across the graphs an. he j output of the last DC box n (phase 3 n Fgure 3) of contaner namely w... w w wll be use as nput to level 2. Followng the same ratonale wll be eual to: where w j j z () j are the samplng matrces across k k w an j respectvely an z s a vector that stacks all DC box measurements. he same process s repeate for all contaners untl the collecte nformaton namely reaches the system controller. can be estmate through the followng euaton: g = w (2) where w g m s a stacke vector wth elements n g w. 3) Reconstructon of the nformaton at the system controller Once the abstracte nformaton reaches the system controller the optmal nter-contaner resource allocaton strateges nee to be entfe. o acheve ths the compresse parameters together wth the ranom g coeffcents m n an are use to recover vector the elements of whch contan nformaton on the utlzaton of each contaner. Now let Ψ be a compressblty bass for w wth Ψ m n beng a bass for the graph. w can be recovere solvng a set of - mnmzaton problems usng the followng steps: g w w... of through the soluton of the ) In the frst step (phase 23) the elements w w graph are recovere from followng problem: mn s ) Once w the elements s g subject to g = w w s (3) has been estmate n the secon step (phase 22) w n of graph element of are estmate through: s n that s connecte wth the mn N s subject to w = n wn wn ns (4) ) In the fnal step followng problem: s w s recovere by the soluton of the mn M s subject to w = w n m w s m nn (5) Problems (3)-(5) can be solve n polynomal tme over the factorze graphs usng nteror pont methos that have 3 N ' computatonal complexty where N ' s the number of components that nee to be montore. For example the complexty for recoverng nformaton per contaner at the N 3 M 3 3. However when system level wthout GF s GF s aopte the orgnal problem s ecompose nto a set of much smaller sub-problems (wth sze eual to the sze of the factorze graphs) leang to sgnfcant computatonal complexty reucton.

8 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 7 Control nformaton (packets) LSE Hybr GF/CS Wthout compresson Number of varables Wthout Compresson Hybr GF/CS Number of contaners 26 Number of DC boxes Fgure 6: Volume of nformaton vs number of servers for the followng schemes: wthout compresson wth statstcal samplng an nformaton reconstructon base on LSE an wth hybr GF/CS (Optmzaton error 8%) 4) Optmzaton n the compresse space In the followng step the recovere nformaton ncatng the average actual usage of all servers an DC boxes belongng to s use to formulate an w optmzaton problem n the compresse space. hrough aggregaton of the resources etals the number of parameters an ecson varables nvolve n the optmzaton phase can be rastcally reuce. Now let ( D) be the emans that nee to be allocate to contaners mm n N. he volume of the traffc eman s enote by. he objectve s to entfy optmal resource allocaton strateges maxmzng performance per space: mn h M N 3 ' l w m n w (6.) Subject to z h D (6.2) M N Q ( ) m n M N ( n ) m n m D (6.3) (6.4) D M N Q m e z n e D ( ) ' h w m M n N (6.5) w' w m M nn (6.6) l where Q s the canate path lst at the system reure to support eman at contaner. hs can be precompute usng the k-shortest path algorthm. enotes the stance n terms of number of hops of from the system controller eman Number of Contaners Fgure 7: Number of ecson varables as a functon of the number of DCs wth an wthout GF/CS. e z s the lnk s the capacty allocate to path for e capacty. ( ) s a bnary coeffcent takng value eual to f eman s assgne to contaner. s a bnary coeffcent that euals f e e lnk belongs to path realzng eman at contaner s the capacty of contaner an s a bnary parameter takng values eual to when contaner l s actve D ( ) ; 0 otherwse. In the above formulaton constrant (6.2) (known as the eman constrants) assures that the volume h of eman wll be realze through flows at the contaner. (6.3) assures that each eman wll be assgne at a sngle contaner whereas (6.4) enotes the network capacty constrants. he necessary processng capacty reure to support emans at contaner s capture through (6.5). Fnally the avalable capacty at each contaner l w shoul be aeuate to support the reueste servces (6.6). An nterestng observaton s that for the objectve functon a cubc evaton cost has been aopte that ams at maxmzng performance per watt per space by packng as many emans as possble at a sngle contaner. o acheve ths n case where a contaner s actve w takes z w' values very close to the avalable capacty l w mnmzng the evaton cost w w' 3 l '. Note that the cubc cost aopte n the objectve functon ams at magnfyng the penalty that s ntrouce when a contaner s unerutlze. hs gves an ncentve to the system to transfer tasks from low to hgh utlze contaners an ntrouces a hgh penalty when contaners reman unerutlze. 5) Optmal Intra-contaner resource allocaton Once the contaners where the emans are processe have been efne the optmal ntra-contaner routng strateges are etermne at each contaner controller. For each contaner the compresse parameters w together wth the ranom

9 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 8 Optmzaton Error (%) CS LSE Control Informaton (Packets) 7 x Servers: Servers: Sample ata (%) Fgure 8: Impact of level of compresson on the accuracy of the obtane results when the orgnal nformaton s reconstructe usng CS an LSE. coeffcents are use to recover vector the elements of whch contan nformaton on the utlzaton of the DC boxes. Followng the ervaton of an optmzaton problem smlar to that presente n (6) s formulate at a contaner level that etermnes the DC boxes where emans are processe. Base on a set of - mnmzaton j z jk problems at a DC box level are formulate estmatng the utlzaton per server wthn the DC boxes ( ). Base on an solvng a problem smlar to ()-(5) the optmal eman allocaton at a server level s etermne. VI. NUMERICAL RESULS he hybr GF/CS optmzaton approach s evaluate for the topology of Fgure 2 wth 20x0 servers per DC box an cubc sze contaners (where K=I=J). Both the system controller an the network controllers are place at the top se of the 3 mesh topologes. raffc statstcs have been generate by approprately mofyng [37] assumng 58.88% usage for nter-contaner lnks 73.77% for ntra-contaner an 57.52% for server-to-server communcaton lnks. he sze of packets generate follow a bmoal strbuton wth peaks aroun 40B an 500B an an average packet sze of 850B. Generate traffc exhbts an ON/OFF pattern wth uraton of the ON/OFF pero followng the lognormal strbuton. he packet nter-arrval tmes wthn ON peros (n mllsecons scale) follow the lognormal strbuton wth parameters (6.4.56). he same also hols for the length of OFF-peros an ON-peros that follow the lognormal strbuton wth parameters ( ) an ( ) respectvely. he performance of the propose hybr GF/CS scheme s compare to the followng baselne approaches: ) Wthout compresson : hs correspons to the case where nformaton s gathere from all harware elements n the system. hs functonalty s supporte by the majorty of the exstng operatng systems.e. Junos OS 5. [38] ) LSE : hs correspons to the case where statstcal samplng s performe (one packet s ranomly selecte n an nterval of n packets.e. CISCO NetFlow [39]). he orgnal nformaton s then reconstructe at the SDN controller usng regresson analyss technues such as Least Suares Error z u r z u r Optmzaton Error (%) Fgure 9: Volume of nformaton reachng the controller uner fferent levels of optmzaton error an number of servers for the propose hybr GF/CS scheme. (analyss) [40]. In both schemes once nformaton has been collecte network optmzaton s performe. Intally the effcency of the propose hybr GF/CS nformaton reconstructon scheme s examne. In Fgure 4 (a) a snapshot of the actual average utlzaton per contaner s prove for a system wth contaners. Fgure 4(b) shows that ths nformaton can be successfully reconstructe wth error % usng the propose CS-base scheme even when a very low number of samples s use (8% samples). However when the LSE analyss s apple over the same number of samples the reconstructon error s (Fgure 4 (c)) n the orer of 25%. It s also observe that the prevalng tren for the CS approach s to unerestmate the utlzaton of the contaners. hs s explane by the low samplng rate (8%)) an the small number of hghly utlze contaners. On the other han n a scenaro where a large number of hghly utlze contaners exsts an overestmaton of unerutlze contaners s expecte. In the next step once the necessary nformaton has been retreve an optmzaton problem that tres to maxmze performance per watt per space s solve at the system controller. A snapshot of the average utlzaton per contaner before applyng the propose optmzaton scheme s llustrate n Fgure 5 (a). Once the system has been optmze for maxmum performance per watt per space t s seen that the majorty of the contaners have been swtche off to save energy an tasks have been consolate to a small number of hghly utlze contaners (Fgure 5 (b)). It s also observe that contaners locate at the top of the system are almost fully utlze whereas contaners locate at the bottom are nactve. hs s explane by the fact that the propose objectve functon allocates tasks to contaners that are locate close to the system controller. hrough ths approach the performance of the system can be mprove through the reucton n the contaner-to-control plane elays. A fferent task allocaton polcy that s also examne tres to eually strbute tasks among contaners (.e. ths can be acheve by maxmzng 20 M N the Jan s farness nex x m n 3 2 /

10 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 9 l M N 2 x m n wth x w w' l [4]). he output of ths process usng the same startng pont s llustrate n Fgure 5 (c) when t s seen that contaners are almost eually utlze. In Fgure 6 we compare the performance of the propose hybr GF/CS scheme n terms of the volume of nformaton that reaches the system s controller wth the tratonal scheme where compresson s not apple an the statstcal samplng scheme. Our results show that the hybr GF/CS scheme rastcally reuces the volume of control ata compare to exstng approaches thus reucng the varables nvolve n the ILP formulaton an the assocate computatonal complexty. It s also observe that the volume of nformaton ncreases almost lnearly wth the number of servers (Fgure 7). When the propose scheme s aopte nstea of all servers senng ther status to the controller rectly nformaton s multplexe. hs allows a constant number of packets contanng the weghte averages of the measurements performe to be relaye across the factorze graphs. It s also observe that the benefts of the propose hybr GF/CS scheme ncreases wth the sze of the DC systems. However for small scale DCs tratonal schemes report lower amounts of packets compare to that for the GF/CS. As alreay mentone n orer for the CS scheme to be effectve the number of measurements shoul be much lower than the number of montore ata. Hence for small scale DCs the number of packets that are relaye contanng the weghte averages of the ranom measurements s hgher than the number of servers leang to suboptmal performance of the propose scheme. Fgure 8 shows the mpact of the number of samples on the optmzaton error when the orgnal nformaton s reconstructe usng the CS an the LSE approach. he optmzaton error s efne as the gap between the result of each one approach (.e. the hybr GF/CS an the LSE) an the orgnal nformaton. As expecte for lower number of samples the estmaton error ncreases. hs may lea to an overestmaton or an unerestmaton of the avalable capacty per server causng suboptmal operaton of the entre system. For example unerestmaton of the actually use resources (.e. unerestmaton of ncatng the average usage per contaner) may lea to an nablty for the system to satsfy resource reuests (especally f the system operates close to ts capacty lmt.e. w takes values close to ). Overestmaton of the actually use resources on the other han may lea to ncrease operatonal expentures snce atonal servers wll be actvate to cover the same traffc emans. However the CS scheme reures a much lower number of samples compare to the LSE scheme to acheve the same level of accuracy. Fnally Fgure 9 llustrates the volume of nformaton that reaches the system controller uner fferent levels of optmzaton error an number of servers for the propose hybr GF/CS scheme. As expecte system controllers that are able to hanle hgher volumes of control nformaton can process more complex optmzaton tasks leang to mprove system performance an lower levels of optmzaton error. Furthermore t s observe that the propose scheme s not w affecte by the ncrease n the DC sze snce a four factor growth n the number of servers ncreases the ata volume by less than 20% wth a 6% optmzaton error VII. CONCLUSIONS hs paper focuse on the esgn of servce provsonng schemes sutable for mega-dc nfrastructures. o aress the scalablty ssues of these nfrastructures ntrouce by the ncrease number of resources avalable n these an the assocate reurements for control an management nformaton we propose for the frst tme to combne graph factorzaton wth the recently reporte compressve sensng theores to montor an optmze ther operaton. hs approach takes avantage of the spatal an temporal correlaton of compute an network resource reuests to montor an optmze metrcs such as server utlzaton wth reuce control an management nformaton. Our moellng results ncate rastcally reuce amounts of traffc transferre from the ata to control plane an number of optmzaton process varables. ACKNOWLEDGEMENS he work was supporte by the EPSRC grant EP/L020009/: OUCAN an the Horzon 2020 project IN2RAIL. he authors woul lke to thank the anonymous revewers for ther constructve comments. REFERENCES [] A. Vahat et al. Scale-out networkng n ata center IEEE Mcro vol. 30 no [2] M. Channegowa et al. Software-efne optcal networks technology an nfrastructure J. Opt. Commun. Netw. nol. 5 no [3] H. Yn et al. Bg ata: transformng the esgn phlosophy of future nternet IEEE Network vol.28 no.4 pp.49 July-August 204. [4] Y. Hao Z. Xu Z. ongyu Z. Yng M. Geyong D. O. Wu "NetClust: A Framework for Scalable an Pareto-Optmal Mea Server Placement" IEEE ransactons on Multmea vol.5 no.8 pp Dec. 203 [5] R. Hammack Hanbook of prouct graphs CRC 20 [6] D. Donoho Compresse Sensng IEEE rans. on Informaton heory vol. 52 no [7] M. Anastasopoulos et al. Scalable Servce Provsonng n Converge Optcal/Wreless n proc. of OFC 205 [8] H. Zheng et al. Capacty an Delay Analyss for Data Gatherng wth CS n Wreless Sensor Networks IEEE rans. Wrel. Commun. vol 2 no [9] M.P. Anastasopoulos A. zanakak D. Smeonou Enablng mega- DCs through scalable montorng an optmzaton n proc. of ECOC Sept Oct. 205 [0] C. Yngyng; S. Jan V.K Ahkar Z. Zh-L; X. Kua A frst look at nter-ata center traffc characterstcs va Yahoo! atasets" n proc. of INFOCOM 20 pp Aprl 20 []. Benson A. Akella an D. A. Maltz Network traffc characterstcs of ata centers n the wl n proc. of ACM IMC pp [2] Pattern-Base Strategy: Gettng Value From Bg Data June 20. [3] R. Vllars et al "From IDC Precatons 203: he new ata centre ynamc" Dec [4] [Onlne] DCD Inustry census 204: Data Centre Power. [5] L. Popa et. al. A cost comparson of atacenter network archtectures'. n Proc. of Co-NEX no [6] H. Abu-Lbeh et al. Symbotc Routng n Future Data Centers In proc. of ACM SIGCOMM 200. [7] W. Dally an B. owles Prncples an Practces of Interconnecton Networks. Morgan Kaufmann Publshers

11 > REPLACE HIS LINE WIH YOUR PAPER IDENIFICAION NUMBER (DOUBLE-CLICK HERE O EDI) < 0 [8] Csco Systems Data Center Desgn IP Network Infrastructure US/ocs/solutons/Enterprse/Data Center/DC 3 0/DC-3 0 IPInfra.pf Oct [9] DC_Infra2_5/DCInfra_.html [20] M. Al-Fares et al. A scalable commoty ata center network archtecture n proc. of SIGCOMM pp [2] A. Greenberg et al. VL2: a Scalable an Flexble Data Center Network n proc. of SIGCOMM pp [22] C. Guo et al. BCube: A Hgh Performance Server-centrc Network Archtecture for Moular DCs n proc. of SIGCOMM vol. 39 no. 4 pp [23] C. Guo et al. Dcell: A Scalable an Fault-tolerant Network Structure for Data Centers ACM SIGCOMM Computer Communcaton Revew vol. 38 no. 4 pp Oct [24] S. Lucente et al. FPGA Controlle Integrate Optcal Cross-Connect Moule for Hgh Port-Densty Optcal Packet Swtch n proc. of ECOC 202 u.3.a.3. [25] H. Zheng et al. Capacty an Delay Analyss for Data Gatherng wth Compressve Sensng n Wreless Sensor Networks IEEE rans. Wreless Commun. vol.2 no.2 pp Feb. 203 [26] C. Luo et al. Effcent Measurement Generaton an Pervasve Sparsty for Compressve Data Gatherng IEEE rans. Wreless Commun. vol.9 no.2 pp Dec. 200 [27] G. Quer et al. On the nterplay between routng an sgnal representaton for Compressve Sensng n wreless sensor networks" In Proc. of IA 2009 pp Feb [28] M. Maran G. B. Gannaks Robust network traffc estmaton va sparsty an low rank n Proc. of IEEE ICASSP. pp [29] Y. Zhang M. Roughan W. Wllnger L. Qu. Spato-temporal compressve sensng an nternet traffc matrces SIGCOMM Comput. Commun. Rev. Vol. 39 no August [30] M P. Anastasopoulos A. zanakak an D. Smeonou "Stochastc Plannng of Depenable Vrtual Infrastructures Over Optcal Datacenter Networks" J. Opt. Commun. Netw. Vol. 5 pp [3] K. Chen et al. Generc an automatc aress confguraton for ata center networks n proc. of SIGCOMM '0 pp [32] X. L G. Shen "Optmal Content Cachng base on Content Popularty for Content Delvery Networks" n proc. of APC 205 AS4G.4. [33] M.H. Frooz S. Roy Network omography va Compresse Sensng n proc. of IEEE GLOBECOM 200 pp Dec. 200 [34] R.G. Baranuk Compressve Sensng [Lecture Notes] Sgnal IEEE Processng Magazne vol.24 no.4 pp.82 July 2007 [35] E. Canès an J. Romberg Sparsty an ncoherence n compressve samplng Inverse Prob. vol. 23 no. 3 pp [36] N. Calabretta et al. "Flow controlle scalable optcal packet swtch for low latency flat ata center network" n Proc. of ICON 203 June 203 [37]. Benson et al. Unerstanng ata center traffc characterstcs ACM SIGCOMM CCR vol. 40 no. pp Jan. 200 [38] [Onlne] Confgurng raffc Samplng - Junper Networks [39] [Onlne] CISCO NetFlow Samplng [40] D. Longfe Y. Wenguo S Gao; X. Ynben Z. Mngmng Zhgang J "EMD-Base Mult-Moel Precton for Network raffc n Software- Defne Networks" n proc. of IEEE MASS pp Oct. 204 [4]. aleb N. Nasser M.P. Anastasopoulos "An Aucton-Base Pareto- Optmal Strategy for Dynamc an Far Allotment of Resources n Wreless Moble Networks" IEEE rans. Vehcular echnology vol.60 no.9 pp Nov. 20