A MapReduce-supported Data Center Networking Topology

Transcription

1 A MapReduce-supported Data Center Networkng Topology Zelu Dng*,, Xue Lu, Deke Guo*, Honghu Chen*, Xueshan Luo* * Natonal Unversty of Defense Technology, Chna McGll Unversty, Canada Abstract Several novel data center networkng (DCN) topologes have been proposed to mprove the topologcal propertes of data centers. Unfortunately, t s gnored that whether these topologes are suted for the onlne applcatons and nfrastructure servces runnng on the correspondng data centers. In ths paper, we propose a novel DCN topology, named Hyper- Fat-tree Network (HFN). HFN ncarnates the good characterstcs of the BCube and Fat-tree topologes, and hence naturally supports the dstrbuted data processng applcaton MapReduce. We then address several challengng ssues facng HFN to support MapReduce. Through analyss and smulatons, we show that HFN possesses excellent propertes and s a vable toplogy for MapReduce.. Introducton Data centers have emerged as the dstrbuted storage and computng nfrastructures for many onlne applcatons and nfrastructure servces, for example the cloud servces []. A data center can storage and process massve data through ts nner servers and then provde many knds of agle and effectve servces to users. A fundamental challenge facng a data center s to desgn a data center networkng (DCN) for effcently nterconnectng a large number of servers va hgh-speed lnks and swtches [2]. In realty, a data center should be equpped wth specfc data management and processng mechansms, such as GFS [3], HDFS [4], Bgtable [5], Dryad [6], etc. to manage and operate massve data effectvely and effcently. One of the most mportant data processng mechansms s MapReduce, whch was proposed by Google n 2004 [7]. MapReduce not only provdes a good control and executon mode for dstrbuted computng and cloud computng, but also extends the servce feld of data centers. In recent years, the emergng dverse servces appeal for the mprovement of topologcal performance of DCN, ncludng scalablty, relablty, etc. In order to meet these requrements, several novel DCN topologes have been proposed, such as Fat-tree [8], Dell [2], FConn [9], and BCube [0]. They do remedy the defects of the tradtonal tree-shaped topology from dfferent aspects. A DCN topology s usually desgned to optmze some fundamental topologcal propertes, and does not consder whether these topologes are suted for the dstrbuted data processng mechansms runnng on data centers. All these novel topologes have not gve the nterconnecton relatonshp of master servers and data servers, whch s the base of dstrbuted data processng mechansms. At the meanwhle, a number of mprovng methods for MapReduce are researched [][2][3]. But they just only descrbe novel procedures and programs for Mapreduce. The relatonshp of the elements that execute these procedures n DCN has also been gnored. Actually, for dfferent dstrbuted data processng mechansms, there should be dfferent correspondng DCN topologes. Only the DCN wth suted topologes for dstrbuted data processng mechansms can meet users ncreasng new servce requrements. To solve ths problem, basng on the BCube and Fat-tree structures, ths paper presents a MapReduce -supported DCN topology named Hyper-Fat-tree Network (HFN). The executon and fault tolerance methods for runnng MapReduce on HFN are researched. In order to be scalable, HFN s recursvely defned basng on the recursve rule used n BCube. Namely, the constructon method uses a low level HFN as a structure unt, and connects many such unts together by means of a hypercube to be a hgh level HFN. What dfferent from BCube s that the smallest recursve unt of HFN s a redundant structure, whch s bult by means of a smlar Fat-tree graph, so as to be relable for runnng MapReduce. In the smallest recursve unt, the nterconnecton relatonshp between master servers and data servers s defned accordng to procedures of Mapreduce, n order to make HFN be suted for the data processng mechansm of MapReduce. HFN s a relable topology wth hghly connectvty and low dameter, for makng use of the

2 advantages of hypercube and Fat-tree. Snce the number of servers n HFN s tmes larger than that of other typcal recursvely defned structures, HFN can scale well to nterconnect the exponentally ncreasng number of servers n DCN. To support a confused MapReduce servce request, the number of servers may be up to one thousand or even more. It s hard for just one master server to control so many other servers smultaneously. The control mechansm adopted n ths paper makes each master server control a number of data servers, and makes many such master servers execute one confused MapReduce task together. Basng on the control mechansm and connecton relatonshp between master servers and data servers, ths paper gves the dstrbuted data operatng and transmttng methods for Map and Reduce operatons. The problem of DCN topology hardly adaptng to Mapreduce s effectvely solved n ths paper. The major contrbutons of ths paper are summarzed as follows. Frst, a MapReduce-supported DCN topology named Hyper-Fat-tree Network (HFN) s presented. Second, the specfc method of runnng MapReduce on HFN for dstrbuted data processng s researched. Thrd, the fault tolerance method for MapReduce on HFN s descrbed. Fnally, by the way of compare, analyss and smulaton, t s proved that HFN s a relable and scalable topology wth hghly connectvty and low dameter. It s also proved that HFN s competent for MapReduce even wth node faults. The rest of ths paper s organzed as follows. Secton 2 ntroduces the related works. Secton 3 presents the physcal archtecture and constructon method of HFN. Secton 4 descrbes the roles of the servers, routng and executon methodology for MapReduce on HFN. Secton 5 gves the fault tolerance routng and fault tolerance approaches for MapReduce on HFN. Secton 6 evaluates the topology propertes and executon tme for MapReduce on HFN through analyss and smulatons. Secton 7 concludes ths paper. 2. Related Works 2. DCN Topologes Network topologes of most current data centers are the tradtonal tree structure. In ths structure, servers are located on leaf nodes. On mddle nodes and root nodes are placed the aggregaton swtches and core swtches respectvely. The servers are nterconnected by aggregaton swtches, whch are connected together by core swtches. The tradtonal tree structure s easy to buld, but can not scale well. Snce the swtches may lead to bottlenecks easly, and a core swtch fault may break down hundreds or even thousands of servers, the tradtonal tree structure s not relable enough. Fat-tree s an mproved structure for the tradtonal tree. Every mddle node n Fat-tree has more than one father node. That means the lnks between the aggregaton swtches and core swtches are ncreased. As the network connectvty s much hgher, Fat-tree s a relable structure. However, just lke the tradtonal tree structure, t stll can not scale well. Recursve herarchy structures are hghly scalable. A recursve herarchy structure can be defned wth the recursve rule and archtecture of the smallest recursve unt. In a recursve herarchy structure, a hgh level recursve unt utlzes a low level recursve unt as a cluster and connects many such clusters by means of the recursve rule. By ncreasng the levels of structure, a large number of servers can be added to a recursve herarchy DCN wthout changng the exstng archtecture. Dell [2], FConn [9], and BCube [0] are typcal recursvely defned topologes. They use completely the same smallest recursve unt, n whch a swtch nterconnects several servers. But ther recursve rules are dfferent. DCell employs the nterconnecton relatonshp of a complete graph as recursve rule. There s a lnk between every two recursve unts of the same level. As a result, DCell gets the advantages and dsadvantages of a complete graph. In order to support hgh connectvty, each server n DCell should be equpped wth multple network ports. FConn looks very much lke DCell. It s constructed based on DCell by debasng the connectvty. The recursve unts n FConn are connected wth only a half of the dle server network ports. Each server n FConn only connects two lnks. Usng FConn as archtecture can make t easy to buld the DCN and reduce the cost. But the relablty may be reduced. BCube employs the nterconnecton relatonshp of a hypercube as recursve rule. The servers of same level and same sequence n dfferent recursve unts are nterconnected by a common swtch. Each server n BCube also should be equpped wth multple network ports. Nevertheless, BCube gets the advantages of hypercube, such as low dameter, hgh connectvty and relablty. All the topologes descrbed above are researched to mprove the structure propertes of DCN from dfferent aspects. Unfortunately, how dose they adapt to the materal data management systems or processng mechansms s gnored. 2.2 MapReduce 2

3 MapReduce ncludes the Map and Reduce two program operatons. MapReduce makes a master server control many data servers to execute concrete Map tasks and Reduce tasks. Map operaton s appled for data classfcaton and preparng ntermedate data for Reduce operaton. Reduce operaton s appled for mergng the ntermedate data accordng to defned programs and preparng data for the new Map operaton. MapReduce can process terabytes of data through many teratve Map and Reduce steps. Wth smple and practcal processng procedure, MapReduce gves a standard mechansm for dstrbuted data processng. The basc MapReduce procedure has been mproved recently for some specal applcatons. However, basc Map or Reduce operaton hasn t been gven up. The mprovements just add some pretreatment or assstant steps for Map and Reduce operatons. Reference [] adds a Merge operaton after Reduce, n order to emphasze the data mergng functon when the data have been processed by Reduce operaton. Reference [3] adds three pretreatments for Reduce, so as to facltate Reduce operaton and get more accurate result. 3. The HFN Structure In ths secton, the physcal structure of HFN s presented. Then the constructon methodology of HFN s proposed. 3. Physcal Structure HFN s recursvely defned n order to be scalable and meet the desgn goals of DCN. The nodes nterconnecton relatonshp n HFN s desgned accordng to the need of basc MapReduce procedure for DCN topology. HFN physcal structure can be presented n terms of the smallest recursve unt and recursve rule as follows. The smallest recursve unt gves the basc buldng block of the whole network topology. Dependng on the controllng mechansm between master servers and data servers, ths paper makes clear the nterconnecton relatonshp of master servers and data servers n the smallest recursve unt. Let HFN 0 (N,M) denote the smallest recursve unt. N denotes the number of master servers n HFN 0. N also denotes the number of swtches n HFN 0. M denotes the number of data servers that each of the N swtches connects. One HFN 0 (N,M) connects N master servers and N swtches by means of a bpartte graph. These N master servers and N swtches can be regard as the two vertces sets of the bpartte graph respectvely. Swtch 0 connects master server 0 and master server. Swtch n connects master server n, master server n, and master server n +, where n N 2. Swtch N connects master server N 2 and master server N. All the swtches don t connect each other drectly. Nether do the master servers. The archtecture of the smallest recursve unt looks lke that of the mddle nodes and leaf nodes n a Fat-tree. The mddle nodes of hgher level are replaced by the master servers. The leaf nodes are replaced by the data servers. The recursve rule determnes the nterconnecton mode of the recursve unts. HFN uses the constructon of BCube [0], namely the hypercube, as recursve rule. Let HFN (K,HFN 0 (N,M)) denotes the level recursve unt ( ), whch uses HFN 0 (N,M) as the smallest recursve unt. K denotes the number of swtches of level n HFN. K equals to the total number of master servers n HFN - (level recursve unt). HFN s constructed by N level recursve unts, whch are connected by the K swtches of level. Master server k n each of the N level recursve unts connects to swtch k of level, where 0 k K. From the constructon of HFN, t s easy to fnd that the number of HFN 0 n HFN - s N. There are N master servers n HFN 0. So the total number of master servers n HFN - s N. That means K = N. Fg. llustrates the nodes nterconnecton relatonshp of HFN (3,HFN 0 (3,4)). Fg.2 llustrates the nodes nterconnecton relatonshp of HFN (K,HFN 0 (N,M)). Fg.. HFN (3,HFN 0 (3,4)) 3

4 MServer,0 MServer N, Constructon Method For the constructon of level HFN, every level recursve unt should be constructed frst. In the fnal analyss, the smallest recursve unts should be constructed frst, and then hgher level HFNs can be constructed by means of the recursve rule. Let DServer m denote the data server m that connects to a swtch n HFN 0 (N,M), where 0 m< M. Let MServer,j denote the master server j n HFN, where j s the sequence of MServer,j among all the master servers n HFN. Consequently, for any master server n HFN, ts unque d can be gven by MServer,j. Let Swtch,k denote the swtch k of level n HFN, where 0 k < N. The constructon algorthm of HFN can be descrbed as the followng pseudo codes. The functons CreateHFN 0 and CreateHFN are used for creatng HFN 0 and HFN respectvely. CreateHFN 0 (nt N, nt M): f (N 0 M 0) return; for (n=0; n<n; n++) for (m=0; m<m; m++) add DServer m to Swtch 0,n ; f (n= =) add MServer 0, to Swtch 0,n ; add MServer 0,2 to Swtch 0,n ; f (<n<n) add MServer 0,n - to Swtch 0,n ; add MServer 0,n to Swtch 0,n ; add MServer 0,n+ to Swtch 0,n ; f (n= =N) add MServer 0,N - to Swtch 0,n ; add MServer 0,N to Swtch 0,n ; CreateHFN (nt, nt N, nt M): f ( 0 N 0 M 0) return; for (n=0; n<n; n++) f (>) CreateHFN (-, N, M)); else CreateHFN 0 (N, M); for (k=0; k< N ; k++) Fg.2. HFN (K,HFN 0 (N,M)) add MServer -,k to Swtch,k ; 4. MapReduce on HFN As a convenent dstrbuted data processng and cloud computng mode, MapReduce has become one of the most mportant applcaton felds of DCN. How to run MapReduce on HFN s proposed as follows. 4. Roles of the Servers for MapReduce In DCN, the servers control and execute the procedure of MapReduce. In order to dfferentate the control functon from executon functon and avod functon conflcts, ths paper dvdes the servers nto master servers and data servers. The runnng process of MapReduce on HFN can be regard as the controllng process of master servers on data servers. In the smallest recursve unt, every master server controls the data servers whch connect to the same swtches wth ths master server. Accordng to the structure of the smallest recursve unt, each data server can be controlled by at most three master servers. The roles of master servers and data servers are descrbed as follows. Master servers are used for controllng the whole procedure of MapReduce. They are responsble for recevng the user MapReduce servce requests. If the number of data servers clamed by a request beyond that controlled by the master server whch receves the request, ths master server wll separate the whole MapReduce nto many sub-mapreduce tasks. Then t wll assgn each of the sub-mapreduce tasks to a master server ncludng tself. Every master server that receves a sub-mapreduce task makes the data servers under ts control execute concrete MapReduce operatons. Fnally, every master server merges the results and sends them to the master server that receves the request. If necessary, ths master server wll start a new turn of MapReduce untl t meets the request. Data servers are used for executng the procedure of MapReduce, whch manly ncludes the Map and Reduce 4

5 operatons. A data server can be n two knds of states, namely the runnng state and dle state. No matter whch state a data server s n, t can receve Map and Reduce tasks from any one of ts master servers. All the Map or Reduce tasks are executed by rule of FCFS (frst come, frst served). Once a Map or Reduce task has been fnshed by a data server, the data server sends the number of tasks watng for executon to ts master servers. If the number equals to zero, t means that the data server s n dle state. Otherwse, the data server s n runnng state. 4.2 Routng and Assgnng Schemes for MapReduce on HFN There are three knds of routngs for MapReduce on HFN. The frst one s the routng between a data server and ts master server. The second one s the routng among data servers whch are controlled by the same master servers. The thrd one s the routng among master servers. Snce the frst and second ones can be schemed just n the smallest recursve unt, ths paper manly ntroduces the scheme of the thrd one. The routng scheme of master servers depends on the assgnng scheme for Mapreduce, whch s summarzed as ths. When a MapReduce servce needs to be assgned to many master servers, the one that receves the request wll choose the nearest one that has not been chosen n the same level as a destnaton to send a sub-mapreduce task, n terms of only one master server n one smallest recursve unt and from a low level to a hgher level. The word nearest here means the smallest number of lnks between the source and the destnaton. Let I denote the total number of levels of HFN. The followng theorems can be gven based on the constructon of HFN. Theorem : For any master server MServer I,j and the recursve unt HFN that MServer I,j belongs to, f MServer I,j also belongs to a HFN - whose sequence n HFN s n, then n can be gven by n=(j/ N )%N, where j/ N s the roundng of N dvdng j, and(j/ j/ N. N )%N s the remander of N dvdng Proof: The number of master servers n one HFN - s N. The sequence of MServer I,j n HFN I s j. Therefore, n HFN I, j/ N s the sequence of HFN - whch MServer I,j belongs to. The number of HFN - n HFN s N. Thus, n HFN, the sequence of HFN -, whch MServer I,j belongs to, equals to the remander of N dvdng j/ N. Thus proved. Theorem 2: If MServer I,j and MServer I,h belong to any gven par of adjacent HFN - s of HFN respectvely, and they connect to the same swtch of level n HFN, then the relatonshp of j and h can be gven by j-h = N, where j-h denotes the absolute value of j subtractng h. Proof: Accordng to the recursve rule, for any swtch of level and n any gven par of adjacent HFN - s of HFN, MServer I,j and MServer I,h are the only two master servers connect to ths swtch. The number of swtches of level n HFN s N. The other master servers between MServer I,j and MServer I,h connect to the other N - swtches respectvely. Therefore, the number of master servers between MServer I,j and MServer I,h s N -. That means the absolute value of j subtractng h s N. Thus proved. Theorem 3: Let L denote the number of sub-mapreduce tasks assgned to HFN. The necessary condton of can be gven by log N L. Proof: There are N + master servers n HFN, and N master servers n HFN 0. So the number of HFN 0 n HFN s N. One HFN 0 can be assgned wth just one sub-mapreduce task. Thus N L, namely log N L. The equalty holds f and only f log N L s an nteger. Suppose that MServer I,j receves a MapReduce servce request, whch needs to be assgned to L master servers. Accordng to above theorems, the routng and assgnng schemes for MapReduce on HFN can be gven by the followng pseudo codes. Functon Assgnment s used for assgnng the sub-mapreduce tasks. Functon FndServers s used for gettng the routng for MapReduce on HFN. FndedServers s an object lst, whose elements are master server objects. It records the master servers that receve sub-mapreduce tasks n order. Path s an attrbute of each master server object. It records the routng path. FndServers(nt, nt L, object MServer I,j, objectlst FndedServers) for (nt n=0; n<n; n++) f (n = = (j/ N )%N) break; f (n < (j/ N )%N) nt h=j-((j/ N )%N-n) N ; f (n > (j/ N )%N) nt h=j+(n-(j/ N )%N) N ; add MServer I,h to FndedServers; f (FndedServers.length = =L) return FndedServers; MServer I,h.Path= MServer I,j.Path; add MServer I,h to MServer I,h.Path; assgn MapReduce k to MServer I,h ; return FndedServers; Assgnment(nt j, nt L ) 5

6 nt k=0; MServer I,j.Path={MServer I,j,}; for (nt =; < nt(log N L)+; ++) objectlst ndedservers={mserver I,j,}; for (nt f=0; f<; f++) nt g= FndedServers.length; for (nt x=g-(n-) f ; x<g; x++) FndedServers=FndServers (-f, K, FndedServers [x], FndedServers); f (FndedServers.length= =L) return; 4.3 Method for MapReduce on HFN Basng on the routng and assgnng schemes descrbed n 4.2, a confused MapReduce servce wll be assgned to many master servers. Then they wll control a number of data servers executng the receved sub-mapreduce task teratvely. Each teraton ncludes Map and Reduce two steps data processng, whch are acheved n the smallest recursve unt. Map on HFN. Suppose that MServer I,h receves a sub-mapreduce task. The number of Map tasks s determned by the number of data blocks that need to be processed n an teraton. The default measure s one Map task for one data block. Frst step, MServer I,h chooses some dle or not busy data servers (named Map data servers) to assgn a Map task to each of them. Second step, Map data servers dvde the correspondng raw data nto ntermedate key/value pars by defned Map programs, and store the ntermedate data on local hard dsks. Thrd step, Map data servers feeds back the types of keys of ntermedate data to MServer I,h, then send the number of ther tasks that watng for executon to all the master servers they belong to. Reduce on HFN. The number of Reduce tasks s determned by the types of keys of ntermedate data. One Reduce task can process one or several types of key/value pars. But one type of key/value pars can only be processed by one Reduce task. Frst step, MServer I,h chooses some dle or not busy data servers (named Reduce data servers) to assgn a Reduce task to each of them. Second step, accordng to the types of keys of ther respectve Reduce task, Reduce data servers fetch the ntermedate data from the correspondng Map data servers. Thrd step, Reduce data servers merge the same type of key/value pars by defned Reduce programs. Fourth step, Reduce data servers feeds back the results of Reduce operatons to MServer I,h. They also send the number of ther tasks that watng for executon to all the master servers they belong to. The whole process of MapReduce on HFN ncludes two level teratons. Frst level teraton s the repeatng of assgnng MapReduce to many master servers and mergng the results from these master servers. Second level teraton s the repeatng of Map and Reduce operatons for each sub-mapreduce. The tmes of the two level teratons are determned by the complexty of the MapReduce servce request. A confused MapReduce may need up to a thousand of servers to run on. However, just hundreds or even tens of servers can support a confuse MapReduce on HFN by the method proposed above. 5. Fault Tolerance for MapReduce on HFN A DCN may consst of hundred thousands of servers, swtches, and lnks. The node and lnk falures are normal when MapReduce runs on DCN. Snce a lnk fault can be regard as the adjonng nodes faults, ths paper focuses on the server and swtch faults. The followng fault tolerance approaches for MapReduce on HFN not only can deal wth server and swtch faults, but also can resolve the congeston problem, so as to serve MapReduce request rapdly and effectvely. 5. Fault Tolerance Routng If a swtch out of the smallest recursve unt goes wrong, all the master servers that nterconnect wth the swtch can be regard as fault to each other. In order to avod the mpacts of server and swtch faults, fault tolerance routng should be used for MapReduce on HFN. Namely, the functon FndServers n 4.2 should be mproved as follows. Broken s a boolean varable whch s used for judgng master server fault. FndServers(nt, nt L, object MServer I,j, objectlst FndedServers) for (nt n=0; n<n; n++) f (n = = (j/ N )%N) break; f (n < (j/ N )%N) nt h=j-((j/ N )%N-n) N ; f (n > (j/ N )%N) nt h=j+(n-(j/ N )%N) N ; f (MServer I,h.broken= =True ) break; add MServer I,h to FndedServers; f (FndedServers.length = =L) return FndedServers; MServer I,h.Path= MServer I,j.Path; add MServer I,h to MServer I,h.Path; assgn MapReduce k to MServer I,h ; return FndedServers; 6

7 5.2 Server Fault Tolerance In the smallest unt of HFN, every data server connects to only one swtch, and every swtch connects to 2 or 3 master servers. Wth ths redundant structure, when a master server or data server goes wrong, the other nodes stll wll be usable. Unfortunately, the runnng MapReduce tasks wll be mpacted by server fault. Master server fault tolerance for runnng task: As soon as a master server receves a MapReduce servce request, the request wll be made backup on another adjacent master server. If the former master server goes wrong when executng ths request, the latter one wll rerun the same MapReduce. When assgnng sub-mapreduce tasks, the master server sets a threshold tme. Suppose that MServer I,h receves a sub-mapreduce task. If MServer I,h has not fed back the result untl the threshold tme, t wll be regard as fault. All the sub-mapreduce tasks that have been assgned passng by MServer I,h wll be reassgned wth the fault tolerance routng. Let Y denote the number of sub-mapreduce tasks that need to be reassgned. The algorthm of gettng these Y tasks s gven by the followng pseudo codes. FndedMapReduce s an object lst, whose elements are sub-mapreduce task objects. for (nt y=fndedservers.mserver I,h.ID; y<fndedservers.length; y++) for (each MServer n FndedServers[y].Path) f (MServer I,h = =MServer) add FndedServers[y].MapReduce u to FndedMapReduce; Data server fault tolerance for runnng task: When assgnng the Map and Reduce tasks, the master server sets a threshold tme. If a data server has not fed back ts state nformaton untl the threshold tme, ths data server wll be regard as fault. The correspondng Map or Reduce task wll be reassgned to another data server. 5.3 Swtch Fault Tolerance Each swtch n a data center connects a number of servers. So a swtch fault may mpact all the servers that nterconnect wth t. Fortunately, n HFN, when a swtch out of the smallest recursve unts goes wrong, the other nodes stll wll be usable. But the runnng MapReduce tasks whch use ths swtch wll be mpacted. The fault tolerance for runnng task n such condton s proposed as follows. A master server can not utlze the faled swtch to transmt any sub-mapreduce task result. As a result, after the defned threshold tme, ths master server wll be regard as fault. The correspondng task wll be reassgned to another master server wth fault tolerance routng. If a master server connects to the fault swtch, for ths reason the master server can not assgn most of sub-mapreduce tasks to other master servers, then the MapReduce servce request wll be executed by the adjacent master server whch stores the correspondng backup. If a swtch n the smallest recursve unt goes wrong, all the data servers connect to t wll be unusable. In ths condton, the data server fault tolerance proposed n 5.2 s avalable. 6. Evaluaton and Implementaton In ths secton, the propertes of HFN and two other typcal recursvely defned topologes are compared. The runnng effect of MapReduce on HFN s analyzed and smulated. 6. Topology Propertes The scalablty of DCN s determned by the number of servers t can accommodate. The more servers a DCN can accommodate, the more scalable t wll be. Accordng to the recursve rule and constructon of the smallest recursve unt, n HFN (K,HFN 0 (N,M)), the number of master severs s N +, the number of data servers s M N +. So the total number of servers n HFN s (M+) N +, much more than that of BCube. The topology dameter of DCN determnes the mnmum dstance between any par of servers. The shorter topology dameter means the better communcaton. If all the master servers and data servers are regarded as the same nodes, the topology dameter of HFN can be gven by ++N/2, where N/2 denotes the maxmal hop count between any par of servers n the smallest recursve unt. N/2 s the roundng of N dvdng 2. The topology dameter of BCube s +. When the value of N s not so bg, there s only small dfference between the topology dameters of HFN and BCube. 7

8 Number of Servers Dameter FConn BCube HFN FConn BCube HFN Fg.3. Dameters Fg.4. Total number of Servers Fg.3 and Fg.4 respectvely llustrate the dameters and total number of servers of several typcal recursvely defned DCN topologes when N=4 and M=8. It s showed that, wth the ncreasng of, HFN connects much more servers wth a lower dameter. Therefore, HFN has better performance advantages than the other typcal recursvely defned topologes. 6.2 Executon Tme of MapReduce on HFN Snce the users MapReduce servce requests are random, ths paper uses the queueng model to evaluate the executon tme of MapReduce on HFN. Accordng to the popular dstrbuted fle systems [3] [4], ths paper supposes that all the data has been stored n data servers. Thus, the executon tme of Map and Reduce programs wll be much longer than that of the communcaton between servers. Ths paper skms the sendng and transmttng tme of the tasks and only focuses on ther executon tme. Suppose the arrval process of Map and Reduce tasks to a data server s Posson. Let ther arrval rates are λ and λ 2 respectvely. Suppose the executon tme for a Map or Reduce task s negatve exponentally dstrbuted. Let the average executon tme of a Map task s and the one of a μ Reduce task s μ. 2 Theorem 4: Let t denote the average executon tme of one sub-mapreduce teraton. Then t s gven by t = + () μ λ μ2 λ2 Proof: Accordng to the queueng theory, the average executon tme for a Map task s. All the Map tasks n μ λ a Map step are parallel, so the average total executon tme of a Map step also s. Smlarly, the average total executon μ λ tme of a Reduce step s. One teraton for a μ2 λ2 sub-mapreduce ncludes Map and Reduce two steps. Thus proved. Actually, the executon tme of a Map step s equal to that of the longest Map task, so does a Reduce step. As a result, the executon tme of one teraton for a sub-mapreduce may be a lttle longer than average. However, when there s lttle dfference between the data servers, ths error does not affect the evaluaton. Let V denote the average total number of data servers needed for a MapReduce servce request. Let R denote the average total number of repeatng tmes of one sub-mapreduce teraton. Let C denote the average total number of repeatng tmes of frst level teraton. Let a and b denote the number of Map tasks and Reduce tasks n a sub-mapreduce teraton respectvely. Theorem 5: If V s bg enough that each smallest recursve unt can get a correspondng sub-mapreduce task, then R s gven by V R= (2) C ( a+ b) N Proof: The number of smallest recursve unts n HFN s N. The number of data servers n the smallest recursve unt needed for one sub-mapreduce teraton s a+b. So R C (a+b) N equals to the total number of data servers needed for achevng a MapReduce servce request. Namely, R C (a+b) N equals V. Thus proved. Accordng to the constructon of the smallest recursve unt, a master can control 3M data servers at most. Suppose a+b =3M, then R s gven by V R= (3) 3 M N C If R s less than, t means that not every smallest recursve 8

9 unt s needed for the MapReduce servce request. But the equatons (2) and (3) are stll avalable at ths condton. Theorem 6: Let T denote the average total executon tme of a MapReduce servce request, then T s gven by T=R t (4) Proof: Accordng to Theorem 4 and 5, the average executon tme of a sub-mapreduce task s R t. Snce all the sub-mapreduce tasks are parallel, the average total executon tme of a MapReduce servce request also s R t. If equaton () and (3) are substtuted nto equaton (4), then T s gven by V T= ( + ) (5) μ λ μ λ 3M N C Fault Tolerance Evaluaton Let P M, P D, and P S denote the probabltes of master server fault, data server fault, and swtch fault respectvely. Theorem 7: Let P denote the probablty of a master server keepng on work, then P s gven by PM = 0 p = n ( PM) ( ( PS )) > 0 n= n Proof: When =0, master servers only connect to the swtches n the smallest recursve unts, whose falures do not mpact the master servers. So the probablty of a master server keepng on work equals to the probablty of t not gong n wrong, namely -P M. When >0, PS s the probablty of n n swtches faults of swtches whch are out of smallest recursve unts and connect to a master server. Then the probablty of these swtches keepng on work s gven n by ( PS ). Only when the master server and all these n= n swtches do not get faled, the master server can keep on work. Therefore, the probablty of a master server keepng on n work s gven by ( PM) ( ( PS )). Thus proved. n= n Theorem 8: Let P 2 denote the probablty of a data server keepng on work, then P 2 s gven by P 2 = (-P D ) (-P S ) (7) Proof: -P D and -P S are the probabltes of a data server and a swtch do not go wrong respectvely. A data server only connects to one swtch. Only when the data server and the swtch both do not break down, ths data server can keep on work. So the probablty of a data server keepng on work s (6) gven by (-P D ) (-P S ). Theorem 9: Let T ' denote the average total executon tme of a MapReduce servce request, when node faults are took nto consderaton. Then T ' s gven by V T ' = ( + ) μ λ μ λ (3 M P) ( N P) C Proof: Accordng to the assgnng scheme proposed n 4.2, and Theorem 7 and 8, for a MapReduce servce request, the number of smallest recursve unts that can be used s gven N by P, namely N P. The number of data servers that can x+ y be used n one smallest recursve unt s gven by P2. Let x+ y x+y=3m, then P2 = 3M P2. If N P and 3M P2 are substtuted nto equaton (5), then ths theorem s proved. 6.4 Implementaton For the smulaton of mplementaton, a test bed s bult, so as to get average executon tme of dfferent MapReduce operatons runnng on HFN. The test bed s composed of 4 Dell PowerEdge 2900 servers as master servers and 6 Dell Vostro 420 commodty computers as data servers. These servers are nterconnected by 6 8-port Lnksys EtherFast Gable/DSL mn swtches n terms of the structure of HFN (2,HFN 0 (2,4)). Hadoop program framework s utlzed n the experment to run MapReduce operatons for Word Count [7], Strng Match [3], and Graph Smplfy [4]. Each of these three types of MapReduce operatons s mplemented 30 tmes. Two counters are set n every data server. They respectvely record the numbers of Map tasks and Reduce tasks that a data server mplements durng a whole MapReduce process. Accordng to defntons n 6.2, μ and μ 2 respectvely denote the average number of Map tasks and Reduce tasks a data server mplements n unt tme. Let μuv denote the counter for Map tasks, where u denotes the number of tmes a MapReduce s repeated, and v denotes the sequence of a data server. Then μ can be gven by the followng pseudo codes, and μ2 can be gven by smlar pseudo codes, where ExecutonTme u denotes the total executon tme of the u th tme a MapReduce s mplemented. for (nt u=0; u<30; u ++) ntμ u = μ v =0; for (nt v=0; v<6; v++) μ = μ + μ ; v v uv (8) 9

10 μ u = μ v /ExecutonTme u /6; μ = μ + μ u ; return μ = μ /30; For the three knds of MapReduce operatons n the experment, when second s used as tme unt, ther respectve μ are 5.8, 4.7, and 6.7, and ther respectve μ 2 are 5., 5.5, and 6.0. Suppose μ and μ 2 do not change when V=4500, N=4, M=8, λ =3, λ 2 =3, P M =0.0, P D =0.02, P S =0.0, C=3 and second s used as tme unt. The varaton of T ' along wth the ncreasng of can be llustrated by Fg.5. T' Strng Match Word Count Graph Smplfy Fg.5. Changes of T ' The total number of servers n HFN s 2304 when =3. However, snce n each HFN 0 there s only one master server and ts correspondng data servers provdng servces for a MapReduce operaton, the actual number of severs n HFN runnng one MapReduce s 536 when =3 n the smulaton. The MapReduce orgnally needs 4500 server to execute. On ths condton, for the three types of MapReduce operatons, ther T ' can decrease to less than one second. So HFN can acheve a great deal of work wth fewer servers and mplement multple MapReduce operatons at one tme. From the smulaton, t s easy to fnd that even wth node faults and low recursve level, HFN s competent for MapReduce. Reference [] Albert Greenberg, James R. Hamlton, Navendu Jan, Srkanth Kandula, Changhoon Km, Parantap Lahr, Davd A. Maltz, Parveen Patel, and Sudpta Sengupta. VL2: A Scalable and Flexble Data Center Network. In ACM SIGCOMM, [2] C. Guo, H. Wu, K. Tan, L. Sh, Y. Zhang, and S. Lu. DCell: A Scalable and Fault-Tolerant Network Structure for Data Centers. In ACM SIGCOMM, [3] Sanjay Ghemawat, Howard Goboff, and Shun-Tak Leung. The Google Fle System. In ACM SOSP, [4] D. Borthakur. The Hadoop Dstrbuted Fle System: Archtecture and Desgn. [Onlne]. Avalable: [5] F. Chang, J. Dean, S. Ghemawat, W. C. Hseh, D. A. Wallach, M. Burrows, T. Chandra, A. Fkes, and R. E. Gruber. Bgtable: A Dstrbuted Storage System for Structured Data. ACM Transactons on Compute Systems, vol. 26, no. 2, [6] CloudStore. Hgher Performance Scalable Storage. [Onlne]. Avalable: [7] Jeffrey Dean and Sanjay Ghemawat. MapReduce: Smplfed Data Processng on Large Clusters. Communcatons of the ACM, vol. 5, no., [8] M. A. Fares, A. Loukssas, and A. Vahdat. A Scalable, Commodty Data Center Network Archtecture. In ACM SIGCOMM, [9] D. L, C. Guo, H. Wu, Y. Zhang, and S. Lu. FConn: Usng Backup Port for Server Interconnecton n Data Centers. In IEEE INFOCOM, [0] C. Guo, G. Lu, D. L, H. Wu, X. Zhang, Y. Sh, C. Tan, Y. Zhang, and S. Lu. BCube: A Hgh Performance, Server-centrc Network Archtecture for Modular Data Centers. In ACM SIGCOMM, [] Thomas Sandholm, and Kevn La. MapReduce Optmzaton Usng Regulated Dynamc Prortzaton. In ACM SIGMETRICS, [2] Marc de Krujf, and Karthkeyan Sankaralngam. MapReduce for the Cell B.E. Archtecture. IBM Journal of Research and Development, 53(5), [3] Colby Ranger, Ramanan Raghuraman, Arun Penmetsa, Gary Bradsk, and Chrstos Kozyraks. Evaluatng MapReduce for Mult-core and Multprocessor Systems. In IEEE HPCA, 2007 [4] Jonathan Cohen. Graph Twddlng n a MapReduce world. Computng n Scence and Engneerng. IEEE Educatonal Actvtes Department, vol. 2, no. 4, Concluson Ths paper presents a MapReduce-supported DCN topology named Hyper-Fat-tree Network (HFN). By the way of compare, analyss and smulaton, t s proved that HFN s a relable and scalable topology wth hghly connectvty and low dameter. It s also proved that HFN s competent for MapReduce even wth node faults. The purpose of ths paper s to make the DCN topology be suted for dstrbuted data processng mechansms and make the avalablty of desgns of DCN topology be mproved. Next work s researchng the mplementaton of dstrbuted fle systems on HFN. 0