Closed form Solution for Scheduling Arbitrarily Divisible Load Model in Data Grid Applications: Multiple Sources

Transcription

1 Amercan Journal of Appled Scences 6 (4): , 2009 ISSN Scence Publcatons Closed form Soluton for Schedulng Arbtrarly Dvsble Load odel n Data Grd Applcatons: ultple Sources onr Abdullah, ohamed Othman, Hamdah Ibrahm and Shamala Subramanam Department of Communcaton echnology and Netork, Unversty Putra alaysa, UP Serdang, Selangor DE, alaysa Abstract: Schedulng an applcaton n data grd as sgnfcantly comple and very challengng because of ts heterogeneous n nature of the grd system. When the Dvsble Load heory (DL) model had emerged as a poerful model for modelng data-ntensve grd problem, ask Data Present (DP) model as proposed based on t. hs study presented a ne Adaptve DP (ADP) for schedulng the ntensve grd applcatons. Ne closed form soluton for obtanng the load allocaton as derved hle computaton speeds and communcaton lnks are heterogeneous. Epermental results shoed that the proposed model can balance the load effcently. Key ords: Data grd, dvsble load theory INRODUCION Grd computng apples the resources of many computers n a netork to a sngle problem at a gven tme-usually to a scentfc or computatonal problem that requres a greater number of CPU cycles or access to large amounts of data []. In data grd envronment, many large scale scentfc eperments and smulatons generate very large amounts of data n the dstrbuted storages, spannng thousands of fles and data sets [2]. Due to the heterogeneous n nature of the grd system, schedulng an applcatons n such envronment ether data- or communcaton ntensves s sgnfcantly comple and challengng. Grd schedulng s defned as the process of makng schedulng decson nvolvng allocatng ob to resources over multple admnstratve domans [3]. A decoupled schedulng archtecture for data ntensve applcatons s also proposed [7]. In ths research, they proposed ask Data Present (DP) model. he results sho that hen the ob s scheduled to a ste here the data s avalable the data transfer s mnmal but the response tme suffers hen there s no data replcaton. hs s because a fe stes hch host the data are overloaded n ths case and hence, makng a case for dynamc replcaton of data. Recently, DL model has emerged as a poerful model for modelng data-ntensve grd problem [4]. DL eplots the parallelsm of a dvsble applcaton hch s contnuously dvsble nto parts of arbtrary sze, by schedulng the loads n a sngle source onto multple computng resources. he load schedulng n data Grd s addressed usng DL model th addtonal constrant that each orker node receves the same load fracton from each data source [5]. Hoever, most of the prevous models do not take nto account the communcaton tme. Whereas, n order to acheve a hgh performance, e must consder both communcaton and computaton tme [6]. In addton, a load balancng algorthm s also developed on a structure of data of netork type WAN, hch guarantees ts portablty on any grd computng. he dstrbuton of loads assures the convergence of the algorthm n an acceptable tme [9]. In ths research, the communcaton tme s not consdered. DP model as eamned th other strateges for non-dvsble applcatons [7,0]. It as modfed to be able to schedule dvsble load applcatons [8]. hs strategy maps tasks only to the stes here the requred data s present. Each task processes the data sets resdng at that ste. here s no nput data transfer n ths case. hey consdered the communcaton tme but not n dvdng the load. Instead, they dvded the load usng DL model then added the communcaton tme to the makespan. Our prevous research n ths area s the Adaptve DL (ADL) for schedulng the arbtrarly dvsble load n data grd applcaton [2]. In ths research, the communcaton tme and communcaton tme are consdered ontly. Correspondng Author: onr Abdullah, ohamed Othman, Department of Communcaton echnology and Netork, Unversty Putra alaysa, UP Serdang, Selangor, alaysa el: Fa:

2 In ths study, a ne Adaptve DP model s proposed as an mprovement of DP model. he obectve s to fnd the optmal allocaton of orkload of processng nodes by the eamnaton of the effect of non-neglgble communcaton delay as ell as computaton tme and the netork confguratons. he desgn of our proposed model adopts the dvsble load paradgm, referred to as the Dvsble Load heory (DL), hch s shon to be effcent n handlng large volume loads. Schedulng model: A generc data grd computng system nfrastructure consdered here comprses a netork of supercomputers and/or clusters of computers connected by Wde Area Netork (WAN), havng dfferent computatonal and communcaton capabltes. We consder the problem of schedulng large-volume loads (dvsble loads) thn n multple stes. Communcaton s assumed to be predomnant beteen such cluster nodes and s assumed to be neglgble thn a cluster node [5,8,]. he target data ntensve applcaton model can be decomposed nto multple ndependent subtasks and eecuted n parallel across multple stes thout any nteracton among sub tasks. For eample, let s consder ob decomposton by decomposng nput data obects nto multple smaller data obects of arbtrary sze and processng them on multple vrtual stes. Hgh Energy Physc (HEP) obs are arbtrarly dvsble at event granularty and ntermedate data product processng granularty [2]. In ths research, assumng that a ob requres a very large logcal nput data set (D) conssts of N physcal datasets and each physcal dataset (of sze L k ) resdes at a data source (DS k, for all k =,2, N) of a partcular ste. Fgure shos ho the logcal nput data (D) s decomposed onto netorks and ther computng resources. he schedulng problem s to decompose D nto datasets (D for all =, 2,..., ) across N vrtual stes n a Vrtual Organzaton (VO) gven ts ntal physcal decomposton. We assume that the decomposed data can be analyzed on any ste. For the notatons, defntons that used n ths research are stated n able. he eecuton tme of a subtask allocated to the ste ( ) and the turn around tme of a ob J (turn_around_tme) can be epressed as follos: = nput _ cm() + cp() + output _ cm(,d) turnaround _ tme = ma{} =,..., he cost ( ) ncludes nput data transfer ( nput_cm ()), computaton ( cp ()) and output data transfer to the clent at the destnaton ste d ( output_cm (,d)): nput _ cm() = ma{ αk } k=..m z cp() = d output _ cm(,d) = f(d ) zd k We assume that data from multple data sources can be transferred to a ste concurrently n the de area envronment and computaton starts only after the assgned data set s totally transferred to the ste. Hence, the problem of schedulng a dvsble ob onto n stes can be stated as decdng the porton of orgnal orkload (D) to be allocated to each ste, that s, fndng a dstrbuton of dstrbuton of { αk} hch mnmzes the turn-around tme of a ob. he proposed SA approach uses ths cost model hen evaluatng solutons at each generaton. DP schedulng model: Frstly, DP model as proposed for schedulng ndvsble load [7]. Consequently, t s modfed to be ork on schedulng dvsble load applcatons [7]. hs strategy maps tasks only to the stes here the requred data s present. Each task processes the data sets resdng at that ste. Fg. : Data decomposton and ther processng 627 able : ermnology, defntons and notatons N he total number of data fles n the system he total number of nodes n the system L he loads n data fle L he loads that node ll receve from data fle N L he sum of loads n the system, here L = L = α he fracton of L that node ll receve from all data fle he nverse of the computng speed of node Z he lnk beteen node and data source () he processng tme n node

3 Fg. 2: Frameork of DP model here s no nput data transfer n ths case. In the begnnng, they calculate the processng tme usng DL model that as proposed n [4]. hen, the transfer output tme s added to calculate the makespan. he equaton of calculatng the load fracton s: = () Fg. 3: Frameork of ADP model In the proposed model, e ll consder both-the communcaton tme fracton as ell as the computaton tme fracton. hus, the Combnaton Fracton () ll be as: z = + = = z (5) So the amount of load that ste gves ste s calculated as: α, = L (2) = o understand ho the DP model orks, the frameork s shon n Fg. 2. Proposed ADP schedulng model: In the prevous DP model, the loads ere dvded by usng the DL model and fnally the makespan as calculated. In the proposed model, e try to balance the loads by consderng communcaton tme. In other ord, the node speed fracton as calculated together th the communcaton tme fracton as follos. he communcaton tme fracton s added nto the ADP model. he fracton of load f e consder the computaton tme only s: = (3) and f e consder the communcaton tme only (tme for transferrng the processed load to the output node), the load fracton ll be as: z = z (4) 628 α = (6) N = = And the closed form soluton s: C α = L N (7) C = = o be clear the frameork of the ADP model s shon n Fg. 3. RESULS AND DISCUSSION o measure the performance of the proposed SAbased approach aganst CDL and GA approaches, randomly generated epermental confguratons ere used. We made the smulaton program usng C++ language. he estmated epected eecuton tme for processng a unt dataset on each ste, the netork banddth beteen stes, nput data sze and the rato of output data sze to nput data sze ere randomly generated th unform probablty over some predefned ranges. he netork banddth beteen stes s unformly dstrbuted beteen bps and 0 bps. he locaton of m data sources DS k s randomly selected and each physcal dataset sze L k s randomly selected th a unform dstrbuton n the range of GB to B. It s assumed that the computng tme spent n a ste to process a unt dataset of sze B s unformly dstrbuted n the range /r cb to 0/r cb seconds, here r cb s the rato of computaton speed to communcaton speed.

4 6.4E E+06 DP ADP 3.0E+06 DP.6E E E E+06.5E+06.0E+06 ADP.0E ccrato Fg. 4: akespan for the DP and ADP models able 2: Percentage makespan mprovements of ADP model aganst DP model th dfferent ccrato ccrato DP (%) Average 20 We eamned the overall performance of each model by runnng them under 00 randomly generated Grd confguratons. hese parameters are vared: ccrato ( ), (20-00), N (20-00), r cb (0-500) and data fle sze ( GB- B). When the number of nodes and the number of data fles are both 00, the results are collected and shon n Fg. 4. he results shoed that the makespan of the proposed model s better than the prevous model, especally hen the ccrato s less than (communcaton-ntensve applcatons). In summary, the model balances the load among the nodes more effcently. he percentage makespan mprovements of ADP model aganst DP model are clearly demonstrated n able 2. From able 2, t as found that ADP model s 20% better than DP model n terms of makespan. hese results shoed that ADP s the best model hch means by applyng ths model ll balance the load effcently for communcaton ntensve applcaton. When e compare the ADP model to the DP model th dfferent sze of data fles, the ADP model produces better result as compared to DP model. he performance of ADP model s mproved hen the sze of data fle s ncreased. he result s shon n Fg E E Data fle sze (GB) Fg. 5: akespan vs. data fle sze for the DP and DP models 5.0E E E E E E E+05.5E+05.0E E E+00 DP ADP No. of Node Fg. 6: akespan vs. no. of node ADP and DP models (N = 00, = 00 and ccrato = 0.00) able 3: Percentage makespan mprovements of ADP model aganst DP model th dfferent data fle sze ccrato DP (%) Average 4 Furthermore, the graphs are also plotted for makespan aganst number of processng node as n Fg. 6. Fgure 6, shos that ADL model s better than DP model hen the number of processng nodes s less than 50. When the number of the processng nodes becomes 50 and above, the ADP s produce better results. In average, the ADP model s better than DP model by 4%, able

5 able 3 shos that, ADP model produce n average better results than DP model. It s also clear that the proposed the ADP model produces better results hen the number of processng node s ncreased. CONCLUSION In ths study, an mprovement verson of DP model called ADP model proposed. he ADP model reduces the makespan and balances the load more effcently than the DP model. he eperment results shoed that ADP model mproved th an average of 20% of the makespan compared to DP model. Wth such mprovement, the proposed model can be ntegrated n the estng data grd schedulers n order to mprove the performance. REFERENCES. Rchard,R.J.A., A.J. Aay and C. Esaran, Implementaton of computatonal grd servces n enterprse grd envronments. Am. J. Appled Sc., 5: Jaechun, N. and P. Hyoungoo, GEDAS: A data management system for data grd envronments. Lecture Notes Comput. Sc., 354: b86/. 3. Venugopal, S., R. Buyya and K. Ramamohanarao, A taonomy of data grds for dstrbuted data sharng, management and processng. AC Comput.Surv.,38: Robertazz,.G., en reasons to use dvsble load theory. IEEE Comput., 36: DOI: 0.09/C Wong, H.., B. Veeravall, Y. Dantong and.g. Robertazz, Data ntensve grd schedulng: ultple sources th capacty constrants. Proceedng of the IASED Conference on Parallel and Dstrbuted Computng and Systems, Nov. 3-5, arna del Rey, USA, pp:7-. =3753&reason= Bharada, V., D. Ghose and.g. Robertazz, Dvsble load theory: A ne paradgm for load schedulng n dstrbuted systems. Clust. Comput.,6: 7-7. DOI: 0.023/A: Ranganathan, K. and I. Foster, Decouplng computaton and data schedulng n dstrbuted data-ntensve applcatons. Proceedng of the th IEEE Internatonal Symposum on Hgh Performance Dstrbuted Computng, July 24-26, IEEE Computer Socety Washngton, DC., USA., pp: Km, S. and J. B. Wessman, A genetc algorthm based approach for schedulng decomposable data grd applcatons. Proceedng of the Internatonal Conference on Parallel Processng, Aug. 5-8, IEEE Computer Socety Press, Washngton DC., USA., pp: DOI: 0.09/ICPP Boukerram, A. and S.A.K. Azzou, Implementaton of load balancng algorthm n a grd computng. Am. J. Appled Sc., 3: akefusa, A., O. atebe, S. atsuoka and Y. orta, Performance analyss of schedulng and replcaton algorthms on grd datafarm archtecture for hgh energy physcs applcatons. Proceedngs on the 2th IEEE Internatonal Symposum on Hgh Performance Dstrbuted Computng, June 22-24, IEEE Computer Socety Press, Washngton DC., USA., pp: Vsanathan, S., B. Veeravall and.g. Robertazz, Resource-aare dstrbuted schedulng strateges for large-scale computatonal cluster/grd systems. IEEE rans. Parall. Dstrbut. Syst.,8: DOI: 0.09/PDS Othman,.,. Abdullah, H. Ibrahm and S. Subramanam, Adaptve dvsble load model for schedulng data-ntensve grd applcatons: Computatonal scence. Lecture Notes Comput. Sc., 4487: DOI: 0.007/ _