Performance Study of Parallel Programming on Cloud Computing Environments Using MapReduce

Transcription

1 Performance Study of Parallel Programmng on Cloud Computng Envronments Usng MapReduce Wen-Chung Shh, Shan-Shyong Tseng Department of Informaton Scence and Applcatons Asa Unversty Tachung, 41354, Tawan {wjshh, Chao-Tung Yang* Department of Computer Scence and Informaton Engneerng, Tungha Unversty Tachung, 40704, Tawan Abstract Dvsble load applcatons have such a rch source of parallelsm that ther parallelzaton can sgnfcantly reduce ther total completon tme on cloud computng envronments. However, t s a challenge for cloud users, probably scentsts and engneers, to develop ther applcatons whch can explot the computng power of the cloud. Usng MapReduce, novce cloud programmers can easly develop a hgh performance cloud applcaton. To examne the performance of programs developed by ths approach, we apply ths pattern to mplement three knds of applcatons and conduct experments on our cloud test-bed. Expermental results show that MapReduce programmng s sutable for regular workload applcatons. Keywords- hgh performance computng; parallel programmng; cloud computng; MapReduce; Hadoop I. INTRODUCTION In recent years, ntegraton of nexpensve commodty computers, such as computng clusters and grds, have become promsng alternatves to tradtonal multprocessors [1, 2]. Among these, cloud computng has emerged as the nextgeneraton hgh performance computng platform. Bascally, cloud platforms are dstrbuted systems whch share resources through the form of nternet servces. On the one hand, users can access more computng resources through cloud technologes wthout knowng low-level detals. On the other hand, cloud envronments requre effectve management to operate n an effcent way. Moreover, the heterogenety and dynamc changng of the cloud envronment make t dfferent from conventonal parallel and dstrbuted computng systems, such as multprocessors and computng clusters. Therefore, t s a challenge to utlze the cloud effcently. Applcatons wth dvsble loads are a rch source of parallelsm [30]. Programmers can dentfy ndependent work unts wthn a program and dspatch them to dfferent processors to reduce ts completon tme. Nowadays, parallelzng a program for cloud platforms manly depends on human efforts. Automatc transformaton of parallel applcatons nto grd-aware ones was nvestgated n [3-5], but ther approach s not sutable for a novce programmer to develop parallel applcatons from scratch. Furthermore, t s dffcult for programmers to acqure real-tme cloud status nformaton and to approprately dstrbute workload wthn a program to heterogeneous workng nodes. Our dea s to provde programmers wth a parallel programmng pattern, whch takes care of detals related to cloud nfrastructure. All the programmers need to do s to fll n the pattern algorthm wth applcaton-specfc code fragments. The resultng program can approprately dstrbute the workload of the program to workng nodes accordng to dynamc node performance. That s, we propose a performance-based pattern algorthm, whch serves as a template for programmers to develop a parallel program on cloud platforms. To verfy ths approach, we have mplemented ths pattern usng Hadoop MapReduce [31, 32] and appled ths pattern to three types of applcatons, Matrx Multplcaton, Assocaton Rule Mnng and Mandelbrot Set Computaton. Expermental results on a cloud test-bed show that programs developed by ths approach can explot the computng power of the cloud. The prmary advantage of ths approach s that a programmer can easly develop hgh performance programs to execute on cloud envronments. The hgh performance results from two features of ths pattern. Frst, t s a hybrd method. In ts frst phase, workload s dstrbuted statcally accordng to node performance to reduce schedulng overhead. In the second phase, the remanng load s dspatched dynamcally to acheve load balance. Second, t utlzes real-tme nformaton to estmate the performance of the cloud. The pattern acqures cloud status nformaton from a montorng tool and estmates the performance of computng and communcaton resources wth the nformaton. Our contrbutons can be summarzed as follows. Frst, ths paper proposes a performance-based pattern for programmers to develop hgh-qualty parallel applcatons wth ease. Programs developed by ths approach can utlze cloud nformaton to adaptvely dstrbute workloads wthn a program. Second, we apply ths pattern to three knds of dvsble load applcatons on our cloud test-bed. Consequently, expermental results show the obvous effectveness of our approach. Note that ths work ams at a general pattern of workload dstrbuton, nstead of proposng a new loop schedulng scheme or a novel data mnng algorthm. II. RELATED WORK In ths secton, the theory of dvsble load s brefly revewed. Then, we present some well-known loop schedulng *Correspondng author /10/$ IEEE

2 schemes. A. Dvsble Load Theory Dvsble Load Theory (DLT) addresses the case where the total workload can be parttoned nto any number of ndependent sub-jobs. In the past, the theory of dvsble load has been wdely nvestgated n statc heterogeneous systems. However, t has not been wdely appled to computng cloud platforms, whch are characterzed by heterogeneous resources and dynamc envronments. Ths problem has been dscussed n the past decade, and a good revew can be found n [6]. In [7, 8], an exact method for dvsble load was proposed, whch was not from a dynamc and pragmatc vewpont as ours. DLT focuses on coarse-gran loads, whch are a pool of jobs or programs. However, the target of ths work s fne-gran loads, whch mght be loop teratons wthn a program, for example. We focus on the problem of parallelzng an applcaton wth dvsble loads for rapd executon on cloud envronments. Snce cloud envronments are dynamcally changng and heterogeneous, the problem s obvously dfferent from the tradtonal DLT problem. B. Loop Schedulng Schemes Conventonally, loop schedulng schemes are classfed accordng to the tme when the schedulng decson s made. Statc loop schedulng schemes make a schedulng decson at comple tme, and equally assgn the total teratons of a loop to processors. It s appled when each teraton of a loop takes roughly the same amount of tme, and the compler knows enough related nformaton before complaton. Its advantage s less overhead at runtme, whle the dsadvantage s possble load mbalance. Well-known statc schedulng schemes nclude Block Schedulng, Cyclc Schedulng, Block-D Schedulng, Cyclc-D Schedulng, etc. However, these schemes are not sutable for dynamc grd envronments. Dynamc loop schedulng schemes make a schedulng decson at runtme. Its dsadvantage s more overhead at runtme, whle the advantage s load balance. Several selfschedulng schemes are restated here as follows. Pure Self-schedulng (PSS) Ths s a straghtforward dynamc loop schedulng algorthm [9]. Whenever a processor becomes dle, a loop teraton s assgned to t. Ths algorthm acheves good load balance but also nduces excessve overhead. Chunk Self-schedulng (CSS) Instead of assgnng one teraton to an dle processor at one tme, CSS assgns k teratons each tme, where k, called the chunk sze, s a constant. When the chunk sze s one, ths scheme s PSS, as dscussed above. If the chunk sze s set to the bound of the parallel loop equally dvded by the number of processors, ths scheme becomes statc schedulng. A large chunk sze wll cause load mbalance whle a small chunk sze s lkely to result n too much runtme overhead. Guded Self-schedulng (GSS) Ths scheme can dynamcally change the number of teratons assgned to each processor [10]. More specfcally, the next chunk sze s determned by dvdng the number of remanng teratons of a parallel loop by the number of avalable processors. The property of decreasng chunk sze mples an effort s made to acheve load balance and to reduce the runtme overhead. By assgnng large chunks at the begnnng of a parallel loop, one can reduce the frequency of communcaton between the master and slaves. Factorng Self-schedulng (FSS) In some cases, GSS mght assgn too much work to the frst few processors, so that the remanng teratons are not tme-consumng enough to balance the workload. The Factorng algorthm addresses ths problem [11]. The assgnment of loop teratons to workng processors proceeds n phases. Durng each phase, only a subset of the remanng loop teratons (usually half) s dvded equally among the avalable processors. Therefore, t balances loads better than GSS does when the computaton tmes of loop teratons vary substantally. In addton, the synchronzaton overhead of Factorng s not sgnfcantly larger than that of GSS. Trapezod Self-schedulng (TSS) Ths approach tres to reduce the need for synchronzaton whle stll mantanng a reasonable load balance [12]. TSS(N s, N f ) assgns the frst N s teratons of a loop to the processor startng the loop and the last N f teratons to the processor performng the last fetch, where N s and N f are both specfed by the programmer or the system. Ths algorthm allocates large chunks of teratons to the frst few processors and successvely smaller chunks to the last few processors. Tzen and N proposed TSS(N/2p, 1) as a general selecton. In [13], the authors enhanced well-known loop selfschedulng schemes to ft an extremely heterogeneous PC cluster envronment. A two-phased approach was proposed to partton loop teratons and t acheved good performance n heterogeneous test-beds. For example, GSS can be enhanced by parttonng percent of the total teratons accordng to ther performance weghted by CPU clock n the frst phase. Then, the remander of the workload s stll scheduled by GSS. Ths enhanced scheme s called NGSS. In [14], NGSS was further enhanced by dynamcally adjustng the parameter accordng to system heterogenety. A performance benchmark was used to determne whether target systems are relatvely homogeneous or relatvely heterogeneous. In addton, the types of loop teratons were classfed nto four classes, and were analyzed respectvely. The scheme enhanced from GSS s called ANGSS. Our prevous work [15, 16] presents dfferent heurstcs to the parallel loop self-schedulng problem. Ths paper extends the dea of performance-based schedulng to desgn a performance-based skeleton for developng hgh performance applcatons on grds. Ths approach s appled to both the parallel loop self-schedulng applcaton and the assocaton rule mnng applcaton. In [30], the dea of performancebased schedulng was extended to desgn a performance-based skeleton for developng hgh performance applcatons on grd platforms. Ths work dffers from the above-mentoned researches n that the computng envronment s focused on cloud platforms.

3 III. APPROACH In ths secton, the MapReduce model s ntroduced frst. Then, the concepts of performance rato and statc-workload rato are revewed. Fnally, we present the programmng pattern for the performance-based cloud computng. A. MapReduce Model The MapReduce programmng model can be used to process large-scale data sets n cloud envronments. It conssts of three types of nodes: Master, Mappers and Reducer, as shown n Fgure 1. The Master dspatches sub-jobs to a set of Mappers. After these Mappers complete ther assgned jobs, the results are merged by the Reducer. Fgure 1. Overvew of the MapReduce model Hadoop [31] mplements the MapReduce model, whch provdes a hgh-level vew to parallel programmers. Wth the help of Hadoop, programmers can focus on hgh-level operatons. In the MapReduce model, the Mappers get subjobs n the form of <key, value>. Takng a large data set as example, a sub-job can be represented by < key, value >, where key s the flename of the data subset, and value s the content of the data subset. B. Performance Rato The concept of performance rato was prevously defned n [15, 16] n dfferent forms and parameters, accordng to the requrements of applcatons. In ths work, the pattern algorthm uses a performance functon to model the heterogeneous performance of the dynamc cloud nodes. The purpose of calculatng performance rato s to estmate the current processng capablty for each node. Wth ths metrc, the program can dstrbute approprate workloads to each node, and load balance can be acheved. The more accurate the estmaton s, the better the load balance s. Assume that m s the number of attrbutes. For example, ths study adopts three attrbutes: CPU speed, CPU loadng, and Bandwdth. Therefore, m s equal to 3. To estmate the performance of each slave node, a performance functon (PF) s defned for a slave node j, as shown n (1): PF j (V 1, V 2,, V m ) (1) where V, 1< <m, s a varable of the performance functon. In more detal, the varables could nclude CPU speed, networkng bandwdth, memory sze, etc. We propose to utlze a Cloud resource montorng tool, Hadoop, to acqure the values of attrbutes for all slaves. The PF for node j s defned as (2). PF j = CS j CL j CS CL node N where N s the set of all avalable cloud nodes. CS s the CPU clock speed of node, and t s a constant attrbute. The value of ths parameter s acqured by the montorng tool. CL s the CPU loadng of node, and t s a varable attrbute. The value of ths parameter s acqured by the montorng tool. The performance rato (PR) s defned to be the rato of all performance functons. For nstance, assume the PF values of three nodes are 1/2, 1/3 and 1/4. Then, the PR s 1/2:1/3:1/4;.e., the PR of the three nodes s 6:4:3. In other words, f there are 13 loop teratons, 6 teratons wll be assgned to the frst node, 4 teratons wll be assgned to the second node, and 3 teratons wll be assgned to the last one. C. Determnaton of Statc-Workload Rato (SWR) Another mportant factor to be estmated s the varaton degree among all unts of workloads. The concept of Statc- Workload Rato (SWR) was prevously defned n [30]. For example, Mandelbrot Set Computaton s a problem nvolvng rregular workloads. In each teraton of a loop, the workload s dfferent and vares sgnfcantly, as shown n Fgure 2. Obvously, a dstrbuton scheme whch does not consder the effect of rregular workload could not estmate PR accurately. No. of Iteratons of Y th teraton of X Fgure 2. The Mandelbrot Set on [-1.8, 0.5] to [-1.2, 1.2] We propose to use a parameter, SWR (Statc-Workload Rato), rangng from 0 to 1, to estmate the proporton of the workload whch can be statcally scheduled, allevatng the effect of rregular workload. In order to take advantage of statc schedulng, the SWR proporton of the total workload s dspatched accordng to Performance Rato. The desgn ratonale s based on a conservatve heurstc to estmate the rregular degree of workloads among all teratons. If the workload of the target applcaton s regular, SWR can be set to be 1. However, f the applcaton has rregular workload, such as Mandelbrot Set Computaton, t s reasonable to (2)

4 reserve some amount of workload for load balancng. We propose to randomly take fve samplng teratons, and compute ther executon tme. Then, the SWR value for the target applcaton s determned by (3). mn SWR = (3) MAX where mn s the mnmum executon tme of all sampled teratons for applcaton. MAX s the maxmum executon tme of all sampled teratons for applcaton. For example, for a regular applcaton wth unform workload dstrbuton, the fve sampled teratons are the same. Therefore, the SWR s 1, and the whole workload can be dspatched accordng to Performance Rato, wth good load balance. However, for another applcaton, the fve samplng executon tme mght be 7, 7.5, 8, 8.5 and 10 seconds, respectvely. Then the SWR s 7/10. Therefore, 70% of the workload would be scheduled statcally accordng to PR, whle 30% of the workload would be scheduled by a dynamc scheme. Module Reduce Intalzaton Whle (a chunk of workload arrves) { receve the chunk of workload Compute on ths chunk Send the result to the Master } Fnalzaton END Reduce IV. EXPERIMENTAL RESULTS To verfy our approach, a cloud test-bed was bult, and three types of applcaton programs were mplemented usng the Hadoop pg language: Matrx Multplcaton, Assocaton Rule Mnng and Mandelbrot Set Computaton. The former two applcatons have regular workloads, whle the last has rregular workload. A. Cloud Test-bed A Cloud computng test-bed has been bult by the Hgh Performance Computng Lab. of Tungha Unversty, Tawan [33], usng Hadoop. The summary of the nodes s shown n Fgure 3. Fgure 4 shows the real-tme status of the cloud testbed acqured by the montorng tool. D. Programmng Pattern Based on the estmated nformaton of workload dstrbuton and node performance, we propose an MapReduce programmng pattern for performance-based workload dstrbuton on cloud envronments. Ths pattern conssts of two modules: a Map module and a Reduce module. The Map module makes the schedulng decson and dspatches workloads to slaves. On the other hand, the Reduce module processes the assgned work. Ths algorthm s just a pattern, and the detaled mplementaton, such as data preparaton, parameter passng, etc., mght be dfferent accordng to requrements of varous applcatons. Intalzaton Module Map /* Stage 1: Gatherng the nformaton */ collect the followng nformaton: CPU_Loadng CPU_Clock_Speed collect the executon tme of 5 sampled teratons /* Stage 2: Calculate schedulng parameters */ calculate SWR of the workload calculate Performance Rato of all slave nodes Fgure 3. The node summary of the cloud test-bed /* Stage 3: Statc Schedulng */ dspatch the (SWR)-percent of workload accordng to Performance Rato probe and receve for returned results /* Stage 4: dynamc Schedulng */ dspatch the (100-SWR)-percent of workload by a dynamc scheme Fnalzaton END Map

5 Fgure 4. The snapshot of the montorng tool on the TIGER Cloud In ths study, we have mplemented several schedulng schemes for the purpose of evaluaton. The conventonal statc schedulng scheme s to equally dstrbute the total workload to each worker at comple tme. However, ths scheme s obvously not sutable for dynamc and heterogeneous envronments. Therefore, a weghted statc schedulng scheme s adopted n ths experment. The prncple of parttonng s accordng to the CPU clock speed of each processor. A faster node wll get more workloads than a slower one proportonally. To reduce errors of expermental results, executon tme n each experment s obtaned by averagng the results of fve repettve executons. B. Applcaton 1: Matrx Multplcaton Matrx Multplcaton s a fundamental operaton n many numercal lnear algebra applcatons. Its effcent mplementaton on parallel computers s an ssue of prme mportance when provdng such systems wth scentfc software lbrares. Consequently, consderable effort has been devoted n the past to the development of effcent parallel matrx multplcaton algorthms, and ths wll reman a task n the future as well. Many parallel algorthms have been desgned, mplemented, and tested on dfferent parallel computers or cluster of workstatons for matrx multplcaton. In ths applcaton, the workload s loop teratons. The Master module s responsble for the dstrbuton of workloads. When a slave node becomes dle, the master node sends two ntegers to the slave. The two numbers represent the begnnng and endng ponters to the assgned chunk respectvely. In other words, every node has a copy of the nput matrces locally, so data communcaton s not sgnfcant n ths knd of mplementaton. Therefore, communcaton cost between the master and the slave s low, and the domnant cost s the computaton of matrx multplcaton. Frst, we want to compare the proposed PWD scheme wth prevous schemes wth respect to the executon tme. Fgure 5 llustrates the executon tme of weghted statc schedulng, GSS, FSS, TSS, NGSS, ANGSS and our PWD scheme, wth nput matrx sze , , and respectvely. The results are shown as follows. Among these schemes, PWD performs better than other schemes. The reason s that PWD accurately estmates the PR, and takes the advantage of statc schedulng, thus reducng the runtme overhead. The statc scheme obvously performs worse than other dynamc schemes. It s reasonable to say that the statc scheme s not sutable for a dynamc cloud envronment, wth respect to performance. It s nterestng that tradtonal self-schedulng schemes (FSS and TSS) perform slghtly better than NGSS and ANGSS. However, ths result s nconsstent wth that of prevous research [13, 14]. The reason mght be that the parameter s set too hgh, 75. If the parameter s set approprately, t s possble for NGSS and ANGSS to perform better, as prevous work has shown. Ths case also ndcates that NGSS and ANGSS suffer from the dffculty of determnng an approprate parameter value. Fgure 5. Executon tme for Matrx multplcaton wth dfferent nput szes C. Applcaton 2: Assocaton Rule Mnng Data mnng, or known as knowledge dscovery, s to acqure nterestng knowledge from large-scale databases [20]. Data mnng technques nclude assocaton rule mnng, classfcaton, cluster analyss, etc. The objectve of assocaton rule mnng s to dscover correlaton relatonshps among a set of tems. The well-known applcaton of assocaton rule mnng s market basket analyss. Ths technque can extract customer buyng behavors by dscoverng what tems they buy together. The managers of shops can place the assocated tems at the neghborng shelf to rase ther probablty of purchasng. For example, mlk and bread are frequently bought together. The formulaton of assocaton rule mnng problem s descrbed as follows [21, 22]. Let I be a set of tems, and D a database of transactons. Each transacton n D s a subset of I. An assocaton rule s a rule of the form A B, where A I, B I, and A B=. The well-known algorthm for fndng assocaton rules n large transacton databases s Apror. It utlzes the Apror property to reduce the search space. As the rsng of parallel processng, parallel data mnng have been well nvestgated n the past decade. Especally,

6 much attenton has been drected to parallel assocaton rule mnng. A good survey can be found n [23]. Tradtonal parallel data mnng work assumes data s parttoned and transmtted to the computng nodes n advance. However, t s usually the case n whch a large database s generated and stored n some staton. Therefore, t s mportant to effcently partton and to dstrbute the data to other nodes for parallel computaton. In ths applcaton, the workload s a database of transactons. We appled the pattern to mplement the Apror algorthm and ts data dstrbuton. Specfcally, the parallelzed verson of Apror we adopt s Count Dstrbuton (CD) [21, 22]. Our datasets are generated by the tool ndcated n [22]. The parameters of the synthetc datasets are descrbed n Table I. Dataset TABLE I. Number of Transactons DESCRIPTION OF OUR DATASET Average Transacton Length Number of Items D10KT5I10 10, D50KT5I10 50, D100KT5I10 100, D150KT5I10 150, Fgure 6 llustrates the executon tme of dfferent scheme, wth nput sze 10K, 50K, 100K and 150K transactons respectvely. Expermental results show that the scheme mplemented by the pattern got better performance than other schemes. From ths experment, we can see the sgnfcant nfluence of workload dstrbuton schemes on the total response tme. In cloud envronments, network bandwdth s an mportant crteron to evaluate the performance of a slave node. The Statc scheme can not adapt to the practcal network status. In contrast to Statc, when communcaton cost becomes a major factor, dynamc schemes would be well adaptve to the network envronment. Moreover, the reason why PWD got the best performance can be attrbuted to the approprate estmaton of node performance, especally for the attrbute of network bandwdth. In cloud computng envronments, CPU speed s not the only factor to determne the node performance. A node wth the fastest CPU s not necessary the node wth optmal performance. Fgure 6. Performance of data partton schemes for dfferent datasets D. Applcaton 3: Mandelbrot Set Computaton The Mandelbrot set computaton s a problem nvolvng the same computaton on dfferent data ponts whch have dfferent convergence rates [24]. Ths operaton derves a resultant mage by processng an nput matrx, A, where A s an mage of a pxels by b pxels. The resultant mage s one of a pxels by b pxels. The Mandelbrot Set Computaton has been mplemented usng the pattern. The Master module s responsble for the dstrbuton of workload. When a slave node becomes dle, the master node sends two ntegers to the slaves. As mplemented n Matrx Multplcaton, communcaton cost between the master and the slave s low, and the domnant cost s the computaton of the Mandelbrot Set. In the followng experment, we want to compare the executon tme of prevous schemes wth the mplemented program. Fgure 7 llustrates the executon tme of GSS, FSS, TSS, NGSS, ANGSS and our PWD scheme, wth nput mage sze 64 64, , and respectvely. The executon tme of weghted statc schedulng s omtted because ts results are sgnfcantly nferor to other schemes. Accordng to the experence n the Matrx Multplcaton applcaton, the parameter n NGSS s set to 30. The results are shown as follows. Among these schemes, PWD stll performs better than other schemes. The reason s also that PWD accurately estmates the PR, and takes the advantage of statc schedulng, thus reducng the runtme overhead. Tradtonal self-schedulng schemes (GSS, FSS and TSS) perform worse than NGSS and ANGSS. The reason s that t s dffcult to effcently schedule rregular workload for conventonal dynamc schemes. If the parameter s set approprately, t s certan for NGSS and ANGSS to perform better than GSS, FSS and TSS, as prevous work has shown.

7 applcatons for cloud envronments, whch s a dffcult task for novce programmers. Nevertheless, wth the pattern, all a programmer need to do s just to fll the applcaton-specfc program codes nto the pattern. If a programmer can code a sequental program, then t s straghtforward to transform t to a cloud applcaton. To extend the pattern dea to non-masterslave applcatons, such as P2P networks, we need to acqure experences and expertse n P2P programmng. In addton, the lack of global statstcal n non-master-slave applcatons s a problem to be solved. In P2P networks, the performancerelated nformaton can be gathered through socal actvtes, such as gossp protocols. Ths wll be an nterestng research topc n our future work. Fgure 7. Executon tme for Mandelbrot Set Computaton wth dfferent nput szes E. Dscusson In ths secton, several ssues are dscussed to clarfy the proposed approach. In general, task schedulng n cloud systems manly focuses on fne gran parallelsm, under the consderaton of the system heterogenety and the messagepassng communcaton. However, one goal of cloud computng s to explot potental parallelsm n nternet-scale cloud envronments. In addton to coarse gran parallelsm, we thnk that t s benefcal to explot fne gran parallelsm n cloud systems. The frst reason s to mprove utlzaton. The proposed approach provdes a mechansm for programmers to effcently utlze the dle resources located n cloud systems. The prelmnary results presented n ths study show that explotng fne gran parallelsm s promsng. Second, the dffcultes resultng from system heterogenety and the message-passng communcaton can be overcome by advanced technques, whch also motvate novel research topcs. Therefore, a number of researches focus on explotng fne gran parallelsm for loop schedulng and data mnng n cloud systems, such as [25-29]. In Secton 3.1, we menton that there are two knds of attrbutes assocated wth nodes, constants and varables. It s an nterestng ssue to nvestgate the relatonshp between these two knds of attrbutes. We thnk that each devce n a cloud system can be assocated wth these two knds of attrbutes. Takng CPU for example, CPU clock speed s a constant attrbute whle CPU loadng s a varable attrbute. Wth respect to the relatonshp between the two knds, t s ntutve that the node wth hgh CPU speed wll get more tasks to execute, resultng n hgh CPU loadng. It s probable that other devces also reveal smlar propertes. However, ths work does not focus on ths topc. We plan to take ths relatonshp nto further consderaton n our future work. In ths work, we prmarly propose a useful cloud programmng pattern, whch adopts a performance-based heurstc to dstrbute workloads, for master-slave applcatons. However, we beleve that t s possble to extend ths approach to non-master-slave applcatons, such as P2P applcatons. We explan the reason as follows. The programmng pattern abstracts our experences n programmng master-slave V. CONCLUSIONS We have proposed a programmng pattern for programmers to easly develop hgh performance applcatons on dynamc and heterogeneous cloud envronments. Ths pattern uses a performance-based approach to dstrbute workloads wthn a program to workng nodes. In ths approach, the system heterogenety s estmated by performance functons, and the varaton of workload s estmated by Statc-Workload Rato. On our cloud platform, programs mplemented by the proposed approach can obtan performance mprovement on prevous schemes. In the near future, we wll mplement more types of applcaton programs to verfy our approach. Also, applcaton of performancebased programmng to e-learnng wll be nvestgated. ACKNOWLEDGMENT The work s supported n part by Natonal Scence Councl, Tawan R.O.C., under grant no. NSC S , NSC S , NSC S MY3 and NSC S MY3. REFERENCES [1] Foster, I., The Grd: A New Infrastructure for 21st Century Scence. Physcs Today, (2): p [2] Foster, I. and C. Kesselman, Globus: A Metacomputng Infrastructure Toolkt. Internatonal Journal of Supercomputer Applcatons and Hgh Performance Computng, (2): p [3] Boeres, C., et al., An EasyGrd portal for schedulng system-aware applcatons on computatonal Grds. Concurrency and Computaton: Practce and Experence, (6): p [4] Boeres, C. and V.E.F. Rebello, EasyGrd: towards a framework for the automatc Grd enablng of legacy MPI applcatons. Concurrency and Computaton: Practce and Experence, (5): p [5] Nascmento, A.P., et al., Dstrbuted and dynamc self-schedulng of parallel MPI Grd applcatons. Concurrency and Computaton: Practce and Experence, n press. [6] Beaumont, O., et al., Schedulng dvsble loads on star and tree networks: results and open problems. Parallel and Dstrbuted Systems, IEEE Transactons on, (3): p [7] Macej, D. and L. Marcn, Mult-nstallment Dvsble Load Processng n Heterogeneous Systems wth Lmted Memory. : Parallel Processng and Appled Mathematcs [8] Macej, D. and L. Marcn, On Optmum Mult-nstallment Dvsble Load Processng n Heterogeneous Dstrbuted Systems. : Euro-Par 2005 Parallel Processng [9] Kruskal, C. and A. Wess, Allocatng ndependent subtaskson parallel processors. IEEE Transactons on Software Engneerng, : p

8 [10]Polychronopoulos, C.D. and D.J. Kuck, Guded self-schedulng: A practcal schedulng scheme for parallel supercomputers. 1987, IEEE Computer Socety. p [11]Susan Flynn, H., S. Edth, and E.F. Lawrence, Factorng: a method for schedulng parallel loops. 1992, ACM Press. p [12]Tzen, T.H. and L.M. N, Trapezod Self-Schedulng: A Practcal Schedulng Scheme for Parallel Complers. IEEE Transactons on Parallel and Dstrbuted Systems, (1): p [13]Yang, C.T. and S.C. Chang, A Parallel Loop Self-Schedulng on Extremely Heterogeneous PC Clusters. Journal of Informaton Scence and Engneerng, (2): p [14]Yang, C.T., K.W. Cheng, and K.C. L, An Effcent Parallel Loop Self- Schedulng Scheme for Cluster Envronments. Journal of Supercomputng, : p [15]Shh, W.-C., C.-T. Yang, and S.-S. Tseng, A performance-based parallel loop schedulng on grd envronments. The Journal of Supercomputng, (3): p [16]Yang, C.-T., W.-C. Shh, and S.-S. Tseng, Dynamc Parttonng of Loop Iteratons on Heterogeneous PC Clusters. to appear n The Journal of Supercomputng, [17]THU. The TIGER Grd [cted 2006; Avalable from: [18]Globus. The Globus Project [cted 2006; Avalable from: [19]MPICH. MPICH-G [cted 2006; Avalable from: [20]Han, J. and M. Kamber, Data Mnng: Concepts and Technques. 2001: Morgan Kaufmann. [21]Agrawal, R. and J.C. Shafer, Parallel Mnng of Assocaton Rules. IEEE Transactons on Knowledge and Data Engneerng, (6): p [22]Agrawal, R. and R. Srkant. Fast algorthms for Mnng Assocaton Rules. n Proc. 20th Very Large Data Bases Conf [23]Zak, M.J., Parallel and Dstrbuted Assocaton Mnng: A Survey. IEEE Concurrency, (4): p [24]Mandelbrot, B.B., Fractal Geometry of Nature. 1988, New York: W. H. Freeman. [25]Herrera, J., et al. Loosely-coupled loop schedulng n computatonal grds. n The 20th Internatonal Parallel and Dstrbuted Processng Symposum (IPDPS 2006) [26]Penmatsa, S., et al. Implementaton of Dstrbuted Loop Schedulng Schemes on the TeraGrd. n IEEE Internatonal Parallel and Dstrbuted Processng Symposum (IPDPS 2007) [27]Cannataro, M., et al., Dstrbuted data mnng on grds: servces, tools, and applcatons. Systems, Man, and Cybernetcs, Part B, IEEE Transactons on, (6): p [28]Folet, V., et al. Optmal Grd Explotaton Algorthms for Data Mnng. n The Ffth Internatonal Symposum on Parallel and Dstrbuted Computng (ISPDC '06) [29]Jang, W.-S. and J.-H. Yu. Dstrbuted data mnng on the grd. n Proceedngs of 2005 Internatonal Conference on Machne Learnng and Cybernetcs [30]Shh, W.C., Yang C.T. and Tseng S.S., Usng a Performance-based Skeleton to Implement Dvsble Load Applcatons on Grd Computng Envronments, Journal of Informaton Scence and Engneerng (JISE), 25(1), pp , [31]Apache, Hadoop, [32]J. Dean and S. Ghemawat, "Mapreduce: Smplfed data processng on large clusters," n OSDI'04: Proceedngs of the 6th Symposum on Operatng System Desgn and Implementaton. San Francsco, Calforna, USA. USENIX Assocaton, pp December 6-8, [33]THU, Cloud Testbed,