A Routing Load Balancing Policy for Grid Computing Environments

Transcription

1 A Routing Load Balancing Policy for Grid Computing Environments Rodrigo Fernandes de Mello University of São Paulo Institute of Mathematic Sciences and Computer São Carlos SP Luciano José Senger State University of Ponta Grossa Department of Information Technology Ponta Grossa PR Laurence Tianruo Yang Department of Computer Science, St. Francis Xavier University P.O.Box 5, Antigonish, NS, B2G 2W5, Canada Abstract The evolution of computers and networks has motivated the development of high performance systems using the distributed system concepts. Such evolution has motivated the study of load balancing techniques, low latency protocols, operating systems and middlewares. Furthermore, the evolution from distributed systems based on clusters to grid environments has motivated new researches to define load balancing algorithms to support scalable and heterogeneous computing capacity environments. In this paper, the load balancing algorithm is proposed, presented and evaluated. This algorithm is designed to equally distribute the workload of tasks of parallel applications over Grid computing environments. Experiments compare the performance of the proposed algorithm to others from literature. The obtained results allow to conclude that the algorithm is indicated for environments where there are several heterogeneous computers and parallel applications are composed of multiple tasks. 1 Introduction During the 197s, 198s and at the beginning of the 199 s, there was a great effort towards the development of process scheduling algorithms for parallel machines. From the 198s on, due to the microprocessors cost decrease and the continuous development of computer networks, researches were addressed to define high performance environments using low cost resources, which would be able to replace the high cost parallel machines. Besides reducing costs, such resources allowed the creation of more flexible and scalable systems. Such systems were named distributed systems [4]. Distributed systems have motivated researches on how to maximize the use of computing resources, obtain high performance and develop communication environments and libraries to simplify the design and implementation of such systems [24]. This generated new researches on process scheduling algorithms for distributed system, aiming the allocation optimization of computing resources to obtain high performance [6]. Among the researchers in this area are: Ranaweera and Agrawal [], Choe and Park [7], Radulescu et al. [19] and Araújo et al. [2]. In the following decades, the distributed system concepts have evolved from local networks to internet environments. In such environments, several networks interact through routers. Then, researches for the development of resources allocation techniques and high performance on large environments have started. Such environments, located on distinct and distant regions, composed of thousands of heterogeneous computing capacity resources are named Grids [14]. The previous process scheduling studies were used as basis for the proposal of new methods and techniques for resource allocation on Grids [2, 17, 5, 1, 18]. Such studies have motivated the development of a new process scheduling algorithm, aiming the load balancing [22] on Grids. This algorithm, named, uses routing concepts from computer networks to define a neighborhood and search the most adequate computers to divide applications workload. The performance of such algorithm has been evaluated and compared to others from literature. Proceedings of the th International Conference on Advanced Information Networking and Applications (AINA 6) X/6 $. 6 IEEE

2 2 A New Load Balancing Algorithm is a load balancing algorithm addressed to grid computing environments where there is a large amount of resources, heterogeneity, high communication latency (when compared to parallel virtual machines executing over LANs), large number of users and distributed location. Load balancing algorithms use dynamic reattribution to equally distribute the applications workload over the available computing resources [22]. Such algorithm uses the message routing concepts from computer networks to define the computers neighborhood. Each computer may distribute the tasks of a parallel application over its neighbors. has been designed to work on high scalable heterogeneous environments. In order to explain, consider three distinct computer networks: α, β and γ. Each computer of the network is composed of different capacity resources such as CPU, memory, hard disk and network. In this environment, each computer executes a process (daemon) to manage the load balancing. In the initialization of this daemon on a computer C n,α (computer C n from the network α), it is calculated the internal communication delay RT T α (RTT - Round-Trip Time) of the network, that is, the cost to transmit and receive a minimum size message between two computers of the network α. After RT T α is calculated, it is possible to define the maximum RTT, RT T neighborhood, used to find out the computer neighborhood of C n,α. This neighborhood is defined by the equation 1, where k is a delay parameter for the communication among computers. RT T neighborhood = RT T α k, k >= 1 (1) The parameter k may be fixed or variable based on the network capacity (latency) of the whole environment. All computers with a communication delay lower or equal to RT T neighborhood to the computer C n,α are defined as neighbors. When a computer C n,α initiates a parallel application composed of tasks, this computer requests, by using broadcast, information about its neighbors load in order to find out the idlest ones. The task distribution is made among neighbors and the neighborhood is defined by the communication delay among computers. A low value to k (equation 1) generates a small neighborhood and consequently causes lower synchronization delays among the tasks of the parallel applications. If a computer is overloaded, it may evaluate its neighbors load and define the transference of a task based on the migration model defined by Mello and Senger [9]. The limitations of the work by Halchor-Balter and Downey [15] have motivated Mello and Senger to create a process transference model that evaluates three issues: the migration cost, the imposed load and the process lifetime. The Halchor-Balter and Downey s model evaluates only the processes age, that is, the execution time. Long-time duration processes do not imply high load processes, consequently, the transference of low occupation processes consumes environment resources and does not allow a good load balance in the system. Mello and Senger propose a migration factor (eq. 2, where: L xpr,i is the load of a task i that still has to be executed; Ca s is the processing capacity of the sending computer; CL s is the load of the sending computer; CL dst is the load of the computer that will receive the task; MC i is the migration cost of the task i). When this factor is equal to 1, the system reaches a perfect balance as each computer executes a load equivalent to its capacity. If this is not possible, an approximation to this value may be done summing the load of n processes whilst the factor 1. If no process of the busy computer results in a migration factor M factor equal or lower than 1, then a process that has a factor (Ca r Ca s ) may be selected. In this case, the factor is compared to the performance difference, in percentage, between the source and the destination computer. M factor,i = 3 Experiments L xpr,i Ca s CL s (CL dst +((L xpr,i Ca s) Ca r)+mc i) (2) Experiments have been carried out over 128 and 512 computers. This allows the evaluation of the algorithm on environments with different scales. Parallel applications of up to 8 and 64 tasks have been evaluated. Such configurations allow the evaluation of the algorithm in situations where there are several tasks synchronizing one each other, that is, communicating among themselves to solve the same computing problem. The workload imposed by such applications follows the Feitelson s workload model [13]. This model is based on the analysis of six execution traces of the following production environments: 128-node ipsc/8 at NASA Ames; 128-node IBM SP1 at Argonne; -node Paragon at SDSC; 126-node Butterfly at LLNL; 512-node IBM SP2 at CTC; and 96-node Paragon at ETH, Zurich. According to this model, the arrival of processes is derived from an exponential probability distribution function (pdf) of mean equal to 15 seconds. However, there are not enough knowledge bases (execution traces) to characterize the arrival times on Grid computing environments. This limitation has motivated this work to evaluate two process arrival distributions, allowing a better evaluation of. The first pdf used to represent the arrival of parallel applications was the one by Feitelson, that is, an exponential with mean equal to 15 seconds. Considering the studies by Feitelson and the fact that the distribution is close to an Proceedings of the th International Conference on Advanced Information Networking and Applications (AINA 6) X/6 $. 6 IEEE

3 exponential, experiments have been proposed with another mean equal to 1 seconds. In the case of the exponential with mean 1 seconds, the applications arrive with much lower intervals (between 1 and 1 seconds). We empirically conclude that the exponential distribution with mean 1 seconds is the most adequate one to simulate the features expected on a Grid, as the distribution with average 15 is too light. In order to carry out experiments and evaluate the scheduling algorithm proposed in this work, we have used the UniMPP (Unified Modeling for Predicting Performance) model for the creation of heterogeneous distributed environments and the evaluation of the parallel applications response time [1]. Such model combines the CPU consumption features of the models by [1] and [11], the time consumed in message transmissions proposed in [8] and [23], the volume of messages and the spatial and message generation probability distributions by [12] and [25]. The UniMPP model generates results in average execution time of the application submitted to the system. In the UniMPP model proposal, Mello and Senger [1] have also proposed an object-oriented simulator 1, which simplifies the implementation of different scheduling algorithms. In order to evaluate the algorithm, a class has to be developed to implement the scheduling algorithm. Such class was aggregated to the simulator to generate results in average response time (in seconds). The results were used to evaluate the performance of compared to other algorithms from literature. 3.1 Environment Parameterization Environments composed of 128 and 512 computers were simulated. The parameters of the UniMPP model have been defined by probability distribution functions (pdfs). The parameters pc i (processing capacity), mm i (main memory capacity), vm i (virtual memory capacity), dr i (disk throughput in reading) and dw i (disk throughput in writing) have been defined by uniform pdfs with the following averages of 15 Mips (million of instructions per second), 124 MBytes (main memory), 124 MBytes (virtual memory), MBytes (transference rate of reading files from hard disk), 3 MBytes (transference rate of writing files on hard disk). Such averages have been defined based on actual values obtained from machines of our research laboratory 2. These measures were extracted by using the benchmark proposed by Mello and Senger [1] 3. The UniMPP model has also parameters to represent processes which are: sm j, the amount of static memory used by the process was defined based on an exponential 1 SchedSim - available at mello/outr.html. 2 Lasdpc Available at mello/ pdf with average of 3 KBytes; pdfdm j, the amount of memory dynamically allocated was defined by an exponential pdf with average of 1 KBytes; pdfdr j, the probability of reading files was defined by an exponential pdf with average of 1 clock ticks, same value used to parameterize the writing in files (pdfdw j ); pdfnet j, the reception and sending of network messages was parameterized by an exponential pdf with an average of 1 clock ticks. In the experiments, it was defined the number of computers by network and, consequently, how many distinct networks were created. Within a same network, computers present a delay (RTT Round-Trip Time, according to the fixed model by Hockney [16]) of.1 (average value extracted by the network benchmark by Mello and Senger [1] for a Fast Ethernet infrastructure). In order to characterize the delay among computers of distinct networks, experiments have been carried out on local, metropolitan and worldwide networks. Such experiments have allowed the definition of exponential probability distribution functions with average of.5 seconds to characterize the delay. The performance of is compared to 6 scheduling and load balancing algorithms from literature: [2, 3],,, [26], [18] and [21]. 3.2 Results The Figures 1 and 2 show, respectively, the results obtained for environments composed of 128 and 512 computers where the application have up to 8 tasks. Each network contains up to 5 computers. The number of tasks per application is defined in the workload distribution by Feitelson [13]. These results are in average response time (in seconds), that is, the average time elapsed for the applications execution. These experiments allowed the observation that the algorithm presents the best results for environments with 128 and 512 computers. Such results have evidenced the benefits of the algorithm on environments in which the parallel applications are not composed of a large number of tasks. This result was obtained by Mello and Senger [21]. As the number of tasks per process increases, the algorithm consumes more processing time to decide on the best scheduling solutions, what jeopardizes the average response time. The Figures 3 and 4 show the obtained results for environments composed of 128 and 512 computers where the application have up to 64 tasks. It may be observed that algorithm starts to present good behavior from environments with 128 computers on. The and algorithms start to slowdown with parallel applications composed of many tasks. This fact can be observed by making a comparison between these experiments and the ones in Proceedings of the th International Conference on Advanced Information Networking and Applications (AINA 6) X/6 $. 6 IEEE

4 Figure computers, applications of 8 tasks, average 15 seconds pdf Figure computers, applications of 64 tasks, average 15 seconds pdf Figure computers, applications of 8 tasks, average 15 seconds pdf Figure computers, applications of 64 tasks, average 15 seconds pdf which the applications have up to 8 tasks. In order to evaluate the algorithm in high load situations, the probability distribution function for the arrival of parallel applications has been changed to an exponential with average of 1 seconds. Such exponential generates a high load behavior in which the users submit most part of the applications in intervals lower than 1 seconds. The Figures 5 and 6 show the results of this arrival distribution with applications composed of up to 8 tasks. Experiments have been carried out on environments composed of 128 and 512 computers, in which each network contains up to 5 computers. Once again the behavior of the algorithm was sim- ilar or higher than the one presented by, since it is more indicated for environments where applications are composed of few tasks. The next step was to carry out experiments with applications of up to 64 tasks. The results showed in Figures 7 and 8 allow the conclusion that the load balancing algorithm has presented good results. The algorithm presents intermediary results and a high variation of response times in two cases (high variation in the submission of 5 and 18 applications). This variation is due to the fact that a computer of any network may initiate a parallel application and distribute its load over the neighbors. In the case neighbors present a high load, it is possible to obtain these significant variations. Proceedings of the th International Conference on Advanced Information Networking and Applications (AINA 6) X/6 $. 6 IEEE

5 Figure computers, applications of 8 tasks, average 1 seconds pdf Figure computers, applications of 64 tasks, average 1 seconds pdf Figure computers, applications of 8 tasks, average 1 seconds pdf Figure computers, applications of 64 tasks, average 1 seconds pdf When the environment of 128 is scaled, the performance of gets better, demonstrating its capacity to work over large environments running applications composed of multiple tasks. 4 Acknowledgments The authors thank the support of Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP 4/2411-9). 5 Conclusions This paper presents the researches that have resulted in the proposal and evaluation of, a new load balancing algorithm for scalable heterogeneous environments such as Grids. The experiments allowed the evaluation of in different load situations. The workload model by Feitelson has been used [13] to define the occupation and number of tasks submitted to the system. As there is no conclusive studies about the behavior of the applications arrival to Grid environments, we have used the exponential probability distribution function with average 15 seconds (obtained from the studies by Feitelson and broadly used Proceedings of the th International Conference on Advanced Information Networking and Applications (AINA 6) X/6 $. 6 IEEE

6 in scheduling studies about parallel machines and clusters) and 1 seconds. Experiments allowed to confirm that presents excellent results in environments composed of many heterogeneous capacity computers, in which, parallel applications composed of multiple tasks are submitted. Such features indicate the algorithm to Grid environments. References [1] Y. Amir. An opportunity cost approach for job assignment in a scalable computing cluster. IEEE Transactions on Parallel and Distributed Systems, 11(7):7 768, Jul.. [2] A. P. F. Araújo, M. J. Santana, R. H. C. Santana, and P. S. L. Souza. : A new load balancing algorithm. In 5th International Conference on Information Systems Analysis and Synthesis - ISAS 99, Orlando, U.S.A., [3] A. P. F. Araújo, M. J. Santana, R. H. C. Santana, and P. S. L. Souza. A new dynamical scheduling algorithm, international conference on parallel and distributed processing techniques and applications - pdpta 99. Las Vegas,Nevada, U.S.A., [4] R. Buyya. High Performance Cluster Computing - Architecture and Systems, volume 1. Prentice Hall, [5] R. Buyya and M. Murshed. Gridsim: A toolkit for the modeling and simulation of distributed resource management and scheduling for grid computing. The Journal of Concurrency and Computation: Practice and Experience (CCPE), 14(13 15):46, 2. [6] T. L. Casavant and J. G. Kuhl. A Taxonomy of Scheduling in General-purpose Distributed Computing Systems. IEEE Transactions on Software Engineering, 14(2): , February [7] T. Choe and C. Park. A task duplication based scheduling algorithm with optimality condition in heterogeneous systems. In International Conference on Parallel Processing Workshops (ICPPW 2), pages IEEE Computer Society, 2. [8] D. E. Culler, R. M. Karp, D. A. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken. LogP: Towards a realistic model of parallel computation. In Principles Practice of Parallel Programming, pages 1 12, [9] R. F. de Mello and L. J. Senger. A new migration model based on the evaluation of processes load and lifetime on heterogeneous computing environments. In International Symposium on Computer Architecture and High Performance Computing - SBAC-PAD, page 6, 4. [1] R. F. de Mello and L. J. Senger. A new slowdown model for heterogeneous multicomputers environments. In submitted, page 11, 5. [11] R. F. M. et. al. Analysis on the significant information to update the tables on occupation of resources by using a peerto-peer protocol. In 16th Annual International Symposium on High Performance Computing Systems and Applications, Moncton, New-Brunswick, Canada, June 2. [12] S. C. et. al. Towards a communication characterization methodology for parallel applications. In Proceedings of the 3rd IEEE Symposium on High-Performance Computer Architecture (HPCA 97), page 31. IEEE Computer Society, [13] D. G. Feitelson and M. A. Jette. Improved utilization and responsiveness with gang scheduling. In D. G. Feitelson and L. Rudolph, editors, Job Scheduling Strategies for Parallel Processing, pages Springer, Lect. Notes Comput. Sci. vol [14] A. J. H. Fran Berman, Geoffrey Fox. Grid Computing: Making the Global Infrastructure a Reality. John Wiley & Sons, 3. [15] M. Harchol-Balter and A. B. Downey. Exploiting Process Lifetimes Distributions for Dynamic Load Balancing. ACM Transactions on Computer Systems, 15(3): , August [16] R. W. Hockney. The Science of Computer Benchmarking. Soc for Industrial & Applied Math, [17] R. P. Ishii, R. F. de Mello, L. J. Senger, M. J. Santana, and R. H. C. Santana. Scheduling based on the impact over process communication of parallel applications. In Proceedings of the 19th International Symposium on High Performance Computing Systems and Applications, page 7. IEEE Computer Society, 5. [18] R. F. Mello, L. C. Trevelin, M. S. Paiva, and L. T. Yang. Comparative study of the server-initiated lowest algorithm using a load balancing index based on the process behavior for heterogeneous environment. In Networks, Software Tools and Application, ISSN Kluwer, 4. [19] A. Radulescu, A. van Gemund, and H. Lin. LLB: A fast and effective scheduling algorithm for distributed-memory systems. In 13th International and 1th Symposium on Parallel and Distributed Processing - IPPS/SPDP, pages , [] S. Ranaweera and D. P. Agrawal. A task duplication based scheduling algorithm for heterogeneous systems. In Parallel and Distributed Processing Symposium - IPDPS, pages ,. [21] L. J. Senger, R. F. de Mello, M. J. Santana, and R. H. C. Santana. Improving scheduling decisions by using knowledge about parallel applications resource usage. page 1, 5. [22] N. G. Shivaratri, P. Krueger, and M. Singhal. Load distributing for locally distributed systems. IEEE Computer, 25(12):33 44, [23] A. Sivasubramaniam, A. Singla, U. Ramachandran, and H. Venkateswaran. An approach to scalability study of shared memory parallel systems. In Measurement and Modeling of Computer Systems, pages , [24] T. Sterling. Cluster computing white paper available at mab/tfcc/whitepaper/, 1. [25] J. S. Vetter and F. Mueller. Communication characteristics of large-scale scientific applications for contemporary cluster architectures. J. Parallel Distrib. Comput., 63(9): , 3. [26] S. Zhou and D. Ferrari. An experimental study of load balancing performance. Technical Report UCB/CSD 87/336, PROGRES Report N.o 86.8, Computer Science Division (EECS), Universidade da California, Berkeley, California 947, jan Proceedings of the th International Conference on Advanced Information Networking and Applications (AINA 6) X/6 $. 6 IEEE