What are Cycle-Stealing Systems Good For? A Detailed Performance Model Case Study

Transcription

1 What are Cyle-Stealing Systems Good For? A Detailed Performane Model Case Study Wayne Kelly and Jiro Sumitomo Queensland University of Tehnology, Australia {w.kelly, j.sumitomo}@qut.edu.au Abstrat The idea of stealing yles has been hyped for some years, boasting unlimited potential by tapping the omputational power of millions of under utilized PCs onneted to the Internet. Despite a few spetaular suess stories (eg SETI@HOME), yle-stealing is today not a widely used tehnology. We believe two prinipal impediments need to be overome. The first is ease of development and use. Most of the problems faed in developing yle stealing appliations are not speifi to those appliations, so generi yle stealing frameworks suh as our G framework an play a vital role in this regard. The seond is unertainty. Potential developers don t know whether if they went to the effort of developing a parallel appliation for a yle stealing environment, it would pay off, i.e. whether they would get a reasonable speedup. To minimize this risk, we propose the development and use of detailed performane models. 1. Introdution Appliations suh as SETI@HOME have demonstrated that yles an be effetively harvested from large olletions of PCs onneted via the Internet. Suh appliations are ideally suited to this environment as the tasks involved are ompletely independent, the sizes of the inputs and outputs to these tasks are relatively small, yet eah task typially requires several hours of proessing. What about other lasses of appliation - appliations that may not be embarrassingly parallel, whih onsume or produe relatively more data or whih require less proessing time per task? Even dediated superomputers with high-performane interonnets have their limits when it omes to ahieving speedups if the omputation to ommuniation ratio is not suffiiently high. Cyle stealing systems whih operate over the Internet have substantially higher overheads, so what is the limit of their appliability? Clearly there are some appliations that work effetively in this environment and others that will not. What is ruial is a means of distinguishing without having to implement and test empirially. In this paper we present a detailed performane model for our Internet yle stealing framework, G [1, ], and show that it an be used to presribe the properties that appliations need to exhibit in order to perform well in this environment. In Setion we outline the arhiteture of G. Despite its seeming simpliity, we found it surprisingly diffiult to aurately model the performane of G appliations. We started by abstrating aspets of parallel exeution suh as average job exeution time and job parameter sizes and modelling the various points at whih jobs and results are queued. It soon beame apparent that we needed to start by developing a disrete event simulator for our framework so that we ould be sure that we were not overly simplifying ruial aspets of the atual system. What started out as a simple disrete event simulator soon turned into a very omplex and detailed simulator. This simulator is desribed in Setion 3. This gave us a good understanding of how the atual system behaved at a more abstrat level, but ultimately we desired an analytial model. As well as allowing preditions to be made more rapidly, analytial models also allow us via investigation of the equations themselves to disover new rules and relationships. What we are modelling is learly a queuing system; however, we had to further simplify some aspets of our disrete event simulator in order to allow state of the art queuing theory tehniques to be applied. To determine the extent to whih suh further simplifiation affeted the auray of the preditions, we onstruted a simpler simulator that refleted only those aspets of our original simulator that ould be modelled using queuing theory tehniques. This simplified simulator is outlined in Setion and a omparison of results obtained by the two simulators is given in Setion 7. In Setion 5 we present our analytial performane model based on Mean Value Analysis (MVA) [3] for multi-lass losed queuing networks. In Setion 7 we ompare empirial performane results to those predited by our omplex and simple simulators and our analytial model, with onlusions in Setion.. G Arhiteture The G Framework is designed to make programming yle stealing appliations as simple as possible. It does so by hiding appliation programmers from muh of the underlying physial topology, job management and ommuniation that takes plae. To model the performane of suh a system we need, however, to delve into these internal implementation details. This setion gives a brief overview of these details. Proeedings of the 5 11th International Conferene on Parallel and Distributed Systems (ICPADS'5)

2 Clients Submit Jobs Feth esults Server Web Server Feth Jobs Submit esults Job epository Figure 1: G Arhiteture A G system omprises a Server that ats as learing house for jobs waiting to be exeuted and results waiting to be olleted. Client appliations use automatially generated appliation speifi proxy lasses to submit jobs to the server via a web servie interfae. A job onsists of exeuting a programmer speified method with a given set of parameters and an be exeuted ompletely independently from all other jobs. use a web browser to launh a generi volunteer host appliation that fethes jobs from the server, exeutes them and submits the results bak to the server (again via a web servie interfae). The volunteer host appliation maintains its own loal queues of jobs waiting to be exeuted and results waiting to be submitted. It does this so as to pipeline the exeution of jobs - overlapping the exeution of one job with the fething of the next job and the submitting of the previous result. If a volunteer attempts to feth a job and there are none urrently on the server then it sleeps for a while before reattempting. When results arrive bak at the server they annot be simply forwarded to the appropriate lient as we assume onservatively that not all lients will be aessible from the server due to firewalls et. The only mahine that we require to be universally aessible to the other mahines making up the yle stealing network is the server itself. This means that lients and volunteers an be loated anywhere on the Internet provided they an ontat the server. This restrition means that the lients need to pull job result off the server by means of a Fethesults web method. To minimize network traffi eah lient only ever has outstanding one Fethesults request. Suh a request returns all results urrently waiting on the server for that lient. If there are no results urrently present then the request will wait (on the server side) until at least one result beomes available. 3. A Disrete Event Simulator In produing our disrete event simulator for G we ended up produing a generi distributed omputing disrete event simulation framework. Only a small part of the simulator deals speifially with the behaviour of the G framework. Our simulation framework models a olletion of workstations onneted by network links along whih IP pakets flow in a FIFO fashion. Eah workstation possesses one or more CPUs whih exeute a olletion of loal threads in a round robin fashion. The behaviour of eah thread is ontrolled by a finite state mahine that expresses an abstration of the appliation s logi. Eah thread state determines a task to be performed. Some tasks are purely CPU bound, requiring a ertain number of yles to omplete, others are entirely memory or disk bound whih means they have no need for the CPU during that period. Other tasks deal with sending and reeiving messages via a network ard. The workstations may possess network interfae ards whih maintain physial I/O buffers and the ability to send an interrupt to the workstation s CPU when a new paket arrives. Suh workstations may implement a TCP/IP stak whih allows soket onnetions to be established and maintained with other mahines. The TCP protool uses speial onnet, aept and aknowledgment messages in order to ontrol data flow suh that buffers do not overflow. The key outome of this mehanism from our point of view is that reeivers are able to exert bakward pressure on senders, preventing them from sending additional pakets until the reeivers have aught up to a ertain extent. Figure 1 shows the omponents that make up our G simulator. The G speifi parts of the system are three sublasses of workstation for the lients, server and volunteers respetively. The lient workstation reates a job submission thread and a feth results thread. The volunteer workstation reates threads for fething, exeuting and submitting. The server workstation simulates the behaviour of the IIS web server and the G web servie implementation via a Listen thread whih listens for TCP onnetion requests on port. A fixed pool of reylable worker threads is used to proess the inoming HTTP requests on the sokets established by the listen thread. The worker threads must first de-serialise the inoming web servie requests before being able to extrat the name of the web servie method and proess it appropriately aording to the abstrated semantis of that method. Proeedings of the 5 11th International Conferene on Parallel and Distributed Systems (ICPADS'5)

3 Figure 1: G Simulator Components Most of these web methods involve a database task whih onsists of a series of CPU phases and memory/disk phases. This detailed modelling was neessary in order to aurately predit the behaviour of the server when large numbers of web requests are being proessed onurrently as the CPU bound phase of one web request an be overlapped with the memory/disk bound phase of another. The transitions whih the state mahines make is determined in part by G speifi state maintained by the lient, server and volunteer omponents, suh as the number of jobs urrently waiting to be proessed on the server or the number of results on a volunteer waiting to be submitted. Figure shows a summary of the state mahine that ontrols the G server worker threads. The length of the tiks that ontrol the flow of the disrete events an be easily adjusted so as to trade-off the auray of the simulation with the length of time required to run the simulator. The number of tiks attributed to eah task for the G threads was determined by extensive experimentation and various methods of profiling designed to isolate the onstituent osts. The experiments were repeated for various parameter values and equations fitted to the empirial data. In most ases, the equations were linear in one or more of the input parameters.. Towards an Analytial Model In order to allow analyti queuing theory tehniques to be applied we had to simplify some of the aspets that we modelled in our original simulator. We made these simplifiations first to our simulator in order measure the exat effet of doing so. If we had attempted to go diretly to an analytial solution we would not have known if the (presumably) different results were due to these simplifiations or due to a flaw in the proess of translating our dynami simulator into a stati analytial model. Figure : Worker Thread State Diagram The following simplifiations were made: o We eliminated the server multithreading. o The CPU and memory/disk phases of the server were amalgamated into a single servie time. o The TCP/IP elements of the ommuniation were dramatially simplified by inorporating all ommuniation osts into the request generation and the response proessing times on the lients/volunteers. 5. An Analytial Performane Model We model the server as a multi-lass FCFS queuing entre with lass dependent servie times. The lasses are Submit Job (), Feth Job (), Submit esult () and Feth esults (). We are able to model the system as losed sine the number of requests for eah lass remains onstant for a given number of lients and volunteers. Mean Value Analysis [3] tells us that the average response time () of a request is equal to the average time the request waits in the queue plus the average servie time (S) spent atually serviing the request. The average time waiting in the queue is approximately equal to the average number of requests (Q) in the queue multiplied by their respetive average servie times: S (1 + Q' ) + S Q d d where the subsripts represent the various lasses. Q' represents the average number of requests that would be in the queue for a losed system ontaining one fewer total requests (N )oflass. In general, this would require us to reursively solve the entire problem with this redued number of requests for this lass. Doing so would be extremely ostly in both time and spae, but in our peuliar senario it is not even valid as the numbers of requests in eah lass are oupled. It is not possible for d Proeedings of the 5 11th International Conferene on Parallel and Distributed Systems (ICPADS'5)

4 example to onsider a G system in whih there is 1 Fethesult request, but SubmitJob requests as without jobs being submitted, no results will ever be fethed. Fortunately there is a well known approximate MVA algorithm [] for multi-lass queuing networks that solves this problem. The approximate MVA algorithm approximates Q' by: N 1 Q. N So, for example, we have: N 1 S 1 Q + + SQ + SQ + SQ N N and N will be equal to the number of lients while N and N will be equal to the number of volunteers. Feth results ( ) is a little different as the Fethesults web servie is implemented asynhronously. If there are no results for that lient urrently waiting on the server, the server will wait until suh a result arrives before proessing the Fethesult request. This adds an additional wait time (W) to the response time. The average wait time W is given by the probability WP that no results are available when the Fethesults request is first servied, multiplied by the average duration WD we will have to wait for a new Submitesult request to arrive. If we assume the arrival of the requests follows a Poison stream we have: X X 1 WP e and WD where X is the average throughput of lass. The average queue length is given by the standard multi-lass MVA equation: Q X The standard MVA equation for throughput (X) is: N X + Z where Z is a lower bound on the time that the requests will spend away from the server. This inludes the time required by the lients/volunteers to generate eah new request, the time to transport the request over the Internet to the server, the time to transport the server s reply bak to the lient/volunteer and the time required by the lient/volunteer to proess the reply before the next request of that lass an be generated. However, we have so far ignored the fat that the four lasses of request are oupled to one another. The atual throughputs (X ) may therefore be lower than the above formula predits due to these other influenes. A FethJob request will return a job if one is available on the server, otherwise it will return a null value. The perentage (r) of FethJob requests that will sueed is X determined by the rate at whih jobs are being submitted and the rate at whih jobs are being fethed: X r min,1 % X This FethJob suess rate r affets the average servie time S on the server as well as the behaviour of the volunteer reeiving the FethJob replies. If a volunteer reeives a null value it will sleep for some period before attempting to request another job. If, however, a job is suessfully reeived, the volunteer will exeute it whih will take some other period of time. We aount for this differene by having two different Z values, one for the ase where the volunteer sleeps (Z sleep ) and one for the ase where the volunteer exeutes the job (Z exeute ). Whilst the volunteer has jobs to exeute, the volunteer s feth thread will be kept busy trying to feth new jobs from the server to replae the ones that the volunteer s exeute thread is removing from the loal job queue. The minimum time required to generate, transport and proess the results of these FethJob requests is represented by Z. Whilst the volunteer s exeute thread is exeuting jobs and plaing their results in the loal result queue, the volunteer s submit thread is kept busy trying to submit those results to the server. The minimum time required to generate, transport and proess the results of these Submitesult requests is represented by Z. If the output queue beomes full, then job exeution pauses until the submit thread an ath up. Any of these three volunteer threads an therefore beome the bottlenek that limits the FethJob throughput. So we have: N X ( Z sleep + )(1 r) + max( Z +, Zexeute, Z + ) r The rate at whih we submit results annot be higher than the rate at whih we suessfully feth jobs, so we have: N X min, rx Z + The rate at whih we feth results annot be higher than the rate at whih we submit results sine Fethesult requests wait on the server side until at least one result beomes available, so we have: N X min, X Z + If we are interested in an optimal steady state then there is no point in submitting jobs at a rate higher than the rate at whih we an feth, so we have: N X min, E X Z + Proeedings of the 5 11th International Conferene on Parallel and Distributed Systems (ICPADS'5)

5 where E is the average number of results returned per Fethesults request. X E X E also affets the average Fethesults servie time (S ) on the server and the time Z required by the lient to proess that number of results. So, S and Z are a funtion of E, whiles and X areafuntionofr. These and other yli dependenes in our analytial equations mean that we annot solve them diretly. We instead solve them in an iterative manner.. Volatile One of the biggest hallenges with yle stealing is that volunteered omputers may (by fault or intention) leave the network without warning at any time. The G framework transparently handles this problem quite simply by retaining a reord of eah job on the server until a result has been submitted. If the volunteer that had been assigned that job leaves, the server an simply reassign it to some other volunteer. This behaviour was easy to inorporate into the simulators but we are still working on inorporating it into our analytial model. This behaviour is atually quite important, as without the assumption that volunteers may leave without warning there is nothing to dissuade us from performing as muh omputation in eah job as possible. Choosing the appropriate job exeution time beomes a balaning at between keeping the omputation to ommuniation ratio high while not making the exeution times so long that large amounts of omputation are lost when a volunteer departs. 7. Experimental Validation Figures 3 through 5 ompare the performane of our atual yle stealing system with the performane predited by our omplex simulator, simple simulator and analytial model as the data size, exeution time and numbers of volunteers are varied respetively. esults per seond Variable task data size, volunteers, 1ms task time 1 Datasize Figure 3: Throughput vs Data Size (i/o) esults per seond esults per seond 1 1 Variable task exeution times, 3K Datasize, Task Exeution Time (ms) Figure :Throughput vs Job Exeution Time Variable, 3K tasksize, 1ms per Task Figure 5: Throughput vs Nr. of. Conlusions Our omplex simulator reasonably aurately predits the performane of the atual system. The simple simulator and analytial model predit well in most ases but underestimate performane when the server is heavily loaded. This is due to the lak of onurreny modelled in those systems. We are urrently investigating multi-server models that may help address these inauraies. eferenes [1] Wayne Kelly, Paul oe and Jiro Sumitomo, G: A Grid Middleware for Cyle Donation using.net, International Conferene on Parallel and Distributed Proessing Tehniques and Appliations, Las Vegas, June. [] Wayne Kelly and Paul oe, A Framework for Automati and Seure Cyle Stealing, International Conferene on High Performane Computing and Grid in Asia Paifi egion, July. [3] M. eiser and S.S.Lavenberg, Mean-Value Analysis of Closed Multihain Queuing Networks, Journal of the ACM, 7():313-3, April 19. [] P.J. Shweitzer, Approximate analysis of multilass losed networks of queues, Proeedings of the International Conferene on Stohasti Control and Optimization, 5-9, Amsterdam, Netherlands, Proeedings of the 5 11th International Conferene on Parallel and Distributed Systems (ICPADS'5)