Performance Improvement in a Multi Cluster using a Modified Scheduling and Global Memory Management with a Novel Load Balancing Mechanism

Transcription

1 Performane Improvement in a Multi Cluster using a Modified Sheduling and Global Memory Management with a Novel Load Balaning Mehanism P. Sammulal, PhD. Assistant Professor, Department of CSE, JNTUH College of Engineering, Jagtial, Andhra Pradesh, INDIA A. Vinaya Babu, PhD. Professor in CSE & Prinipal JNT University Hyderabad, Hyderabad, Andhra Pradesh, INDIA. ABSTRACT In Cluster Computing Environment the data lateny time has signifiant impat on the performane when the data is aessed aross lusters. In this ase, streamlining data aess through the usage of the memory management tehnique with a proper sheduling mehanism will improve the performane of the entire operation. Memory management beomes a prerequisite riterion while handling appliations that require large volume of data in various sientifi appliations. If memory management is not properly handled the performane will have a proportional degradation, even if the other fators perform to the maximum possible levels. Hene it is ritial to have a fine memory management tehnique. The existing sheduling algorithms onsider only data availability as the sole riterion in allotting an inoming job to a node in a luster. But this proess would not yield optimum performane beause bandwidth is also a major fator in determining the performane level. So to overome this problem a new sheduling algorithm is what required. Load balaning is a key tehnique used to improve the performane of luster appliation by utilizing mahines to the full extent without any idle or underutilized resoures. We have tested our CWA load balaning algorithm for fae reognition system and results are enouraging. Keywords High Performane Cluster Computing, Job Sheduling, Global Memory Management, Loal Memory Management, Distributed Shared Memory, Load balaning. 1. INTRODUCTION The first inspiration for luster omputing was developed in the 1960s by IBM as an alternative of linking large mainframes to provide a more ost effetive form of ommerial parallelism [1]. Cluster omputing an be desribed as a fusion of the fields of parallel, high-performane, distributed, and high availability omputing. In some sientifi appliation areas suh as high energy physis, bioinformatis, and remote sensing, we enounter huge amounts of data [2]. Managing suh huge amounts of data in a entralized manner is almost impossible due to extensively inreased data aess time. But in multi luster, the required data may be sattered aross several lusters. In this ase, streamlining data aess through the usage of the proposed memory management tehnique will improve the performane of the entire operation. Memory management beomes a prerequisite when handling appliations that require immense volume of data for e.g. satellite images used for remote sensing, defense purposes and sientifi appliations. If memory management is not properly handled the performane will have a proportional degradation. Hene it is ritial to have a fine memory management tehnique deployed to handle the stated senarios. Sheduling is a hallenging task in this ontext. The dataintensive nature of individual jobs means it an be important to take data loation into aount when determining job plaement. Despite the other fators whih ontribute performane in a luster omputing environment, optimizing memory management an improve, the overall performane of the same. To address this problem, we have defined a ombined memory management tehnique. To improve the performane of the appliation, there is a need to move data from one node to another among the lusters. It indiates that effiient data movement through the interonnetion of a luster an beome a major through. Dynami load balaning an solve the load imbalane problem of luster system and redue its ommuniation time period. In this paper, different new algorithms have been proposed one eah for Memory Management, Sheduling of an in oming job and Load Balaning among multi lusters. These methods assign weights for eah node to shedule an inoming job and then balaning the load dynamially using memory loality as the main fator. The main parameters onsidered in Load Balaning algorithm are partition size, CPU usage, memory usage and exeution time. 2. RELATED WORK Ann Chervenak et al. [3] review the priniples that they are following in developing a design for data grid arhiteture. Then, they desribe two basi servies that they believe are fundamental to the design of a data grid, namely, storage systems and metadata management. Kavitha Ranganathan and Ian Foster [4] desribe a simulation framework that they have developed to enable omparative studies of alternative dynami repliation strategies. They present preliminary results obtained with this simulator, in whih they evaluate the performane of five different repliation strategies for three different kinds of aess patterns. Kavitha Ranganathan and Ian Foster [5] develop a family of job sheduling and data movement (repliation) algorithms and use simulation studies to evaluate various ombinations and they desribe a sheduling framework that addresses the problems. William H. Bell et al. [6] find the design of the simulator OptorSim and Various repliation algorithms. Houda Lamehamedi et al. [7] introdue a set of repliation management servies and protools that offer high data availability, low bandwidth onsumption, inreased fault tolerane, and improved salability of the overall system. 15

2 Christine Morin [8] gives the design of Gobelins,a luster operating system, aiming at providing these two properties to parallel appliations based on the shared memory programming model and their experimentations are arried out on a luster of dual-proessor PC interonneted by a SCI high bandwidth network. Mihael R. Hines, Mark Lewandowski and Kartik Gopalan[9] address the problem of harnessing the distributed memory resoures in a luster to support memoryintensive high-performane appliations and they presented the arhitetural design and implementation aspets of Anemone - an Adaptive NEtwork MemOry Engine whih is designed to provide transparent, faulttolerant, low-lateny aess distributed memory resoures. KA-PO Chow and et.al. proposed a load balaning sheme alled COMET: Communiation Effiient Load Balaning Strategy for Multi agent luster omputing[10]. In this they used direted graphs, but they were onfined to multi agent appliations only. Alex K. Y. Cheung and Hans-Arno Jaobsen,[11] "Dynami Load Balaning in Distributed Content-based Publish/Subsribe", White papers on load balaning, University of Toronto, pages. 21, July Ch. Satyanarayana and et.al. proposed a new fae reognition system using PCA whih is assoiated to the appliation and aomplished a noteworthy performane development on a hronologial exeution [12]. 3. DESIGN OF COMBINED MEMORY MANAGEMENTTECHNIQUE 3.1 Global and Loal Memory The proposed memory management tehnique omprises global memory and loal memory. Global memory (Mg) is a distributed shared memory and is ommon for all the lusters whereas, loal memory (M l ) is speifi to the nodes of every individual luster. The Global memory (Mg) takes are of moderating data aessibility within and among lusters and its goal is to optimize the use of nodes memory for global performane. The Loal Memory takes are of data availability within the node and its goal is to optimize the use of node or workstation memory for loal performane. The global memory (Mg) onstitutes of persistent storage and temporary storage units [10]. In Global Memory Management, the data aess rate of all the data in the temporary memory and permanent memories are onsidered. The data whih is frequently aessed is stored in the persistent part and the less frequently aessed is stored in the temporary part. The goal of global memory management is to provide the best overall system performane by minimizing aess time for all jobs in the lusters. 3.2 The Sheduler and Memory Management While making the request, the user speifies the appropriate resoures, the estimated exeution time and the deadline. Then request is forwarded to the sheduler. The sheduler onsists of a resoure management system whih maintains details regarding the resoure availability and the resoure whih is under utilization. Global Memory Cluster i d 1, d 2, d 3 Cluster j d 8, d 9 Cluster k d 1 d 2 d 3 d 4 N i1 N i2 N i3 d 5 d 6, d 7 N j1 N j2 N j3 -- d 8 d 10 N k1 -- d 9 N k2 N k3 Job J i d1 d2 d3 d5 d6 d8 d9 Fig. 1 Example for Proposed approah for seletion of Cluster 16

3 The information is updated periodially and / or after the ompletion of running requests by the resoure management system. The sheduler after the reeption of a new request makes an analysis to identify a partiular luster to whih the request an be forwarded. The sheduler primarily takes in to onsideration about the load of the proessors of the nodes in the onerned lusters before the task is assigned. But this proess of designating lusters for proessing tasks would not yield optimum performane beause bandwidth is also a major fator in determining the performane levels. So to overome this problem a new algorithm is proposed using global memory and loal memory. The onventional sheduling algorithm blindly fixes a partiular luster taking into aount the availability of data as the sole riterion and it would lead to performane degradation. To illustrate the above senario onsider a partiular request that requires a ertain luster that is identified for the given request whih is based on the presene of major portion of required data and the ost for aessing remaining data is not onsidered. At the same time, if the task is designated to a luster irrespetive of the perentage of data present in that luster and onsidering the ost of aessing the remaining data from the rest of the lusters through global memory the performane an be optimized further. We have proposed a new sheduling algorithm that also gives the needed importane to the ost of aessing data as shown in fig Sheduling Algorithm Assumptions a) SD Mg is a set of data segments to be transferred from SS C to global memory Mg. b) SS C is the set of lusters having the data segments in global memory SD Mg that are required by the job J i. ) SS Ci is the luster C i having data segments in the global memory SD Mg that are required by the job J i. d) SD WC is a set of data segments available within the onerned luster C Ji. e) SQ C is the information of all the lusters update between lusters in the system For data segments within a Cluster: For eah data segments in SD Mg For eah luster in SS C t = Calulated time to transfer data segments from SS Ci through Mg t min = min (t) Update SQ C The total time needed to transfer the files between the nodes within the luster is alulated as follows. sizeof ( SD WC ) T WC t i0 min For data segments between Clusters: For eah data segments in SD Mg For eah luster in SS C t = Calulated time to transfer data segments from SS Ci through Mg t min = min (t) Update SQ C The total time needed to transfer the data segments between the lusters through the global memory Mg is alulated as follows. T BC sizeof ( SD Mg ) i0 t min The total time needed to transfer the data segments required by the job is alulated as follows. T T WC T BC Repeat the above steps for all the lusters S T T Q ( T0, T1, T2... TNC min(s T ) The luster with minimum time is hosen to allot the job. At regular intervals the data aess times in Global Memory is analyzed. If the data stored in temporary portion has been aessed more frequently then it is shifted to the permanent storage part. 4. GLOBAL AND LOCAL MEMORY MANAGEMENT TECHNIQUE 4.1 Memory management of nodes within a luster Data is normally transferred between nodes and lusters when it is not found in a partiular node. When the data is available in the same node it uses its loal memory. But, within the luster and among the lusters, all the data is transferred through the temporary memory in the global memory where permanent memory ontains the data that is aessed more frequently. Assume that the data aess rate of eah data segment is maintained in a vetor V DAR by every node in a luster. There is a threshold value (Thresh) for the size of the node memory. Using V DAR vetor the used memory of the node an be alulated. If the used memory of a node exeeds threshold value, then the data segments whih are not frequently aessed is to be removed from that node. Eah node in a luster performs the following steps: (1) Sort the vetor V DAR in asing order (2) Retrieve the V DAR of all the remaining nodes within that luster (3) Find the non repliated data in the node and remove that data from the vetor S MN is the size of the node memory within the luster i.e. number of data segments residing in the onerned node. Remove the data segment from the node with in the luster S MN = size of all the remaining data segments in that node if (S MN < (Thresh - θ)) Perform the above algorithm for the nodes of all the lusters in the Cluster Computing Environment Memory Management of Global Memory The global memory onsists of both permanent memory and temporary memory. All the data segments are transferred between the lusters through the temporary memory in the global memory. Assume that the data aess rate of all the data segments in the temporary memory and permanent memory are maintained in a vetor. If a data in temporary memory is more frequently aessed, it must be transferred from temporary memory to permanent memory. After transferring the data, they must be removed from temporary memory. ) 17

4 4.2.1 Temporary Memory Management: SDA R DA sorted in desing order. R P M, T Permanent and Temporary Memories M S TM is the size of the temporary memory i.e. number of data segments (amount of data) residing in the temporary memory. for eah value of SDA R Move thedata segment fromt to S TM =size of all the data segments in the temporary memory If ( STM ( Thresh )) M P M Permanent Memory Management: If the used memory of the permanent memory exeeds a predefined threshold value, the data segments whih are not frequently aessed are removed from permanent memory. Assumptions: DA R Aess rate of all the data segments in the P M SDA R DA R sorted in asing order. S PM is the size of the permanent memory i.e. number of data segments (amount of data) in the permanent memory. for eah value of SDA R Remove data segment from permanent memory S PM =memory of all the remaining data segments in the permanent memory If ( SPM ( Thresh )) 5. EFFICIENT LOAD BALANCING IN FACE RECOGNITION SYSTEM The main fous is on the load distribution and load balaning aspets of implementing a omplex fae reognition system using PCA whih is an image proessing appliation. Fae reognition system has pratial and potential appliation in areas suh as seurity systems, riminal identifiation, video telephony, mediine, and so on. In the proposed system, the image database is deomposed into several sub databases, eah plaed in individual nodes of all lusters in a multi luster environment. Eah node alulates the features of all images of the sub database present in that node. When an input query image is given, its features are extrated and repliated to all the individual nodes in the different lusters. The features of the query image are ompared against the features of the images in the sub databases. The results of the nearest mathing images are bak propagated to the master node in the multi luster. The algorithm proposed for this appliation onsiders the load from eah node, the node with minimum load is seleted to make that node as load manger. The load manager is responsible for the distribution of image database and to all nodes. The Cluster Weightage Alloation algorithm gives minimum and maximum loaded nodes in the proedure. The proposed load balaning algorithm for fae reognition is explained in setion A Algorithmi steps for Load Balaning Sheme The assumptions of sheduling algorithm for this system are as follows: SN WC : Set of nodes within the luster SP WN : Set of proesses running on the node N NL i : The load on the node i SQN WC : The information of all nodes at the load manager A, C : Vetor of available lusters, Current luster For eah C luster in A For eah node in SN WC For eah Proess in SP WC M=alulate the load of a node based on the proposed Cluster Weightage Alloation (CWA) algorithm NL i =NL i + M Update SQN WC NL min = min {NL i } NL heavy =max {NL i } Here NL min and NL heavy are the nodes with minimum and maximum load respetively. And the load information of all nodes is olleted by the load manager. This information is used for distribution and balane of the load and the threads are to be migrated from overloaded nodes to the under loaded nodes dynamially. If a node fails in the middle of proessing, then the load manager finds the failure node and it takes the responsibility to share the load aross the luster with very less performane degradation. The CWA algorithm is explained in setion B. The new load J L i.e. fae database has to be distributed to every node in the multi luster environment. After distribution of fae data base, the load of eah node is alulated as K= ( (LC ij ) + J L )/N The distribution of load on eah node is done by the algorithm given below For eah C luster in A For eah node in SN w If (LC ij <k) LC ij =LC ij + (K - LC ij ) Where: L ij Load on i th luster, j th node J L LC ij load on all the lusters.where 1<=i<=n and 1<=j<=N N number of nodes in multi-lusters SN WC set of nodes having the SF WC in Mg within Cluster SF WC set of files available with in the onerned luster 5.2. Cluster Weightage Alloation (CWA) Algorithm The proposed algorithm is mainly defined to alloate the jobs in the orresponding luster with minimum memory utilization (µ lm) to balane the load [10]. To illustrate the above senario, let us onsider a number of available lusters with many nodes. The main problem is how to alloate the inoming job to the orresponding luster with immediate exeution as well as the minimum load. Analyze the µ lm of every luster and sort it in asing order. The luster having least µ lm has faster time than the other is hosen initially to allot the job. If the job is alloated to the partiular node, it should always be notied 18

5 that the load balanes eah luster. The total t e of eah node in a luster is evaluated for assigning the job to the equivalent luster. The weightage of eah node in a luster is alulated on the basis of time for whih the job is alloated to the node whih is having maximum weightage. The luster weightage alloation algorithm is learly explained as follows: Algorithm for Weightage alulations of Clusters: Set µ = 0, s=0; for eah C C luster in A C s = size (µ lmn ) for eah memory utilization in µ lmn µ m = µ + ; for For eah CPU utilization in µ lmn µ l = µ + ; for µ lmn= µ m+ µ 1; C[i]= µ lmn /s; for SA C = sort (A C using C) de; for eah index i of SA C W C [i] = (i+1) / N C ; for Where: A C Vetor of all available lusters N C, C C No of available lusters, Current luster W C Weightage of all lusters C Vetor of total available memory utilization from all available lusters µ lmn Memory utilization of eah node in a luster In the above pseudo ode, the CPU and memory usage of eah node in the available lusters are sorted and stored it in SA C and the weightage of eah luster is alulated Algorithm to selet a job based on Partition size: for i=1 to N for eah node in N N µ=µ+ µ lmn [j] C A =µ/s p µ lmsj =Sort (µ lmj ) As t sj = t j (µ lmsj [i]) for i = 0 to N J -1 j = j + µ lmsj [i] if j >=C A then Index=urrent index of µ jp for i=0 to index µ lmjp [i] = µ lmsj [i] t jp [i] = t sj [i] Where: N C Number of available lusters N N Number of nodes in eah luster t n Total exeution time of eah node t r Remaining time of eah node t ne Elapsed time of eah node µ lmn Memory utilization of eah node in a luster N j, S P Number of jobs, Partition size µ lmj Memory needed in job utilization t j Time required in job exeution µ lmj Memory utilization of Current job C A Available memory utilization µ lmsj Memory utilization of sorted jobs µ lmjp Memory utilization of job s partition size µ lmsn Memory utilization of sorted nodes In the above pseudo ode, the job s CPU usage and the Exeution time is seleted by one-third of the jobs in the partition size. for j=1 to N for eah node i in N N Nj t r i ( t k t k ) k 0 n t sr =Sort (t r ) As µ lmsn [i] = µ lmn (t sr [i]) for eah memory utilization in µ lmsn µ=µ+µ lmsn [i] if µ>µ lmj then Index = urrent memory utilization of µ lmsn for eah remaining time in t sr t = t + t sr [i] If i++ > index then Break From this, the luster with minimum time is hosen to allot the job. Then the weightage of eah node is alulated with the help of the memory usage. The formula for finding the weightage of eah node is given as, ne 1 ( lmn [ i]/100) W The weightage is alulated for eah node in a luster and the seleted job is allotted to the node whih is having maximum weightage. In order to hek the balanes of eah luster, we have used to hek with the following onditions. N J 2N J J J i J i 1 N N Where J is the number of jobs given to the luster. 6. RESULTS The algorithms are implemented and tested on PelianHPC Operating System. The luster and the file information are stored in the tables of a database. The data base is designed with MySQL tool. It uses three tables to store the information of all the data segments and lusters. There are three lusters in a multi-luster arhiteture, they are {C i, C j, C k } and the data segments that reside on system are {d 1, d 2, d 3 d 10 } as shown in fig 1. Let the distribution of these data segments be C i ontaining {d 1, d 2, d 3, d 4 }, Cj ontaining {d 5, d 6, d 7 } and C k ontaining {d 8, d 9, d 10 }. The onventional algorithm blindly assigns the job to the luster C i as this luster ontains more number of data segments as shown in fig.1. Before the atual experiment, a large database has been reated whih inludes the data segments with various sizes and omputed the data aess times by transferring the data segments among the nodes in a multi-luster environment, onsidering average 19

6 network traffi. When a job is assigned to a partiular node in a luster its orresponding data segments may be present in other nodes of the same or remote lusters. The size of eah data segment present in a node is ompared with those data segment sizes that are in the database to get the nearest aess time. Based on these aess times, the total aess times have been alulated for eah node of a luster. The data segment is moved in terms of data hunks. Data hunk is a maximum transmission unit. The size of a data hunk is fixed and varies from one file system to another. The numbers of data hunks for any data segment an be alulated by taking eil value of the ratio of data segment size in kb and data hunk size. The transfer time of data segment is the time required to transfer all its data hunks. For example data segment d4 of size 438 kb requires 4 data hunks and its transfer time is the time required to transfer 4 data hunks. If we onsider the link bandwidth of 1Gbps and data hunk size of 128KB for NTFS then the minimum time required for transferring 128KB segment is 1048 miro seonds. The proposed sheduling algorithm first alulates the minimum aess times taken for the data segments within the lusters (from data base tables) by eah node in that luster. If a data segment resides on a node then the aess time for that data segment by that node is treated as zero. Similarly, the aess times are alulated for all the data segments in eah luster. The aess times between the nodes of the luster C i is shown in table I. Table I: The Aess Times Between eah Pair of Nodes In Cluster C i. C i d1 d2 d3 D4 NT i1-i NT i2-i NT i1-i Here NT i1-i2 represents the aess time for a data segment from node N i1 to node N i2. In our experiment luster C i, the node N i1 ontains the data segments d 1, d 2, node N i2 ontains d 3 and node N i3 ontains the data segment d 4. The luster C i ontains the data segments d 1, d 2, d 3 whih are required by job J i. So here the set SD WC is {d 1, d 2, d 3 }. Now the total aess times for eah node in luster C i is as follows: t (N i1 ) = = 1053 µse t (N i2 ) = =5263 µse t (N i3 ) = =6406 µse From the above values the minimum aess time is 1053 µse from N i1 and this is T WC for luster C i. The information in table II shows the aess times of the eah pair of nodes N j1, N j2 and N j3. And these aess times are only for data segments that reside on luster C j. The luster C j ontains the data segments d 5, d 6 whih are required by job J i. So here the set SD WC is {d 5, d 6 }. Table II: The aess times between eah pair of nodes in luster C j. C j d5 d6 d7 NT j1-j NT j2-j NT j1-j Now the total aess times for eah node in luster C j is as follows: t (N j1 ) = = 4947 µse t (N j2 ) = = 7346 µse t (N j3 ) = =12354 µse From the above values the minimum aess time is 4947 µse from N j2 and this is T WC for luster C j. Table III: The aess times between eah pair of nodes in luster C k. C k d8 d9 d10 NT k1-k NT k2-k NT k1-k The information in table III shows the aess times of the eah pair of nodes N k1, N k2 and N k3. The luster C k ontains data segments d 8, d 9 whih are required by job J i. So SD WC is {d 8, d 9 }. Now the total aess times for eah node in luster C k is as follows: t (N k1 ) = = 3528 µse t (N k2 ) = =9713 µse t (N k3 ) = = 6396 µse From the above values the minimum aess time is 3528 µse from N k1 and this is T WC for luster C k. In the seond part of algorithm, for eah luster, the minimum of the aess times for the data segments that reside on other lusters is alulated. Table IV: The aess times of data segment between eah pair of lusters. CT ij CT jk CT ik d d d d d d d d d d The aess times of all data segments for eah pair of lusters are given in Table IV. Here in table CT ij represents the aess time between luster C i and luster C j. CT jk and CT ik represent the aess times between remaining pair of lusters (C j, C k) and (C i, C k ) respetively. The data segments are transferred in between nodes aross the remote luster using global memory. The aess times between different lusters are shown as follows: T BC (C i ) = =22438 µse T BC (C j ) = = µse T BC (C k ) = = µse To designate a luster the algorithm uses the summation of T BC and T WC as the parameter. The minimum of these are onsidered to assign the job. T (C i ) = = µse T (C j ) = = µse T (C k ) = = µse In our experiment N i1 has more data segments ompare to all other nodes. So existing sheduling algorithm assigns the job j i to N i1 node in the luster C i. As per the above alulations of data aess times, it takes µse to gather other segments from the remote lusters. Based on the proposed sheduling algorithm it is lear that minimum of these total aess times is for luster C j that designated for the job J i as shown in fig. 1. The luster C j takes µse to gather other segments from 20

7 Speed Up Speed Up Transfer Times(µse) Exeution Time International Journal of Computer Appliations ( ) the remote lusters. The proposed sheduling algorithm assign the job j i to node N j2 in luster C j. Thus the proposed algorithm shedules the jobs to the lusters onsidering the data aess times as the key fator. The fig. 2 shows the transfer time between nodes of various data segments sizes from 0kb to 512kb between the exeution times of sequential and proposed system is relatively small Sequential On Cluster Series Work Load Data Size in KB Fig. 2 Transfer time between nodes of various data segment sizes from 0kb to 512kb. A snapshot of data aess times between the lusters and within the luster is illustrated in fig. 3. Fig.4 Exeution time of Appliation in sequential and on luster environment with 6 nodes. Beyond 600 images the differene between these two is notieable, with 2000 images; the differene between the exeution times is very high. At heavy workloads, the luster is performing very well than the other approahes. The appliation is tested on the PelianHPC luster under the proposed system varying number of nodes and with workload of 2000 images Number of Nodes Fig.5 Relative Speed up for the orresponding number of nodes. Fig. 3 snapshot of proposed sheduling algorithm. Among three lusters, luster C j has minimum total aess time and the job is alloated to the luster C j. Several experiments have been performed on PelianHPC. First the fae reognition appliation is exeuted on a stand alone system of onfiguration Pentium IV with 512MB memory. Fig.4 shows the evaluation of the exeution. At eah time, the work load i.e., the number of images, for whih the Eigen faes are to be alulated, is inreasing and the orresponding exeution time has to be onsidered. The exeution time for 200 images is relatively small i.e., 3.8 se. The load of images is inreased with regular interval and tested. Beyond 2000 images the system s response time is very high and the appliation is exeuted up to 2000 images. For heavy work load, the exeution time is 24.8 se whih is very high. The same appliation is exeuted on desribed luster under the proposed system, varying work load as before. The results have shown that, under the small work load the differene At eah time a node is onneted to the luster, and the speedup is alulated under the same workload, the results have shown that the luster with 6 nodes exhibits high speedup in the response time as in fig.5. When there are many nodes, the load management overhead should be onsidered. With two nodes the exeution shows a speed up of 1.5 to 2.5, for six nodes it has shown speedup 3 in exeution as shown in fig Clusters Fig.6 Relative Speed up for the orresponding number of lusters. The effiient distribution at initial step, improved the performane. There is a threshold on the number of nodes 21

8 aording to the appliation, beause the overhead onerned with the load management and monitoring ould inrease with the number of nodes in the luster, whih in turn shows impat on the performane. The same appliation has been tested on multi luster having three lusters with three nodes eah. With single luster the speed up is 2, the 2 nd luster it is 2.8 and for 3 rd luster the speed up is 3.2 shown in the bellow fig CONCLUSION The Proposed Sheduling Algorithm onsiders data aess times as the main fator to deide a luster or node on whih that partiular request is to be served. Experimental results show a substantial improvement in data aess time aross the lusters by reduing the exeution time. Providing due onsideration to data aess lateny besides omputational apability proves worthwhile. The proposed, Memory Management Tehnique uses a ombination of Loal Memory and Global Memory that buffers data, based on aess patterns. The global Memory takes are of moderating data aessibility aross nodes within the lusters and among the lusters. The Loal Memory takes are of data availability within the node and its goal is to optimize the use of node or workstation memory for loal performane. The Global Memory is bifurated into a Temporary Storage part and a Persistent Storage part. Data plaed in these two parts is updated dynamially and at regular intervals, based on the pattern of usage. The onept omes handy when the data to be aessed from another luster is markedly time taking than the same data aessed through the Global Memory. An effiient Dynami Load Balaning Mehanism has been developed with a new load metri that onsiders Partition Size, CPU usage, Memory usage and Exeution time of jobs as main fators to deide the load of eah workstation. These fators are optimized by proposed Cluster Weightage Alloation Algorithm. Based on the CPU and Memory usage, CWA assigns weights to eah node in the lusters whih in turn helps in assigning a new inoming job. CWA also onsiders the remaining time at eah luster in assigning weights whih balanes the load aross the lusters. And using the number of page faults as parameter to represent memory loality for effiient proess migration from heavily loaded to low loaded node will show the optimum performane. A number of tests were performed on different senarios and from these results we an onlude that the ombination of the proposed Sheduling Algorithm, Memory Management Algorithm and Load Balaning Algorithm exhibit better performane than traditional shemes. The task distribution and load balaning of a omplex appliation inreases the speedup in the exeution time and also redues the response time. The fae reognition system using PCA is the image proessing appliation whih undergoes many more omputations and also heavy I/O, memory intensive loads. The exeution of this appliation using Load Balaning Algorithm on a pelianhpc luster has shown muh improvement in speedup in the exeution rather than sequential exeution. 7. REFERENCES [1] R. Buyya (ed.), High Performane Cluster Computing: Arhitetures and Systems, vol. 1, Prentie Hall, [2] Wolfgang Hosehek, Franiso Javier Jaen-Martinez, Asad Samar, Heinz Stokinger, and Kurt Stokinger. Data Management in an International Data Grid Projet, Proeedings of First IEEE/ACM International Workshop on Grid Computing (Grid 2000), Vol. 1971, pages Bangalore, India, Deember [3] A.Chervenak, I. Foster, C, Kesselman, C. Salibury, and S. Tueke, The Data Grid: Towards an Arhiteture for Distributed Management and Analysis of Large Sientifi Datasets, Journal of Network and Computer Appliations, vol.23, Pages ,2000. [4] I. Foster, and K. Ranganathan, Identifying Dynami Repliation Strategies for High Performane Data Grids, Proeedings of 3rd IEEE/ACM International Workshop on Grid Computing, vol of Leturer Notes on Computer Siene, Pages 75-86, Denver, USA, November [5] I. Foster, and K. Ranganathan, Deoupling Computation and Data Sheduling in Distributed Data-intensive Appliations, Proeedings of the 11th IEEE International Symposium on High Performane Distributed Computing (HPDC-11), IEEE CS Press, Pages , Edinburgh, U.K., July [6] W. H. Bell, D.G. Cameron, L. Capozza, P. Millar, K. Stokinger, and F. Zini, Simulation of Dymani Grid Repliation Strategies in OptorSim, Proeedings of the Third ACM/IEEE International Workshop on Grid Computing (Grid2002), Baltimore, USA, vol of Leture notes in Computer Siene, Pages 46-57, November [7] E. Deelman, H. Lamehaedi, B. Szymanski, and S. Zujun, Data Repliation Strategies in Grid Environments, Proeedings of 5th International Conferene on Algorithms and Arhiteture for Parallel Proessing (ICA3PP 2002), IEEE Computer Siene Press, Pages , Bejing, China, Otober [8] Christine Morin, Global and Integrated Proessor, Memory and Disk Management in a Cluster of SMP's in IRISA/INRIA Campus universitaire de Beaulieu, Rennes edex (FRANCE) [9] Mihael R. Hines, Mark Lewandowski and Kartik Gopalan, Anemone: Adaptive Network Memory Engine in proeedings of the twentieth ACM symposium on Operating systems priniples Brighton, United Kingdom Year of Publiation: [10] KA-PO CHOW, YU-KWONG KWOK1, HAI JIN, AND KAI HWANG, Comet: A Communiation-Effiient Load Balaning Strategy for Multi-Agent Cluster Computing. [11] Alex K. Y. Cheung and Hans-Arno Jaobsen, "Dynami Load Balaning in Distributed Content-based Publish/Subsribe", White papers on load balaning, University of Toronto, pages. 21, July [12] Ch.Satyanarayana, D.Haritha, P.Sammulal and L.pratap Reddy Updation of faespae for Fae Reognition using PCA, in the proeedings of IEEE international onferene on RF and Signal Proessing Systems, RSPS onduted in 1 st -2 nd February at KLCE, Vijayawada.AP