Dependable and Efficient Scheduling Model with Fault Tolerance Service for Grid Applications

Transcription

1 Dependable and Efficient Scheduling Model with Fault Tolerance Service for Grid Applications C. Suhasini, B. DineshKumar Reddy, Sathyalakshmi and Roberts Masillamani Abstract Owing to uncertainty of the grid resource and service, rational grid resource scheduling has been becoming a difficult problem at all times. In this paper, grid resource scheduling problem is researched based on grid trust model and availability model. Moreover, a high dependability grid resource scheduling algorithm based on credibility and availability with Fault Tolerance is proposed. We use checkpoint mechanism for fault tolerance in this paper. The proposed approach effectively schedule the grid jobs with fault tolerant way. Also, it increases the percentage of jobs completed within specified deadline and making the grid trustworthy. Keywords Grid Resource, Credibility, Availability, Grid Job Scheduling, Checkpoint, Fault Tolerance. G I. INTRODUCTION RID computing or the use of a computational grid, is applying the resources of many computers in a network to a single problem at the same time usually to a scientific or technical problem that requires a great number of computer processing cycles or access to large amounts of data. Compared to other distributed environments, such as clusters, complexity of grid mainly originates from decentralized management and resource heterogeneity. These characteristics often lead to strong variations in availability, which in particular depends on resource and network failure rates, administrative policies, and fluctuations in system load. Apparently, runtime changes in system C.Suhasini, is with School of computer Science, Hindustan University Chennai, INDIA grhasini@gmail.com B.DineshKumar Reddy, is with School of computer Science, Hindustan University Chennai, INDIA bdineshd@gmail.com, Sathyalakshmi is with School of computer Science, Hindustan University Chennai, INDIA slakshmis@hindustanuniv.ac.in Roberts Masillamani is with School of computer Science, Hindustan University Chennai, INDIA deancs@hindustanuniv.ac.in availability can significantly affect job execution. Since for a large group of time-critical or time consuming jobs delay and loss are not acceptable, fault tolerance should be taken into account. Providing fault tolerance in a distributed environment, while optimizing resource utilization and job execution times, is a challenging task. The number of dynamic resources in the grid system increases continuously, so fault tolerance becomes a critical property for applications running on these resources. However, in traditional implementations, when a failure occurs, the whole application is shutdown and has to be restarted from the beginning. A technique to avoid restarting of the application from the beginning is rollback recovery which is based on the concept of checkpoint. Checkpoint mechanism is used to reduce the limitations imposed by the high volatility of resources.it periodically saves the application s state to stable storage. So, whenever a failure interrupts a volunteer computation, the application can be resumed from the last stable checkpoint. Some of the existing fault tolerance and recovery mechanisms like checkpointrecovery and over provisioning. Checkpoint-recovery techniques make it possible for the job to resume execution from the last checkpoint instead of restarting from the beginning, whenever a failure occurs. Over provisioning techniques replicate a job in more than one resource to increase the probability of successful execution. Although these techniques address the reliability challenges to some extent, no large-scale study has been done on how effective they are when coupled with scheduling. scheduling methods, the system can get better performance, as well as applications can avoid unnecessary delays. The emergence of grid computing further increases the importance of fault tolerance. Grid computing will impose a number of unique new conceptual and technical challenges to fault-tolerance researchers. 9

2 Some of the factors due to which the probability of faults in a grid environment is much higher than a traditional distributed system are lack of centralized environment, predominant execution of long jobs, highly dynamic resource availability, diverse geographical distribution of resources and heterogeneous nature of grid resources. Thus, fault tolerance related features must be incorporated in grid job scheduling to improve the performance of the grid system. The number of dynamic resources in the grid system increases continuously, so fault tolerance becomes a critical property for applications running on these resources. However, in traditional implementations, when a failure occurs, the whole application is shutdown and has to be restarted from the beginning. II. PROBLEM DEFINITION Suppose Q is a problem, all possible solutions are representing using a set A, set A is called the solution set of the problem. Considering A = (T,-T), where T represents a grid node to another grid node was not only credible but also available and -T represents a node set which a grid node to another grid node wasn t entirely credible and available, is a set of grid nodes i Grid computing has emerged as the nextgeneration parallel and distributed computing methodology that aggregates dispersed heterogeneous resources for solving various kinds of large-scale parallel applications in science, engineering and commerce [6]. A Grid enables sharing, selection, and aggregation of a wide variety of geographically distributed resources including supercomputers, storage systems, data sources and specialized devices owned by different organizations. Management of these resources is an important infrastructure in the grid computing environment. It becomes complex as the resources are geographically distributed, heterogeneous in nature, owned by different individual or organizations with their own policies, have different access models, and have dynamically varying loads and availability. To achieve the promising potentials of computational grids, an effective and efficient scheduling system is fundamentally important. A grid becomes useful and meaningful when it both encompasses a large set of resources and serves a sizable community. The different phases of grid scheduling process have been discussed [14]. In grid, the abilities of the computing resources vary, and jobs often arrive dynamically. Because of these, scheduling methods of parallel computing [4] may not be applicable in a grid. Through good in line with the requirements of Solutions. For example, a grid node has high credibility, but it is not available, or a node has high availability, but it isn t credible, such nodes do not meet the requirement nodes. For the credibility and availability of the grid nodes, we adopted an integrated scheduling function IS(i) to evaluate, this integrated scheduling function is evaluated via two functions of the credibility function R(i) and availability function U(i). IS(i) is defined as formula (1): Is(i)=λ R(i) + γ U(i) (1) The λ and γ are particularly the weights of credibility and availability, the sum of the λ and γ is 1, the size of λ and γ is determined by the extent of our requirements for the credibility and availability. If there is a low requirement level for credibility, then we can take the value of λ smaller; if higher requirement of the credibility is needed, then the value of λ will be set bigger, we could have a flexible set. Definition 1. The credibility is an integrity assessment of grid node status and behaviors characterized by the value of credibility. It is to make each other's satisfaction degree evaluation of the trusted status after interacting information between the nodes. We define a triple R(i)=( I(i), D(i), S(i) ) to indicate the credibility of the node. I(i), D(i), S(i) are respectively called the identity satisfaction degree, direct satisfaction degree and indirect satisfaction degree of the service request of the call node i to the node i. Definition 2.The identity satisfaction degree denotes the true identity, the owned authority, or the delegation of authority in line with the requirements of its own pre-set that the call node evaluates the objective nodes, and its value is 1 or 0, which means satisfied or dissatisfied. For example, there are data collection nodes, data analysis nodes, alarm aggregation nodes and there source scheduling nodes and so on in the grid intrusion detection system. If we want to conduct data analysis, we need to call data analysis node, if the called node isn t a data analysis node, it does not possess the required identification and authorization, then its value of identity satisfaction degree will be set to 0; if the identity is suitable, then its value of identity satisfaction degree will be set to 1. If the value of identity satisfaction degree is 0, we consider the node is not credible, the value of credibility function directly to beset to 0. If the value of identity 10

3 satisfaction degree is 1, then we consider its direct satisfaction degree and indirect satisfaction degree, and come to the credibility function R(I). R(i) is defined as formula (2): R(i) = α D(i) + β S(i) I(i)=1 (2) 0 I(i)=0 The α and β are the weights of direct satisfaction degree and indirect satisfaction degree, the sum of α and β is 1, the size of α and β is determined by our trust extent of direct satisfaction and indirect satisfaction. Generally speaking, we have higher confidence for direct satisfaction degree; therefore, we usually set the value of α larger, while the value of β smaller. Definition 3.Direct satisfaction degree is a direct evaluation of trusted status of the other node after interacting with each other, and comes from the historical interaction record; its value is the ratio of the satisfaction times of interactions and the total times of interactions. If the interaction each other is successful, then the satisfied times plus one. If it fails, then the dissatisfied times plus one. Suppose the interaction node set with node Pi is P=( Pj, j=1,2,3 m) in a certain fixed period of time recently, the total times of historical interaction are Sj between node Pi and node Pj, the satisfied times are Dsj, thus, we define direct satisfaction degree function D(i) of node Pi as formula (3): (3) Definition 4. Indirect satisfaction degree is an indirect evaluation of trusted status of the indirect interacting node to this node, and comes from the indirect historical interaction records, its value is the ratio of the satisfaction times and the total times of indirect evaluation. For any node Pi, Suppose the indirect evaluation node set with node Pi is P=(Pj, j=1,2,3 m) in acertain fixed period of time recently, The satisfied time of the node Pi evaluating node Pjis Dtj, the total time of evaluation is Tj, thus, we define indirect satisfaction degree function S(i) of node Pi as formula (4): (4) Definition 5. Through Rule, direct satisfaction degree function with the weights (3) and indirect satisfaction degree function (4) drag-in formula (2), and get the credibility function R(i) as formula (5): R(i)= α 1 0 I( i)=0 (5) Definition 6.Resource availability function U(i) is defined as the characteristics of resource performance and resource busy degree characterization. It is decided by resource performance P(i),resource busy degree B(i) and resource availability benchmarks Resource performance P(i) is defined as formula (6): (6) Among them, P(i) is the resource performance of the No. i grid node; Rijis the No. j resource attribute oft the No. i grid node, such as the size of the CPU frequency, memory size, hard disk size, etc. Aj is the weight of the No. j resource attribute, which shows the degree of the designer s emphasis on each attribute. Once the grid nodes are identified, P(i) generally does not need to change. The degree of resource busy is defined as the formula (7) B(i) ] W j (0,1) (6) Therein, B(i) is the busy degree of the No. i grid node, Uijis the usage of No. j category resource of the No. i grid node, such as CPU, memory usage, etc. Tijis the total amount of No. j category resource of theno. i grid node, W j represent the weight of resource usage of No. j category resource. B(i) is constantly changing over time, so the scheduling algorithm needs to periodically check the use of various resources of the grid nodes, which can be obtained by DCS (data collection system ) that developed through laboratory oneself. Therefore, the function of resources availability is defined as the formula (8): U(i)= 11

4 (7) Definition 7. The function of the credibility (5) and the function of the availability (8) drag-in formula (1)is integrated scheduling function III. GRID RESOURCE INTEGRATED SCHEDULING BASED ON CREDIBILITY AND AVAILABILITY WITH FAULT TOLERANCE In this paper, the basic ideas of high dependability grid resource integrated scheduling algorithm based on availability and reliability With fault tolerance are: there exists an integrated trust relationship between service requestor and service providers. The integrated trust relationship is determined by the availability function U and credibility function R. The availability function is determined by the performance of the grid resource and used to determine the strength of processing ability of grid resource. The credibility function is evaluated by a triad (identity satisfaction degree, direct satisfaction degree, indirect satisfaction degree) and used to determine the extent of grid resources trusted. High credibility shows that the average trust efficiency of the grid nodes is high and the failure rate of the task implementation is low, able to meet the security requirements of task executing. Thus, the bigger of integrated scheduling function value, the more stable of the service execution after scheduling, the more credible of the results of the execution. At the same time scheduling algorithm can guarantee not only demand i.e., during job/resource failure. For a Fig. 1 The basic structure of grid scheduling algorithm with fault tolerance 12

5 Particular job, the checkpoint server discards the result of the previous checkpoint when a new value of checkpoint result is received.fault tolerance manager is included to make the grid work more efficient. According to the integrated scheduling algorithm analysis, the overall objective of grid integrated scheduling algorithm is to make assignment optimization of service requests and grid nodes. It not only ensures the maximizing of the implementation of security operations but also takes into account the efficiency of the implementation. A. Checkpoint Manager It receives the scheduled job from the scheduler and sets checkpoint based on the failure rate of there source on which it is scheduled. Then it submits the job to the resource. Checkpoint manager receives job completion message or job failure message from the grid resource and responds to that accordingly. during execution, if job failure occurs, the job is rescheduled from the last checkpoint instead of running from the scratch. Checkpoint manager implements checkpoint setter algorithm to set job checkpoints. B. Checkpoint Server On each checkpoint set by the checkpoint manager, job status is reported to the checkpoint server. Checkpoint server save the job status and return it on C. Fault Tolerance Manager Fault Tolerance Manager maintains the fault index value of each resource which indicates the failure rate of there source. The fault index of a grid resource is incremented every time the resource does not complete the assigned job within the deadline and also on resource failure. The fault index of a resource is decremented whenever the resource completes the assigned job within the deadline. Fault index manager updates the fault index of a grid resource using fault index update algorithm. Algorithm: Integrated scheduling algorithm (Integrate) Input: service requests T, a set of the grid nodes G. Output: The corresponding table Map of service request and the assignment of the grid nodes. 1) Initialization. Set the service request set T, a set of the grid nodes G, the weight λ of the credibility, the weight γ of the availability, initialize the value of comprehensive scheduling list of R, etc. 2) Repeat. 3) For each request, it is to find out the host node element of the service, namely, to generate a mapping table of service request. 4) For all grid nodes Gj except for Gi. If the identity satisfaction degree of the node Gi to all nodes is 0 Then T-Ti // Task Ti can t assign the nodes, to the exclusion of the service request dispatching set. Else For all nodes, to calculate the integrated scheduling function value, if identity satisfaction degree of the node isn t 0, then it adds the value to the list of integrated scheduling. 5) To descending the value of the integrated scheduler generates grid nodes resource scheduling priority queue. 6) The Ti will be assigned to the head node of integrated scheduling priority queue. 7) T-Ti 8) Until the task queue T empty. Algorithm: Check point mechanism. 1. IF checkpoint manager receives the job completion result from resource THEN (i) IF resource fault index >= 1 THEN Send a message to fault tolerance manager to decrement the fault index of resource that completes the assigned job. Send details of the finished job to the scheduler. (ii) GOTO Step3 2. IF checkpoint manager receives the job failure message from resource THEN (i) Send a message to fault tolerance manager to increment the fault index of resource that fails to complete the assigned job. (ii) Send a message to checkpoint server, whether there is any checkpoint result of this job. (iii) IF checkpoint result of the job exists in checkpoint server THEN Submit the remaining part of job after last checkpoint received to the scheduler for rescheduling. GOTO Step 3. (iv) IF checkpoint result of the job does not exist in checkpoint server THEN Submit the job from start to the scheduler for rescheduling. GOTO Step EXIT IV. CONCLUSION Based the theory of resource credibility and availability with fault tolerance, this paper presents a high dependability grid resource integrated scheduling algorithm with fault tolerance suitable for grid environment, and describes and analysis the algorithm in detail in this paper. The algorithm can dynamically adjust scheduling policy according to the requirement of service requests for the credibility and availability of resources. It guarantees the performance requirements of resources, while taking into account the needs of the credibility of resources. The dependability of algorithm has greatly improved, thereby reduce the probability of 13

6 service requests implementing failure and deception, and ensure the security of the implementation of the task. This algorithm improved fault tolerance using check point mechanism. Next work will take into account a variety of service quality requirement of the user, mainly we are trying to simulate the mathematical model using Gridsim and Globus tool kit. REFERENCES [1] Benoit Anne, Cole Murray, Gilmore Stephen and ills ton Jane. (2005): Enhancing the effective utilization of Grid clusters by exploiting on-line performability analysis, IEEE International symposium on Cluster Computing and the Grid(CCGRID), pp [2] Buyya. R, Murshed. M, Abramson. D. (2002): A deadline and budget constrained cost-time optimization algorithm for scheduling task farming applications on global grids, In Proceedings of the international conference on parallel and distributed processing techniques and applications, Las Vegas, USA, pp [3] Elnozahy. E. N., AlvisiLorenzo, Wang Yi-min, Johnson. B. (2002): A survey of rollback recovery protocols in messagepassingsystems, ACM Computing Surveys, 34: [4] Feitelson D.G, Rudolph. L (1995): Parallel Job Scheduling: issues and Approaches. Springer Lecture Notes in Computer Science, 949: 1-18.ISSN: [5] Beaumont, L. Carter, J. Ferrante, A. Legrand, Y. ]onheterogeneours Platforms. In: Proc. of the Int l Parallel and Distributed Processing Symp, [6] R. Buyya, D. Abramson, J. Giddy. Nimrod/G: An Architecture for a Resource Management and Scheduling System in a Global Computational Grid. In: Proc. of th, [7] W.M. Zheng, B. Yang, W.J. Lin, Z.G. Li. An Improved Communication of Parallel Task Scheduling on SMP System. Science in China (Series E), 31(5): , [8] L.L. Yuan, G.S. Zeng, L.L Jiang. C.J. Jiang. Dynamic Level Scheduling Based on Trust Modeling in Grid. Chinese Journal of Computers, 29(7): ,