CHAPTER 3 GRID MONITORING AND RESOURCE SELECTION

Transcription

1 31 CHAPTER 3 GRID MONITORING AND RESOURCE SELECTION This chapter introduces the Grid monitoring with resource metrics and network metrics. This chapter also discusses various network monitoring tools and the various challenges being faced by Grid schedulers. 3.1 RESOURCE MONITORING IN GRID By maintaining the resource status constantly, the necessary information can be quickly provided as requested. However, the cost of maintaining resource status is highly related to the number of resources and the frequency of status updating. Therefore, trade-off between maintenance cost and data accuracy should be considered. Resource monitoring involves monitoring the available resources whenever a job is submitted to the resource broker. A submitted job is often executed whenever sufficient amount of resources are freed by other jobs. Resources are nothing but the CPU, Physical Memory, Kernel Memory, Data, Memory, Node, Network of storage, servers, and database servers Resource Metrics CPU Utilization The CPU time required for executing some job is called the CPU utilization. Multi-Tasking can be done whenever many jobs are submitted on

2 32 a particular node. CPU utilization is one of the important parameters for executing a job Memory This metric decides how many jobs data and how much of the data can be stored locally on a system. Memory has its associated metrics namely page faults, peak memory usage, virtual memory size. This may further be refined as the Kernel memory and the user level used Data The metrics associated with this are the data cache size, amount of free space, total available free space and the type of data currently handled Node This includes the following metrics namely, uptime, request processed, and the transaction throughput. Wu-chun and Ruay-shiung (2009) were proposed an efficient protocol called the Grid Resource Information Retrieving (GRIR) protocol, which is based on the push data delivery model to obtain the accurate network status. Here they have avoided useless updates to the Mediators and have reduced the consumption of bandwidth by fixing some sort of threshold for updating purpose. 3.2 NETWORK MONITORING IN GRID Network monitoring is very important in the operation of networks of significant size. Grid Monitoring Architecture involves formation of the cluster in a computational grid. Grid middleware and applications to make intelligent use of the network, optimizing their performance by adapting to changing network conditions (including the ability to be self healing ). Some

3 33 of the factors affecting TCP throughputs are transfer size, maximum sender/receiver buffer size, path characteristics (RTT, packet loss rate, available bandwidth, nature of traffic etc). Collecting, relating and analyzing of network information are one of the important aspects of effective grid application and services. Ferrari and Giacomini (2004) proposed the network monitoring for Grid performance optimization by introducing the cost function, called closeness which was realized by the network metrics RTT, packet loss and throughput between any nodes in the grid cluster. This thesis converses about the cost function by means of combining the various metrics such as bandwidth, latency, packet loss rate, jitter and RTT to ensure the efficient transfer of input file /output data for the submitted job Tools for Network Monitoring Ping The ping utility of UNIX is mainly used to test the connection and availability of the destination host. It also measures the round trip time (RTT) and packet loss by transmitting and receiving ICMP echo packets. This round trip time measured by ping command is also used to estimate the capacity of the pipe. Though ping measures the packet loss, it is not much effective since it uses only small load of ICMP echo packets Traceroute Traceroute is another important tool, which is used to discover the route to various destination hosts and determine the RTT to each hop in the network. This tool is used to identify where the problems occur in the network.

4 Pchar Pchar is another important monitoring tool, which is used to measure bandwidth, RTT, router buffer space on all the links in the network. The bottlenecks in network path between two different hosts can also be identified Iperf Iperf is a tool used to measure the bandwidth and quality of a network link. Iperf uses both TCP and UDP transmission patterns. The use of different capacities of TCP and UDP helps to get better statistics about the network. Iperf is used to measure the maximum bandwidth available for an application. Jitter and datagram loss can be measured using the Iperf UDP test. Iperf can be easily installed and used in the UNIX system. It makes use of a client and server available at two different hosts. The analysis will be provided to the client in case of TCP tests and to the server in case of UDP tests (Iperf 2008) UDPmon UDPmon is a network performance-monitoring tool by identifying various network metrics. UDPmon uses the UDP packets to measure the endto-end performance (UDPmon 2005). The programs available for UDPmon use the socket interface in a simple way and do not require a root privilege. UDPmon identifies the following network metrics such as request response latency, packet-loss ratio, and jitter. UDPmon also provides one more flavour known as udpmon_bw_mon. This provides achievable UDP throughput, packet loss ratio, inter-packet jitter, relative one-way delay.

5 TCPmon TCPmon, an open source utility for monitoring the data, uses the TCP/IP protocol to send and receive the packets between the two end hosts (TCPmon 2005). TCPmon helps in measuring the round trip time latency between the end hosts using the TCP packets. TCPmon with GUI interface is also available PingER The mechanism used in PingER is the Internet Control Message Protocol (ICMP) Echo mechanism (PingER 2003). This is also referred as the Ping facility. This allows users to send data packets of desired length to a remote host and have it echoed back. It uses only 100 bits per second (bps) per monitoring-remote-host-pair approximately. PingER measures RTT and packet loss ratio. 3.3 A STANDARD RESOURCE SCHEDULING ALGORITHM The standard resource scheduling algorithm followed by most of the schedulers is explained below. The scheduling is carried out at time intervals called scheduling events. These events can be determined by either regular intervals, called as poll-based or in response to certain conditions, called as event-based (Casanova et al 2000). The proposed network aware resource selection strategy improves the scheduling because it takes into account resource as well as network performance estimations in selection of suitable resource for the submitted jobs.

6 36 while (there is any unsubmitted jobs) { Update the resource performance based on job scheduled in previous intervals; foreach (unsubmitted job) { Match the job to a resource set to satisfy the requirements at the job level; Schedule the jobs; } do { Assign mapped jobs to each compute resource heuristically; }while (all jobs are submitted or no more jobs can be submitted in job queue); wait for the next scheduling event; } 3.4 CHALLENGES IN GRID SCHEDULING The Grid Computing falls into the category of distributed and parallel computing environments. They have a lot of unique characteristics but scheduling in Grid is highly difficult and challenging task. An adequate Grid scheduling system should overcome these challenges to influence the promising potential of grid providing high performance services. Resource Heterogeneity - Grid resources are usually heterogeneous in nature and the heterogeneous resources may have different hardware, such as instruction set, computer architecture, number of processors, physical memory size, CPU speed, different software such as different operating systems, file systems, cluster management software, different type of networks, bandwidth availability, network latency and so on. A centralized scheme is not scalable because of the risk in single point of failure.

7 37 Decentralized - In the decentralized scheme, every resource is responsible for maintaining its current state information locally and answering queries from different clients. A decentralized scheme may not be efficient due to more number of queries. However, the decentralized scheme is more reliable because there is no single point of failure. Hybrid: Using the hybrid scheme resources are categorized into several groups. Within each group, centralized scheme is applied. Thus, each group has representative entity, which is in charge of the information of all resources in its group. Over the groups, the decentralized scheme applies. An efficient grid scheduler or resource broker should ensure the following features while optimization such as it should be adaptable and scalable according to the dynamism of resources and jobs, it should have the facility to predict and estimate the performance of jobs and resources, it should have the capability for taking the cost of resources into account during optimization, and it should have the ability to give user preferences and local resource policies. 3.5 CARE RESOURCE BROKER CARE Resource Broker (CRB) is a grid metascheduler or broker was proposed by (Thamarai Selvi et al 2010) and it is deployed over the GT4 Middleware. The CRB is implemented by the Centre for Advanced Computing Research and Education (CARE), Anna University, India. It addresses several scheduling issues encountered when virtualization technology is integrated with Grid. CRB supports creation and management of virtual resources in existing physical resources and it is capable of deploying Grid middleware and other related software on the fly. This feature allows CRB to make several application decisions that conventional grid schedulers cannot do. CRB obtains user s requirement request for running an

8 38 application in grid resources. It then aggregates grid resource information and discovers suitable physical resources that exactly match with the requirements. If no such resource is discovered, the broker determines the reason that it is due to non-availability of sufficient computing nodes in a cluster, it then identifies a physical cluster (A) that comes closely with the request. It then determines another cluster (B) and suggests formation of required number of CPUs in the form of virtual machines in B. The virtual machines will then be added to Cluster B as computing nodes of A to meet the application requirements and it is due to non-availability of execution environment such as simulation tools, operating system and other libraries in any of the cluster. The broker then determine potential physical cluster that can host virtual cluster that can meet the application requirements.