A STUDY OF THE PERFORMANCE TRADEOFFS OF A TRADE ARCHIVE

Transcription

1 A STUDY OF THE PERFORMANCE TRADEOFFS OF A TRADE ARCHIVE CS737 PROJECT REPORT Anurag Gupta David Goldman Han-Yin Chen {anurag, goldman, han-yin}@cs.wisc.edu Computer Sciences Department University of Wisconsin, Madison Madison 53706, WI Dec 17, 1999

2 1. Introduction A Tape Archive is a collection of files stored on magnetic tapes. An archive is usually maintained for files that are of importance and have to be preserved over time. Since these files are interesting, over time the users are likely to access them. The task is then to develop a multiuser retrieval system for the files in the archive. In this project we have tried to address the various tradeoffs in the performance metrics of such a system using discrete event simulation studies. We have created a tape archive consisting of tape drives, a disk cache with disk drives and a robot arm that mounts and dismounts the tapes from the archive. The archive serves a community of users who submit multi-file retrieval requests to the archive. The archive moves the requested files from tape to disk and notifies the user. Once the files are consumed by the user who ordered it, the archive is notified and the files can be removed from the cache. 2. The Questions and Intuition Modeling a tape archive involves some complex and detailed study of the various parts of the system and how they interact. Changing one part of the system can dramatically affect another part. In the next two sub-sections we try to raise some of the issues that come up in performing a simulation of such a system and have presented some intuitive results that can be expected based on our assumptions and implementation. 2.1 The Questions A discrete event system that simulates multiple accesses to a tape archive can be designed in several ways. There are benefits and tradeoffs that must be considered: a) Single Vs Multi Threaded Model The system can be multi-threaded or sequential (single threaded). If it is multi-threaded a separate Time Manager has to be maintained to keep track of the overall system time. Separate threads have to wait and notify each other according to a possibly complex schedule. A multithreaded system will not scale well with a large number of customers. If it is sequential the 2

3 system has to accurately model behavior that is essentially concurrent. A sequential system will scale well with a large number of customers. b) Scheduling Policies The system can choose one of any number of policies for the requests to be scheduled, or for the tape archive files to be mapped to disks. For the request scheduler is it First-Come-First- Serve, Shortest-Job-First, Random, etc? What are the potential tradeoffs for each policy? For mapping files to disk do we use Hashing, Range or Random, etc? What can be gained or lost from using one instead of the other? c) Multiple Users We can choose to model N customers or just one, thereby losing concurrent accesses but gaining simplicity. We can choose to model no customers at all by assuming the requested files are consumed as soon as they are written to disk. d) Real Time Simulation We can choose to keep the system closed and static, forcing the user to view statistics and graphs of a singly determined instance of simulation. Or we can allow the user the utmost flexibility in choosing the structure of the system, perhaps even while the system is currently processing. If we allow the user to change the system layout, what variables can be changed? Can the user specify specific scheduling and mapping policies on the fly? Can the user change the number of files, disks, tapes, and other features of the archive system? In the Implementation section, we answer some of these questions and also discuss our assumptions and what aspects of the system we have decided to study. 2.2 The Intuition The objective of our experiment is to study different job-scheduling policies and their impact on selected performance metrics. Specifically, we have chosen to implement the First- Come-First-Serve (FCFS), the Shortest-Job-First (SJF), and the Last-Come-First-Serve (LCFS). Also of interest are various possible file-to-disk-mapping strategies and their potential effect on system performance. To this end we implement three different policies: Random, Hash, and Range mapping. To gain insight into the relative merits of these policies, we choose 3

4 the Average Response Time, the Average Waiting Time, and the Average Normalized Waiting Time of the serviced jobs to be the main performance metrics on which we focus our study. For the job-scheduling policies, we expect LCFS to perform worst with respect to average waiting time of individual jobs. This is because LCFS consistently favors jobs with the smallest waiting times, which in turn leaves jobs with longer waiting times to wait even longer for service. Further, we expect FCFS to perform better than SJF with respect to average waiting time of individual jobs. This is due to SJF s potential problem of starvation, which is detrimental to response time of long jobs. Since both the number of files and the assortment of files of an individual request are generated randomly in our simulation, long jobs are generated as often as are short jobs. Thus, running shortest jobs first at the cost of the response time of longer jobs significantly hurts the average response time. For the file-to-disk-mapping policies, we expect more or less equal behavior from all of them. Our intuition is based on the fact that since our file requests are generated randomly none of the file-to-disk-mapping policies can substantially take advantage of any kind of reference pattern. 3. Implementation In this and the following sections, we describe our assumptions and give a detailed view of our simulation model. We choose to implement a robust and flexible system that displays graphically to the user a comparison between the performance metrics of the multiple scheduling policies. For instance, the user can choose to compare a system with N customers to one with N*2 customers, or a system with a Random disk mapping policy to one with a Range disk mapping policy. Each system receives the same file requests and the corresponding data displayed clearly shows the tradeoffs between them. Our archive is sequential and is therefore scalable, models multiple customers and is therefore realistic, and allows the user to set specific scheduling and mapping policies and several system variables through a graphical interface. 4

5 3.1 Overview The Tape Archive simulation model consists of the following main components: a) Generator The Generator generates the user requests and puts them in a queue from which the scheduler can choose the next request to be serviced b) Queue The Queue contains the user requests. Each request is a multiple-file request c) Scheduler The Scheduler chooses a request from the Queue according to the designated scheduling policy and gives it to the Disk Manager to process it d) Disk Manager The Disk Manager takes a request from the Scheduler and coordinates with the Disk Cache and the Tape Archive to process the files in the request. It returns the total service time for that request e) Tape Archive The robot arm retrieves a tape from the tape archive, seeks to the requested file on the tape, and reads the file from the tape and gives the file to the Disk Cache f) Disk Cache The Disk Cache writes a file to disk using the replacement policy and pins it to be read by the customer. If a file has to be replaced and there is no unpinned one it waits until a customer unpins one g) Customer Customer read files from disk and unpins them after a random amount of think time h) Analyzer Reads the serviced requests from the file, calculates all the system statistics and displays them to the user dynamically 5

6 3.2 Details Generator: Associated with each generated order are user id, arrival time, set of files requested, service time, and waiting time, of which the first three attributes are produced by the generator. The user id and the set of file ids of an order are generated randomly, namely according to the uniform distribution. The arrival time of each order is generated such that the inter-arrival time between orders is exponential. For the purpose of our study, the arrival rate is carefully selected so that it is not greater than the average service rate. Scheduler: We implement the following three scheduling policies: a) First Come First Serve (FCFS): The request that arrived first is selected for service b) Shortest Job First (SJF): The request with the shortest service time in the waiting queue is selected. The expected service time of the request is calculated in accordance with the file selection algorithm utilized by the Tape Archive (described below) c) Last Come First Serve (LCFS): The request that arrived last is selected for service Disk Manager: The Disk Manager is given a queue of files to fetch from the archive and write to disk. It returns the total service time for that request. For each file in the queue it does the following: 1. Call the Tape Archive to return a file from the queue of files 2. Call the Tape Archive to retrieve this file from the archive 3. Check with the Disk Cache to make sure this file can be written to disk 4. If all the disks are full and no file can be replaced (because the customer hasn t read it yet) wait for the customer to unpin it 5. Tell the Disk Cache to write the file to disk Tape Archive: The Tape Archive simulates the archive that consists of a series of d tape drives and a robot arm that fetches the tapes from the archive. The files are mapped to tapes sequentially. The robot arm moves files from the tape to disk based on a policy set by the user (Hash, Range or Random). The algorithm to fetch a file from tape is: 1. Pick a file from the queue based on the Shortest-Seek-First and calculate its cost in time units 2. Fetch the tape if the given file is not on the current tape 3. Update the head of the tape to point to the end of the given file 4. Update the service time for the seeking and the read 6

7 Disk Cache: The Disk Cache simulates the array of c disks. It provides functionality to the Disk Manager to check if a file can be written to disk and to write that file to disk. Files are written to disk according to the First-Find replacement policy that replaces the first unpinned page it finds. Analyzer: As requests are being completed, the analyzer computes (and displays) running statistics of the following performance metrics for each scheduling policy: 1) number of jobs completed so far 2) expectation of waiting time of serviced jobs 3) variance of waiting time of serviced jobs 4) expectation of response time of serviced jobs 5) variance of response time of serviced jobs 6) expectation of waiting time normalized by service time of serviced jobs Customer: The N customers in the system have to consume their requested files so the files can be removed from disk by the replacement policy as the disks get full. There are two sources of contention: 1) A customer reading a file can cause contention for the disk with other customers or the archive robot arm writing files to the disk. However, a customer waiting for another customer or waiting for the robot has no effect on the overall system service time. This is because the robot can service the request unimpeded by customers who are only reading files that have already been serviced. The service time is affected only when the robot has to wait for a customer to finish accessing a disk. Since the time the robot takes to seek to a file on tape is several orders of magnitude more than the time it takes to write to a disk, this contention time is insignificant. Furthermore, because the robot takes so long to access files on tape we can usually schedule customers read requests without any contention. Therefore this type of contention can be ignored. 2) If customers are slow to consume their requested files these files will accumulate on disk and hamper the replacement policy. For example, the customer reads the file but decides to go to sleep before stating that he has consumed the file. This is the second source of contention. Customers consume files according to a random number that is generated for each file. 7

8 Customers consume one file at a time, concurrently with other customers. A file can be replaced after it is consumed, but not before. Therefore, we need to keep track of time for the consumption of a file to simulate customers consuming their requested files. At every system event we must follow these steps: 1) Increment the total service time by the time of this event 2) For each customer find the file it is currently consuming 3) The time of the current event is subtracted from the random number generated for each of these files 4) If the file time becomes zero then unpin the file 5) If the event time is more than the file time then find the next file the customer is consuming and subtract the remaining event time from this files randomly generated time. Keep unpinning files that have no remaining time. Loop through this step until the event time has been used up or there are no more files for this particular customer 5. Simulation Analysis As jobs are being completed, the analyzer computes (and displays) running statistics of the following: 1) Number of jobs completed so far 2) Expectation of waiting time of serviced jobs 3) Variance of waiting time of serviced jobs 4) Expectation of response time of serviced jobs 5) Variance of response time of serviced jobs 6) Expectation of waiting time normalized by service time of serviced jobs namely, the expectation of the penalty ratio Thus, six graphs are produced to display these statistics. Each of the statistics is graphed as a function of time, which, in our discrete-event system, is based on the event of service completion. So, on the X-axis is the number of serviced requests. Furthermore, in order to display a complete history of these statistics, each point of the curves is computed over all jobs that are completed at the analysis instance and the entire history of that statistic is shown (e.g. the expectation of the waiting time over serviced requests). 8

9 5.1 Quantitative Analysis of One Simulation Run Each simulation run spawns three threads corresponding to each of the three scheduling policies (FCFS, SJF and LCFC). All the three threads are given the same random number generator seed so that they generate the same random sequence stream so that the scheduling policies can be compared. We discuss the graphs of one such simulation instance: a) Expectation of the Waiting Time For an input of requests, we see that the expectation of the waiting time of the serviced requests is about 110 time units for LCFS, 75 for the SJF and almost 5 for FIFO. The value of LCFS is significantly more than the other policies because the last request arrived is serviced first, increasing the waiting times of the requests that arrived earlier. LIFO performs better than SJF because the first job arrived is serviced first. b) Variance of the Waiting Time The variance values for the waiting time for LCFS is very large as expected because the expected waiting time is large. Similarly for the other two policies. The variance of FIFO is better than SJF. c) Expectation of the Response Time For an input of requests, we see that the expectation of the response time of the serviced requests is about 160 time units for LCFS, 130 for the SJF and almost 35 for FIFO. The response time for LCFS is greater because the waiting times for the requests are longer. Here, the waiting time affects the response time more than the service time. Since the waiting times were very small for FIFO, and the response times are in the range 30-35, it gives a good estimate of the service times for this policy. d) Variance of the Response Time The variance values for the response time for LCFS is very large as expected because the expected response time is large. Similarly for the other two policies. The variance of FIFO is better than SJF. 9

10 e) Penalty Ratio The expectation of the normalized waiting time (waiting time over service time or the penalty ratio) shows expected trends based on the expected waiting and response times. The value is more for LCFS than SJF which is more then for LIFO for the reasons discussed above. f) Number of Serviced requests The number of serviced requests is shown as requests get processed for the various scheduling policies. The serviced requests are more at any given instant for FIFO and FCFS (comparable for them) compared to SJB. The results of our experiment agree with our prior intuitions. FCFS performs much better than SJF and LCFS in both expected waiting time and expected normalized waiting time. SJF, in turn, performs better than LCFS. The same is true for the variance of waiting time and the variance of normalized waiting time. Moreover, FCFS displays a steady history for all of the performance metrics, while SJF shows rising curves for the initial 1000 to 4000 jobs (depending on the job streams) before it stabilizes. The rising curve lasts long and runs steep especially for the variance statistics of LCFS. This is due to the nature of the LCFS policy, which progressively widens the gap in waiting time among jobs in the most dramatic way possible. And as we predicted, the three file-to-disk mapping policies we choose to implement (Hash, Range and Random) do not appear to affect the performance metrics in any significant measures. 6. Results and Conclusions We simulated a Tape Archive Discrete Event System Model and studied some performance metrics for the various scheduling and file-to-disk mapping polices. We implemented the First Come First Serve, Shortest Job First and the Last Come First Served scheduling polices. For the file to disk mapping, we implemented the Hash, Range and Random policies. We studied the average waiting time, average response time and the normalized waiting time as our main performance metrics. Conforming our intuition, the FCFS performed better than SJF and LCFS in both expected waiting time and normalized waiting time. The file to disk mapping policies did not appear to affect the relative performance metrics. 10

11 7. Acknowledgements We would like to thank Prof. Miron Linvy for his help, guidance and practical suggestions throughout the life span of this project. We would also like to thank Zhiqi Qiu for her thoughtful pointers. 11