Improving the Performances of Proxy Cache Replacement Policies by Considering Infrequent Objects Hon Wai Leong Department of Computer Science National University of Singapore 3 Science Drive 2, Singapore 117543 leonghw@comp.nus.edu.sg Bo Guan Department of Computer Science National University of Singapore 3 Science Drive 2, Singapore 117543 ABSTRACT In this paper, we perform a careful study of the effect of infrequent objects on the performance of many well-known web proxy cache replacement policies including,,, and (an offline policy). Using a frequencyaware version of these policy (one that is aware of these infrequent objects), we show that significant improvement in the performance (hit rates and byte hit rates) can potentially be achieved. We also present -Pred, which is a modified replacement policy that attempts to predict single access objects that it will not cache. The algorithm, though simple, achieve better performance than. The results are encouraging and point to more research on designing more sophisticated replacement policies that can predict infrequent objects. 1. INTRODUCTION The World Wide Web (WWW) has grown at a phenomenal pace and web requests have become the predominant service and dominant workload component for Internet traffic [10]. Many previous studies have shown that a significant proportion of Web requests are static web pages and many of these do not change frequently. Thus, proxy web caching is a standard method to enhance the performance. In proxy Web caching, the proxy server acts as an intermediary between a number of client browsers and the Internet web server as shown in Figure 1. The proxy server has a cache to store frequently (and usually recently) accessed web objects. When a request is made from a client browser, the request first goes to proxy server. If the requested object is already stored in the cache (called hit), the object requested This research was supported in part by the National University of Singapore under grant R252-000-128-112. This research was done while this author was an undergraduate at the National University of Singapore. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC 06 April 23-27, 2006, Dijon, France Copyright 2006 ACM 1-59593-108-2/06/0004...$5.00. 791 Client A Client B Web Cache Internet Figure 1: Proxy Cache Server Web Server is sent to the client browser. If the object requested is not in the cache (called a miss), the request is forwarded to the target web server. The benefits of proxy web caching are clear: the client browser enjoys faster response and there is a reduction in traffic demand made on the network, benefiting all parties on the Web users, ISPs, and original Web servers and the Internet at large [10]. An important optimization problem in proxy web caching is the design of the replacement policy which decides which object(s) to replace when a new arrives. Much work has been done in this research area and a number of replacement policies (or algorithms) have been proposed. Our study focuses on a very specific feature of many web access traces namely, the existence of a substantial proportion of infrequent objects, namely, objects that are accessed very infrequently (say, once, twice, or thrice) in the entire trace. This phenomena has been reported in many other studies of web workloads [7, 2, 4, 10]. In the two traces used in our study, we found that as much as 53% of the objects are single access objects and they account for about 45% of the total byte count. We study the effect of these infrequent objects on the performances of some well-known replacement policies that were reported to have good performances [6, 2, 10]. These includes the (least frequently used), (least recently used), and (greedy-dual-size-frequency) policies. Two offline policies, -P (perfect least frequently used), and (longest forward distance) were also included in our study. [6, 10] To study the effect of infrequent objects, we first ask the general question what if we do not cache the infrequent objects?. To answer this question, for each replacement policy studied, we introduce an offline frequency-aware version, namely, one that is aware of these infrequent objects and will not cache them. The performance gap (in terms of hit rate and byte hit rate) between these two versions will give a good indication of the potential gain in performance that can be achieved with good replacement policies that can predict these infrequent objects.
Our experimental results show that there is a significant performance gap (in terms of hit rates and byte hit rates) between the two versions of the,, and replacement policies. The significant performance gaps indicate that (a) to improve proxy cache performance, replacement policies should handle infrequent objects, particularly, single-access objects, (b) there can be potential gain in not caching all missed objects, and (c) a fruitful research direction is the design and analysis of replacement policies that can predict these infrequent objects. More specifically, we find that the single-access objects account for most of this performance gap that seem to suggest that research should be targetted at prediction of single-access objects. We also present preliminary results on a simple modified policy that attempts to predict single access objects. The algorithm will not cache the predicted single access objects. The results show that even a simple prediction algorithm can improve the performance of the replacement policy, thus illustrated the power of prediction. Work is ongoing to refine our simple prediction algorithm with the goal of getting closer to the performance of -A(1). 2. ASSUMPTIONS AND SETUP It is well known (see for example [6]) that getting a fair comparison of various proposed replacement policies is complicated. Therefore, in this section, we precisely state all the assumptions made in our own trace-driven simulation study to make it possible for others to do their own implementation and verification of our results, if so desired. These assumptions are, by and large, the same as those adopted by many previous work. However, since not all previous work clearly state all their assumptions, definitions and measurements used, it is sometimes hard to make fair and consistent comparisons. We use a trace-driven simulation [10] on the following traces obtained from the National Laboratory for Applied Network Research [1]: ST [Feb 16, 2001 to Mar 1, 2001] SJ [Jan 27, 2001 to Feb 9, 2001] The SJ trace was collected from a server at MAE-West Exchange Point [sj.cache.nlanr.net] in San Jose, California while the ST trace was collected from the International Exchange Point [startap.us.ircache.net] in Chicago, Illinois. The key characteristics of these two trace files are shown in Table 1. We now state the assumptions made in this paper: We consider a single cache of a finite, fixed size and assume that the Web objects cannot be divided. Note that if an object (such as an html page) is modified, the modified object is considered a new object in our simulation. We also assume that the trace file has been cleaned up to remove the uncacheable objects (such as dynamic content objects, network failure, or unqualified HTTP requests) these are very well discussed in many previous work ([10]) and will not be discussed in detailed here. Since these uncacheable objects do not affect the performance study of replacement policies, they can be safely removed from the trace. We further assume that the requests to the cache must be served in the order of arrival. Most replacement policies also make the further assumption that every missing item (that are not oversized) must be loaded into cache at the time of request. In this paper, we relax this assumption by allowing the replacement policy to decide whether or not a 792 Property SJ ST Duration of Trace 14 days 14 days Total Requests 713,460 729,890 Total Unique Requests 452,108 565,510 Total Request Size 6.67 10 9 1.81 10 10 Total Unique Req Size 3.32 10 9 1.03 10 10 Table 1: Characteristics of the SJ and ST traces missing item should be cached these are discussed further in Section 3. Every time item is loaded into the cache, some items may be evicted to make room for it. The set of items to be evicted is determined by the replacement policy used. 3. PROBLEM FORMULATION We give a precise formulation of the proxy web caching problem following the notations of [8]. Let U = {1, 2,..., n} be the finite universe of the n objects in trace. 1 For each object i, we define s(i) to be its size. Let S be the size of the cache. Let R = (r 1, r 2,..., r m) be the sequence of m requests where r k U is the object requested at time k. The state of the cache at time k is denoted by B k U and is the set of objects in the cache at time k and must satisfy the property i B k s(i) S. Therefore, the sequence (B 0, B 1,..., B m) represents the consecutive states of the cache in response to the sequence of request R. B 0 is the initial state and B m is the final state of the cache. Define the hit sequence to be (h 1, h 2,..., h m) where h k = { 1 if rk B k 1 (a hit) 0 if r k / B k 1 (a miss) Given a request sequence R, a cache of size S and at initial state B 0 =, a replacement policy produces a cache state sequence (B 1, B 2,..., B m), where for k = 1,..., m, we have (B k 1 E k ) {r k } if r k / B k 1 and has B k = to be cached otherwise B k 1 where E k B k 1. In the first case, the requested object r k is a miss and has to be cached and E k is the set of objects to be evicted from the cache. In the second case, the cache remains unchanged under two conditions: (a) if the requested object r k is already in the cache (a hit) or (b) if it not the cache but the replacement policy decides that it should not be loaded. In (b), we relax the usual assumption (made by many replacement policies) that all missing objects must be cached. The replacement policy determines whether or not to load a missing object r k and also the set E k of evicted objects when r k is loaded. Performance Metrics: The two most commonly used metrics to evaluate proxy cache performances are the hit rate (HR) the ratio of the number of hits to the total number of requests, and the byte hit rate (BHR) the ratio of the total bytes of all hits to the total bytes for all requests. More formally, we have m k=1 HR = h m k m, BHR = k=1 h k s(r k ) m k=1 s(r k). Since the hit rate measures only the number of objects served, replacement policies that favour small objects remaining in the cache will perform better on the hit rate. 1 As discussed above, these are all cacheable objects.
The byte hit rate tries to account for the object sizes and so is a better measure of the amount of network traffic reduced due to the caching strategy. In our paper, we will evaluate both, but give higher consideration to byte hit rate. A Theoretical Upper Bound: Theoretically, the best performance for a given trace is obtained by an infinite cache a cache that is big enough for all the objects in U. Let B = n i=1 s(i) denote the size of the infinite cache. Then the corresponding upper bound for the hit rate and byte hit rates are given as follows: HR = (m n)/m, BHR = ( m k=1 s(r k) B ) m k=1 s(r k). 4. REPLACEMENT POLICIES STUDIED We first give an overview of some existing replacement policies proposed by the scientific community. We first note that the problem of find a replacement policy that optimizes any of the metrics used (hit rate, byte hit rate, or other commonly used metrics) is known to be NP-complete problem. Thus, there have been many replacement policies proposed in the literature. A good survey of replacement policies can be found in [5] and [6]. Due to space limitation, we shall only give details of the policies studied in this paper and refer the reader to the above for information on other policies. Least-Recently-Used () replaces the object which was requested least recently. Least-Frequently-Used () replaces the object that was accessed the least number of times. Perfect (-P) is the same as, except that it does not use in-cache frequency, but the frequency of the object from the beginning of the cache simulation. Greedy-Dual-Size-Frequency () [6] replaces the object that has the lowest priority where the priority of an object f is given by P r(r k ) = Clock + freq(r k ) Cost(r k )/Size(r k ), where Clock is a running queue clock, freq(r k ) is the frequency of object r k (in the cache), Cost(r k ) is the cost and Size(r k ) is the size. Longest Forward Distance () replaces the object for which the next request is furthest away in the future. It has been reported [6, 10, 9] that the above are among the better replacement policies for proxy caching. Here, we note that is an offline replacement policy. The has been shown to be optimal for the special case in which the objects are of a uniform size and cost [3]. However, it was shown [8] that the policy is no longer optimal for the general case. Nevertheless, still produces very good results (in terms of hit rate and byte hit rate) even for the general case. 5. INFREQUENTLY ACCESSED OBJECTS Our study of a number of Web traces indicates that there are many objects in the trace that are accessed very infrequently in the entire trace. We are interested in infrequent objects that have frequency f for small f = 1, 2, 3. Figure 2 shows the distribution of request frequencies of the objects in the two traces SJ and ST. It confirms that there is a significant proportion of infrequent objects. In the SJ trace, 53% of the requested objects are single-access ones and they 793 (a) By number of request. (b) By total request size/ Figure 2: Distribution of Request Frequency. account for 45% of the total request size. More importantly, they also account for 90% of the total size of unique object B. 2 In other words, only about 10% of B comes from multiple-access objects. One contributing factor to this is the fact that, like most other simulation studies, we consider each modified object to be a new object and this will increase the number of single-access objects [4]. Single-access objects are never requested again, and so caching them will not give any hit. On the other hand, it may force other objects to be evicted and therefore could reduce the overall hit rate and byte rate. Furthermore, since their proportion is significant, the negative impact may also be potentially large. However, for objects with frequencies 2 or 3, the situation is not so clearcut. There is the risk, of course, that the all the accesses occur fairly close to each other and so caching these infrequent objects will be beneficial. On the other hand, if the intervals between consecutive accesses are too far apart, then caching these infrequent objects will also have a negative impact on the hit rate and byte bit rate. 5.1 Idealized Frequency-Aware Policies To study this effect, for each policy studied, we propose a idealized frequency-aware version of the policy. The frequency-aware version knows if an object is infrequent and will not cache these infrequent objects. Figure 3 shows the algorithm used to process the frequency-aware policy. For each replacement policy studied, say P, let P -A(k) denote the frequency-aware version of policy P where we do not cache objects with access frequency k. The original policy can also be considered P -A(0). We use small values 2 For the ST trace, the numbers are 71%, 54%, and 95% respectively.
New Requst HIT? No Infrequently Accessed? Cache No Yes Yes Record as HIT Do not cache Figure 3: Strategy for Frequency-Aware Policy of k = 1, 2, 3. We define the performance gap (in HR and BHR) for a policy P as follows: PG-in-HR(P) = HR(P-A(k)) - HR(P) PG-in-BHR(P) = BHR(P-A(k)) - BHR(P) The performance gap of a replacement policy P gives an indication of the potential gain in performance if we can introduce good online versions of policy P that can predict these infrequent objects. 6. EXPERIMENTAL RESULTS We have implemented a web proxy cache simulator that allows us to experiment with all the policies described. The simulator is written in C++. New replacement policies can also be easily added if needed. The two traces, SJ and ST, were used. The cache is initially empty (i.e. we do not perform any warmup of the cache). The cache size S is given by S = β S. To obtain sufficient data points, we have used β = 0.1%, 0.2%,..., 0.9%, 1.0%, 2%,..., 10%. 6.1 Baseline Comparison We first do a baseline evaluation of all the replacement policies used in our study. The results are shown in Figures 4 and 5. The figures show that while the two traces have different slightly characteristics, the general performance trend of the various policies is very similar: Rank by HR :, -P,,, Rank by BHR:,, -P,, We give some observations here: The two offline policies, and -P perform well, as expected. In particular, performs the best in both HR and BHR (and very close to optimal) on both traces. Among the online policies, is the best in terms of both HR and BHR. In fact, the performance of is close to that of the offline policy -P. The performance of in terms of HR is better than but worst than. In terms of BHR, is close to. 6.2 Performance of Frequency-Aware Policies We now present results on the effect of infrequent objects by measuring the performance gaps between each replacement policy and its idealized frequency-aware version. In this section, only results on the SJ trace will be presented. The performance on the ST trace is similar, but is not shown due to space limitation. To aid the analysis, we have separated the replacement policies into two groups: the offline (, -P) and online ones (,, ). The offline replacement policies, like and -P, make use of the past frequency information in its eviction policy. Thus, it is reasonable to expect the performance of the frequencyaware version to be similar to that of the original. In contrast, we expect larger performance gaps for the online replacement policies like, and. of Different Policies for SJ trace of Different Policies for st trace -P 0.05 -P of Different Policies for SJ trace of Different Policies for st trace 0.44 0.42 0.38 0.36 0.34 -P Figure 4: Comparison of different policies on SJ using hit rate (above) and byte hit rate (below). 0.32 -P Figure 5: Comparison of different policies on ST using hit rate and byte hit rate. 794
of offline Policy and Policy-A1 for SJ trace of online Algo and Policy-A1 for SJ trace -P -P-A1 -A1 -A1 -A1 -A1 of offline Policy and Policy-A1 for SJ trace of online Policy and Policy-A1 for SJ trace -P -P-A1 -A1 Figure 6: Performance of idealized frequency-aware offline policies (, -P) for SJ trace. -A1 -A1 -A1 Figure 7: Performance of idealized frequency-aware online policies (,, ) for SJ trace. 6.2.1 Effect of Single-Access Objects. We first study the effect due to single-access objects by comparing the performance of each replacement policy with their corresponding A(1) version the version that does not cache single-access objects. The results are shown in Figures 6-8. For all the graphs shown in Figure 6-8, solid lines are used to represent the original policies, while dotted lines represent their frequency-aware versions. Figure 6 shows the results for offline policies and their A(1) versions on the SJ trace. From Figure 6, it is clear that the performance gaps for these offline policies are small, as expected. In particular, the gap between and - A(1) is almost negligible, which shows that it is able to deal with single-access file even though it does not directly use frequency information. The results for online policies (,, ) are shown in Figure 7. Figure 7 clearly shows that the A(1) versions of,, and perform significantly better in terms of HR and BHR. The performance gaps for online policies is significant typically increase with the cache size. For example, for a cache size of 4%S, the performance gaps in HR for,, and are 4.4%, 9.8%, and 5.6%, respectively. Similarly, the performance gaps in BHR are 2.2%, 3.2%, and 3.4%, respectively. In fact, all the A(1) versions converge to HR and BHR when the cache size reaches about 10%S. This can be explained as follows: Recall from Section 5 that the singleaccess files accounts about 90% of S. This means that the multiple-access files (those with frequency 2 or more) has a total size of only about 0.1%S. Therefore, when the cache size is at 10%S, the size of the cache is fast approaching 795 the total size of all uniques files considered by the A(1) policies. Therefore, almost all the replacement policies will have performances converging to HR and BHR. The results in Figure 6-7 suggest that, are good choices for further study since they both have good BHR and HR and reasonably large performance gaps. Therefore, for these two replacement policies, we also run - A(k) and -A(k) for k=1,2,3. The results (not shown here due to space limitation) suggest that the gain comes mostly from considering the A(1) version. 7. PREDICTION OF SAO Given the significant impact of single-access-objects on HR and BHR, it is natural to ask if we can design replacement policies that can predict them. We now present some preliminary results on the prediction of single access objects. Our prediction algorithm, -Pred, actually attempts to predict multiple access objects (MAO s) since they easier to predict than single access objects (SAO s). Our simple prediction algorithm is based on two observations: (a) Objects that have appeared before are not SAOs. (b) Popular objects usually come from popular hosts. For a more practical online implementation, we use an appearance window W app that keeps track of all the objects accessed within a window of size N app and a host ranking window W host, which tracks the set of popular hosts based on history of past accesses. In our current implementations, we have set N app = 50, 000 and W host to be the set of all hosts with more than 500 previous accesses. The details of -Pred are given below:
of, -P and -A for SJ trace of, -P and -A for SJ trace -P -A Figure 8: Performance of prediction algorithm - Pred for SJ trace. procedure P redict(r k ) begin if (Cache can accomodate r k ) then PredMAO:=true elseif (r k W app) then PredMAO:=true elseif (host(r k ) W host ) then PredMAO:=true else PredMAO := false; if (PredMAO=false) then Do not cache r k else Cache r k using replacement policy; end We note that the assumptions made by -Pred are rather simplistic. For example, while a host may be very popular, not all the files under it are equally popular. Also, -Pred do not factor in other factors such as file sizes, recency, and so on. However, the aim here is to illustrate the power of prediction in improving performance namely, that a simple prediction algorithm can already improve performance. We compare the performance of -Pred with that of and -A(1). The -Pred algorithm requires a warm-up period to fill initialize W host and W app. Therefore, to get a fair comparison, we set the same warm-up period for all three algorithms. The results are shown in Figure 10. The HR for -Pred lies between that of and -A(1), about 1/3 of the gap distance. For BHR, it performs uniformly better than and especially well for small caches (up to 2%S ). The results in Figure 10 clearly show (a) the power of prediction namely, that even a simple prediction algorithm like -Pred performs better than, (b) that there is still room for improvement as shown by the distance (for both HR and BHR) from -A(1). The latter point is not -P -A 796 unexpected since -Pred is a very simple algorithm and did not take into account many other important factors in its prediction. We expect that more sophisticated prediction algorithms should be able to do much better, although it is not known how close to -A(1) one can achieve with an online -variant. 8. CONCLUSION In this paper, we show that it is beneficial not to cache infrequent objects in proxy caching. To gain insights to this issue, we first propose offline frequency-aware versions of well-known replacement policies like,, and so on. In particular, we show that not caching singleaccess files leads to improvements in the hit rates and byte hit rates of and. This shows that the is potential for improving these replacement policies. We also present preliminary results using a simple online prediction algorithm, -Pred, that outperforms. We are currently researching more sophisticated prediction algorithms to improve proxy cache performance. 9. REFERENCES [1] Proxy trace log, national laboratory for applied network research. http://www.nlanr.net. [2] M. Arlitt, L. Cherkasova, J. Dilley, R. Friedrich, and J. Tai. Evaluating content management techniques for web proxy caches. HP Lab. Technical Paper, pages 98 173, 1998. [3] L. A. Belady. A study of replacement policies for virtual storage computers. IBM Systems Journal, (5):78 101, 1966. [4] L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker. On the implications of zipf s law for web caching. In 3rd Intl. WWW Caching Workshop, June 1998. [5] P. Cao and S. Irani. Cost-aware www proxy caching algorithms. USENIX Symp. on Internet Technology and Systems, pages 193 206, Dec 1997. [6] L. Cherkasova. Improving www proxies performance with greedy-dual-size-frequency caching policy. HP Computer Systems Laboratory, 1998. [7] S. Glassman. A caching relay for the world wide web. First Intl. Conf. on the World-Wide Web, pages 69 74, May 1994. [8] S. Hosseini-Khayat and J. R. C. Jr. Optimal solution of off-line and on-line generalized caching. Technical Report WUCS-96-20, 1996. [9] H. H. Z. Qiang Yang and T. Li. Mining web logs for prediction models in www caching and prefetching. Proc. ACM SIGKDD, August 2001. [10] M. Rabinowich and O. Spatscheek. Web caching and replication. Addison-Wesley, 2002.