An Efficient LFU-Like Policy for Web Caches

Transcription

1 An Efficient -Like Policy for Web Caches Igor Tatarinov Abstract This study proposes Cubic Selection Sceme () a new policy for Web caches. The policy is based on the Least Frequently Used () heuristic but takes into account ect size and possibly ect retrieval costs. We show that can be efficiently implemented and it results in high cache performance.. Introduction The World Wide Web (WWW) is often considered a major invention of the 99s. Since its introduction in 99, the Web has been constantly growing and so has the load on the Internet and Web servers. Today, many popular sites serve millions of requests every day. High network/server load is the major factor that negatively affects response time observed by Web surfers. Caching proved to be an easy and inexpensive way to make the Web work faster. Web ects (HTML documents, images, multimedia clips, etc.) can be cached at three different levels: on the Web server (a main-memory server cache), in the network (a proxy cache), and on the client (a built-in browser cache). Data caching in databases and file systems has been the subject of a tremendous number of research studies. Unfortunately, the results of those studies turned out to be inapplicable to Web caching. The traditional cache policies: Least Recently Used (LRU), Least Frequently Used (), etc. which showed good performance in database and file systems, do not perform well for the Web. The main reason for that is a different granularity of Web data items. Web caches deal with whole ects (of different size) whereas database and file system caches deal with data blocks (of the same size). Another important characteristic of Web caching is the possibility to use admission control, i.e., to not cache some of requested ects. A file system/database buffer manager always places requested data blocks in the cache (the cache serves as an interface between the storage subsystem and the application). On the contrary, a Web cache manager may choose not to admit a ect if this can increase cache performance (hit ratio).. Web Cache Performance Metrics Traditional storage systems use a single cache performance metric: the cache hit ratio, that is defined as the fraction of requests satisfied from the cache. When used in Web servers, this metric will be called the ect hit ratio (OHR). Another performance metric applicable to Web server caches is the byte hit ratio (BHR). The byte hit ratio is the ratio of the number of bytes of data fetched from the cache to the total number of bytes requested. In traditional caches, these two metrics are equivalent because all data items have the same size the size of the disk block. When ect retrieval time is independent of ect size, i.e., excessive bandwidth is available and latency dominates (as in a Web server cache), the ect hit ratio directly reflects the response time observed by the users. When ect

2 retrieval time is proportional to ect size (as in a proxy cache in a low-speed network), the byte hit ratio becomes a better metric. In many real-life situations, a more general approach may be needed. Several recent studies [CI97, S+97, WA97] proposed integrating an ect retrieval cost formula into the cache management policy. This results in new performance metrics, e.g., the delay savings ratio [S+97]. It is, therefore, essential that a Web cache policy be easily adjustable to different performance metrics (retrieval costs)., the cache policy proposed in this study, can easily account for various retrieval cost..2 Previous Work: the Early Years [W+96] proposed a simple taxonomy of Web cache policies. Cache policies are viewed as sorting problems that vary in the sorting key used. The following basic ect attributes (sorting keys) can be used: ect size, the last access time (last access recency), and the number of accesses. A combination of attributes can be used to select a victim. For example, (LRU-SIZE) removes the largest ect from the cache first. Objects of the same size are removed in the LRU order. This approach of using a combination of one or more basic ect parameters as a sorting key may seem to be universal. It does have a serious problem, however. If the first key parameter says that ect A is better than ect B, it does not matter how much better ect B might be in the second criterion. Object B will be removed from the cache before ect A. For example, suppose, ect A is bytes and it was last accessed, say, a week ago. Object B is only byte larger ( bytes) but it was accessed only a second ago. If an LRU-SIZE cache manager needed to choose which of the two ects to replace first, it would replace ect B only because it is byte larger. This problem can be partially solved by applying the integer logarithm function to SIZE (see [W+96] for details). () LRU-SIZE tends to discard large ects even when a small amount of free space is needed to accommodate a new ect. [W+96] is a more efficient version of LRU-SIZE. It takes the size of the incoming ect into account. first tests whether there are any ects equal or larger than the incoming ect. If yes, one of them is replaced using LRU. Otherwise, all ects larger than half the size of the incoming ect are considered. If such ects exist in the cache, one or more of them are removed by LRU. If not, the procedure is repeated using one quarter the ect size, and so on. This algorithm yields better cache performance but incurs more CPU overhead than the simpler LRU-SIZE., as well as LRU-SIZE, have one common problem: small ects may never be replaced regardless of how long ago they were accessed last time. (LRU-TH), proposed in [AS+95], avoids the above problem. LRU-TH requires a parameter, threshold, that determines the maximum size of a ect that can be cached. Other than that it is equivalent to the pure LRU. The OHR performance of LRU-TH is comparable to that of in many cases. However, it depends on the value of the threshold that can not be optimized a priori. Additionally, LRU-TH results in a very low byte hit ratio which makes this policy unattractive for use in Web proxies. 2

3 .3 The Knapsack Approach to Web Caching Several studies [AS+96, S+97] independently pointed out that the task of maximizing the expected performance of a Web cache can be considered as a knapsack problem. Indeed, if we assume that caching a ect has a certain benefit for the cache manager (future accesses to the ect will be hits) and that each ect has a weight (its size), then the cache manager has to maximize the total expected benefit from the cache given that the total weight of the cached ects cannot exceed the maximum capacity of the knapsack (cache size). More formally, let D be the set of all ects stored on the Web server, and let C, C D, be the set of cached ects. The task of the cache manager is to find the solution, C, to the following knapsack problem: max benefit( D) such that size( D) CacheSize. The benefit function actually determines the performance D C D C metric that is being optimized. It may also be adjusted (divided) by the cost of ect retrieval [S+97]. The knapsack problem is NP-hard. However, an approximate solution can be found by putting items in the cache in the value order where value benefit =. The approximate solution happens to be very precise when the number of weight items is large. By organizing ect entries as a heap, the task of handling a request (hit or miss) can be performed in O(log n) operations, where n is a number of ect in the cache [S+97]., on the other hand, requires a constant number of operations to handle a hit (only one ect entry is re-queued, see below). (SLRU) SLRU (Self-adjusted LRU) is proposed in [AE+96]. This is a knapsack policy that defines ect benefit as the ratio: where t is the current time, t last is the last ect access time (second to last, if the ect is ( t t last ) being accessed at moment t). SLRU results in a high cache hit rate but this algorithm is computationally very expensive since sorting has to be performed on every cache miss. () Pyramidal Selection Scheme () was designed by the authors of SLRU to reap the benefits of the latter while avoiding its high CPU overhead. divides cached ects into groups based on their log 2 SIZE value. Each group is maintained as an LRU queue. Only the head elements of each queue can be chosen as victims. Each time a replacement has to be made, a head element with the lowest value (see SLRU s value formula) is ejected from the cache. Note that SLRU can not be implemented using an ordinary priority queue because the ect value function (non-linearly) includes current time. (Static caching) [TRS97] proposes static caching, a different approach to Web caching. In static caching, the set of cached ects is updated periodically. The Web server s request log is used to compute the ects that should be cached during the following period. For example, if the period is one day, the values of all accessed ects are computed at the end of each day. The most valuable ects are placed in the cache. No new ects enter the cache until the end of the following day so the cache policy is indeed static. The ect value is defined as number of accesses ect size. Although, the study does not mention it, static caching essentially tries to optimize cache 3

4 performance by periodically solving the above knapsack problem. Static caching has very low CPU overhead and, in many cases, outperforms other policies. Its main disadvantage is slow adaptivity to changes in the workload. The dynamic version of the above cache policy was studied in [T98] where it was referred to as Weighted (). can be implemented using a heap-based priority queue as described above. One serious problem of the cache policies that are based on access counters is their slow adaptivity. An ect that is accessed many times may remain in the cache for a very long time despite that it is no longer accessed. Such situations can frequently occur in the Web. Web documents containing news are extremely popular on the day they are published but not popular at all on the following days. An simple solution to a similar problem is described in [CI96]. That approach requires however that a used-to-be-popular ect be requested again for its value to be downgraded., the policy that we propose, automatically downgrades values (access counters) of stale ects in the cache regardless whether those ects are requested again or not..4 Cache Implementation and Admission Control Issues Before we describe how works, let us mention that any ect cache requires a lookup table. The lookup table is usually implemented as a hash table. Given an ect identifier (oid), the lookup table quickly retrieves an ect handle associated with the give oid. The ect handle contains a pointer to the ect s location in the cache, and possibly other information, e.g., access counter. Several research studies [M96, AE+96, T98] pointed out the importance of admission control in Web caches. It has also been observed that efficient admission control requires retaining access statistics for some ects not currently in the cache. Since Web servers typically store a relatively small number of ects (<,), Web server caches can afford keeping access statistics for all stored ects. Web proxies and browsers, on the other hand, may potentially access any ect on the Internet. Clearly, a different approach is required in this case. For example, the cache manager can maintain another, auxiliary small cache that would contain fixed size entries. Each entry can be used to retain access statistics for a single ect (the ect itself is not cached). The policy that governs the auxiliary cache may be different from that of the main cache. 2. Cubic Selection Scheme () - an Efficient Implementation of Our implementation of has a certain resemblance to the way SLRU is implemented in [AE+96]. In that study a pyramidal selection scheme () is used to select which ects to eject from the cache. / being somewhat more complex than /SLRU results in significantly better performance especially with smaller cache sizes. is based on a cube-like data structure shown in Figure. The cube is formed by a matrix whose elements are groups (queues) of ect entries. Each ect entry is a pointer to the ect s handle in the cache lookup table. Object entries are pictured as smaller cubes within the cube. 4

5 I= log MaxX i Q I Q I Q IJ log X 2 Q i Q Q Q j Q J j LRU queue containing ects with the same log X and log S values X - ect s access counter value S - ect s size 2 log S J= log CacheSize Figure : The Cubic Selection Scheme. Each group contains (entries of the) ects that have the same value of log S and log X where S is an ect s size and X is an ect s access counter value. Accordingly, the height of the cube is log MaxX + where MaxX is the maximum possible value of an access counter whereas the width of the cube is log CacheSize +. (The cubes in our figures have the same width and height only for easier presentation.) The cube does not have a welldefined depth because the ect groups may have different sizes (depths). MaxX is used to prevent overflows of ects access counters. For example, if MaxX=255 ( byte is used to store the counter), an access counter is not allows to increase over 255. The cube will have a height of 8. The ect groups within the cube may be implemented in many different ways as long as the following two operations can be efficiently performed. Firstly, it should be possible to remove a given ect from a group. Secondly, it is essential that groups can be quickly merged (added to one another). (The reason why these two operations are important will be described later.) As can be seen from Figure, our implementation of uses LRU queues for ect groups. A double-linked LRU queue meets both of the requirements but also makes the algorithm of victim selection more intelligent. 2. How works When an ect is requested, the cache manager uses the lookup table to get an ect handle. If the ect is found, it is sent to the user and its access counter is incremented. The ect s entry in one of the queues is moved to the end of the same or different queue depending on whether incrementing the counter made the ect eligible for the next, higher level in the cube. Victim Selection If the requested ect is not cached, the cache manager uses conditions (*) to select a set of victims. If no suitable 5

6 i = log X d = d = d = - d = i j Q Q 2 Q 23 Q 34 Q 34 Top (lru) ects Q 23 Q 2 d = -J Q j = log S Figure 2 How selects victims. Queues on a diagonal d are not considered until all queues on smaller diagonals (-Jd-) have been exhausted.. Figure 3: One diagonal slice of the queue cube. The top ect with the smallest value is removed first on each iteration. set can be found, the ect is passed to the user without being cached. Else, the victims are ejected and the requested ect is placed in the cache. To select a set of potential victims, the cache manager scans the cube in a diagonal fashion as shown in Figure 2 and 3. A higher diagonal is only considered when all lower diagonals have been exhausted. On each diagonal, the top ect with the lowest value (access counter to size ratio) is selected first. If not enough space can be freed by removing that ect, another top ect is selected as a potential victim and so on until the diagonal is exhausted. A higher diagonal is considered next. The above process stops when enough cache space can be freed for the new ect or the total benefit of the potential victims overweighs that of the requested ect. The latter is an important condition that enforces admission control. The second condition also makes handling of large ects more efficient. Large Web ects are not usually popular (have a low benefit). As a result, such ects are rejected very quickly without considering many cached ects Cube Splitting Clearly, the above policy has a problem. If left unmodified, it will eventually result in all ects having the same (MaxX) value of the access counter and ect size will become the only criterion for victim selection. To avoid this problem, ect access counters should be split periodically. The cube allows for doing that very efficiently without actually updating each and every ect s counter. Whenever a split is desired, the cube is re-organized by moving all layers (but the lowest) one level down. As a result, layer 2 queues are appended to the corresponding layer queues whereas the highest layer becomes empty, see Figure 4. The split operation essentially performs a division of all ect access counters by two. Next time an ect entry is accessed, the counter has to be adjusted (divided by two). To implement this, a LastSplitTime value is stored in each ect entry. If LastSplitTime<CacheSplitTime, the counter and LastSplitTime are adjusted. One may notice that if 6

7 log X 4 2 Cube layers split log S Figure 4: A part of a typical cube before and after a split. Numbers in squares are queue lengths. an ect has not been accessed while the cube has been split twice, the ect s access counter will only be split (divided by 2) once. Such small mistakes however do not prevent the algorithm from achieving very high performance. To decide whether a split is necessary, the cache manager maintains a sum of logarithms of the access counters of all cached ects (Total). Therefore, given a cube, one can compute the value of Total as a sum of LayerNumber*ObjectsOnThatLayer. For example, the cube in Figure 4 has a Total of *2+*5+2*+3*+4*5=85. When an ect is moved one layer up in the cube, Total is incremented by one. When a new ect is added to the cache, Total is incremented by the number of the cube layer where the ect was put. Whenever the value of Total #ectsincache (TotalAvg) becomes greater than log MaxX /2, i.e., more weight shifts to the upper layers of the cube, the cube is split and Total is decremented by #ects in cache #ects in layer. For example, in Figure 4, left, TotalAvg=85/42 which is greater than log MaxX /2 = 2, hence, a split was necessary. After the split the value of Total becomes 85 (42-2)=4. Auxiliary Cache Operation In order for to work efficiently, it is essential that an auxiliary cache is maintained. As discussed in Section, such a cache should retain statistics for non-cached ects. If no auxiliary cache is maintained, new ects will always have their access counters equal to one and may never enter the cache. Such an auxiliary cache should also split access counters similarly to the main cache. 3. Simulation Environment 3. Characteristics of Traces Used in This Study For performance analysis, we used trace-driven simulation and request logs (traces) from the following three Web sites: ) the Computer Science Division of the University of California at Berkeley (UCB), 2) ClarkNet Internet service provider (this trace was also studied in [AW96]), and 3) InfoArt content provider. Table shows the characteristics of the studied traces. Data set size represents the total size of unique ects, i.e., the amount of disk space occupied by all requested ects. 7

8 Characteristic/Trace UCB ClarkNet InfoArt Trace start date Jan /97 Aug 28/95 Dec /96 Trace duration month 2 weeks 3 weeks Total valid requests 2,29,646 2,936,945,493,689 Avg request/day 73,924 29,78 64,943 Total Mbytes transferred 43,572 27,567,542 Avg ect size (bytes) 9,937 9,842 7,76 Number of unique ects 4,24 3,575 33,42 Data set size, DSS (Mbytes), Ideal Object Hit Ratio (cache size = DSS) Ideal Byte Hit Ratio (cache size = DSS) Table : Trace Characteristics 3.2 Simulation Details To detect cache hits we used a technique similar to that described in [W+96]. Objects are identified by their file names (paths). If an ect appears in the trace with a new size, it is assumed that the ect has been modified. Thus, a cache hit may only occur if the requested ect is in the cache and its size has not changed. Each simulation had a one-day warm-up period after which all statistical counters were reset to zero. Files that had cgi in their names were ignored since CGI scripts often mark their output as non-cacheable. 3.3 Studied policies and Results The list of studied cache policies is given in Table 2. Figure 5 shows the performance of the policies. Policy name Object attributes used when replacing when admitting LRU last access time always admits access counter always admits size and last access time always admits,, ect value ect value Table 2: Studied cache policies. Both and result in a very high ect hit ratio. is the best policy in terms of the byte hit ratio which can be explained by the fact that approximates a knapsack cache policy with a benefit function of ect_size*access counter. Such a benefit function is clearly oriented towards decreasing the number of bytes missed. The resulting value function is equal to access counter which is the same as in except that the latter does not enforces any admission control. seems to be a good policy for the InfoArt-like sites (news providers) where it performs better than other policy. As was mentioned above, this can be explained by s ability to quickly remove no-longer-unpopular ects from the cache. 8

9 .9.8 ClarkNet.9 InfoArt.9.8 UCB ect hit ratio ect hit ratio ect hit ratio ClarkNet.9.8 InfoArt.9.8 UCB byte hit ratio byte hit ratio byte hit ratio Figure 5: Performance of the cache policies. References [AS+95] M. Abrams, C. Stanbridge, G. Abdulla, S. Williams, and E. Fox. Caching Proxies: Limitations and Potentials. In Proc. of 4 th Int l Conference on WWW, Boston, USA, December 995. ei.cs.vt.edu/~succeed/www4/www4.html [AE+96] C. Aggrawal, M. Epelman, J. Wolf, P. Yu. On Cache Policies for Web Objects. IBM Research Report 269, [AW96] M. Arlit and C. Williamson. Internet Web servers workload characterization and performance implications. IEEE/ACM Trans. Networking, 5(5), 997. ftp://ftp.cs.usask.ca/pub/discus/paper.96-3.ps.z [CI97] P. Cao and S. Irani. Cost-Aware WWW Proxy Caching Algorithms. In Proc. Of USENIX Symposium on Internet Technologies and Systems, [M96] E.P. Markatos. Main Memory Caching of Web Documents. In Proc. of 5 th Int l Conference on WWW, Paris, France, May 996. www5conf.inria.fr/fich_html/papers/p/overview.html [RD9] J. Robinson, M. Devarakonda. Data Cache Management Using Frequency Based Replacement. In Proc. of ACM SIGMETRICS, Boulder, CO, USA, May 99. [S+97] P. Scheuermann, J. Shim, R. Vingralek: A Case for Delay-Conscious Caching of Web Documents. In Proc. of 7 th WWW Conf., Brisbane, Australia, April [T97] I. Tatarinov. Performance Analysis of Cache Policies for Web Servers. In Proc. of 9th Intl. Conf. on Computing and Information, ICCI 98, Winnipeg, Canada, June [TRS97] I. Tatarinov, A. Rousskov, V. Soloviev. Static Caching in Web Servers. In Proc. of 6th Intl. Conf. on Computer Comm. and Networks, IC3N 97. Extended version: [W+96] S. Williams, M. Abrams, C. Stanbridge, G. Abdulla, and E. Fox. Removal Policies in Network Caches for World-Wide Web Documents. In Proc. of ACM Sigcomm96, 996. ei.cs.vt.edu/~succeed/96waasf/ [WA97] R. Wooster and M. Abrams. Proxy Caching That Estimates Page Load Delays. In Proc. of 6 th WWW Conf.,