Measurement Based Intelligent Prefetch and Cache Technique in Web

Transcription

1 Measrement ased Intellient Prefetch and Cache Techniqe in Web Zhen Zhao, Jie Yan, Son Wan, Gan Zhan, Yantai Sh Department of Compter Science, Tianin University, Tianin 30007, P.R.China Tel.: , Fax: Abstract In Web, sers in the same workrop miht have similar interests and habits. In this paper we stdy intellient proxy techniqes for people who se a proxy to access Web. Or intellient proxy has two parts: cache and prefetch. We researched three replacement policies for proxy cache: LRU and two variations of LRU. Or simlation showed that basic LRU prodces worst hit rate, while the other two policies achieve rohly the same reslts. Therefore, we choose a mixtre of them as or proposed scheme. y introdcin the prediction alorithm and threshold alorithm, The proxy can predict which Web files will be needed in the near ftre and download some of them before they are really reqested by the ser rop. We can et a more accrate probability by rnnin it on the proxy server than on the client site, when client ser has not visited a file in a Web server often enoh. y implementin the techniqes of prefetch and cache on the proxy server, we can redce the Web latency perceived by sers and also the total nmber of access reqests to Web server, ths propose a reasonable way of decreasin their cost to access the Web for developin contries.. INTRODUCTION Generally speakin, people workin in the same rop may have similar interests and habits. In other words, it's very likely that the paes they've reqested may relate to each other or even be the same one from the Web. Accordin to this characteristic, we proposed an intellient proxy techniqe for these people, who access the Web throh a proxy server. Compared to the existin proxy techniqes, or intellient proxy has two main distinishin featres. First, we enhanced the eneral proxy by addin a prefetch fnction to it. The prefetch techniqe has been sed to combine with the client browser [], bt has never been implemented in a proxy. The proxy, bein exposed to the Web accesses of mltiple sers who have similar interests, has the potential to predict more accrately which paes a ser miht access next, ths it can prefetch those paes to the ser's machine on behalf of the ser. Second, we made some improvements on the cache replacement alorithm of the eneral proxies, which sally se simple FIFO or pre LRU (Least Recently Used) alorithm. We se a variant of LRU alorithm, which achieved a mch hiher hit rate in the limited cache space. In this paper, we first analysis rop interests distribtion, then we desin or cache alorithm to be sed on the proxy and investiate an approach to redce the ser-perceived latency by prefetch from the Web server to the proxy. In the last section, we present the implementation scheme of or intellient proxy server.. THE ANALYSIS OF GROUP INTERESTS DISTRIUTION We assme that members in one rop may have similar interests and will access the Web for similar paes. To analyses this sitation we have stdied three different rops for their Web access traces. In stdyin the lo files of three months, we cont statistical data for all the visited URLs and their freqencies. We define Q n P n specified pae access cont all paes access cont pae cont havin specified Q total cont of access paes X n P n Q n > q n The relation between X n (%) X n and n Qn is shown in Fire Q n (%) Fire : Distribtion of rop interests

2 The fire can be sed to determine the similarity of the rop. In this fire, we have observed that X n (q n 5%) of the three rops are 0%~0%, which denotes that rop sers have enoh similarity. 3. IMPROVEMENT TO CACHE ALGORITHM If a docment is cached when it is accessed first time by a ser, it will be retrieved more efficiently on sbseqent accesses by the same ser or by the same rop members. A cachin proxy does a difficlt ob. First, in order to have a cache hit, the same docment has to be either reqested by one ser twice or more, or be reqested by two or more sers. Second, since most web browsers today have a bilt-in cache, a ser is nlikely to reqest a sinle docment twice in its life cycle. Ths the cache proxy can only have a hit when two or more sers reqest the same docment. This redces the fraction of reqests that the proxy can satisfy from its cache, known as the "hit rate". How effective cold a cachin proxy ever be? We first simlate a proxy server with nlimited disk space, so that cached docments are never deleted de to space limitations. This ives an pper bond on the hit rate a real proxy server can ever achieve. The simlation inpt was athered from several traces of all Web accesses drin 3 months. We observed 30-50% hit rate. We then considered the sitation when we have only limited disk space. Some files have to be replaced de to space limitations. Three replacement policies were examined: LRU and two variations of LRU. Or simlation showed that basic LRU prodces worst hit rate, while LRU-MIN and LRU-THOLD [] achieve rohly the same reslts. So we choose a mixtre of LRU-MIN and LRU-THOLD as or proposed cache scheme. 3.. Cache Replacement Policy Sppose that the size of an incomin reqest docment is S, and the file is not in the cache LRU Policy Classic least recently sed: When the free space in the cache is smaller than S, repeat the followin ntil the free space is at least S: replace the LRU docment. (LRU may discard many small docments to make room for one lare docment.) 3... LRU-MIN Policy A variant of LRU that tries to minimize the nmber of docments replace: Let L and T denote, respectively, a list and an inteer. () Set T to S. () Set L to all docments eqal to or larer than T. (L may be nll.) (3) Remove the LRU docments of list L ntil the list is empty, or the free cache space is at least T. (4) If the free cache space is not at least S, set T to T/ and oto () LRU-THOLD Policy A variant of LRU can avoid the sitation in which a docment that is lare compared to the cache size cases replacement of a lare nmber of smaller docments. This policy is identical to LRU, except that no docment larer than a threshold size is cached. (Even if the cache still has room.) 3.. Simlation Reslts We compared docment replacement policies LRU, LRU-MIN and LRU-THOLD, to identify which maximizes hit rate and minimizes disk size reqired for the cache. The reslts are in Table. Table : Comparison of three cache replacement policies Hit Rate (%) Cache Size (Mbytes) Lifetime (Hors) Min Max Min Max Min Max LRU LRU-M LRU-T In this table, iven an infinite disk space we confirmed that a cachin proxy has an pper bond of 30-50% in its hit rate as described in the reference [3]. In no case did LRU otperform the other replacement policies in Table. Given that the optimal threshold for LRU- THOLD is a fnction of the workload and the ratio of the disk size available for the cache compared to the cache size needed for no replacement, LRU-MIN is the best policy. It reqires no parameters and achieves the best performance most of the time. On the other hand, LRU-THOLD achieves dramatically smaller cache sizes with a small penalty in hit rate compared to LRU-MIN, ths LRU- THOLD is recommended when disk size is limited compared to the cache size reqired for no replacement. Finally, the lifetime reported in the table shows that a docment stays in the cache for reatly varyin times as mch as ten times loner for LRU-THOLD compared to LRU. The short lifetimes for LRU miht explain why its hit rates are never hiher than the other policies: docments are replaced too freqently. Therefore we decide to se the adaptive policy: se LRU-MIN ntil the cache size approached 00% of the available disk size and then chane to LRU-THOLD with a threshold that is radally redced ntil the cache size reaches a low water mark. 4. THE PREDICTION ALGORITHM

3 The prediction alorithms in this stdy observe information collected from past accesses from all the sers to predict what each ser miht access next. The proxy needs to lo all HTTP reqests of the sers and adapts its prefetch activity based on them. In or scheme, the proxy is reqired to maintain two kinds of conters, the pae conters and the link conters, to keep track of each ser's access history. Each pae A is associated with a pae conter C A. In addition, if pae can be accessed directly from pae A, in other words, there exists a hyperlink of on pae A, then we se conter C (A,) to denote this link. Whenever pae A is accessed, the conter C A is increased by one. Similarly, the conter C (A,) is also increased by one, if pae is accessed by clickin on the correspondin link on pae A. The cmlative vale of conter C A and C (A,) can be achieved by addin p each ser's C A and C (A,) respectively. We se conditional probability P( A) to denote the probability of "a ser is to access pae riht after he or she accesses pae A". Therefore, the personal access probability is obtained in the followin way. When a pae A is bein viewed by the ser, for each pae i linked to A, the access probability of i is compted as P( i A) C (i,a) /C A. As described above, the rop sin the proxy has very similar interests. Therefore, the definition of rop access probability has sinificant meanin in the proxybased prediction alorithm. Assmin there are k members in the rop, the rop access probability of i is defined as where P ( i A ) k k C C denotes the th rop member's C A A. ( C A We have three choices to define or final sed access probability. The first one is to se personal access probability p. It is known that the accracy of prediction is closely related to personal interests, which is well indicated by this probability. However, when the ser's Web access history information is rather scant, the probability may not really reflect his or her real interests. The second choice is to se the rop access probability p. Jst as the rop has some common interests, the accmlative data can represent the poplarity of the paes over the rop members. Therefore, this probability may reflect the ser's interests to a certain extent, especially, the more similar the rop members' interests are, and the better this probability fits. Last bt not the least, we introdce the third choice, which we have adopted in this paper, interatin the advantaes of first two choices. y introdcin a weiht β, here we present the interated access probability as p β p + ( - β ) p (), A ) where 0< β <. We can emphasize the personal data or rop data by adstin the vale of β. 5. THE THRESHOLD MODULE Or threshold alorithm is based on that described by Zhimei Jian []. However, there are a few noteworthy differences. First, their scheme was desined for se by a sinle ser to prefetch files on the Web, in which the prefetch enine was located on the ser's local machine. Or model is to pt the prefetch enine on the proxy server, takin advantae of the accmlative data collected on the proxy to achieve a better prediction reslt. Therefore, or threshold alorithm is desined for sers in a rop, who share the same proxy server. Second, by adstin the parameter β, we can obtain their reslts as a limit case of or alorithm; that is to say, or model has a wider application in some senses. In order to et a better tradeoff between system resorce sae and latency, or prefetch stratey first predicts which files are likely to be needed soon and chooses some of them to download beforehand. The first part of this task is accomplished by the prediction modle that we discss in the last section. In the threshold modle, we determine the prefetch threshold for the Web server in real time to determine which files to prefetch. We se the term cost to measre the system performance, which is comprised of the delay cost (α T $/time nit) and the system resorce cost (α $/time nit). The delay cost indicates how valable the time is to the ser. The system resorce cost incldes the cost of processin the packets at the end nodes and that of transmittin them from the sorce to the destination. In this section, we stdy how to determine which files to prefetch in order to minimize the averae cost of reqestin a file for several system models. 5. The cost fnction C For a iven system, let be the arrival rate of ser reqests for paes when no prefetch is applies. We assme that prefetch does not affect the ser's behavior reardin the likelihood of accessed paes. In other words, when prefetch is employed, sers still isse reqests at rate in the same pattern, althoh they can et some paes faster de to prefetch. Let the arrival rate of normal reqests and prefetch reqests are and respectively. Hence the rate at which ser reqests are satisfied by the prefetched files is -, which is simple p becase prefetched paes are eventally reqested by the ser with probability p, where p is the access probability of the prefetched paes. Ths + p, or + + ( - p) () In a Rond-Robin processor-sharin system, the averae response time for reqests reqirin an averae of

4 x time nits of processin is x s t (3) ρ b ( ρ ) where ρ is the system load, s is the averae file size, and b is the system capacity. For the system shown in Fire, ρ s ( + ) / b. Therefore, the cost of a normal reqest, which is the sm of the system resorce cost and the delay cost, becomes s c α s + α t α s + α (4) T T b ( + ) Normal reqests Prefetch reqests Fire : The prefetch system model We can see that in the above eqation, as more files are prefetched, the cost of normal reqests increases becase prefetch increases the system load, which will increase the delay of retrievin files. The averae cost of a prefetched reqest is c α s (5) We obtain the final averae fnction of cost as follows: C c + c s ( p ) αt {[ + ( p) ] α + } b [ + ( p) ] s (6) Pt () into (6), we obtain or obective cost fnction. C s {[ + [ β p - ( - β )p ] ] α (7) + [ ( β p + ( - β ) p ) ] α T } b [ + [ β p - ( - β ) p ] ] s Processor sharin S will et its maximm vale at the zero point of Therefore, we can et the critical vale ', which is dc. d αt ( b( β p + ( β) p) s) ' ( b s ) s ( β p ( β) p) α( β p ( β) p) (9) Since fnction (7) is maximized at ', where dc 0 for d b (β p +( - β) p ) <s, it follows that the cost decreases as increases for > '. Specifically, if ' 0 for the iven p,, and r (α T /α ), then for any in the rane of [0,/p], the hiher the, i.e. the more that files with access probability p are prefetched, the lower the cost is. Ths, for the iven p,, and r, if ' <0, then prefetch all the files with access probability p will minimize the cost. 5.3 The prefetch threshold H Let s now find the prefetch threshold H sch that the cost can be minimized by prefetch all the files with access probabilities p, for p reater than H. From eqation (9), we obtain that ' 0 if and only if (0) ( β p ( ρ ) r + ( β ) p ) ( ρ ) b + r where ρs / b. We then set the prefetch threshold to be ρ () H ( ( ρ ) ) r b + Eqation (0) shows that if the access probability p is reater than or eqal to the threshold H, then ' 0 accordin to (9). Moreover, followin or previos analysis, this implies that prefetch all the files with access probability p minimize the cost, for p H. The threshold H is plotted in Fire 3 as a fnction of system tilization ρ for several different vales of r. 0.9 r 5. The optimm vale of the prefetch rate Let's assme the vale of p and are known, we wish to find at which point of, the averae reqest cost in the system can et its minimized vale. From the eqation (7), we take the second order derivative of C with respect to, and obtain d C s αt ( s b( β p + ( β) p ))( β p ( β) p ) [ ](8) 3 d ( b s ( + ( β p ( β) p ))) The prereqisite of a system to be stable is that + mst be less than b/s, that is to say s ( + ( β p - ( - β) p ) ) < b Therefore, as lon as s < b (β p +( - β) p ), the vale of (8) will definitely below the zero. It implies if s < b (β p +( - β) p ), the eqation (7) Prefetch Threshold H Utilization Fire 3: Prefetch threshold H as a fnction of tilization for different vale of r. Fire 3 shows that as system load ρ increases, the prefetch threshold tends to increase as well, which means that fewer files shold be prefetched. t the increase is r5 r0 r0 r50 r00 r00 r

5 not monotonic for small vales of r. The reason is, in those cases, when the load is low, prefetch does not save mch time. As the load increases, it takes loner to transmit files and prefetch can save more time, ths the threshold decrease. As the load contines to increase, prefetch files will add relatively lon delay to the normal ser reqests, so the threshold needs to be increased aain. In determinin the prefetch threshold, we shold also take into consideration other network performance parameters, sch as Web server load, nmber of roters in the end-to-end connection, bandwidth of links between roters and volme of traffic on each roter. With the aid of some existin measrement software, we can obtain the accrate vale of these data. Here we se Response-time as composite index of network performance. We introdce a network performance factor f n. The new threshold H R is obtained by the followin fnction: H R f H () n The relationship between Response-time and f n is shown in Table, which is determined by the thosands of measrement. Or reslts showed that settins in the table cold achieve optimized effects. Table : Relationship of Response-time and network performance factor f n Response Time (ms) < >000 f n IMPLEMENTATION OF INTELLIGENT PROXY Conceptally, a proxy server transfers characters between the clients network application and the remote web servers. To provide a fll-dplex connection between them, a proxy server mst wait for and read data from two sides. ecase the proxy cannot know which sorce of data will become available first, it cannot block indefinitely waitin for inpt from one of the two sorces withot also checkin for inpt from the other. The proxy server shold be sed for a rop of people and an iterative implementation can perform poorly becase it reqires a iven client to wait while it handles all prior reqests. And the prefetch fnction will increase the response time for mltiple sers. A concrrent implementation avoids lon delays becase it dose not allow a sinle client to hold all resorces. Finally, concrrency can help desiner separate control and processin from normal inpt and otpt. Ths, we choose the concrrent and connection-oriented implementation for better-observed response. We employ mltithread method to realize sch taret. We have two main threads: one to read and maniplate console commands and the other to accept clients connectin reqest. When a client send reqests, the server bein two new threads for every client, one for transferrin data from the web server to the clients and the other for transferrin data reverse direction. The proxy server responses as soon as possible whenever data is available on either end. The prefetch fnction incldes three main parts: the manaement of prefetched files, the interaction of prefetch threads and normal fetchin threads, and prefetch alorithm. In order to control all the threads, we desin lobal strctre links to control all prefetch threads and other threads. Every ser has one link. The link node is called as Control Node. Each thread has one Control Node. A Control Node incldes data fields to record threads stats and properties. It also has data fields for sinals in order to deal with mltithread synchronization and mtal exclsion. In the next section we discss the details of or implementation. 6. The comptation of predictive access probability: In order to compte the final sed accessed probability in formla () we have to record all the access history for every ser s personal conters and rop conters as well. Usin all these data, we can et the personal access probability and the access probability of the special rop. As to the weiht, we choose an adaptive method to determine it. When one special ser ses the proxy for loner time, we can et enoh historical data to predict for him. That means the weiht will be increased slowly. We choose the followin steps to determine the weiht, and the steps are proved effective in practice. In followin, we se C to denote the access freqency for a special pae of one sinle ser and se C for the collective access freqency of rop. C, C < 0 and C > 00 C C β tan, C > 0 π 0 C tan, C < 0 and C < 00 π 0 When C is less than 0 and C is more than 00, the rop access probability will take the sinificant part in the final probability. If C is less than 0 and C is also less than 00, we consider that sin C is more accrate. This sitation will vanish when the proxy server is sed. 6. The modification of compted H R In formla () we et the threshold H R for predictive

6 prpose. In practice, we have to inclde the inflence of compter performance. We mst take into accont the nmber of concrrent mltithreads, memory sed and other network performance parameters. To enhance the proxy server s whole performance, we arrane the threads to several ranks havin different processin privilee. All the normal fetchin threads have the hihest level privilee and the prefetch threads have lower privilee sorted accordin to the access probability. The privilee will indce a bias in maniplation of perspective prefetch acts. When the CPU is rather bsy to deal with clients normal reqests, we have to redce prefetch threads. So in practice we modify the threshold H R to a little hiher as follows: H ' H R + 0.* H R * U. The U is defined as the tilization of compter resorce and can be compted approximately sin a iven nmber dividin the nmber of crrent threads. In or realization, we se a compter with Intel Pentim II 350, 8M RAM and 6.3G HD as a proxy server for 0 sers. We set the iven nmber to 500. In practice, the nmber of threads existed enerally is less than 50 and the clients are satisfied with the response time. Fire 4 is the loic diaram of or prefetch proxy server. In Cache, not in prefetch Consistency Verify Not Expire Expire Initialization Listenin Cache and Prefetch File Searchin Not in both Waitin for Web Server In oth Not in cache, in prefetch Prefetch Take-over sally no more than 30% to 50%. The reason is simple: Most people browse and explore the web, tryin to find new information. One way to frther raise the cachin hit ratio is to anticipate ftre docment reqests and prefetch these docments in a local cache. Ths, sccessfl prefetch pls cache redce the web latency observed by sers, and lower both server and network load. It is known that limited network bandwidth in the developin contries are the main reason to the Web access latency perceived by the sers. Or intellient proxy techniqes in the Web can redce the sers' waitin time, and redce the total nmber of access reqests to Web server and network commnication cost. One practical application is for people in developin contries to make freqent access to the Web servers to obtain technoloy and economic information from developed contries. ACKNOWLEDGMENTS This research was spported in part by the National Natral Science Fondation of China (NSFC) nder rant No Here we thank Oliver W. W. Yan, professor of School of Information Technoloy and Enineerin at University of Ottawa, who provided sefl comments on a draft of this paper. REFERENCES: [] Zhimei Jian, An Adaptive Network Prefetch Scheme, IEEE International Conference on Commnications, Part (of 3), vol., Jne [] Marc Abrams, Charles R. Standride, Cachin Proxies: Limitations and Potentials, [3] S. Glassman, A Cachin Relay for the World-Wide Web, In First International World Wide Web Conference, pp , May 994. [4] J. Gwertzman, Atonomos Replication in Wide- Area Networks, Technical Report, Harvard University, pp. 7-95, 995. Send Data to Client Prefetch alorith Prefetch chosen link Fire 4: The implementation of intellient proxy server 7. CONCLUSION The Web traffic contines to increase at exponential rates [4]. Cachin docments closer to sers redces the nmber of server reqests and the traffic associated with them. Unfortnately, several recent stdies sest that the maximm hit rate achievable by any cachin alorithm is