Clients Proxy Cache Servers Inter-proxy communication distance Proxy utilization and load balance Size of client population served Proxy Clients

Transcription

1 achig for a Better Web Large-Scale Web achig ad otet Delivery Jeff hase PS 22: Distributed Iformatio Systems Fall 2000 PerformaceisamajorcoceritheWeb Proxy cachig is the most widely used method to improve Web performace Duplicate requests to the same documet served from cache Hits reduce latecy, etwork utilizatio, server load Misses icrease latecy (extra hops) Hits Misses Misses Iteret liets Proxy ache Servers ache Effectiveess Previous work has show that hit rate icreases with populatio size [Duska et al. 97, Breslau et al. 98] However, sigle proxy caches have practical limits Load, etwork topology, orgaizatioal costraits Oe techique to scale the cliet populatio is to have proxy caches cooperate ooperative Web Proxy achig Sharig ad/or coordiatio of cache state amog multiple Web proxy cache odes Effectiveess of proxy cooperatio depeds o: Iter-proxy commuicatio distace Size of cliet populatio served Proxy Iteret Proxy utilizatio ad load balace liets liets liets Hierarchical aches otet-sharig Amog Peers Idea: place caches at exchage or switchig poits i the etwork, ad cache at each level of the hierarchy. Resolve misses through the paret. origi Web site (e.g., U.S. ogress) upstream Idea: Sice sibligs are close i the etwork, allow them to share their cache cotets directly. dowstream

2 Harvest-Style IP Hierarchies Issues for ache Hierarchies Examples Harvest [Schwartz96] Squid (NLANR) NetApp Netache cliet Idea: multicast probes withi each family : pick first hit respose or wait for all miss resposes. object request object respose query (probe) query respose With IP: query traffic withi families (size ) Iter-siblig IP traffic (ad aggregate overhead) is quadratic with. Query-hadlig overhead grows liearly with. miss latecy Object passes through every cache from origi to cliet: deeper hierarchies scale better, but impose higher latecies. storage A recetly-fetched object is replicated at every level of the tree. effectiveess Iterior cache beefits are limited by capacity if objects are ot likely to live there log (e.g., LRU). Hashig: ache Array Routig Protocol (ARP) Issues for ARP Microsoft Proxy Server GET a-f hash fuctio g-p q-u v-z Advatages. sigle-hop request resolutio 2. o redudat cachig of objects 3. allows cliet-side implemetatio 4. o ew cache-cache protocols 5. recofigurable o way to exploit etwork locality at each level e.g., relies o local browser caches to absorb repeats load balacig hash ca be balaced ad/or weighted with a load factor reflectig the capacity/power of each server must rebalace o server failures Reassigs (/)th of cached URLs for array size. URLs from failed server are evely distributed amog the remaiig - servers. misspealtyad cost to compute the hash I ARP, hash cost is liear i : hash with each ode ad pick the wier. Directory-based: Summary ache for IP ASummary-IP Hierarchy Idea: each cachig server replicates the cache directory ( summary ) of each of its peers (e.g., sibligs). [ao et. al. Sigcomm98] Query a peer oly if its local summary idicates a hit. e.g., Squid cofigured to use cache digests miss Summary caches at each level of the hierarchy reduce iter-siblig miss queries by 95+%. To reduce storage overhead for summaries, implemet the summaries compactly usig Bloom Filters. May yield false hits (e.g., %), but ot false misses. hit Each summary is three orders of magitude smaller tha the cache itself, ad ca be updated by multicastig just the flipped bits. object request object respose cliet query query respose 2

3 Issues for Directory-Based aches Servers update their summaries lazily. Update whe ew etries exceed some threshold percetage. Update delays may yield false hits ad/or false misses. Other ways to reduce directory size? Viciity cache [Gadde/hase/Rabiovich98] Subsettig by popularity [Gadde/hase/Rabiovich97] What are the limits to scalability? If we grow the umber of peers? If we grow the cache sizes? O the Scale ad Performace... [Wolma/Voelker/.../Levy99] is a key paper i this area over the last few years. first egative result i SOSP (?) illustrates tools for evaluatig wide-area systems simulatio ad aalytical modelig illustrates fudametal limits of cachig beefits dictated by referece patters ad object rate of chage forget about capacity, ad assume ideal cooperatio ties together previous work i the field wide-area cooperative cachig strategies aalytical models for Web workloads best traces UW Trace haracteristics A Multi-Orgaizatio Trace Trace UW Duratio 7 days HTTP objects 8.4 millio HTTP requests 82.8 millio Avg. requests/sec 37 Total Bytes 677 GB Servers 244,2 liets 22,984 Uiversity of Washigto (UW) is a large ad diverse cliet populatio Approximately 50K people UW cliet populatio cotais 200 idepedet campus orgaizatios Museums of Art ad Natural History Schools of Medicie, Detistry, Nursig Departmets of omputer Sciece, History, ad Music A trace of UW is effectively a simultaeous trace of 200 diverse cliet orgaizatios Key: Tagged accordig to their orgaizatio i trace ooperatio Across Orgaizatios Treat each UW orgaizatio as a idepedet compay Evaluate cooperative cachig amog these orgaizatios How much Web documet reuse is there amog these orgaizatios? Place a proxy cache i frot of each orgaizatio. What is the beefit of cooperative cachig amog these 200 proxies? Ideal Hit Rates for UW proxies Ideal hit rate - ifiite storage, igore cacheability, expiratios Average ideal local hit rate: 43% 3

4 Ideal Hit Rates for UW proxies Sharig Due to Affiliatio Ideal hit rate - ifiite storage, igore cacheability, expiratios Average ideal local hit rate: 43% Explore beefits of perfect cooperatio rather tha a particular algorithm Average ideal hit rate icreases from 43% to 69% with cooperative cachig UW orgaizatioal sharig vs. radom orgaizatios Differece i weighted averages across all orgs is ~5% acheable hit rate - same as ideal, but does t igore cacheability acheable hit rates are much lower tha ideal (average is 20%) Average cacheable hit rate icreases from 20% to 4% with (perfect) cooperative cachig acheable Hit Rates for UW proxies Scalig ooperative achig Orgaizatios of this size ca beefit sigificatly from cooperative cachig But we do t eed cooperative cachig to hadle the etire UW populatio size A sigle proxy (or small cluster) ca hadle this etire populatio! No techical reaso to use cooperative cachig for this eviromet I the real world, decisios of proxy placemet are ofte political or geographical How effective is cooperative cachig at scales where a sigle cache caot be used? Hit Rate vs. liet Populatio urves similar to other studies [e.g., Duska97, Breslau98] Small orgaizatios Sigificat icrease i hit rate as cliet populatio icreases The reaso why cooperative cachig is effective for UW Large orgaizatios Margial icrease i hit rate as cliet populatio icreases I the Paper.... Do we believe this? What are some possible sources of error i this tracig/simulatio study? What impact might they have? 2. Why are ideal hit rates so much higher for the MS trace, but the cacheable hit rates are the same? What is the correlatio betwee sharig ad cacheability? 3. Why report byte hit rates as well as object hit rates? Is the differece sigificat? What does this tell us about referece patters? 4. How ca it be that byte hit rate icreases with populatio, while badwidth cosumed is liear? 4

5 Trace-Drive Simulatio: Sources of Error What about Latecy?. Ed effects: is the trace iterval log eough? Need adequate time for steady-state behavior to become apparet. 2. Sample size: is the populatio large eough? Is it represetative? 3. ompleteess: does the sample accurately capture the cliet referece streams? What about browser caches ad lower-level proxies? How would they affect the results? 4. liet subsets: how to select to represet a subpopulatio? 5. Is the simulatio accurate/realistic? cacheability, capacity/replacemet, expiratio, latecy From the cliet s perspective, latecy matters far more tha hit rate How does latecy chage with populatio? Media latecies improve oly a few 00mswithidealcachig compared to o cachig. Questios/Issues. How did they obtai these reported latecies? 2. Why report media latecy istead of mea? Is the differece sigificat? What does this tell us? Is it cosistet with the reported byte hit ratios? 3. Why does the magitude of the possible error decrease with populatio? 4. What about the future? What chages i Web behavior might lead to differet coclusios i the future? Will latecy be as importat? Badwidth? Large Orgaizatio ooperatio What is the beefit of cooperative cachig amog large orgaizatios? Explore three ways Liear extrapolatio of UW trace Simultaeous trace of two large orgaizatios (UW ad MS) Aalytic model for populatios beyod trace limits Extrapolatio to Larger liet Populatios UW & Microsoft ooperatio Use least squares fit to create a liear extrapolatio of hit rates Hit rate icreases logarithmically with cliet populatio, e.g., to icrease hit rate by 0%: Need 8 UWs (ideal) Need UWs (cacheable) Low ceilig, though: 6% at 2.M (UW cacheable) A city-wide cooperative cache would get all the beefit Use traces of two large orgaizatios to evaluate cachig systems at medium-scale cliet populatios We collected a Microsoft proxy trace durig same time period as the UW trace ombied populatio is ~80K Icreases the UW populatio by a factor of 3.6 Icreases the MS populatio by a factor of.4 ooperatio amog UW & MS proxies Gives margial beefit: 2-4% Beefit matches hit rate vs. populatio curve 5

6 UW & Microsoft Traces UW & MS ooperative achig Trace UW MS Duratio 7 days 6.25 days HTTP objects 8.4 millio 5.3 millio HTTP requests 82.8 millio 07.7 millio Avg. requests/sec Total Bytes 677 GB N/A Servers 244,2 360,586 liets 22,984 60,233 Populatio ~50,000 ~40,000 Is this worth it? Aalytic Model Iside the Model Use a aalytic model to evaluate cachig systems at very large cliet populatios Parameterize with trace data, extrapolate beyod trace limits Steady-state model Assumes caches are i steady state, do ot start cold Accouts for documet rate of chage Explore growth of Web, variatio i documet popularity, rate of chage Results agree with trace extrapolatios 95% of maximum beefit achieved at the scale of a mediumlarge city (500,000) [Wolma/Voelker/Levy et. al., SOSP 999] refies [Breslau/ao et. al., 999], ad others Approximates asymptotic cache behavior assumig Zipf-like object popularity caches have sufficiet capacity Parameters: λ = per-cliet request rate µ = rate of object chage p c = percetage of objects that are cacheable = Zipf parameter (object popularity) Zipf [Breslau/ao99] ad others observed that Web accesses ca be modeled usig Zipf-like probability distributios. Rak objects by popularity: lower rak i ==> more popular. The probability that ay give referece is to the ith most popular object is p i Not to be cofused with p c, the percetage of cacheable objects. Zipf says: p i is proportioal to /i,forsomewith 0<<. Higher gives more skew: popular objects are way popular. Lower gives a more heavy-tailed distributio. I the Web, rages from 0.6 to 0.8 [Breslau/ao99]. With =0.8, 0.3% of the objects get 40% of requests. acheable Hit Ratio: the Formula N is the hit ratio for cacheable objects achievable by populatio of size N with a uiverse of objects. N = ÿ x = ÿ µ x + λ N x 6

7 Iside the Hit Ratio Formula Iside the Hit Ratio Formula, Part 2 Approximates a sum over a uiverse of objects......of the probability of access to each object x... times the probability x was accessed sice its last chage. What is the probability that i was accessed sice its last ivalidate? = (rate of accesses to i)/(rate of accesses or chages to i) = λnp i /(λnp i + µ) N is just a ormalizig costat for the Zipf-like popularity distributio, which must sum to. is ot to be cofused with N. = ÿ x = ÿ µ x + λn λ x =/Ω i [Breslau/ao 99] 0<< N = ÿ x = ÿ µ x + λ N x Divide through by λnp i. Note: byzipfp i =/i so: /(λnp i )=i /λn Hit Rates From Model Extrapolatig UW & MS Hit Rates acheable Hit Rate Focus o cacheable objects Four curves correspod to differet rate of chage distributios Believe eve Slow ad Mid-Slow are geerous Kee at 500K M These are from the simulatio results, igorig rate of chage (compare to graphs from aalytic model). What is the sigificace of slope? [Graph from Geoff Voelker] Latecy From Model Rate of hage What is more importat, the rate of chage of popular objects or the rate of chage of upopular objects? Straightforward calculatio from the hit rate results Separate popular from upopular objects Look at sesitivity of hit rate to variatios i rate of chage 7

8 Popular docs sesitivity Top curve Upopular low R-of- Issue is miutes to hours Upopular docs sesitivity Bottom curve Popular low R-of- Days to weeks to moth Upopular more sesitive tha popular! ompare differeces i hit rates betwee A, ad B, Rate of hage Sesitivity Hierarchical aches ad DNS What are the implicatios of this study for hierarchical caches ad otet Delivery Networks (e.g., Akamai)? Demad-side proxy caches are widely deployed ad are likely to become ubiquitous. What is the margial beefit from a supply-side DN cache give ubiquitous demad-side proxy cachig? What effect would we expect to see i a trace gathered at a iterior cache? DN iterior caches ca be modeled as upstream caches i a hierarchy, give some simplifyig assumptios. A Idealized Hierarchy N 2 N 2 N Level (Root) Level 2 Assume the trees are symmetric to simplify the math. Igore idividual caches ad solve for each level. Hit Ratio at Iterior Level i N gives us the hit ratio for a complete subtree coverig populatio N The hit ratio predicted at level i or at ay cache i level i over R requests is give by: hits at level requests to level i i h Rpc ( N i i Ni + = = r i ri + hi + the hits for N i (at level i) mius the hits captured by level i+, over the miss stream from level i+ ) Root Hit Ratio Predicted hit ratio for cacheable objects, observed at root of a two-level cache hierarchy (i.e. where r 2 =Rp c ): h N N2 = r N 2 Request Routig Fuctio ƒ(leaf, object, state) Geeralizig to DNs ƒ Iterior aches (supply side reverse proxy ) N I Leaf aches (demad side) N L N L N L N Symmetry assumptio: ƒ is stable ad balaced. 8

9 Hit ratio i DN caches acheable iterior hit ratio Give the symmetry ad balace assumptios, the cacheable hit ratio at the iterior (DN) odes is: NI NL NL N I is the covered populatio at each DN cache. N L is the populatio at each leaf cache. cacheable hit ratio fixed faout N I /N L Iterior hit rates improve as leaf populatios icrease... icreasig N I ad N L --> Iterior hit ratio as percetage of all cacheable requests margial cacheable hit ratio...but, the iterior cache sees a decliig share of traffic. icreasig N I ad N L --> 9