Exploring non-typical memcache architectures for decreased latency and distributed network usage.

Transcription

1 Syacuse Univesity SURFACE Electical Engineeing and Compute Science Technical Repots College of Engineeing and Compute Science Exploing non-typical memcache achitectues fo deceased latency and distibuted netwok usage. Paul G. Talaga Syacuse Univesity Steve J. Chapin Syacuse Univesity Follow this and additional woks at: Pat of the Compute Sciences Commons Recommended Citation Talaga, Paul G. and Chapin, Steve J., "Exploing non-typical memcache achitectues fo deceased latency and distibuted netwok usage." (20). Electical Engineeing and Compute Science Technical Repots This Repot is bought to you fo fee and open access by the College of Engineeing and Compute Science at SURFACE. It has been accepted fo inclusion in Electical Engineeing and Compute Science Technical Repots by an authoized administato of SURFACE. Fo moe infomation, please contact

2 SYR-EECS-20-0 Septembe 5, 20 Exploing Non-typical Memcache Achitectues fo Deceased Latency and Distibuted Netwok Usage Paul G. Talaga Steve J. Chapin ABSTRACT: Memcache is a distibuted in-memoy data stoe designed to educe database load fo web applications by caching fequently used data acoss multiple machines. While memcache aleady offes excellent pefomance, we exploe how data-locality can incease pefomance unde cetain envionments and wokloads. We build an analytical model, then compae typical configuations to a handful of poposed schemes which pomise to educe latency and/o educe coe netwok taffic. An example web application, MediaWiki, is pofiled and shown to benefit fom altenate Memcache configuations when used in a lage web fam. We develop metics to pedict any web application s pefomance unde ou schemes. KEYWORDS: Memcache, Web Technologies, Caching, Data Locality Syacuse Univesity - Depatment of EECS, CST, Syacuse, NY 3244 (P) (F)

3 Exploing Non-typical Memcache Achitectues fo Deceased Latency and Distibuted Netwok Usage Paul G. Talaga Syacuse Univesity Steve J. Chapin Syacuse Univesity Septembe 5, 20 Abstact Memcache is a distibuted in-memoy data stoe designed to educe database load fo web applications by caching fequently used data acoss multiple machines. While memcache aleady offes excellent pefomance, we exploe how data-locality can incease pefomance unde cetain envionments and wokloads. We build an analytical model, then compae typical configuations to a handful of poposed schemes which pomise to educe latency and/o educe coe netwok taffic. An example web application, MediaWiki, is pofiled and shown to benefit fom altenate Memcache configuations when used in a lage web fam. We develop metics to pedict any web application s pefomance unde ou schemes. Intoduction Oiginally developed at Danga Inteactive fo LiveJounal, the Memcache system is designed to educe database load and speed page constuction by poviding a scalable key/value caching laye available to all webseves. The system consists of memcache seves (memcached instances) fo data stoage, and client libaies which povide a stoage API to the web application ove a netwok o local file socket connection. No duability guaantees ae made, thus memcache is best used to cache egeneable content o in font of a eliable stoage system. Data is stoed and etieved via keys, that uniquely detemines the stoage location of the data via a hash function ove the seve list. High scalability and speed ae achieved with this scheme as a key s location (data location) can easily be computed locally. Complex hashing functions allow addition and emoval of memcache seves without seveely affecting the location of the aleady stoed data. This wok exploes diffeent data placement techniques on top of Memcache to educe latency and netwok usage. We pesent six achitectue vaiants, evaluate thei pefomance in a deived usage model, and offe insights into applications which could benefit fom these vaiants. Key to this wok is the fact that moving data any distance takes time and netwok esouces. While moden LAN netwoks in a data cente envionment allow easy and fast tansmission, locating the data vey close to whee it is used can lowe latency and oveall netwok usage esulting in bette oveall system pefomance [2]. The benefits ae amplified fo lage o multiple data centes. Just as CPU caches speed computation, we apply the same ideas to data caches in the webfam. The est of the pape consists of: Section 2 eviews the cuent Memcache seve and client achitectue, with example code. Section 3 sets up an analytical model fo latency, cache usage, netwok utilization, and configuation vaiables fo a memcache instance in a datacente which we use late to compae diffeent configuations. A specific usage example using MediaWiki [7], used by Wikipedia, is also given. Standad memcache achitectues, obvious extensions, as well as ou poposed achitectues inspied by multi-cpu caches [4] is outlined in Section 4. In Section 5 we estimate pefomance fo each achitectue unde ou model looking at aveage latency, netwok usage, and elative available space. We futhe discuss the esults and futue wok in Section 6, elated wok in Section 7, and conclude in Section 8. Appendix A contains all fomula used to estimate latency.

4 2 Memcache Details As peviously mentioned, Memcache is built using a client-seve achitectue. Clients, in the fom of an instance of a web application, set o equest data fom a memcache seve. A Memcache seve consist of a daemon listening on a netwok inteface fo TCP client connections, UDP messages, o altenatively though a file socket fo local clients. Daemons do not communicate with each othe, but athe pefom the equested memcache commands fom a client. An expiation time can be set fo data to automatically emove stale data. If the allocated memoy is consumed, data is geneally evicted in a least-ecently-used manne. Fo speed, data is only stoed in memoy (RAM) athe than pemanent media. The location of a memcache daemon can vay. Fo small deployments a single daemon may exist on the webseve itself. Lage multi-webseve deployments geneally use dedicated machines to un multiple memcache daemons outfitted with lage amounts of memoy (RAM). This facilitates management and allows webseves to use as much memoy as possible fo web scipt pocessing. Clients access data via a libay API. As of Memcached vesion.4.7 (8/6/20), 5 commands ae suppoted, categoized into stoage, etieval, and status commands. Vaious API s witten in many languages communicate via these commands, but not all suppot evey command. Below is an example of a PHP snippet which uses two of the most popula commands, set and get to quickly etun the numbe of uses. function get_num_uses(){ $num = memcached_get("num_uses"); if($num!== FALSE){ etun $num; }else{ $num = get_num_uses_fom_databaase(); memcached_set("num_uses", $num, 60); } etun $num; } Rathe than quey the database on evey function call, this caches the esult in memcache. A timeout is set fo 60 seconds so at most once a second the database would be queied, with the function etuning a numbe at most a minute stale. In this case no elation to the session is made, but session identifying infomation could be integated into the key and thus stoe session data. Using memcache to stoe session infomation combined with sticky load balancing (equests fom the same session end up in the same seve o ack) can allow data to be stoed close to whee it will be used fo geate speed, which we exploe late. Altenatively, as in the above example, some data may be used by all clients. Rathe than each webseve equesting commonly used data ove the netwok, it may make sense fo a local copy to be stoed, negating the need fo epeated netwok access. Some memcache clients, including PHP Memcache, include bulk set and get commands (multi-get) which sets o gets data to memcache seves in paallel, geatly inceasing pefomance. Rathe than loop ove a set of keys, this allows an aay of keys to be sent to the command. We exploit this featue as much as possible in ou implementations as should any memcache enabled web application. 3 Analytic Model To analyze the stengths and weaknesses of diffeent configuations we devise an analytical model to measue pefomance. Fo a ealistic evaluation, we pofile a popula open-souce web application (MediaWiki [7]) fo a sample of memcache usage. We then use the model and usage sample to analyze the pedicted pefomance of diffeent configuations. 3. Assumed Netwok Topology Netwok topology geatly influences the pefomance of any lage inteconnected system. We assume a netwok topology consisting of multiple web, database, and memcache seves oganized into a physical 2

5 Rack... Rack n Figue : Assumed Web Fam Topology hieachical sta topology. See Figue fo an example. Web seves, unning application code, ae gouped conceptually into acks. These can eithe be physical acks of machines, sepaate webseve theads in a single machine, o an entie data cente. Instances within the same ack can communicate quickly though one netwok switch, o within the same machine. The choice of what a ack descibes depends on the oveall size of the webfam. Whateve the ganulaity, communication is faste inta-ack ove ine-ack. Racks ae connected though a backbone link. Thus, fo one webseve in a ack to talk to anothe in a diffeent ack at least 3 switches must be tavesed. Goups of acks can be aggegated togethe into floos, ooms, o datacentes. To genealize the diffeent possible configuations futhe, we assume netwok latency is linealy elated to the switch (o othe device) count a signal must tavel though. Thus, in ou ack topology the estimated latency (ound-tip time) fom one ack to anothe is l 3 because thee switches ae tavesed, whee l 3 = 3 2 switch fo some switch delay value. Similaly l 2 epesents a equest tavesing 2 switches, such as fom a ack to a node on the backbone and back. l epesents tavesing a single switch, but we ll use the tem l local to epesent this closest possible netwok distance. In some cases we ll use l localhost to epesent l 0, whee no physical netwok laye is eached. This metic diffes fom hop count as we count evey device a packet must pass though, athe than only counting outes. 3.2 Netwok & System Constants We define a set of vaiables and constants which ae used to compae memcache configuations. Note we include aveage memcache command and key length in size object and size message to simplify ou calculations. This estimates the on-wie data size. Using these values fo stoage is not exact, but useful due to exta meta-data in the memcache seve pe data item. 3

6 l localhost Aveage latency to localhost s memcached(l 0 ) l local Aveage latency to a neaby node though one device(l ) l n Aveage latency tavesing n devices (ound-tip) whee l n < l m n < m Numbe of acks k Replication value ps Popotion of keys which ae only used fo a specific session [0 ] w Pecent of commands which ae eads switch Delay pe switch tavesed (ms) size object Aveage size of stoed object (bytes) size message Size of message used in Snoop and Di (bytes) 3.3 Assumptions To simplify ou model we assume the following taits:. Computation is instantaneous - Memcache lookup and code evaluate do not take any time. While false in pactice, computes keep getting faste (o moe coes), and the speed of signal tansmission will foeve emain fixed. Netwok latency is about equivalent to cuent compute time, with Facebook claiming a 73µs aveage latency fo Memcache handling UDP equests, well within the µs latency fo ethenet. [0] [3] [6] Thus, we conside this a constant cost. 2. Linea Netwok Latency - Netwok latency is linealy elated to netwok device tavesal count. [6] 3. Sticky Sessions - A web equest in the same session can be outed to a specific ack o webseve. We define a usage sceneio as the elative fequency of memcache commands an application uses. Some commands ae futhe boken down into a hit o miss as these may incu diffeent costs totaling 7 diffeent equest types. In ou case we will use the open souce application MediaWiki, which has built-in suppot fo Memcache, fo an example usage sceneio. Obtaining the fequencies via memcache instumentation is discussed next. 3.4 MemcacheTach Pedicting memcache usage is not easy. Use demand, web application code changes, netwok usage and design all can influence the pefomance of a memcache system. Instumentation of a unning system is theefoe needed, but few tools exist. The memcache seve itself is capable of etuning the keys stoed, numbe of total hits, misses, and thei sizes. Unfotunately this is not enough infomation to answe the following questions: What keys ae used the most/least? How many diffeent clients use the same keys? How many memcache equests belong to a single HTTP equest? How much time is spent waiting fo a memcache equest fom the client pespective? Which memcache seves ae esponding slowly? Which webseves ae expeiencing slow netwok pefomance? To answe these and othe questions we developed MemcacheTach, a memcache client wappe fo PHP which intecepts and logs all equests. While cuently analyzed afte-the-fact, the data could be steamed and analyzed live to give insight into memcache s pefomance and allow live tuning. Analysis povides values fo ps, switch delay, stoed data size, the atio of the 7 memcache equest types, as well as othe 4

7 switch (ms) 3 ps 0.9 Avg. data size (bytes) 92 w 0.6 Mem. equests pe page 6.3 Set 4% Add hit 0% Add miss 0% Replace hit 0% Replace miss 0% CAS hit match 0% CAS hit missmatch 0% CAS miss 0% Delete hit 0% Delete miss % Inc hit % Inc miss 24% Dec hit 0% Dec hit 0% Flush 0% Get hit 45% Get miss 4% Figue 2: MediaWiki usage values useful infomation about a set of memcache equests. Figue 2 shows the measued values fo MediaWiki ove a set of thee sessions of viewing and editing content totaling 2 pages. The aveage page used 6.3 memcache equests and waited 25ms fo memcache equests in ou expeiment. Notice the only memcache commands obseved wee set, incement, delete, and get, a athe limited selection. Supisingly 60% of keys wee only used by one seve, showing heavy use of session stoage, and thus a good fit fo location awae caching. MemcacheTach is available at with installation and usage instuctions. 4 Memcache Achitectues Hee we descibe the achitectue and ules fo configuations we analyze late using the pevious netwok model. As mentioned in the Intoduction, stoage location is detemined by each client. Similaly, all altenate configuations below ae implemented in client code via wappes aound existing clients Standad Deployment Cental - SDC The typical deployment consists of a dedicated set of memcached seves existing on the ack backbone (l 2 ). Thus, all memcache equests must tavese to the memcache seve(s). Data is stoed in one location only, not eplicated Standad Deployment Spead - SDS This deployment places memcache seves in each webseve ack. Thus, some potion of data (/) exists close to each webseve (l ), while the est is fathe away (l 3 ). Remembe the key dictates the stoage location, which could be in any ack, not the local Standad Deployment Replicated - SDR To add some duability to data, we stoe data on moe than one memcache daemon (pefeably on a diffeent machine o ack). While solutions do exist to handle duplication on the memcache seve (epcached [5]), we look at the moe geneal case whee the client pefoms eplication on its own. This allows easy memcached updates and keeps management close to the application. Implementation can be eithe though multiple stoage pools o, in ou case, modifying the key in a known way to choose a diffeent seve o ack. Reads have the advantage of choosing which pool/key to use and thus distibuting load o educing latency by choosing a neaby copy. 5

8 4.0.4 Snooping Inspied - Snoop Based on multi-cpu cache snooping ideas, this achitectue places memcache seve(s) in each ack allowing fast local eads. Wites ae local as well, but a data location note is sent to all othe acks unde the same key. Thus, all acks contain all keys, but data is stoed only in the ack whee it was witten last. An update cleas all notes and data with the same key. To avoid ace conditions deleting all data, notes ae stoed fist in paallel, followed by the actual data. Thus, in the wost case multiple copies could exist, athe than none. A etieval equest fist ties the local seve(s), eithe finding the data, o a note. If a note is found the emote ack is queied and the data etuned. Duability and speed can be gained by pobabilistically copying the data to the local seve if a note was oiginally found, though this limits available stoage space and is not analyzed late Diectoy Inspied - Di An altenate mulit-cpu caching system uses a cental diectoy to stoe location infomation. In ou case, a cental memcache cluste is used to stoe shaing/duplication infomation. Each ack has its own memcache seve(s) allowing local wites, but eads may equie etieval fom a distant ack. A etieval equest will ty the local seve fist, and on failue quey the diectoy and subsequent etieval fom the emote ack. A stoage equest fist checks the diectoy fo infomation, cleas the emote data if found, wites locally, and finally sends a note to the diectoy. The diectoy note can contain a pimay and multiple seconday ack locations allowing duplication fo duability and distibuted load. We diffeentiate this scheme with the Dik option Wite Though (l ) - WTR Hee a small memcache seve(s) exists in the same ack, with a Standad Deployment Cental (SDC) fathe away. Data is witten locally as well as globally. A etieval equest queies the local cache, and on failue eads the SDC fathe away. Due to multiple caches and no emoval system it is impeative to have shot expie times locally othewise stale data could be etieved Wite Though (l localhost ) - WTL Simila to the above WTR, but the local memcache seve exists on the same webseve machine and intefaces though the faste socket inteface. 5 Compaisons We evaluate the above achitectue options by estimating latency, netwok usage, and available stoage space ove each memcache command using ou above descibed model and vaiables. Web application usage heavily detemines pefomance of any memcache system. Thus, given a usage scenaio, netwok model, envionment vaiables, and the above achitectue policies we can estimate the aveage memcache latency, netwok usage, and available stoage. This allows a level compaison between achitectues and lets us exploe how altenate envionment vaiables could impove o hinde pefomance. 5. Latency Fomulas wee deived which pedict the estimated latency fo each of the 7 equest types ove all achitectues given the netwok model and envionment vaiables. Fo example the set command would have a latency of l 2 unde SDC, while the SDS would be l local + because some potion of the keys would be local. SDR and Snoop would take l 3 because multiple sets can be issued simultaneously. Di equies a ound tip to the diectoy to lean about othe acks which may aleady contain the same key. Theefoe, on a set the othe acks must be cleaed of that key so the new data is used in the futue. Thus ps of the time the key is l local, and the othe potion it is 6

9 Figue 3: MediaWiki pofile unde diffeent switch speeds and ps values. l 3 in the non-duplicated case. Finally, the data can be set and a note sent to the diectoy. This esults in a latency of 2 l 2 + ps l local + ( ps)l 3 + l local. Wite Though (both vesions) can set in paallel but must wait fo both commands to etun, thus taking l 2 time. Appendix A povides fomulas fo pedicted latency of all 7 equest types ove all achitectues. With all fomula, weightings fo commands used, and envionment vaiables we can compae how each achitectue pefoms unde diffeent paametes. We fist look at how netwok switch(device) speed can effect pefomance. Remembe we assumed the numbe of devices linealy elates to netwok latency, so we vay the single device speed between 2.7µs and.85ms, with an additional ms OS delay, in the Figue 3 plots. Response time measues ound tip time, so a switch speed of ms would esult in a 2ms latency. Thee plots ae shown with ps values of 0%, 50%, and 90% with weightings deived fom ou MediaWiki pofile above. As seen in Figue 3, ps dives favoable pefomance fo many of these schemes. Next we take a close look at how ps changes esponse time in Figue 4 using a fixed switch latency of 0.925ms and ou MediaWiki usage pofile. Pedictibly all 3 location avese schemes (SDC, SDS, and SDR) exhibit no change in pefomance as ps inceases. Additionally, SDC is the best when ps = 0 due to all othes having added ovehead. As ps inceases the othe 5 schemes impove with Snoop and WTL pefoming bette than SDC. So fa we ve analyzed pefomance using MediaWiki s usage pofile. Now we look at the moe geneal case whee we split the 7 possible commands into two types: ead and wite, whee ead consists of a get equest hit o miss, and wite is any command which changes data. MediaWiki had 60% eads, o 6 eads pe 4 wites. Figue 5 vaies the ead/wite atio while looking at thee ps values. 5.2 Netwok Usage While low latency is ou end goal, analyzing netwok usage is also impotant to pedict hotspots o moe cost-effective configuations. Recall ou assumed sta netwok topology. Webseves ae put in acks possibly including a memcache seve, with each ack connected to a backbone. A cental memcache seve may exist on the backbone. Hee we discuss the pedict the total bytes tansfeed fo the backbone switch compaed to SDC. Fewe bytes tansfeed equates to fewe packets, meaning less usage. We ignoe ack switch analysis fo most schemes since lowe backbone usage usually equates to lowe ack switch usage SDC In the typical situation a single (o cluste of) memcache seve sits on the backbone which sevices all equests. All equests and data must theefoe pass though the ack switch as well the backbone switch. 7

10 Figue 4: MediaWiki pofile unde diffeent ps values. Figue 5: Vaying ead/wite atio and ps values. 8

11 Depending on the numbe of webseves, the memcache link(s) may become a bottleneck as all taffic is funneled though them SDS Since object placement is not location awae, the majoity ( ) of data will still pass though the ack and backbone switch. Unfotunately each ack switch will handle twice the memcache load the majoity of the time ( ) as each equest will leave one ack and ente anothe. The emaining / will only pass though one switch SDR Lacking a location awae policy, the majoity of data ( ) will pass though the ack and backbone switch. Each wite will put k times (eplication value) as much load on all switches. Reads ae the same, though possibly local due to clients having an option whee to quey the data (choose least loaded o closest) Snoop This achitectue is location awae and will stoe data close to whee the the stoe was issued. The downside to this scheme is each set causes a stoe command note to be sent to all acks. Luckily a set command s data is stoed in its own ack s memcache seve, thus bypassing the backbone switch. Depending on the ps value of the application, eads can eithe be identical to SDS (all infomation passing though the backbone and two ack switches) when ps = 0, to the exteme ps = of no ead using the backbone because it can be seviced locally. To pedict backbone switch usage of Snoop we develop the following atio of total backbone taffic to that of SDC fo both ead and wite: o w ( ps) size object + ( w) size message size object w ( ps) + ( w) size message size object Fo example, in ou MediaWiki scenaio we calculate a 94% decease in backbone taffic using the following paametes with Snoop ove SDC: size message 20 (bytes) size object 5 (Kbytes) ps 0.9 w Clealy the diffeence between message, object size, and numbe of acks is impotant. Fo a wite in Snoop to have equivalent load on the backbone switch we have the following equality, size message = size object. Given ou values above we could have 750 acks befoe a wite is equivalent. A small message, lage objects, an application that uses moe eads, o a highe ps value will offe lowe netwok usage in this achitectue Di Since objects ae stoed close to whee they wee changed, the ps value geatly influences the oveall netwok load. Fo a ps value of no object data need coss the backbone switch, with only a single message sent to the epositoy pe change, athe than one message pe ack fo the Snooping inspied. Simila to Snoop, when ps = 0 all objects tavese the backbone. On a local memcache miss the epositoy will be queied, adding some additional netwok load, though all object location messages should be less than 20 bytes and thus smalle than the aveage object. The k option, whee multiple object copies ae stoed, will have slightly less backbone fo eads due to additional copies being local to some queies. 9

12 5.2.6 Wite Though Both vesions, WTR & WTL, pefom identically to the SDC in the wost case, o on statup, but as moe local copies ae stoed we d expect backbone usage to decease fo eads. Pefomance is quite difficult to pedict due to individual key ead/wite atios, local lowe expiation values, and global expiation values, plus oveall memcache usage. In the best case we d expect the backbone to cay each key s change, as well as a single quey fo each ack to fill its local vesion (ps = 0), o only a single ack if ps =. 5.3 Available Stoage Space Each achitectue exhibits diffeent pefomance, but at what additional hadwae cost to stoe the same amount of data? Hee we analyze available space fo each achitectue SDC This seves as ou baseline metic. No exta meta-data is stoed, so all available space is utilized fo object stoage. We ignoe intenal memcache efficiencies as these may change ove time SDS The only diffeence between this and Nomal Rack is the location of memcache seves, thus no exta infomation is stoed. 00% efficiencey compaed to SDC SDR Duplicating data educes available space. With a specific k value (eplication value), we aive at a utilization value of k 00% efficiency compaed to SDC. Fo example with k = 3, you would need thee times the the stoage space to stoe the same amount of data as a SDC Snoop Snoop s stoage of messages educes the efficiency of this achitectue. Fo values of size object, size message, and we aive at the following stoage efficiency fomula: size object size message ( ) + size object = Snoop efficiency compaed to SDC Fo example in ou MediaWiki case whee size object = 5Kb, size message = 20b, and = 0, we aive at an efficiency of 98%, baely enough to waant additional hadwae ove using SDC. Again, a lage aveage data size compaed to message size will incease efficiency Diectoy Inspied Rathe than stoe many messages though out the system like Snoop, Diectoy Inspied only stoes a single note in a cental location, inceasing stoage efficiency futhe. and if some k duplication is used, size object size message + size object = Di efficiency compaed to SDC size object size message + k size object = Di k efficiency compaed to SDC With the same MediaWiki 0 ack example a non-duplicated diectoy configuation has an efficiency of 99%, while simple duplication (k = 2), gives a value of 49%. 0

13 5.3.6 Wite Though Figue 6: Vaying ead/wite atio and ps values with an East and West coast DC Many factos influence the pefomance of wite-though (WTR & WTL), such as w, expiation values, and individual key ead/wite ations, as well as the fact that local stoage will neve be used fo pimay object stoage and thus is only useful fo educing netwok load and latency. Thus, the efficiency is given by the simple fomula: SDC Stoage Space SDC Stoage Space + Rack Stoage Space 6 Discussion & Futue Wok 6. Race Conditions As with any distibuted stoage system ace conditions must be addessed. Luckily fo ou achitectues, and memcache in geneal, eos ae allowed to occu. Data is not guaanteed to be pemanent, o coect, and as a esult geat speed can be gained. Applications using memcache must be awae of its limitations and deal with failues when they occu, usually by equesting the data fom a eliable souce and e-setting the object in memcache. As mentioned befoe, implementation changes can be used to minimize conflicts in specific achitectues. 6.2 Multi-Datacente Case Thus fa we have assumed a Memcache installation within the same datacente, with appopiate estimates on latency. In geneal unning a standad memcache cluste spanning datacentes is not ecommended due to high (elative) latencies and expensive bandwidth. The location-avese achitectues, SDC, SDS, and patly SDR would not be good choices fo this eason. We can apply ou same analysis to the multidatacente situation by viewing the datacente as a ack, with a high l 3, though ealistic, value fo intadatacente latency. SDC is no longe possible with its l 2 latency, with SDS taking its place as the typical off-the-shelf achitectue. If we assume a l 3 value of 40ms, a best case CA to NY latency, with l = 4ms inside the datacente, we aive at Figue 6 giving aveage latencies between the diffeent achitectues ove diffeent ps and ead/wite atios. Fo Di s diectoy we assume it spans both datacentes like SDS. Hee the diffeence between locality awae and avese is moe ponounced. Snoop, Di(k), and WTR ae able to outpefom SDC fo most cases except high wite volume and low ps values. SDR pefoms pooly due to consistency checks between the two copies. Inteestingly as moe datacentes ae added SDC becomes wose due to a highe popotion of data being fathe away while the location awae achitectues can keep it close.

14 Regadless of the lowe latencies ove a standad deployment in the multi-datacente case, caching may not be beneficial ove many isolated caches. 6.3 Selective Replication Replication of a elational database can incease pefomance by distibuting eads. Unfotunately entie tables must be eplicated, possibly including seldom used data. In a key/value system such as Memcache eplication can offe speed benefits as we saw in SDR. We looked at the static case whee all data is eplicated, but why not selectively eplicate fequently used data so we don t waste space? As mentioned in the Snoop desciption, data could be pobabilistically copied locally. Thus, fequently used but infequently changed data would be eplicated allowing fast local eads. Unused Memcache memoy is a waste, so by changing the pobability of eplication on the fly memoy could be used moe optimally. We intend to investigate this in futhe wok. 6.4 Object Expiation In Memcache, if the allocated memoy is consumed objects ae geneally emoved in a least-ecently-used manne. In a standad deployment this woks well, but in ou case whee meta infomation is sepaate fo the data a possibility exists whee meta expiation may cause a miss followed by a set on an aleady stessed node. Adjusting expie times, o modifying Memcache to etun the set expie time could alleviate this issue by pobabilistically etuning a miss when the expie time is nea zeo. This would have the added benefit of placing the subsequent set on a heavily accessed node and place the data close to whee it is used the most. 6.5 Use Space Caching As mentioned in Section 2, the Memcache seve is a sepaate pocess even when used on the same machine as the client. Inte-pocess communication is theefoe necessay, eithe though the loopback o file socket intefaces. Eithe way, tanspotation costs ae incued. By moving the memcache seve, o emulating it, within the same pocess space as the client the tanspotation costs could be eliminated. Object seialization ovehead may also be educed. We plan on investigating this option futhe. 6.6 Oveflow Memcache s standad deployments (SDC,SDS, SDR) all behave well when they each memoy satuation by emoving objects evenly in a least-ecently-used manne due to the hashing stategy. Ou location awae achitectues (Snoop & Di) may ove-stess some acks by stoing all objects on them while neaby acks emain empty. While possible, this situation should not become a poblem as long as web equests ae evenly spead acoss all web seves. Thus, the cache as a whole should not be filled fom a single client, but athe fom each ack. 7 Related Wok Memcache is pat of a lage key/value NoSQL data movement which povides enhanced pefomance ove elational databases by elaxing ACID guaantees. One way key/value systems can scale well is by using a hashing system based on the key to uniquely identify an object s stoage location. To achieve localeawae caching we must use some othe key/location mapping system. Hee we discuss simila systems to Memcache and concentate on those which has some locality component. The Hadoop Distibuted File System [2] is designed to manage distibuted applications ove extemely lage data sets while using commodity hadwae. They employ a ack-awae placement policy able to place data eplicas to impove data eliability, availability, and netwok bandwidth utilization ove andom placement. Reads can ead fom a close copy athe than a emote one. Additionally they identify bandwidth limitations between acks, with inta-ack faste, suppoting ou achitectues. Thei manta of Moving 2

15 Computation is Cheape than Moving Data woks fo lage data sets, but in ou web case we can wite the data close to whee it will be used instead. File meta-data is stoed and managed in a cental location simila to ou Di achitectues. Cassanda can also use a ack and datacente awae eplication stategy [] fo bette eliability and to ead locally. This is convenient when multiple datacentes ae used so a ead will neve quey a emote data cente. Voldemot uses a simila system fo eplicas using zones [9] [2]. While the above thee systems use some locality awae policy fo eplicas they all use a hashing stategy fo pimay stoage, thus possibly witing the data fa fom whee it will be used. We plan on investigating weathe locality awae pimay stoage could be beneficial to these systems, o some mixtue. Micosoft s AppFabic povides vey simila sevices to Memcache with in-memoy seializable object stoage acoss many computes using key/value lookup and stoage [8]. No mention of key hashing is given to locate data, though they do mention a outing table to locate data simila in pactice to ou Snoop achitectue. No mention of how this table is updated. Thei outing table enties efeence a specific cache instance, wheeas ou Snoop note efes to a hash space, o ack, possibly containing thousands of memcache instances, theeby giving moe stoage space and flexibility. Duability can be added by configuing backup nodes to eplicate data, though unlike ou achitectues all eads go to a single node until it fails, unlike ous whee any copy can be ead. EHCACHE Teacotta is a cache system fo Java, containing a BigMemoy module which pemits seializable objects to be stoed in memoy ove many seves. Java s gabage collection (GC) can become a bottleneck fo lage in-application caching, thus a non-gabage collected self-managed cache system can use lage amounts of memoy while ignoing typical Java GC issues. Essentially BigMemoy implements a key/value stoe using key hashing fo Java objects simila to the Memcache client and seve. Unlike memcache, it allows configuations othe than the standad memcache case. Fo example it allows a ead and wite though mode, backed by a duable key/value stoage system, theeby emoving all cache decisions fom the application code. Replication is done by default (2 copies) fo availability, with moe available. Additionally, it can delay witing to the duable stoe fo faste appaent wite speed. Simila to ou wok, a small local cache can be maintained, with many policies available. [] 8 Conclusion We ve seen that when a web application has a high ps value, many eads pe wite, o slow netwoking equipment, a location-awae Memcache achitectues can lowe latency and netwok usage without significantly educing available space. Ou example application, MediaWiki, showed that a significant amount of keys and data is session specific and thus can exploit such achitectues. Ou tool, MemcacheTach, can be used to measue a web application s detailed Memcache usage to estimate pefomance with altenate achitectues unde a simple model. As web applications gow, and thei use base becomes moe geogaphically divese, the need fo systems which can keep latencies low fo all uses is needed. Ou poposed efomulation of multi-pocesso cache systems fo Memcache show that bette pefomance can be cuently gained within the web datacente unde cetain cicumstances, with futhe gains found fo moe geogaphically distibuted data centes ove cuent techniques. Refeences [] N. Bailey. Fontpage - cassanda wiki, [2] D. Bothaku. Hdfs achitectue guide, design.html#data+blocks. [3] S. Cheshie. It s the latency, stupid, [4] J. L. Hennessy and D. A. Patteson. Compute Achitectue, Fouth Edition: A Quantitative Appoach. Mogan Kaufmann Publishes Inc., San Fancisco, CA, USA, [5] K. Inc. epcached - add data eplication featue to memcached, 3

16 [6] R. Inc. Latency on a switched ethenet netwok, notes/latency_on_a_switched_ethenet_netwok.pdf. [7] mediawiki. Mediawiki, [8] A. N. Nithya Sampathkuma, Mualidha Kishnapasad. Intoduction to caching with windows seve appfabic, [9] sumbaly. Voldemot topology awaeness capability, wiki/topology-awaeness-capability, 20. [0] P. Saab. Scaling memcached at facebook, , [] I. Teacotta. Ehcache documentation cache-topologies, distibuted_caching.html. [2] P. Voldemot. Poject voldemot, A Latency Costs ove each Memcache command fo each Achitectue See Section 3.2 fo vaiable definitions. 4

17 Nomal Nom Spead Replicate Snoop Di Di k WT Set l 2 l local + l 3 l 3 2 l 2 + l 3 2 l 2 + l 3 + max(l local, l 2 ) l local Add l 2 l local + 2 l local l 2 l 2 ps l local + hit ( ps) (l 2 + l local ) Add l 2 l local + 2 l local + l 3 l 2 + l local l 2 + l 3 l local + l 2 miss Replace l 2 l local + 2 l local + ps l 2 + ps l 2 + l 3 l local + l 2 hit l local + ( l local + ( ps) ps) Replace l 2 l local + 2 l local l 2 l 2 l local + l 2 miss CAS l 2 l local + 2 l local + ps 2 l 2 + l 3 2 l 2 + l 3 l local + l 2 hit l local + ( match ps) CAS l 2 l local + l 3 l local + ps l 2 l 2 l 2 hit no l local + ( ps) CAS l 2 l local + l 3 l local l 2 l 2 l 2 miss Delete l 2 l local + l 3 l 3 2 l 2 + ps 2 l 2 + l 3 max(l local, l 2 ) hit l local + ( ps)l 3 Delete l 2 l local + l 3 l 3 l 2 l 2 max(l local, l 2 ) miss Inc hit l 2 l local + 2 l local + ps ps l local + l 2 + l 3 l local + l 2 l local + ( ( ps)(l 2 + ps) l 3 ) Inc l 2 l local + l 3 l local l 2 l 2 l 2 miss Dec hit l 2 l local + 2 l 3 ps l local + l 2 + l 3 l local + l 2 ( ps)(l 2 + l 3 ) Dec l 2 l local + l 3 l local l local + l 2 l local + l 2 l 2 miss Flush l 2 l local + l 3 l 3 l 3 l 3 l 2 Get hit l 2 l n local + l local + ( l local + ( l local + ( ps l local + l local + ps) ps)(l 2 + l 3 ) ps)(l 2 + l 3 ) ( ps) (l 2 + n l local ) Get miss l 2 l local + k l local l local + l 2 l local + l 2 l local + l 2 5