A Non-blocking Directory Protocol for Large-Scale Multiprocessors. Technical Report

Transcription

1 A Non-blocking Diectoy Potocol fo Lage-Scale Multipocessos Technical Repot Depatment of Compute Science and Engineeing Univesity of Minnesota EECS Building 200 Union Steet SE Minneapolis, MN USA TR A Non-blocking Diectoy Potocol fo Lage-Scale Multipocessos Jinseok Kong, Pen-chung Ye, and Gyungho Lee Apil 08, 1999

2

3 A Non-blocking Diectoy Potocol fo Lage-Scale Multipocessos Jinseok Kong Pen-Chung Ye Gyungho Lee y Dept. of Compute Sci. and Eng. Univesity of Minnesota Minneapolis, MN , USA fjkong, yeg@cs.umn.edu y Division of Engineeing Univesity of Texas San Antonio, TX , USA glee@voyage1.eng.utsa.edu Abstact This pape pesents a non-blocking diectoy-based cache coheence potocol to impove the pefomance of lage-scale distibuted shaed-memoy multipocessos. In the poposed non-blocking diectoy potocol, all subsequent equests can poceed ithout being blocked at the diectoy. The citical path of a data access and the amount of netok tansactions needed to complete a memoy equest (including the tansactions needed to maintain coheence) ae also educed. To suppot the non-blocking diectoy potocol, the histoy of data accesses is maintained ith a small histoy table at the diectoy. Using detailed simulations, e evaluate the pefomance of the potocol compaed ith othe blocking diectoy potocols. Key ods { cache coheence, diectoy potocol, distibuted shaed memoy multipocesso, memoy achitectue. 1

4 1 Intoduction As pocesso clock ates appoaching 1GHz, memoy latency becomes a sevee pefomance bottleneck. The situation in a distibuted shaed memoy (DSM) multipocesso is even moe exacebated by its equied ovehead in enfocing cache coheence potocol in addition to the memoy access latency. In this pape, e look at ays to educe the memoy access latency and the ovehead in enfocing cache coheence potocols. Fo netok-based DSM multipocessos, diectoy-based cache coheence potocols ae still the dominant potocols today because they don't ely on snooping mechanisms hich ae vey dicult to implement eciently on netoks. In this pape, e addess to issues egading the ovehead eduction in the diectoy-based potocols. In the st issue, e focus on to metics: (1) the numbe of netok tansactions equied on the citical path in a potocol, and (2) the total numbe of netok tansactions needed (including all coheence-elated netok tansactions) in ode to complete a memoy equest. The citical path in a potocol is the longest path fom the issuance of a pocesso's equest to the aival of the equested cache line. It detemines the latency of a memoy equest in the potocol. The second metic detemines the netok bandidth and othe use of esouces needed to suppot the potocol. It may also aect the latency of memoy equests because a lage numbe of netok tansactions needed to maintain cache coheence in a potocol may ceate moe contention at the system esouces, hich could advesely aect the memoy latency. The second issue addessed in this pape is the seialization of memoy accesses. The time beteen the issuance of a memoy equest fom a cache miss o a eplacement of a cache line, and its aival at the home diectoy is usually nondeteministic because of vaious possible netok conditions. Most diectoy-based potocols use the ode of memoy accesses in hich they aive at thei home diectoy as the basis fo seialization. In most existing potocols, hen to equests to the same cache line aive at the home diectoy at about the same time, one has to be blocked until the st equest is completed - e call such potocols as blocking potocols. This could aect the latency of memoy accesses to the shaed cache lines. Those shaed cache lines could be the esult of shaed vaiables such as synchonization vaiables, o the esult of false shaing [9]. 2

5 In this pape, e popose a non-blocking potocol hich minimizes the citical-path length of a memoy equest, in the mean time, minimizing the total numbe of netok tansactions needed to complete a memoy equest (including the tansactions needed to maintain coheence). It also allos multiple equests to the same cache line to be pocessed simultaneously at the home diectoy by keeping the ode of those equests in a histoy table fo coect memoy updates. Using simulations, e sho such a non-blocking potocol can achieve a bette pefomance than existing blocking potocols. In the est of the pape, e give some backgound to motivate ou study in Section 2. In Section 3, e give a detailed desciption of ou potocol. Some coectness issues of the poposed potocol ae addessed in Section 4. The simulation setup and some of the simulation esults ae pesented in Section 5 folloed by conclusions in Section 6. 2 Backgound In the discussion of potocols, e use the folloing naming conventions fo the system components involved in data tansactions. A equeste node is the node hich contains the pocesso oiginating a given equest, hile a home node is the node containing the diectoy infomation of the equested cache line. A emote node is any node hich is not the equeste. An one node is the node hich has the oneship copy among the copies of the equested line. If the cache of a node does not have the valid copy equested by its pocesso, the one node is esponsible fo eplying to the equest. Thus, the diectoy enty of a line should have infomation about the one node as ell as the nodes shaing the cache line. All cache misses go to thei home diectoy st. The data equest fo a ead miss is deliveed to the home node as a ead equest. A ite miss is deliveed to the home node as a ead-exclusive equest to indicate that the pocesso ishes to modify the cache line. If the memoy copy at the home node is valid, the oneship is given to the copy, i.e., the home node is the one node. The equest is etuned to its equeste ith a valid copy. If the memoy copy is invalid, the oneship is in the copy at a emote cache. Thus, the equest is foaded to the emote node. When a valid copy needs to be eplaced fom the equeste, a elocation equest is deliveed to 3

6 (a) Simple (b) DASH (c) DVSM (d) Wite- Only (e) Non- Blocking N 0 N 0 / H / N 1 N0 H N 1 Ack Sig H + N 0 / / / / H + N* 0 H N 1 H + WB / / / / Sig N 0 H N 1 N* 0 H + N* 0 N 0 H + N 1 N* 0 N* 0 N* N 0 1 Ack Sig N 0 N 0 H H + N 0 H + N 1 N* 0 N 0: equeste node N 1: one node N*: completion of a equest H: home node H : locking diectoy enty + H : unlocking and updating diectoy enty Sig Ack WB : ead equest : ead-exclusive equest : home update signal : acknoledgment : ite-back equest Figue 1: Flo of data equests in diectoy potocols the home node if the eplaced copy has the oneship. The elocation equest is futhe tansmitted to a ne one node, in most cases, it is the home node itself. If the eplaced copy does not have the oneship, a eplacement-hint equest is initiated to update the diectoy at the home node. Some machines, such as DASH [6], do not geneate the eplacement-hint equests because data consistency and integity can still be guaanteed ithout such equests. When a pocesso ites to a shaed cache line, all shaed copies in emote nodes should be invalidated by issuing invalidation equests unde a ite-invalidation coheence potocol. The diectoy potocols poposed in the liteatue die mainly in the ay the data equest is issued fo a cache line hose valid copy is pesent in a emote node. One possible appoach is to send a equest to the home node, have the home node foad the equest to the one node, etieve the equested data to the home node, and then have the home node etun the data to the equeste node. This o of equests is shon in the simple potocol of Figue 1 (a). In the gue, N 0 is the equeste node and N 1 is the one node. When N 0 's equest is successfully foaded to the one node N 1 fom the diectoy, the associated diectoy enty is maked to indicate that a equest fo the enty is being seviced (shon as H? in the gue). The subsequent equests hich aive at the diectoy fo the same cache line ae suspended. A technique fo the suspension is to bue the subsequent equests at the diectoy. To eliminate the bueing hadae, bouncing the 4

7 subsequent equests back to thei equestes as unsuccessful equests is anothe possible technique. An unsuccessful equest ill foce the equeste to ety late. Those to techniques equie a lock bit in each diectoy enty indicating hethe the enty is locked o not. When the sevice is completed, the diectoy is unmaked hile updating the diectoy infomation (shon as H + in the gue). The completion of N 0 's equest is denoted as N 0 in the gue. Afte the completion, the equested cache line is in N 0 and is eady to be used. The equest o of the DASH potocol [6] is shon in Figue 1 (b). Simila to the simple potocol, only the one node can update the diectoy infomation. The one node N 1, upon eceiving a ead equest, geneates to messages: a message containing a copy of the equested line is etuned to the equeste node, and a message (a ite-back (WB) equest) is sent to the home node to update its copy and the diectoy infomation. The oneship of the cache line is etuned to the home node. Upon eceiving a ead-exclusive equest, the one node N 1 sends a message containing the copy of the line to the equeste node N 0 hile dispatching a message to the home node to update the diectoy infomation. Afte updating the diectoy infomation, the home node foads an acknoledgement to the equeste node N 0. The equeste node N 0 is not alloed to eplace the nely equested line until it eceives the acknoledgement fom the home node indicating the diectoy update has been completed. This is to avoid the ace condition in hich the equested line is eplaced in N 0 and updates the home diectoy again befoe the diectoy is updated by N 1 [6]. Instead of blocking the diectoy hile it is foading a equest to the one node, the DASH potocol allos the subsequent equests fo the same cache line to be foaded to the one node as ell. Hoeve, the equests ill be etuned to thei equestes as unsuccessful equests if the emote one node no longe has the oneship of the equested cache line. As can be seen in Figue 1, the DASH potocol ould educe the data access latency in the citical path fom fou to thee tansactions compaed to the simple potocol. Hoeve, the total numbe of equests inceases fom fou to ve in a ead-exclusive equest. The softae-based potocol implemented in distibuted vitual shaed memoy (DVSM) [7] adopts a dieent appoach. In the xed distibuted manage potocol of DVSM, the esponsibility of updating the diectoy infomation is given to the equeste node, not to the one node. On 5

8 eceiving the equested copy, the equeste node launches a message to the home node to update the diectoy infomation. This is shon in Figue 1 (c). Although data access latency in the citical path consists of thee tansactions, the total numbe of equests is fou. Also, the diectoy locking time is longe than the othe blocking potocols (see Figue 1). It has a highe pobability of blocking the successive equests. Befoe descibing the ite-only blocking potocol (Figue 1 (d)), let us take a look at the states of a cache line in the cache memoy. Unde a ite-invalidation potocol, a state hich indicates the validity of a cache copy is needed. One o moe than one memoy location can have the same copy of data. The data in the cache can be up-to-date but dieent fom the copy in the home node. If e allo seveal caches to shae an up-to-date cache copy hich is dieent fom the copy at the home node, the ite-back equest in the DASH potocol can be eliminated. In this case, anothe state hich indicates the oneship of the shaed cache copies is needed fo ecient eplacement of the up-to-date cache copy. Given this condition, if the pevious equest is a ead equest, the subsequent equests can be foaded to the cuent one node befoe the completion of the pevious ead equest. That is, if the pevious equest is a ead equest, blocking the diectoy is not necessay. This is because the one node pesistently has the same copy befoe and afte the completion of the ead equest. With this obsevation, the ead equest only needs up to thee netok tansactions. Note that the diectoy infomation is updated hen the ead equest st aives at the diectoy (see the ite-only potocol in Figue 1 (d)). If the pevious equest is a ead-exclusive equest, fo data consistency, all subsequent equests should have the copy updated by the ead-exclusive equeste node. Thus, the ite-only potocol in Figue 1 (d) blocks the subsequent equests of a ead-exclusive equest. The signal to unblock the diectoy is issued by the one node as in the DASH potocol. Afte it is unblocked, the diectoy sends an acknoledgement to the equeste node N 0. The acknoledgement is just to pevent an ealy eplacement of the nely acquied cache line, hich it no has the oneship. That is, node N 0 cannot eplace the nely acquied cache line until afte it eceives the acknoledgement fom the diectoy. Anothe inteesting case can aise hen the unblocking signal aives too ealy fom N 1 to the home diectoy. The acknoledgement issued fom the diectoy aives at N 0 even befoe the 6

9 Time: t t t t t t t t t t t t t t t t t t (a) Simple (buffe) H N 0 N 1 H + N* 0 H N 2 H N 0 H + N* 2 (b) DASH N 0 H N 1 Sig N 2 H N 1 N* 0 Ack H + N 0 Fail N* 2 N2 H N 0 WB H + (c) DVSM (no buffe) (c ) DVSM (buffe) (d) Wite- Only (no buffe) N 0 N 0 N 0 Sig H N 1 N* 0 H + Fail Fail Sig H N 2 H N 2 H N 2 N 0 N* 2 H + Sig H Sig H H N 1 Sig N 1 N* 0 H + N 2 H N 0 N* 2 H + N* 0 H + Ack N 0 Fail N 2 H N 2 H + N 0 N* 2 (e) Non- Blocking N 0 H + N 1 N* 0 N 2 H + N 0 N 0 N* 2 N i: nodei N* i : completion of node s equest i H: home node H : locking diectoy enty + H : unlocking and updating diectoy enty WB Fail : ead equest : ead-exclusive equest : ite-back equest : unsuccessful equest Sig Ack : home update signal : acknoledgment : aiting equest Figue 2: Flo of to data equests hen made simultaneously equested cache line aives fom N 1 to N 0, and tuns N 0 to a ne one node of the equested cache line. The acknoledgement signal has to be peseved. Futhemoe, any subsequent equest to the cache line has to be bueed. The blocking schemes descibed so fa have a minimum of thee tansactions on thei citical paths. Hoeve, they equie exta signals to unblock the diectoy enty. Anothe main daback of the blocking schemes occus hen simultaneous accesses ae made to the same cache line. This is shon in Figue 2. The gue assumes that a subsequent ead equest fom N 2 aives at the diectoy ight afte a ead-exclusive equest fom N 0. As can be seen, both the length of the citical path and the numbe of equests fo the second ead equest is substantially inceased compaed to those in Figue 1. Table 1 summaies the esults hen to equests to the same cache line ae in pogess simultaneously. In the table, e assume each equest tansaction on a datapath takes to cycles and the second equest stats one cycle afte the st equest (see Figue 2). Geneally, the 7

10 Latency (cycles) # of equests potocols st second equest (/) st second equest (/) equest st eq. st eq. equest st eq. st eq. (/) is a ead is a ite (/) is a ead is a ite Simple (bue) 8 7/ /6 4 DASH 6 10/ /5 5/9 7/8 DVSM (no bue) /10 8 DVSM (bue) /13 4 4/6 4 Wite-Only /5 3/7 5/7 Non-blocking /5 3 Table 1: Simple compaison among potocols fo to simultaneous equests. N 0 H + N 1 Rel Fail N 2 H + N 0 N 0 Rel N 1 H N 0 N* 0 N* 2 t 0 t 1 t 2 t 3 t 4 t 5 t 6 N0 N 1 N0 N 1 + N N i: nodei H : updating diectoy enty 1 - N1 - N - N* i : completion of 2 N N 2 0 : ead-exclusive equest node i s equest Rel : elocation equest Nn-1 - Nn-1 - H: home node Fail : unsuccessful equest : aiting equest Histoy table at t 2 Histoy table at t 3 Figue 3: A simple example of a non-blocking diectoy potocol second equest ceates a lage numbe of equests and inceases the data access latency compaed to the st equest. Bues at the diectoy usually educe the numbe of equests and the data access latency. Ou poposed non-blocking diectoy scheme updates the diectoy infomation hen a ead o ead-exclusive equest st aives at the diectoy (see Figue 1 (e)). This eliminates the nal signal to unblock the diectoy enty. The non-blocking potocol equies up to thee tansactions fo all data accesses (including ead-exclusive equests) ithout geneating exta tac. It pefoms quite ell especially hen seveal equests ae made simultaneously (see Figue 2 (e) and Table 1). 8

11 3 Non-blocking Diectoy Potocol Figue 3 shos an example of the non-blocking potocol. In the gue, N 1 is the one node of a line at time t 0. A ead-exclusive equest is issued to the home node H at time t 0, and is foaded to N 1 afte the diectoy changes the oneship fom N 1 to N 0 at time t 2. A subsequent ead-exclusive equest fom N 2 aives at the home node at time t 3. In the blocking potocols, the equest is blocked, i.e., it is bueed at the diectoy o etuned to N 2 as an unsuccessful equest. In the non-blocking diectoy potocol, the ead-exclusive equest is foaded to N 0 afte the diectoy changes the oneship fom N 0 to N 2. When N 2 's ead-exclusive equest aives at N 0 at time t 5, the equest aits fo the completion of N 0 's equest. If N 0 's equest successfully aives in N 0 fom N 1 ith the equested cache line, the cache line is updated at N 0 and foaded to N 2. Hoeve, a poblem aises hen the cache line oned by N 1 is eplaced at time t 2 befoe N 0 's ead-exclusive equest aives at t 4 as in Figue 3. N 0 's equest ill not be able to nd the equested copy at N 1. When the eplaced cache line in a elocation equest is deliveed to the diectoy at time t 4, it should be foaded to N 0. Hoeve, since the N 2 's equest has updated the one node at time t 3 in the non-blocking potocol, the infomation about N 0 's equest is lost. Although this may happen vey infequently, the diectoy must povide a means to nd the coect node in hich such eplacement occus. To tackle this poblem, the histoy of the one nodes could be maintained in a small table, called histoy table, at the diectoy. As can be seen in Figue 3, the one node N 1 is ecoded in the table enty of the ne one node N 0 at time t 2, and the one node N 0 is ecoded in the enty of the ne one node N 2 at time t 3. When the elocation equest fom N 1 aives at the diectoy at time t 4 and nd that the one node has changed in the diectoy, it ill seach the table enty of the cuent one node N 2. The histoy table enty of N 2 contains N 0 hich is not the eplacement equeste node N 1. It continues to look fo the table enty of N 0. Since N 0 's enty points to N 1, the seach stops. The elocation equest is foaded to N 0. Hee, note that the sequence of data updates is based on the ode of the equests hich aive at the diectoy. In the folloing to subsections, the non-blocking diectoy is descibed in moe detail. 9

12 P addess elocate bit invalidate bit sync. counte aiting equests Cache Memoy Local Memoy Requeste Status Registe (RSR) One Shaed nodes nodes Diectoy infomation state bit node # N0 - N1 - N - 2 Nn-1 - Histoy table Figue 4: Hadae equiements fo the non-blocking potocol 3.1 Hadae equiements Figue 4 shos some basic hadae equiements fo the non-blocking potocol. The infomation about the one nodes and the nodes shaing the cache line is maintained in the diectoy. The diectoy infomation can be kept eithe in a full-map [3] o pointe-based [5] diectoy stuctue. The histoy table descibed above is also located in the diectoy. The histoy table has n enties, hee n is the numbe of nodes in the system. Each enty has a node numbe and a state bit. When the oneship of a cache line is changed by a ead-exclusive equest o by an invalidation equest at the diectoy, the pevious one node is saved in the table enty of the equeste node (ne one node). The state bit in the histoy table indicates hethe the change of oneship is caused by a ead-exclusive equest o an invalidation equest. Although ou non-blocking scheme can suppot multiple equests fom one node by elaxing the efeence ode in a pogam context [1], hee e assume the the pocesso can issue only one equest at a time unde ou sequential consistency model to simplify ou pesentation. To suppot the non-blocking diectoy potocol, e also need some exta hadae at the pocesso cache. The infomation of the line being equested by the pocesso needs to be specied in a egiste, called equeste status egiste (RSR) (see Figue 4). To indicate hethe the equested line is invalidated o elocated hile the pocesso is aiting fo the completion of the equest, to bits, called invalidate and elocate bits, ae used. The synchonization counte in RSR counts the numbe of equests that the pocesso needs to eceive. The synchonization infomation is also 10

13 I: Invalid state S: Shaed state D: Dity state O: shaed-one Issue: Issue state : ead equest : ead-excl. equest inv: invalidation equest Rel: elocation equest ep: eplacement-hint eq. F: failed ead equest Ack: acknoledgement flush: flush singal (sync): condition on sync. counte tansition on a local equest tansition on a emote eq. Input/Output /F,,inv Cache and RSR I, inv /F,inv Rel ep /,/, inv /invo S Issue Rel O /invs F/,flush/,/, inv/,(syncj0) inv O Rel/,Ack/, (syncj0) Rel,Ack, (sync=0),inv, (sync=0) inv D S inv, Rel/,invS/, invo/(, flush) inv, /inv, ep/ Diectoy I ep/, Rel/Ack o Rel/(Rel,flush) Figue 5: State tansition diagams fo the cache and the diectoy needed in most of the blocking potocols. The non-blocking potocol equies some bues to keep the aiting equests at the pocesso cache. In the folloing section, e ill see ho the hadae components descibed hee ae used. 3.2 Diectoy potocol Figue 5 shos ho the states in the cache and the diectoy ae changed based on input equests. The output equest fo each tansition ae also shon in the gue. We have ve cache states of each cache line; invalid (I), shaed (S), dity (D), shaed-one (O), and issue (Issue) states. The st fou states ae the same as in [4]. Hoeve, note that the I-state also includes the situation hen the equested cache line is missing in the cache due to a eplacement. The Issue-state epesents the situation hen the cache is issuing a equest and the equest is not completed yet. The home diectoy maintains the states fo each equeste node. If the equeste node is the one node, the diectoy is in the one (O) state. If the equeste node is a node shaing the cache line, it is in 11

14 the shaed (S) sate. Otheise, the diectoy is in the invalid (I) state. That is, the equeste node does not have a valid copy fo the given equest ead equest Initially, both the cache and the diectoy states ae in the I-states. When a pocesso's ead equest hits at the cache, thee is no state change. A ead miss launches a ead equest to the diectoy afte changing the cache to the Issue-state. When a ead equest fom N i aives at the home node, the diectoy foads the equest diectly to the one node N j and consides N i as a shaing node (i.e., change to the S-state). The histoy table emains unchanged. When N i 's ead equest aives at the one node N j, N j could be in thee possible situations. (1) If N j is accessing the same cache line (i.e., in the Issue-state), N i 's equest aits at N j, and ill be etuned ith a valid copy hen N j 's equest is completed. The state of N i ill be changed to the O-state. (2) If the equested data is missing in N j due to a eplacement (i.e., the I-state) o does not have the oneship (i.e., the S-state), N i 's equest is etuned to N i as an unsuccessful equest. The unsuccessful equest compels N i to eissue the ead equest. (3) If N j has the oneship copy (i.e., the O-state o the D-state), N j sends a eply message containing the cache line to N i. When the eply message aives at N i, its pocesso eties the equest if the cache line is invalidated by a emote invalidation equest duing the issue of the N i 's equest (i.e., invalidate bit in RSR is set). Otheise, the pocesso esumes its execution afte changing the cache to the S-state ead-exclusive equest When a ead-exclusive equest fom N i eaches the home node, the equest is diectly foaded to the cuent one node N j. The diectoy also sends invalidation signals to all of the nodes hich have the same cache line. Pevious one N j is saved in the histoy table enty of the equeste N i hile setting its state bit to zeo. The diectoy consides N i as the one node (i.e., the O-state). On eceiving a ead-exclusive equest, the one node handles it almost the same ay as a ead equest. Hoeve, the equeste N i does not ety the access even if it eceives an unsuccessful equest fom N j. This is because the equested cache line ill be deliveed to 12

15 N i: nodei N* i : completion of node s equest i H: home node + H : updating diectoy enty inv : ead-exclusive equest : invalidation equest flush : equest flush signal : aiting equest N H N + 1 N 1 N* 0 0 inv inv N N 2 0 inv N N 1 H + 0 N 0 N* 1 flush N 1 t 0 t 1 t 2 t 3 t 4 t 5 t 6 t 7 Figue 6: An invalidation equest hich ceates a ush signal N i as a elocation equest (see Figue 3). The equeste node N i collects all of the invalidation acknoledgements as ell as the equested cache line using the synchonization counte. When all of the acknoledgements and the equested line have aived (i.e., the synchonization counte is deceased to 0), the pocesso continues its execution afte updating the cache line (i.e., change it to the D-state). If thee ae any emote ead o ead-exclusive equests aiting at N i, the equests ae etuned successfully afte changing N i 's cache to O-state o I-state espectively invalidation equest If N i 's ite hits on a shaed line (i.e., it is in the S-state o the O-state) in its on cache, N i launches an invalidation equest to the diectoy. When the invalidation equest aives at the diectoy, it is possible that the diectoy no longe consides N i as a shaing node (i.e., it is in the I-state). This could happen if a node N j 's ead-exclusive o invalidation equest aives at the diectoy befoe N i 's invalidation equest. Since the ode of the equests aiving at the diectoy detemines the sequence of the data updates, the equested cache line should be updated by N j st. Thus, N i 's invalidation equest needs to be foaded to the cuent one N j as a ead-exclusive equest. At the same time, if the invalidation equest is fom the pevious one node (O-state in N i ), a ush signal is sent to N i to ush the N j 's ead-exclusive equest back to N j ith a valid cache copy. Figue 6 shos an example of an invalidation equest hich ceates the ush signal. At time t 0, N 1 and N 2 shae a line, and N 1 has the oneship. When N 0 's ead-exclusive equest aives at 13

16 the diectoy at time t 2, it is foaded to N 1 hile sending an invalidation signal to N 2. Befoe N 0 's equest aives at N 1, if N 1 ants to update the shaed-one line, an invalidation equest is launched, and aives at the diectoy at time t 3. At this time, the invalidation equest is deliveed to N 0 as a ead-exclusive equest since N 1 is not a shaing node. Also, the diectoy sends a ush signal to N 1. At time t 4, N 0 's ead-exclusive equest aits fo the completion of N 1 's invalidation equest. By eceiving the ush signal at N 1, N 0 's ead-exclusive equest is etuned to N 0 ith the equested cache line hile invalidating the copy in N 1 at time t 5. When N 1 's invalidation equest aives at N 0 at time t 5, it also aits fo the completion of N 0 's ead-exclusive equest. N 1 's invalidation equest aiting at N 0 is etuned to N 1 ith the cache line updated by N 0 at time t 7. If the diectoy consides N i as a shaing node (i.e., the S-state o the O-state) hen N i 's invalidation equest aives at the diectoy, the diectoy dispatches invalidation signals to all of the othe shaing nodes. The equeste node N i becomes the one node (O-state). If the pevious one node is dieent fom N i, the pevious one node is saved in the histoy table. The table state bit is set to 1 indicating the line is invalidated. Unlike the ead and the ead-exclusive equests, the invalidation equests tansmitted to the shaing nodes do not have to ait fo the completion of any pending actions of the shaing nodes. This is because the copy to be updated by N i is in N i. They just invalidate the same cache copies (i.e., change to the I-state) in the shaing nodes, if any. To invalidate the copy being issued by a shaing node (i.e., in the Issue-state), the invalidate bit in RSR is set to elocation equest When a elocation equest bings a eplaced copy fom the equeste node N i into the diectoy, the diectoy st compaes the cuent one node ith the equeste. If they ae the same node (i.e., it is in the O-state), the eplaced copy is simply itten into the local memoy. If not (i.e., it is in the I-state), then thee exists a node N j hose ead-exclusive o invalidation equest aives at the diectoy befoe the elocation equest, and changes the one node fom N i to N j. The copy in the elocation equest must be updated by N j. Since the diectoy foads all equests afte updating the diectoy infomation, it is possible that a dieent node N k can be the one 14

17 N i: nodei N* i : completion of node s equest i H: home node + H : updating diectoy enty Rel : ead-exclusive equest : elocation equest : ead equest Fail : unsuccessful equest flush : equest flush signal : aiting equest (a) Deadlock N 0 H + N 1 Rel Rel N 1 H N 0 N 1 H + N 0 t 0 t 1 t 2 t 3 t 4 t 5 (b) No deadlock N 0 H + N 1 Rel H Rel flush N 1 Fail N 1 N* 0 N 0 N 1 N 1 H + N 0 N 0 N* 1 t 0 t 1 t 2 t 3 t 4 t 5 t 6 t 7 Figue 7: A elocation equest involved in a deadlock node hen the elocation equest aives at the home diectoy. To nd the node hose equest st changes the oneship (in this case, it is node N j ), the diectoy seaches the histoy table ecusively stating fom the enty of the cuent one node N k. When the ecusive seach aives at the enty of N j, the diectoy knos that N j took the oneship fom N i. If the state bit of the enty is zeo, the diectoy foads the elocation equest to N j and sends a ush signal to N i. O, if the state bit is set to one, the eplaced copy is discaded and an acknoledgment is sent to N j. The acknoledgment pevents N j fom geneating anothe equest befoe the eplaced copy is discaded. Nothing in the diectoy needs to be modied at this time (i.e., it emains in the I-state). If e omit the ush signal in the above case, a deadlock could occu. Figue 7 (a) shos an example of the possible deadlock. At time t 0, N j is the one node and N 0 geneates a ead-exclusive equest. Befoe N 0 's ead-exclusive equest aives at the one node N 1 at time t 3, the oneship copy is eplaced fom N 1 at time t 1 and anted by N 1 again at time t 2. Thus, N 0 's equest aits fo the completion of N 1 's equest at time t 4. Wheeas, N 1 's equest, hich follos the elocation equest, aits at N 0 fo N 0 's equest to be completed. That is, a deadlock occus. The ush signal at time t 3 in Figue 7 (b) akes up N 0 's equest aiting at N 1. The N 0 's unsuccessful equest is 15

18 etuned to its equeste. When the anted copy is deliveed by a elocation equest, it is updated by N 0, and sent to N 1 though the aiting equest. If the copy that a pocesso ants to fetch is in a elocation equest, the pocesso does not geneate anothe equest until its cuent equest is successfully completed. Also, the equeste hose equest st changes the oneship alays eceives the equested copy. Thus, the histoy of one nodes fo a line in the table is not coupted by a dieent equest. This is the eason hy the ecusive seach alays nds the node hose equest st changes the oneship of the line. If a eplacement-hint equest fom N i aives at the diectoy, N i is deleted fom the shaing node list (i.e., it is changed to the I-state). Hee, note that the maximum numbe of ead-exclusive equests hich could be aiting fo a pending equest at a node is one because no othe aiting equests to the node can be made afte the st ead-exclusive equest. Thus, the bue to keep the aiting equests at a node can be esticted to a small size. When a ead-exclusive equest aives at the node and the equest has to ait fo the completion of the node's equest, all equests existing in the bue have to be ead equests. Theefoe, if the bue is full, the ead-exclusive equest can be kept in the bue by victimizing a ead equest fom the bue. The victimized ead equest is etuned to its equeste node as an unsuccessful equest. To suppot multiple equests fom a node, e assume that the numbe of equests issued by a pocesso is limit by a constant numbe, c, and each equest has a unique equest numbe associated ith it. We only need to incease the numbe of enties in the histoy table fom n to cn, and the diectoy must keep the infomation about the equest numbe as ell as the node numbe. When a ead-exclusive o an invalidation equest changes the oneship of a cache line, the pevious one node and its equest numbe ae saved in the table enty of the cuent equeste. 4 Potocol Veication The numbe of dieent instances fo a memoy system is nite due to its limited size. If e look at all possible instances that can be eached fom an initial instance ith a given potocol, e can check the coectness of the potocol. Hoeve, a manual constuction of all eachable instances 16

19 is almost impossible because of the huge numbe of instances. Thus, in most cases, e ely on an automatic fomal potocol veication tool. Hee, e use the state enumeation method poposed in [12] in ou automatic veication tool. The main obstacle of this method is the state explosion poblem. To tackle this poblem, e st choose an abstaction model fom the oiginal system. The abstaction model assumes thee nodes (Node 0, 1 and 2) and an addess space of 2 memoy locations (addess x and y) hich ae mapped to the same one-line cache. Node 0 is the home node of the addesses. The communication among the equeste node, the home node, and the one node is modeled as a completely connected FIFO queue system in both diections. The size of each queue is not limited. Each node can have only one emote equest pending at the node. To educe the numbe of instances, e exploit to types of symmety; addess symmety and stuctual symmety. In the addess symmety, e exchange to addesses (addess x and y) hile leaving the stuctual conguation unchanged. In the stuctual symmety, to nodes (Node 1 and 2) ae sapped. This means the name of the to nodes ae exchanged. Thus, all node numbes in an instance should be sapped and the queue positions elated to the nodes ae also changed accodingly. The instance hich adopts both symmeties at a time is also consideed in ou enumeation method. Ou veie exploes 26,174,955 dieent eachable instances ith maximum depth of 79 in a beadth-st seach. The maximum numbe of equests that can be in one queue at a time is six. In the bead-st seach tee, evey instance has a successo othe than itself (i.e., no deadlock). In each instance, the veie examines to essential invaiant conditions. The st invaiant condition is hethe the instance has one and only one oneship copy fo each addess o not. The second invaiant condition is to check if thee is any valid copy in a dieent location hen a pocesso updates the data in the cache. Fo data consistency, thee must not be a valid copy in any dieent location hen the copy is updated. Anothe consideation in the veication of a potocol is the detection of deadlocks due to limited system esouces. To avoid deadlocks, e utilize the fact that the poposed non-blocking scheme allos evey pocesso's equest to be completed in thee tansactions. Thus, if a sepaated queue is dedicated fo each tansaction and if the equests in the queue fo the last tansaction 17

20 Pogam Pogam Desciption Poblem Size Banes Banes-Hut N-body simulation 16K paticles Cholesky blocked spase factoization d750.o FFT complex 1D adix- n six-step FFT 64K points FMM Fast Multipole N-body simulation 16K paticles LU(Cont.) blocked dense LU factoization matix Ocean(Cont.) ocean movement simulation ocean Radiosity computation of light equilibium disti. oom Radix intege adix sot 1M integes Raytace endeing a 3D scene teapot Volend endeing a 3D volume head (ende only) Wate-Nsq Evaluating a ate molecule system 1000 molecules Wate-Sp impoved vesion of Wate-Nsq 1000 molecules Table 2: Chaacteistics of the taces. ae seviced st (i.e., have the highest pioity in the use of the datapath), e can pevent the deadlock. 5 Pefomance Study To see ho dieent potocols aect the pefomance of a memoy achitectue, e compaed the non-blocking potocol ith to blocking potocols: the DVSM potocol ithout bues at the diectoy and the ite-only blocking potocol (see Section 2). The ite-back scheme in the DASH potocol can educe the data access latency if the copy itten back to the memoy is continuously valid hen the next equest fo the same cache line aives at the diectoy. Hoeve, if the copy is invalidated by a shaing node, the ite-back equest can aste netok and memoy bandidth. Although the ite-back scheme may cause dieent consequences in pefomance, e did not simulate it because pevious studies sho that the pefomance advantage of the ite-back schemes is maginal [2]. We developed a pogam-diven simulato using the Splash-2 pogams [11] as the okload. Some chaacteistics of the Splash-2 pogams ae illustated in Table 2. We use MINT [10] as the font end pat of ou simulato. In the back end, e implemented the thee dieent potocols; DSVM, ite-only, and non-blocking potocols. The default simulation paametes ae shon in 18

21 System Unit Paametes default values # of pocessos = 32, page size = 4K bytes, line size = 64 bytes size (pe node) 16K bytes access time 1 cycle cache cache ll time 4 cycles set associativity 4-ay eplacement andom selection memoy line access time 24 cycles diectoy pocessing time 16 cycles netok sevice time 110/90 cycles Table 3: Paametes in simulation. Table 3. Thee ae 32 pocessos ith one pocesso in each node. Each pocesso can geneate one equest at a time. The size of a cache line is 64 bytes, and the page size is 4K bytes. In ode to eect the oking set sizes (elative to the cache size) in ou simulation, the cache size is set at 16K bytes so as to t beteen the st and the second impotant oking sets of the Splash-2 pogams [11]. The cache access time is the unit of ou simulation clock cycle. Fou cycles ae assumed fo the cache ll time, and fou-ay set associativity fo the cache is used. The eplaced line fom the cache is andomly selected. The memoy access time is 24 cycles. The diectoy pocessing time pe equest is assumed to be 16 cycles. Each non-memoy instuction completes in a single cycle as assumed in MINT. To simplify ou simulation, e do not simulate the netok topology in detail, but athe assume the communication time beteen nodes (netok latency o netok sevice time) to be a constant egadless of thei distances. The netok latency fo equests ith data is 110 cycles, and the netok latency fo equests ithout data is 90 cycles. The sizes fo the equests ith and ithout data ae assumed to be 68 and 4 bytes espectively. The contention at the system esouces (i.e., cache, memoy, diectoy, and netok) is modeled using queues. Thee is almost no dieence in cache hit ates among the potocols since the potocols hadly aect locality of the data. Among the pogams, Ocean and Radix have elatively high cache miss ates. Figue 8 illustates dieences in the equest ates at the diectoy. In the gue, (blocking fo eads and ites) denotes the DVSM potocol, denotes the ite-only blocking 19

22 Request Rate Banes FFT LU Radiosity Raytace Wate-Nsq Cholesky FMM Ocean Radix Volend Wate-Sp legend Inv Rep. Data Figue 8: Numbe of equests to the diectoy pe efeence. Taffic (Byte) Banes FFT LU Radiosity Raytace Wate-Nsq Cholesky FMM Ocean Radix Volend Wate-Sp legend Coh. Rep. Data Figue 9: Amount of netok tac pe efeence. potocol, and denotes the non-blocking potocol. We classify the equests into thee categoies; data equests, eplacement equests, and invalidation equests. Since the potocol locks the diectoy enty fo both ead and ead-exclusive equests, it needs a lage numbe of equests to unlock the enty. Also, the subsequent data equests ae likely to be blocked at the diectoy. The blocked equests need to be eissued befoe the equests can nally be foaded to the one nodes. Thus, the potocol has the lagest numbe of data equests to the diectoy. Since the potocol also locks the diectoy fo the ite accesses, some exta equests ae needed. In the potocol, no equests ae blocked at the diectoy, and hence no exta equests to unlock the diectoy enties ae needed. Thus, the potocol has the smallest data equest ate at the diectoy. Thee ae almost the same eplacement and invalidation equest ates among the potocols. This is because the eplacement and invalidation equests ae handled almost the same ay among the potocols in most cases. The aveage amount (bytes) of netok tac pe efeence fo each diectoy potocol can be found in Figue 9. Hee, e divide the netok tac into thee categoies: data tac, eplacement tac, and coheence tac. The amount of netok tac vaies acoss the potocols even though the dieence is small compaed to the diectoy equest ates. The main eason fo the small 20

23 Latency (Cycle) aiting time 20 emotememoy 16 hit emotecache 12 hit localmemoy 8 hit 4 localcache hit 0 Banes FFT LU Radiosity Raytace Wate-Nsq Cholesky FMM Ocean Radix Volend Wate-Sp legend Figue 10: Aveage data access latency including the aiting time. dieence is that the lage numbe of equests hich have impact on the diectoy equest ates does not contain data. The potocol has the smallest amount of data tac among the thee potocols in most of the pogams (the atio of eduction compaed to the potocol is fom 5.4% in Banes up to 10.7% in Radiosity). Figue 10 pesents the aveage data access latency pe efeence. The aveage latency consists of local cache hit, local memoy hit, emote cache hit, and emote memoy hit latencies. On the citical path of a data access, the aiting time caused by contention at the system esouces is also included in the aveage latency. If e exclude the aiting time, thee is almost no dieence in the aveage latency among the potocols. This is because the citical path of a data access is the same among the potocols, and the simultaneous accesses to the same cache line happen infequently. The dieence among the potocols comes mainly fom the aiting time at the system esouces especially fo emote accesses. The exta equests hich ae not on the citical path in the blocking potocols incease the aiting time. Radiosity has a lage numbe of equests hich get the equested data fom emote caches. Thus, the potocol in Radiosity signicantly inceases the emote cache hit latency. This means that, in Radiosity, simultaneous ead equests ae made fequently and the oneship copy is usually in a emote cache. The aveage latency in the potocol is fom 3.5% (in Wate-Sp) up to 57.1% (in Radiosity) smalle than the latency in the potocol. Figue 11 depicts the speedup based on the potocol, i.e., speedup = exec. time of exec. time of a potocol. A pocesso hich aives at a synchonization point (ait(), baie(), lock(s), etc) is blocked until 21

24 Speedup blocking Banes FFT LU Radiosity Raytace Wate-Nsq Cholesky FMM Ocean Radix Volend Wate-Sp legend busy Figue 11: Speedup: pogam execution times nomalized to the case of. the blocking event is eliminated. The maximum blocking time among the pocessos is pesented in Figue 11. The potocol povides supeio pefomance stability ove the othe potocols. The potocol is especially eective hen the cache miss ates ae high (34% and 15% speedups in Ocean and Radix espectively) and hen simultaneous equests ae made fequently (37% speedup in Radiosity). Finally, e found that the pefomance gap among the potocols is idening as the netok sevice time and the diectoy pocessing time incease. 6 Conclusion In this pape, e popose a non-blocking diectoy potocol. In this potocol, all equests can poceed ithout being blocked at the diectoy. To suppot this potocol, the diectoy maintains a histoy of one nodes in a small table, called histoy table. Thee ae to main advantages in the poposed non-blocking potocols. Fist, due to the shot citical path in the potocol, it allos fast emote accesses; i.e., up to only thee netok tansactions. Second, it educes the amount of netok tansactions needed in the potocol since the signal fo unlocking the diectoy enty is not needed. The non-blocking diectoy oks ell especially hen seveal nodes make equests to the same cache line at the same time. We veied the non-blocking potocol using a state-enumeation method on an abstaction model. With detailed pogam-diven simulations, the pefomance of the poposed non-blocking diectoy potocol is studied. The simulation esults sho that the poposed non-blocking diectoy potocol could povide bette pefomance than the existing blocking diectoy potocols. 22

25 Refeences [1] S. Adve, K. Ghaacholoo, \Shaed Memoy Consistency Models: A Tutoial," IEEE Compute, pp , Dec [2] J. Achibald, J. L. Bae, \Cache Coheence Potocols Evaluation Using a Multipocesso Simulation Model," ACM Tans. on Comp. Sys., Vol. 4, No. 4, pp , Nov [3] L. Censie, P. Feautie, \A Ne Solution to Coheence Poblem in Multicache Systems," IEEE Tans. Computes, Vol. C-27, No. 12, Dec 1978, pp [4] R. Katz, S. Wood, D. A. Wood, C. Pekins, R. G. Sheldon, \Implementing a cache consistency potocol," Poc. of the 12th Annual Int'l Sym. on Comp. Achi., 1983, pp [5] J. Kong, P.-C. Ye, G. Lee, \Minimizing the Size of Diectoy fo Lage-scale Multipocessos," in pepaation. [6] D. E. Lenoski, J. Laudon, K. Ghaacholoo, A. Gupta, J. Hennessy, \The Diectoy-Based Cache Coheence Potocol fo DASH multipocesso," Poc. of the 17th Annual Int'l Sym. on Comp. Achi., pp , [7] K. Li, P. Hudak, \Memoy Coheence in Shaed Vitual Memoy Systems," ACM Tans. on Compute Systems, 7(4): , Nov [8] P. Seazey, A.J. Smith, \A Class of Compatible Cache Consistency Potocols and thei Suppot by the IEEE Futuebus," Poc. of the 13th ISCA, pp , [9] J. Toellas, M.Lam, and J.Hennessy, \False Shaing and Spatial Locality in Multipocesso Caches," IEEE Tans. on Computes, IEEE, Inc., Vol. 43, Numbe 6, pp , June [10] J. Veensta, R. Fole, \Mint: A Font-End fo Ecient Simulation of Shaed-Memoy Multipocessos," Poc. of 2nd MASCOTS, Jan.-Feb., [11] S. C. Woo, M. Ohaa, E. Toie, J. P. Singh, A. Gupta, \The SPLASH-2 Pogams: Chaacteization and Methodological Consideations," Poc. of the 22th Annual Int'l Sym. on Comp. Achi., [12] P. Zaopulo, C.H. West, H. Rudin, D.D. Coan, D. Band, \Toads analyzing and synthesizing potocols," IEEE Tans. on Communications, 28(4),