The Difference-bit Cache*

Transcription

1 The Difference-bi Cache* Toni Juan, Tomas Lang~ and Juan J. Navarro Deparmen of Compuer Archiecure Deparmen of Elecrical and Universia Poli&cnica de Caalunya Compuer Engineering Gran CapiiJ s/n, Modul D6 Universiy of California a rvine E Barcelona, (Spain) langhci. edu {anonioj, juanjo}@ac.upc. es Absrac The difference-bi cache is a wo-way se-associaive cache wih an access ime ha is smaller han ha of a convenional one and close or equal o ha of a direc-mapped cache. This is achieved by noicing ha he wo ags for a se have o differ a leas by one bi and by using his bi o selec he way. n conras wih previous approaches ha predic he way and have wo ypes of his (primary of one cycle and secondary of wo o four cycles), all his of he difference-bi cache are of one cycle, The evaluaion of he access ime of our cache organizaion has been performed using a recenly proposed on-chip cache access model. nroducion Since he cycle ime of a pipelined processor is usually deermined by he cache access ime [4], [ O], [2], he bes performance is ob ained wih a direc-mapped firs-level cache [], [7], [5], even hough for mos programs he miss raio of his cache is somewha greaer han ha of a se-associaive cache [3], [5], [6], [3]. A clear performance improvemen could be obained if i is possible o have a cache wih he access ime of he direc-mapped cache and he miss raio of he se-associaive cache. Cache organizaions ha modify a se-associaive cache o achieve an average access ime close o ha of a direcmapped cache are presened in [9], [4], [2], [] and a design framework is presened in [6]. All hese proposals are based on he same idea, namely, a candidae line is seleced in a ime corresponding o he direc access, while i akes longer o deermine wheher i is he correc line. Becanse of he speculaive naure of he iniial selecion, hese schemes have wo kinds of his: primary his having a laency of one cycle, and secondary his wih a laency from wo o four cycles. As a consequence, he average hi ime is somewha *This work was suppored by he Minisry of Educaion and Science of Spain (CCYT TC-0429/95) and by he EU (ESPRT Projec APPARC 6634) Permieaion o make digialhard mpy of per or all of his work for personal or classroom use is raned wihou fee provided ha copies are no made or disribued for pro or commercial advana e, he copyrigh noice, he ile of he ubikxdion and ik dae appear, an i noioe is given ha copying S i y permission of ACM, nc. To copy oherwise, o republish, o pos on serwm, or o redisribue o liss, requires prior specific permission and/or a fee. WA PA, 99S ACM s6-s/9s/ $3.50 larger han ha of a direc-mapped cache. The proposals menioned differ in he funcion used o predic he candidae line. Oher relaed approaches are he vicim cache [8] and he virual lines [5]. n he vicim cache he miss rae is he same of he direc-mapped cache bu here are wo kinds of misses, he normal ones and faser ones served by a small fully-associaive cache placed beween he firs and he second level caches. The proposal of virual lines exends he vicim cache mainly by reducing he line size of he main cache o increase he emporaj localiy and by increasing he line size in he auxiliar cache o improve he spaial localiy. We furher describe previous approaches in Secion 3. n he organizaion presened here he hi ime is faser han ha of a wo-way se-associaive cache and close or equal o ha of a direc-mapped cache, he acual value depending on he echnology. n conras wih previous approaches ha predic he way and have wo ypes of his (primary of one cycle and secondary of wo o four cycles), all his of he difference-bi cache are of one cycle. Moreover, he miss rae is equal o a wo-way se-associaive cache. On he oher hand, his proposal is only well suied for he woway case, whereas he previous ones do no have his limi aion. However, he reducion in miss rae obained by a higher associaiviy is small [6], [2]. Furhermore, increasing he degree of associaiviy in he oher proposals increases he number of secondary his, worsening he average memory access ime. The organizaion presened requires a cache wih virual addresses and ags, since he bis needed would no be available in ime if he addresses or ags have o be ranslaed.. Muliarray implemenaion The organizaion we propose is based on he fac ha an opimal (fases) realizaion of he daa par of a cache memory consiss of several subarrays as shown in Figure [7], [8]. The number of subarrays and heir size are a funcion of he cache size and of echnology characerisics and implemenaion resricions. For his realizaion, in he direc-mapped case (see Figure 2) he index bis are pariioned ino wo pars: one par is used o access a row from each subarray and he oher o selec he desired subarray. Moreover, he word bis are used o selec he word from he line. Shown in Figure is only he pariion of he bi line, alhough an acual implemenaion would also pariion he word line. We do no show his pariion because i does no have an effec in our proposal. 4

2 !X 3 a) ndex b) m, ,, l,- f ( n- + J L Figure : deal memory array (a) and opimal aspec raio for same memory (b), The same pariioning of he index bis is used in he w-way se-associaive case (see Figure 3 for w = 2), wih he difference ha w bis of he second par are obained from he ag comparisons, insead of from he index (afer decoding). The access ime of he se-associaive cache is larger han ha of a direc-mapped cache because usually he criical pah is hrough he ag par and includes he access of ags and he comparisons [5], [8]. From he muliarray organizaions shown, he following characerisics can be observed: For he direc-mapped cache, he delay of he signals o enable he ri-sae gaes is smaller han he ime o access he daa from he memory. Consequenly, he ime o obain he resuling daa line is equal o he ime in which he daa is available a he oupu of he subarray plus he delay of he ri-sae gae. Moreover, he ag comparaor is no in he criical pah since he daa can be ransferred o he nex sage of he pipeline wihou knowing wheher he access is a hi. This informaion is only required before he use of he daa, usually o sore i in a processor regiser. To have a se-associaive cache wih he same word access ime w he direc-mapped cache of he same capaciy and line size, i is sufficien ha he enable signals of he ri-sae gaes are obained wih a delay which is smaller han he access ime of he diiri subarray, To achieve his, schemes [4], [9], [2] h a,ve been proposed in which he correc way is predice(i. However, because of he predicion, here are wo ypes of his and he average hi ime is somewha larger han ha of a direc-mapped cache. n his paper we describe a new cache organizaion which achieves he hi ime of he direc-mapped cache for a woway se-associaive cache. Tha is,. We deermine he enable signals of he he ri-sae gaes wih a suiably small delay, and We do no perform predicion, bu selec he correc word if here is a hi: The comparisons of he ags are only used o deermine wheher here is a hi, bu no o choose he line. Consequenly, all his are of one cycle. 2 The difference-bi cache Our realizaion of a wo-way se-associaive cache is based on he fac ha he wo sored ags ha correspond o a se have o differ in a leas one bi. We call dhf-index he posiion in he ag of he leas-significan bi in which hese wo ags differ and cliff-value he value of he bi in he ag of way O of he se. These cliff-index and cliff-value are used o deermine he enable signals of he ri-sae gaes as shown in Figure 4. To do his, he pairs (cliff-index, diffwdue) are sored in he Diff memory of size S x r, where S is he number of ses of he wo-way associaive cache and he value r depends on he code used o represen cliff-index. f is he number of bis of he ag, he minimum value of r is [Zog,] +, wih he binary code, and he maximum +, wi h he l-ou-of- code; inermediae values are obained wih oher codes, as discussed laer. The enable signals of he ri-sae gaes are obained as follows: The corresponding enry of he Diff memory is read, simulaneously wih he daa (and wih he ags, alhough hese are no in he criical pah). The obained cliff-index is used o selec he corresponding bi of he ag porion of he address. The seleced bi and cliff-value are used o deermine he way: if he bis are equrd hen way O of he se is seleced, whereas if hey are differen way is seleced. The way bis (for way O and for way ) are used o drive he enable signal of he ri-sae gaes ha pass he corresponding word. 5

3 @l 4--J--+ il, i h l $? cmp Hi? Daa Word Figure Muli-array realizaion ofadirec-mapped cache (only hepariioning of he daa par is shown). Noe also ha only one ag comparaor is needed since heago compare can deseleced as shown in Figure4. n a miss, i is necessary o deermine he new cliff-index and cliff-value. The simple hardware required is no shown in he Figure. consiss of an array ofxor gaes, o compare he bis of he ags, andaprioriy encoder, for he paricular code used for he cliff-index. For he replacemen policy, here are he same choices as for he convenional wo-way se-associaive cache, resuling in he same miss raio. 2, Deerminaion ofcriical pah We now deermine hecriical pah in order o argue ha i is plausible ha he access ime of he described woway se-associaive cache is equal o ha of a direc-mapped cache of he same capaciy and line size. The criical pah is.= max(~a~a,.na~~e)+~ri as shown in Figure 4, Consequenly, he access ime corresponds o ha of a direc-mapped cache if As described above, enable < d~~~ enable 7 diff +.$e~ec + way + drive where diff is he access ime of he Diff memory. The erms diff and,elec are relaed and depend on he code used o represen cliff-index. n general, if he code has more bis difis larger since he memory is wider bu..i.c is smaller because he decoding is simpler. The opimal combinaion depends on he increase of cijj wih he memory widh and on he complexiy of he corresponding decoder. s e~able < da~a? This depends on he echnology and on he implemenaion resricions. Alhough an implemenaion or circui-level modeling is required o give a definiive answer, we claim ha his is reasonable because he Diff memory is significanly smaller han one subarray of Daa. This is so because he daa memory has a widh of one line (L bis), whereas he Diff memory has a widh of T bis (and r < -L). Consequenly, in pracical cases, he widh of Diff is several imes smaller han he widh of D a ~ for example, for a processor wih = 30 and L = 256 resuling in a Diff widh of beween 6 and 3 bis, depending on he code, and a Daa widh of 256 bis. Moreover, he number of rows of Diff is one half of ha of Daa. Finally, he opimal pariion of Daa using [8] produces from wo o eigh subarrays, so ha Diff is significanly smaller han one Daa subarray. As a consequence, he access ime of Diff is smaller han ha of a Daa subarray, so ha he way selecion can be performed in ime. To confirm ha i is plausible o conclude ha he resuling hi access ime corresponds o ha of a direc-mapped cache, we evaluae he delay using he deailed analyical access model for on-chip caches presened in [8] and apply i o a cache of he characerisics similar of ha of he Alpha processors (8 Kbyes, line of 32 byes, ag of 3 bis) and o caches wih wice and four imes his capaciy. The evaluaion of he direc-mapped cache and he convenional wo-way se-associaive cache are performed using ac i, he sofware associaed wih [8], whereas he evaluaion of he difference-bi cache is obained analyically using he expressions given in [8]. Following he approach used in [8], in Figure 5 we develop an implemenaion for he fully-decoded scheme of he general idea shown in Figure 4 (we do no include he ag memory and he ag comparaors, since his does no affec he daa selecion par). The word-selecion par of he implemen aion has he following componens: 6

4 @ EK\\Y J Way Way O, , * Daa Word Figure 3: Muli-array realizaion of a convenional wo-way se-associaive cache (only he physical pariioning of he daa par is shown). The Diff memory. A decoder o decode he cliff-index (his decoder does no appear in Fig 5 since i is no necessary for he fully-decoded case). b A selecor of he corresponding ag bi. This is implemened as a column of a memory, wih one bi line and a sense amplifier. To obain he bi and is complemen, wo sense amplifiers are used. A 2x2 crossbar o obain he wayo and wayl bis. This crossbar is conrolled by he cliff-value. The driver of he enable signals of he word ri-sae gaes, As in [8], his is implemened in hree levels of gaes. Moreover, we include here he delay of an inverer ha is par of he ri-sae gae. The evaluaion of he criical pah is done for lines of 4 words, words of 64 bis, addresses of 43 bis and he echnology parameers of he model presened in [8] (for a.8,um CMOS echnology )2. The only varying pimameer is he cache capaciy. Neverheless, for he delay of he enable signals in he difference-bi cache, we have considered hree coding opions: fully-encoded (5 bis), fully-decoded (from 29 o 3 bis depending on he cache capaciy) and parially-encoded (6+6 = 2 bis and a decoding of one level of wo-inpu gaes). To simplify he presenaion of he resuls, we divide he delay of he enable signal ino an invarian par (no dependen on he capaciy nor on he coding) and a variable par. The componens of he invarian par correspond o he following imes: 2 According o [8] numbers for a.5pm echnology can be obained dividing all delays by.6, so ha he conclusions remain he same. Capaciy Coding di f f decodeinvariun ~nable Kb [ns] [ns] [ns] [ns] [ns] / , Table : Delay for 8, 6 and 32 Kbyes difference-bi caches for he fully-encoded (5+), parially-encoded (2+ ) and fully-decoded (3+, 30+ and 29+) schemes. selecion of he ag bi, eelec = 0.6 nsec. crossbar, w.g= 0.2 nsec. Q driver of enable signals, d~i~~ = 2. nsec. This oal invarian par is hen of 2.9 nsec. Table gives he delay of he enable signal for differen capaciies and coding schemes. Using his daa, in able 2, we compare he delay of he enable signal for a convenional wo-way se-associaive cache and for he differencebi cache wih he delay of he daa par of he cache. From hese ables we conclude ha he difference-bi cache is considerably faser han he convenional wo-way seassociaive cache and ha, choosing a suiable encoding, i is reasonable o argue ha he access ime of he differencebi cache is equal o ha of a direc-mapped cache. 7

5 Way Way O ! r Hi? Daa Word Figure 4: The difference-bi cache. Capaciy Conv. wo-way Bes Diff Daa Kb [ns] [ns] [ns] , Table 2: Delay of he enable signal for a convenional woway se-associaive cache and for he bes difference-bi cache and delay of he daa subarray. 2.2 Area increase The area requiremens of he new wo-way se-associaive cache implemenaion are somewha larger han hose of he convenional we-way se-associaive cache. This exra area corresponds o he shaded porion of Figure 4. The main conribuion o his area is he Diff memorv of size S x T bis. n comparison, he Daa cache has 2S x L bis (.L is he line size in bis) and he ag memory has area S x (2) so ha he fracion of increase is 2(LT+ ) Table 3: Delay of he criical pah for convenional wo-way se-associaive caches (~o~~ = ~g + cmp+ dr;oe. ) To clarify furher he difference beween he wo-way convenional cache and he difference-bi cache, in Table 3 we give a breakdown of he imes of he former, The idea can be applied o any cache and line sizes. For a given echnology, he access ime would be closer o ha of a direc-mapped cache for larger cache size since he invarian par of he delay in he difference-bi cache becomes less significan when he size increases. This is also he case for longer lines (for he same capaciy) since he number of ses is reduced, resuling in a shorer Diff array. For pracical cases his fracion is small; for example for he values used in he evaluaion, similar o hose of he Alpha family of processors, i is beween 0.0 and 0.06 depending on he Diff implemenaion. Table 4 shows oher ypical values. Since he added area depends on he widh of he Diff memory, i is convenien o choose he minimum widh ha achieves he required access ime. From he access ime daa given in Table, we would choose T = 3 for he 8K case (a 2% area increase) and r = 6 for he 6K and 32K cases (a % area increase). 3 Relaed work As menioned in he nroducion, several previous proposals have considered a se-associaive cache wih he access ime of a direc-mapped cache. The common denominaor among hese proposals is ha a predicion of he way is performed and he corresponding word is seleced. Laer, afer he ag comparisons, he correc way is deermined and a new selecion has o be performed if he predicion failed. Consequenly, here are wo ypes of his w follows: s Primary his, which occur when he predicion is correc. These his are served in one cycle. 8

6 ag index h~n..3 L To driver of enable signal Figure 5: A possible hardware implemenaion for he ag selecion and way signals. ; yfl + 2[/l + +ll ().04 3 O.O CL06 Table 4: Fracion of area increase of he difference-bi cache compared wih a direc-mapped cache wih he same capaciy and line size. Secondary his, when he predicion fails. n his case, anoher selecion has o be performed, so ha he hi requires from wo o four cycles. The proposals differ in he funcion used for selecion and on he replacemen policy. n cen ras, he difference-bi cache proposed here achieves he miss rae of a wo-way seassociaive cache and he hi ime of a direc-mapped cache wih all he his being primary his. We now describe in somewha more deail hese previous proposals. n he MRU cache [4] he prediced line is he mosrecenly used one of he se. Secondary his require wo cycles, This scheme can be used for any degree of associaiviy bu as he associaiviy increases he probabiliy o have a primary hi decreases, worsening he average hi ime. n he column-associaive cache [] wo hashing funcions are applied o an address. The daa is accessed using he firs hashing funcion (similar o direc mapping). f his firi funcion misses, a second funcion is used for a secondary hi. f he second funcion is a hi, he lines corresponding o hese wo funcions are swapped. n a miss, he las line referenced is placed according o he firs hashing funcion. A secondary hi requires hree cycles. Moreover, due o he sequenial applicaion of wo hashing funcions, he miss cycle ime is increased in hree cycles. Anoher propos.d is he DASC cache [2]. This is a ses.wociaive cache in which he predicion is done assuming a direc-mapped cache. f he ag side deecs a hi in anoher posiion of he se, he daa use is abored and he line in he accessed posiion and in he correc posiion are swapped. A secondary hi requires four cycles, n case of miss, he line is wrien according o he replacemen algorihm and hen is swapped wih he line ha is accessed in a direc-mapped cache. Again i can be used for any degree of associaiviy bu he probabiliy of firs-ime hi decreases. The las proposal is he PAD cache [9]. The ag side is divided ino wo pars. The firs par holds he k leassignifican bis of he ags and he oher par keeps he remaining bis. The way is prediced comparing he ags of he firs par. n case of more han one hi in his par, any of he ways (for example he mos-recenly used) is accessed while he second par of he ags are compared o deermine if he correc way was prediced. The penaly of secondary his is of one addiional cycle. can be used for any degree of associaiviy bu he probabiliy of primary hi decreases. 4 Conclusions We have presened he difference-bi cache, a new organizaion of a wo-way se-associaive cache wih he access ime of a direc-mapped cache of he same capaciy and line size. This access ime is obained by separaing he selecion of he proper way from he deecion of a hi, and selecing he way using he leas-significan bi in which boh ags of a se differ. The performance obained wih he difference-bi cache is beer han he performance obained wih a direcmapped cache, a convenional wo-way se-associaive cache and wih any of he previous proposals ha cause wo ypes of his. Our proposal has been evaluaed using he implemenaion approach and he deailed cache model of [8]. The resuls of his evaluaion show ha he desired access ime is achieved for parameers corresponding o pracical firs-level caches. The addiional area of he selecion mechanism is small and has been esimaed a abou 3 ZO of he cache area. The difference-bi cache can be direcly used for virualindexed/virual-agged caches. requires a cache wih virual ags, since he delay of he address ranslaion would no allow a selecion of he way in ime. Moreover, he index has o be virual o permi a fas daa access. However, hese virualindexed/virual-agged caches have wo drawbacks: a) a conex swich may invalidae all cache lines unless he cache lines are agged wih idenifiers of heir address space, and b) wo or more virual addresses can map o he same real address inroducing synonym problems [9]. Virual-indexed bu real-agged (V/R) caches are preferred because hey do no suffer from he conex-swiching problem. Moreover, he synonyms problem is minimized. We are invesigaing he possibiliy of adaping he differencebi cache o V/R caches. Noice ha he ranslaed ag is no needed unil he bi selecion so ha he ranslaion can overlap wih he access o he Diff memory. is no clear a his poin wheher his overlap is sufficien o ge a suiable access ime. The difference-bi idea can be applied o any degree of associaiviy bu he addiional area required increases significanly and also he access ime. 9

7 Acknowledgemens We hank Dr. Andre Seznec for his help in clarifying he applicabiliy y for he V/R case and he anonymous reviewers for heir useful commens. References [] Anan Agarwal and Seven D. Pudar. Columnassociaive caches: A echnique for reducing he miss rae of direc-mapped caches. n Proc. of he n. Syvnp. on Compuer Archiecure, pages 79 90, 993. [2] Keih Boland and Aposollos Dollas. Predicing and precluding problems wih memory laency. EEE Micro, pages 59 67, Aug 994. [3] Jeffrey D. Gee, Mark D. Hill, Dionisios N. Pneumaikaos, and Alan Jay Smih. Cache performance of he spec92 benchmark suie. EEE Micro, Vol. 3(4):8-6, Aug 993. [6] Kevin B. Theobald, Herber H. J. Hum, and Guang R. Gao. A design framework for hybrid-access caches. n Proc. of he s n. Symp. on High-Performance Compuer Archiecure, pages 44 53, Jan 995. [7] Tomohisa Wada, Suresh Rajan, and Seven A. Przybylski. An analyical access ime model for on-chip cache memories. EEE Journal of Solid-Sae Circuis, Vol. 27(8):47-56, Aug 992. [8] Seven J.E. Wilon and Norman P. Jouppi. An enhanced access and cycle ime model for on-chip caches. Research Repor 93/5, Digial WRL, Jul 994. [9] C. Eric Wu, Yarsun Hsu, and Yew-Huey Liu. A quaniaive evaluaion of cache ypes for high-performance compuer sysems. EEE rans. on Compuers, Vol. 42(0):54-62, O 993. [4] John L. Hennessy and Norman P. Jouppi. Compuer echnology and archiecure: An evolving ineracion. EEE Compuer, pages 8-29, Sep 99. [5] Mark D. Hill. A case for direc-mapped caches. EEE Compuer, pages 25-40, Dec 988. [6] Mark D. Hill and Alan Jay Smih. Evaluaing associaiviy in CPU caches. EEE rans. on Compuers, Vol. 38(2):62-630, Dec 989. [7] Norman P. Jouppi. Tradeoffs in he design of he muliian cpu. n Proc. of he n. Symp on Compuer Archiecure, pages , 989. [8] Norman P. Jouppi. mproving direc-mapped cache performance by he addiion of a small fully-associaive cache and prefech buffers. n Proc. of he n. Symp. on Compuer Archiecure, pages , 990. [9] Lishing Liu. Parial address direcory for cache access. EEE rans. on Very Large Scale negraion Sysems, Vol. 2(2): , Jun 994. [0] Kunle Olukoun, Trevor Mudge, and Richard Brown. Performance opimizaion of pipelined primary caches. n Proc. of he n. Symp. on Compuer Archiecure, pages 8 90, 992, [] Seven A. Przybylski, Mark Horowiz, and John Hennessy. Performance radeoffs in cache design. n Proc. of he n. Symp on Compuer Archzecw-e, pages , 988. [2] Andre Seznec. DASC cache. n Proc. of he id n. Symp on High-Performance Compuer Archiecure, pages 34 43, Jan 995. [3] Alan Jay Smih. Cache memories. Compuing Sw-veis, Vol. 4(3): , Sep 982. [4] Kimming So and Rudolph N. Rechschaffen. Cache operaions by MRU change. EEE rans. on Compuers, Vol. 37(6): , Jun 988. [5] O. Temam and Y. Jegou. Using virual lines o enhance localiy exploiaion. n Proc. oj he d. Conf. on Supercompuing, pages 2,