Energy/Performance Design of Memory Hierarchies for Processor-in-Memory Chips

Transcription

1 Energy/Performnce Design of Memory Hierrchies for Processor-in-Memory Chips Michel Hung, Jose Renu, Seung-Moon Yoo, Josep Torrells Deprmen of Compuer Science Deprmen of Elecricl n Compuer Engineering Universiy of Illinois Urbn-Chmpign hp://icom.cs.uiuc.eu/flexrm Inroucion Merging processors n memory ino single chip hs he well-known benefis of llowing high-bnwih n low-lency communicion beween processor n memory, n reucing energy consumpion. As resul, mny ifferen sysems bse on wh hs been clle Processor In Memory (PIM) rchiecures hve been propose [, 3, 7, 8,, 2 6, 8]. Recen vnces in echnology [, ] pper o mke i possible o inegre logic h cycles nerly s fs s in logic-only chip. As resul, processors re likely o pu much pressure on he relively slow on-chip DRAM. To hnle he spee mismch beween processors n DRAM, hese chips re likely o inclue non-rivil memory hierrchies in ech DRAM bnk. Wih mny on-chip high-frequency processors, ll of hem poenilly ccessing he memory sysem concurrenly, hese chips will consume much energy. In iion, hese chips re likely o be use in non-riionl plces like he memory of server [3, 7, 2] or he I/O subsysem [], which my no hve hevy-uy cooling suppor. Consequenly, i is imporn o esign he chips for energy efficiency. In his bsrc, we exmine, from performnce n energy-efficiency poin of view, he esign of he memory hierrchy in muli-bnke PIM chip wih mny simple, fs processors. Our resuls sugges he use of per-processor memory hierrchies h inclue moes-size cches, simple DRAM bnk orgnizions h suppor segmenion, n no prefeching. 2 Memory Hierrchies for PIM Chips Our focus rchiecure is PIM chip h inclues ens of relively simple, highfrequency processors, ech of which is ssocie wih bnk of DRAM. Such esign hs been suggese for sysems like Acive Pges [2, 3], FlexRAM [7], n DIVA [3] mong ohers. The chip cn be moele s in Figure -(), where he orgnizion of he This work ws suppore in pr by he Nionl Science Founion uner grns NSF Young Invesigor Awr MIP-9736, MIP-9693, n CCR-99788, DARPA Conrc DABT63-9-C-97, n gifs from IBM n Inel.

2 processors, memory, n nework my vry. We feel, however, h currenly-propose esigns re relively conservive in logic spee. Recen vnces in echnology pper o llow logic o cycle nerly s fs s in logic-only chip [, ]. This mens h hese chips my soon inclue processors cycling bou 8- MHz. Such processors re likely o pu much pressure on he slower DRAM. Processor Nework DRAM Bnk () Processor SRAM I-Mem D-Cche Sub-bnke DRAM (b) DB RB RB RB RB RB RB D Buffer Row Dec Row Dec Row Dec Row Dec D-Cche 26 DRAM 26 (c) Fig.. Exmple of chip rchiecure consiere. RB, DB, n Row Dec sn for row buffer, buffer n row ecoer, respecively. To hnle he spee mismch beween processors n DRAM, hese chips re likely o ssocie non-rivil memory hierrchy o ech DRAM bnk. In his pper, we ssume per-bnk bseline memory hierrchy s in Figure -(b). In he figure, he insrucion memory hierrchy inclues fs SRAM memory. The memory hierrchy inclues cche wih hrwre sequenil prefech of line. The DRAM bnk iself is sub-bnke n hs row n buffers. For exmple, Figure -(c) shows he DRAM orgnize ino 8 sub-bnks, wih row buffers, n 2 26-bi buffers. Unlike in memory-only chips, where he DRAM orgnizion is ofen limie o snr esigns, embee sysems llow mny ifferen orgnizions for he DRAM rry. For exmple, esigners cn chnge he wih n lengh of DRAM sub-bnk, n he number of sub-bnks. These chnges cn ffec he performnce elivere n he energy consume by DRAM ccesses, n he re uilize.

3 ) 876 : ;: In riionl DRAM rry orgnizion, when bnk is ccesse, every oher sub-bnk is cive. Consecuive sub-bnks re no cive becuse hey shre row buffer. Figure 2-() shows sub-bnk orgnizion. We now consier hree improvemens: segmenion, inerleving, n pipelining. 2 3!!!! Acive $ $ $ $ 2 % % % % $ % (,) (,) (,2) D Bus W Bi Line () Triionl (,) (,2) (,3) ;<<<< 2 3,- &' &' Acive ( () W,- 6 7././ = = W/2 = = 8 9 Acive (c) Inerleve Segmene (b) Segmene Wor Line : Aress Acivion Time : Sub-bnk Occupncy Time : D Trnsfer Time (x,y) y sub-bnk in bus x (,) (,) (,2) (,3) (,) (,) (,2) (,3) (,y) (2,y) (e) () Fig. 2. Differen DRAM bnk orgnizions n imings. Wih segmenion (Figure 2-(b)), only one sub-bnk is cive ime. The resuling row buffer ecoupling chnges he hi re of he row buffers. In iion, DRAM ccesses consume less energy: becuse only hlf of he bi lines re cive, bou % of he energy is sve. Wih inerleving, ech sub-bnk is vericlly slice n bus is ssigne o ech of he resuling slices. Figure 2-(c) shows 2-wy inerleve sysem. The performnce is higher becuse boh busses work in prllel (Figure 2-() shows iming igrm wih he mximum overlp, ssuming single ress bus). As for energy, lhough row buffer his now cos bi more, DRAM ccesses gin sve bou % of he energy becuse only hlf of he cells re cive. The re use increses. Finlly, one problem shown in Figure 2-() is h res from ifferen sub-bnks h shre bus re serilize by long sub-bnk occupncy imes. Wih pipelining, hese sub-bnks cn overlp heir occupncy imes (Figure 2-(e)). The only serilizion hppens in he shre ress bus n bus. The resul is higher performnce. As for energy, pipelining hs only smll impc. 3 Evluion Environmen We evlue he PIM chip of Secion 2 using MINT-bse simulion sysem [9]. The rchiecure moele is single chip wih 6 processors connece in ring. Ech processor is ssocie wih -Mbye DRAM bnk like in Figure -(b). The bseline

4 prmeers of ech processor-bnk pir re shown in Tble. The rge echnology is IBM m Blue Logic SA-27E ASIC [] wih he eful volge of.8 V. Processor D-Cche I-Memory D Buffer Row Buffer Sub-Bnk 2-issue in-orer 8MHz Sz: 8KB, WB Size: Kins. Number: Number: Number: BR Penly: 2 cycles Assoc: 2 Line: ins. Size: 26 b Size: KB Cols: 96 In,L/S,FP Unis: 2,, Line: 32 B RTrip:.2ns Bus: 26 b Bus: 26 b Rows: 2 Pening L,S: 2,2 RTrip:.2ns RTrip:3.7ns RTrip:7.ns RTrip: ns Tble. Prmeers for single memory bnk n processor pir. In he ble, BR n RTrip sn for brnch n conenion-free roun-rip lency from he processor, respecively. Appl. Wh I Does Problem Size D-Cche Averge Hi Re Power(W) GTree D mining: ree generion MB bse, 77.9 K recors, 29 ribues/recor.7.2 DTree D mining: ree eploymen. MB bse, 7. K recors, 29 ribues/recor BSOM BSOM neurl nework 2 K enries, ims, 2 iers, 6-noe nework, 832 KB b.97. BLAST BLAST proein mching 2.3 K sequences,. MB ol, query of 37 byes Mpeg MPEG-2 moion esimion 2x26 frme plus reference frme. Tol 2 KB FIC Frcl imge compressor 2x2 imge, 2x2 inernl srucure. Tol 2 MB Tble 2. Applicions execue. The nmes for he DRAM bnk orgnizions h we evlue re Tr, S, SP, IS, n ISP, which refer o riionl, segmene, segmene pipeline, inerleve segmene, ACBEDGFIH n inerleve segmene pipeline, respecively. In ech cse, we o refer o B -wys inerleve wih F sub-bnks per wy. To esime he energy consume in he chip, we hve pplie scling-own heory o on exising evices repore in he lierure, s well s use severl echniques n formuls repore in he lierure [6, 7, 9, 2]. We he conribuions of he processors, clock, memory hierrchies, n oher moules. A eile iscussion of he mehos h we hve followe cn be foun in [2]. In [2], we hve iionlly vlie our esimes wih CACTI [9] n wih publishe resuls on he ARM processor []. For he experimens, we use 6 pplicions h re suible o he ineger-bse PIM chip consiere: hey ccess lrge memory size, re very prllel, n re ineger bse. They come from severl inusril sources. We hve prllelize ech pplicion ino 6 hres by hn. Tble 2 liss he pplicions n heir chrcerisics. They inclue he omins of mining, neurl neworks, proein mching, mulimei, n imge compression. Ech pplicion runs for severl billions of insrucions.

5 Evluion The bes memory hierrchy orgnizion epens on he meric being opimize. We consier wo merics: performnce n energy-ely prouc. In our evluion, we sr wih he bseline rchiecure of Secion 3 n hen vry i. As reference, we use n iel rchiecure (Perf): los n sores re sisfie wih zero lency n consume no energy in he memory sysem. IPC Averge BLAST BSOM DTree FIC GTree Mpeg Tr(,) S(,) SP(,) IS(2,) ISP(2,) IS(2,8) ISP(2,8) Perf () Tr(,) S(,) SP(,) IS(2,) ISP(2,) IS(2,8) ISP(2,8) Perf (b) Fig. 3. Effec of he DRAM bnk orgnizion on he IPC in sysems wih -Kbye () n 8- Kbye (b) cches. Mximizing Performnce To compre performnce, we mesure he verge IPC elivere by he combine 6 processors for he urion of he pplicion. We firs evlue he effec of he memory bnk orgnizion. Figure 3 shows he IPC of he pplicions running on he bseline rchiecure for ifferen memory bnk orgnizions. Chrs () n (b) correspon o sysems wih - n 8-Kbye D-cches, respecively. The memory orgnizions re orere from he simpler ones on he lef sie o he more sophisice ones on he righ sie. Ech chr hs n Averge line h rcks he verge of ll pplicions. Figure 3-() shows h performnce improves slighly s we move o he more sophisice esigns. Going from Tr(,) o ISP(2,8) increses he IPC by n verge of 8%. However, for 8-Kbye cches (Figure 3-(b)), he chnges re very smll. This is becuse, wih lrge cches, here re relively few cche misses n, s resul, he ype of DRAM bnk orgnizion mers less. Compring he IPC in Perf n ISP(2,8), we see he IPC los in he mos vnce memory sysem. This frcion is on verge 8% n % in Figures 3-() n (b). Figure -() shows he effec of he cche size n prefeching suppor. We consier he bseline rchiecure wih hree ifferen DRAM bnk orgnizions: conservive (Tr(,)), ggressive (ISP(2,8)), n in-beween (IS(2,)). The figure shows he IPC verge over ll pplicions. We nlyze cches of 26 byes, Kbye, 8 Kbyes, n 6 Kbyes, ll wih n wihou prefeching. For ech memory orgnizion, here re 8 brs, lbele wih he cche size in byes followe by P or NP for prefeching or no prefeching, respecively.

6 The bes performnce is chieve wih he lrges cche size (6 Kbyes). However, lrge cches eliver iminishing reurns. The figure lso shows h ing he simple prefeching suppor consiere here mkes lile ifference o performnce. Energy-Dely Prouc Averge BLAST BSOM DTree FIC GTree Mpeg 2 2 Tr(,) S(,) SP(,) IS(2,) ISP(2,) IS(2,8) ISP(2,8) Perf Tr(,) S(,) SP(,) IS(2,) ISP(2,) IS(2,8) ISP(2,8) Perf () (b) Fig.. Effec of he DRAM bnk orgnizion on he energy-ely prouc in sysems wih - Kbye () n 8-Kbye (b) cches. Minimizing he Energy-Dely Prouc In embee sysems, common figure of meri is he energy-ely prouc [2]. A low prouc implies h he sysem is boh fs n energy-efficien. Consequenly, in his secion, we compre he energy-ely prouc of he chips wih ifferen memory hierrchy esigns. To compue he energy consume, we up he conribuions of ll he subsysems in he chip. Figures -() n -(b) show he energy-ely prouc of he chip uner he bseline rchiecure for ifferen DRAM bnk orgnizions. Chrs () n (b) correspon o sysems wih - n 8-Kbye D-cches respecively, n re orgnize s in Figures 3- () n 3-(b). For ech pplicion, he chrs re normlize o Perf. In sysems wih -Kbye cches (Figure -()), he verge energy-ely prouc ecreses for he more vnce memory orgnizions. For exmple, he prouc in ISP(2,8) is only 6% of h in Tr(,). The reson is h vnce DRAM bnk orgnizions eliver slighly higher IPCs n consume much less energy in he process. However, s cches increse o 8 Kbyes (Figure -(b)), he chnges re smller. Overll, for 8-Kbye cche sysems, only segmenion (going from Tr(,) o S(,)) mkes significn ifference. Supporing inerleving n incresing he number of sub-bnks from (2,) o (2,8) hs only smll effec. Figure -(b) mesures he energy-ely prouc for he verge of ll pplicions for ifferen cche sizes n prefeching suppor. The brs re normlize o Perf. From he figure, we see h esigns wih lrger cches en o hve lower energy-ely proucs. For exmple, in Tr(,), he prouc wih 6-Kbye cches is bou 3% of he prouc wih 26-bye cches. The reson is h cches hve ouble effec: hey spee up he progrm n, in iion, elimine energy-consuming memory ccesses. We observe, however, h for he more vnce memory orgnizions n lrge cches, he ren reverses: 6-Kbye cches re slighly worse hn 8-Kbye cches. The reson is

7 IPC NP 2NP 892NP 638NP 26P 2P 892P 638P 26NP 2NP 892NP 638NP 26P 2P 892P 638P 26NP 2NP Tr(,) IS(2,) ISP(2,8) () 892NP 638NP 26P 2P 892P 638P ED NP 2NP 892NP 638NP 26P 2P 892P 638P 26NP 2NP 892NP 638NP 26P 2P 892P 638P 26NP 2NP Tr(,) IS(2,) ISP(2,8) (b) 892NP 638NP 26P 2P 892P 638P Fig.. Effec of he cche size n prefeching suppor on IPC () n energy-ely prouc (b). h he iminishing reurns in lower miss res elivere by lrger cches o no compense for he higher energy consumpion h lrger cches require. We lso see h simple prefeching oes no help. Discussion In PIM chip like he one nlyze here, minimizing he energy-ely prouc is likely o be he op prioriy. Our resuls sugges o use moes-size D-cches (8 Kbyes), simple DRAM bnk orgnizion h suppors only segmenion, n no prefeching. Moes-size cches re effecive: hey spee-up he pplicion, re energy-efficien, consume moes re, n rener fncy DRAM bnk orgnizions lrgely unnecessry. If re is no n issue, he energy-ely prouc cn be improve slighly by supporing inerleving in he DRAM bnk n incresing he number of sub-bnks. References. A. Brown e l. ISTORE: Inrospecive Sorge for D-Inensive Nework Services. Workshop on Ho Topics in Opering Sysems, Mrch R. Gonzlez n M. Horowiz. Energy Dissipion In Generl Purpose Microprocessors. IEEE Journl on Soli-Se Circuis, 3():277 28, Sepember M. Hll e l. Mpping Irregulr Aplicions o DIVA, PIM-Bse D-Inensive Archiecure. In Supercompuing, November IBM Microelecronics. Blue Logic SA-27E ASIC. hp:// news/999/s27e, Februry S. Iyer n H. Kler. Embee DRAM Technology: Opporuniies n Chllenges. IEEE Specrum, April M. Kmble n K. Ghose. Anlyicl Energy Dissipion Moels for Low Power Cches. In Inernionl Symposium on Low Power Elecronics n Design, pges 3 8, Y. Kng, W. Hung, S. Yoo, D. Keen, Z. Ge, V. Lm, P. Pnik, n J. Torrells. FlexRAM: Towr n Avnce Inelligen Memory Sysem. In Inernionl Conference on Compuer Design, pges 92 2, Ocober 999.

8 8. P. Kogge, S. Bss, J. Brockmn, D. Chen, n E. Sh. Pursuing Peflop: Poin Designs for TF Compuers Using PIM Technologies. In Froniers of Mssively Prllel Compuion Symposium, V. Krishnn n J. Torrells. An Execuion-Driven Frmework for Fs n Accure Simulion of Supersclr Processors. In Inernionl Conference on Prllel Archiecures n Compilion Techniques, pges , Ocober K. Mi e l. Smr Memories: A Moulr Reconfigurble Archiecure. In Inernionl Symposium on Compuer Archiecure, June 2.. J. Monnro e l. A 6-MHz, 32-b,.-W CMOS RISC Microprocessor. IEEE Journl of Soli Se Circuis, 3():73 7, November M. Oskin, F. Chong, n T. Sherwoo. Acive Pges: A Compuion Moel for Inelligen Memory. In Inernionl Symposium on Compuer Archiecure, pges 92 23, June M. Oskin e l. Exploiing ILP in Pge-Bse Inelligen Memory. In Inernionl Symposium on Microrchiecure, D. Person e l. A Cse for Inelligen DRAM. IEEE Micro, pges 33, D. Person n M. Smih. Workshop on Mixing Logic n DRAM: Chips h Compue n Remember S. Rixner e l. A Bnwih-Efficien Archiecure for Mei Processing. In Inernionl Symposium on Microrchiecure, November C-L. Su n A. Despin. Cche Design Tre-offs for Power n Performnce Opimizion: A Cse Suy. In Inernionl Symposium on Low Power Elecronics n Design, pges 63 68, April E. Wingol e l. Bring I All o Sofwre: Rw Mchines. IEEE Compuer, pges 86 93, Sepember S. Wilon n N. Jouppi. CACTI: An Enhnce Cche Access n Cycle Time Moel. IEEE Journl on Soli-Se Circuis, 3(): , My N. Yeung e l. The Design of SPECin92 RISC Processor uner 2W. ISSCC Diges of Technicl Ppers, pges 26 27, Februry S-M. Yoo, J. Renu, M. Hung, n J. Torrells. FlexRAM Archiecure Design Prmeers. Technicl Repor CSRD-8, Deprmen of Compuer Science, Universiy of Illinois Urbn-Chmpign, Ocober 2. hp://icom.cs.uiuc.eu/flexrm/publicions.hml.