UH-MEM: Utility-Based Hybrid Memory Management. Yang Li, Saugata Ghose, Jongmoo Choi, Jin Sun, Hui Wang, Onur Mutlu

Transcription

1 UH-MEM: Utility-Based Hybrid Memory Maagemet Yag Li, Saugata Ghose, Jogmoo Choi, Ji Su, Hui Wag, Our Mutlu 1

2 Executive Summary DRAM faces sigificat techology scalig difficulties Emergig memory techologies overcome difficulties (e.g., high capacity, low idle power), but have other shortcomigs (e.g., slower tha DRAM) Hybrid memory system pairs DRAM with emergig memory techology Goal: combie beefits of both memories Processor i a cost-effective maer Problem: Which memory do we place each Memory A Memory B page i, to optimize system performace? Our approach: UH-MEM (Utility-based Hybrid MEmory Maagemet) Key Idea: for each page, estimate utility (i.e., performace impact) of migratig page, the use utility to guide page placemet Key Mechaism: a comprehesive model to estimate utility usig memory access characteristics, applicatio impact o system performace UH-MEM improves performace by 14% o average over the best of three state-of-the-art hybrid memory maagers 2

3 Outlie Backgroud Existig Solutios UH-MEM: Goal ad Key Idea Key Mechaisms of UH-MEM Evaluatio Coclusio 3

4 State-of-the-Art Memory Techologies DRAM scalig has already become icreasigly difficult Icreasig cell leakage curret, reduced cell reliability, icreasig maufacturig difficulties [Kim+ ISCA 2014], [Liu+ ISCA 2013], [Mutlu IMW 2017], [Mutlu DATE 2017] Difficult to sigificatly improve capacity, speed, eergy Emergig memory techologies are promisig 3D-Stacked DRAM higher badwidth smaller capacity Reduced-Latecy DRAM (e.g., RLDRAM, TL-DRAM) Low-Power DRAM (e.g., LPDDR3, LPDDR4) No-Volatile Memory (NVM) (e.g., PCM, STTRAM, ReRAM, 3D Xpoit) lower latecy lower power larger capacity higher cost higher latecy higher cost higher latecy higher dyamic power lower edurace 4

5 Hybrid Memory System Pair multiple memory techologies together to exploit the advatages of each memory e.g., DRAM-NVM: DRAM-like latecy, NVM-like capacity Cores/Caches Memory Cotrollers Chael A Memory A (Fast, Small) Memory B (Large, Slow) Chael B 5

6 Hybrid Memory System Pair multiple memory techologies together to exploit the advatages of each memory e.g., DRAM-NVM: DRAM-like latecy, NVM-like capacity Cores/Caches Memory Cotrollers Chael A Memory A DRAM (Fast, Small) Memory B NVM (Large, Slow) Chael B Separate chaels: Memory A ad B ca be accessed i parallel 6

7 Hybrid Memory System: Backgroud Cores/Caches Memory Cotrollers Chael A Row Buffer Bak Chael B Memory A (Fast, Small) Memory B (Large, Slow) Bak: Cotais multiple arrays of cells Multiple baks ca serve accesses i parallel Row buffer: Iteral buffer that holds the ope row i a bak Row buffer hits: serviced ~3x faster, similar latecy for both Memory A ad B Row buffer misses: latecy is higher for Memory B 7

8 Hybrid Memory System: Problem Cores/Caches Chael A Memory Cotrollers IDLE Chael B Memory A (Fast, Small) Memory A is fast, but small Load should be balaced o both chaels Memory B (Large, Slow) Page 1 Page 2 Which memory do we place each page i, to maximize system performace? Page migratios have performace ad eergy overhead 8

9 Outlie Backgroud Existig Solutios UH-MEM: Goal ad Key Idea Key Mechaisms of UH-MEM Evaluatio Coclusio 9

10 Existig Solutios ALL [Qureshi+ ISCA 2009] Migrate all pages from slow memory to fast memory whe the page is accessed Fast memory is LRU cache: pages evicted whe memory is full FREQ: Access freuecy based approach [Ramos+ ICS 2011], [Jiag+ HPCA 2010] Access freuecy: umber of times a page is accessed Keep pages with high access freuecy i fast memory RBLA: Access freuecy ad row buffer locality based approach [Yoo+ ICCD 2012] Row buffer locality: fractio of row buffer hits out of all memory accesses to a page Keep pages with high access freuecy ad low row buffer locality i fast memory 10

11 Weakesses of Existig Solutios They are all heuristics that cosider oly a limited part of memory access behavior Do ot directly capture the overall system performace impact of data placemet decisios Example: Noe capture memory-level parallelism (MLP) Number of cocurret memory reuests from the same applicatio whe a page is accessed Affects how much page migratio helps performace 11

12 Importace of Memory-Level Parallelism Before migratio: Before migratio: reuests to Page 1 Mem. B reuests to Page 2 Mem. B reuests to Page 3 Mem. B After migratio: After migratio: reuests to Page 1 Mem. A reuests to Page 2 Mem. A T reuests to Page 3 Mem. Mem. A BT time Migratig oe page reduces stall time by T time Must migrate two pages to reduce stall time by T: migratig oe page aloe does ot help Page migratio decisios eed to cosider MLP 12

13 Outlie Backgroud Existig Solutios UH-MEM: Goal ad Key Idea Key Mechaisms of UH-MEM Evaluatio Coclusio 13

14 Our Goal A geeralized mechaism that 1. Directly estimates the performace beefit of migratig a page betwee ay two types of memory 2. Places oly the performace-critical data i the fast memory 14

15 Utility-Based Hybrid Memory Maagemet A memory maager that works for ay hybrid memory e.g., DRAM-NVM, DRAM-RLDRAM Key Idea For each page, use comprehesive characteristics to calculate estimated utility (i.e., performace impact) of migratig page from oe memory to the other i the system Migrate oly pages with the highest utility (i.e., pages that improve system performace the most whe migrated) 15

16 Outlie Backgroud Existig Solutios UH-MEM: Goal ad Key Idea Key Mechaisms of UH-MEM Evaluatio Coclusio 16

17 Key Mechaisms of UH-MEM For each page, estimate utility usig a performace model Applicatio stall time reductio How much would migratig a page beefit the performace of the applicatio that the page belogs to? Applicatio performace sesitivity How much does the improvemet of a sigle applicatio s performace icrease the overall system performace? Utility = StallTime. Sesitivity. Migrate oly pages whose utility exceeds the migratio threshold from slow memory to fast memory Periodically adjust migratio threshold 17

18 Key Mechaisms of UH-MEM For each page, estimate utility usig a performace model Applicatio stall time reductio How much would migratig a page beefit the performace of the applicatio that the page belogs to? Applicatio performace sesitivity How much does the improvemet of a sigle applicatio s performace icrease the overall system performace? Utility = StallTime. Sesitivity. Migrate oly pages whose utility exceeds the migratio threshold from slow memory to fast memory Periodically adjust migratio threshold 18

19 Estimatig Applicatio Stall Time Reductio Use access freuecy, row buffer locality, ad MLP to estimate applicatio s stall time reductio Access freuecy Row buffer locality Number of row buffer misses Why ot row buffer hits? Hits have eual latecy i both memories, so o eed to track them. Access latecy reductio for the page if migrated 19

20 Calculatig Total Access Latecy Reductio Use couter to track umber of row buffer misses (#RBMiss) Calculate chage i umber of memory cycles if page is migrated Reductio i miss latecy if we migrate from Memory B to Memory A Access Latecy 6789 = #RB Miss 6789 (t?7@ A, 6789 t?7@ D, 6789 ) Access Latecy F6.G7 = #RB Miss F6.G7 (t?7@ A, F6.G7 t?7@ D, F6.G7 ) Need separate couters for reads ad writes Each memory ca have differet latecies for reads ad for writes Allows us to geeralize UH-MEM for ay memory 20

21 Estimatig Applicatio Stall Time Reductio Use access freuecy, row buffer locality, ad MLP to estimate applicatio s stall time reductio Access freuecy Row buffer locality Number of row buffer misses Access latecy reductio for the page due to migratio MLP Applicatio stall time reductio 21

22 Calculatig Stall Time Reductio MLP characterizes umber of accesses that overlap How do overlaps affect applicatio performace? Determiig MLP 6789 ad MLP F6.G7 for each page: Total umber of cycles reduced for all reuests Stall Time. = Access Latecy 6789 MLP p Access Latecy F6.G7 MLP F6.G7 Cout cocurret read/write reuests from the same applicatio Amout of cocurrecy may vary across time à use average cocurrecy over a iterval More reuests overlap à reductios overlap à smaller improvemet from migratig the page Not all writes fall o critical path of executio due to write ueues iside memory: p is probability that applicatio stalls whe ueue is beig draied 22

23 Key Mechaisms of UH-MEM For each page, estimate utility usig a performace model Applicatio stall time reductio How much would migratig a page beefit the performace of the applicatio that the page belogs to? Applicatio performace sesitivity How much does the improvemet of a sigle applicatio s performace icrease the overall system performace? Utility = StallTime. Sesitivity. Migrate oly pages whose utility exceeds the migratio threshold from slow memory to fast memory Periodically adjust migratio threshold 23

24 Estimatig Applicatio Performace Sesitivity Objective: improve system job throughput Weighted speedup for multiprogrammed workloads Number of jobs (i.e., applicatios) completed per uit time [Eyerma+ IEEE Micro 2008] For each applicatio i, Speedup. = N OPQRS,T N UVOWSX,T Weighted Speedup = ].^_ Speedup. How does each applicatio cotribute to the weighted speedup? 24

25 Estimatig Applicatio Performace Sesitivity Sesitivity: for each cycle of stall time saved, how much does our target metric improve by? How much of the weighted speedup did the applicatio save i the last iterval? Sesitivity. = Performace. T`a8679,. = Speedup. T`a8679,. Number of cycles saved i last iterval T`a8679,. ca be couted at rutime Derived mathematically (see paper for details) Speedup. ca be estimated usig previously-proposed models [Moscibroda+ USENIX Security 2007], [Mutlu+ MICRO 2007], [Ebrahimi+ ASLPLOS 2010], [Ebrahimi+ ISCA 2011], [Subramaia+ HPCA 2013], [Subramaia+ MICRO 2015] 25

26 Key Mechaisms of UH-MEM For each page, estimate utility usig a performace model Applicatio stall time reductio How much would migratig a page beefit the performace of the applicatio that the page belogs to? Applicatio performace sesitivity How much does the improvemet of a sigle applicatio s performace icrease the overall system performace? Utility = StallTime. Sesitivity. Migrate oly pages whose utility exceeds the migratio threshold from slow memory to fast memory Periodically adjust migratio threshold Hill-climbig algorithm to balace migratios ad chael load More details i the paper 26

27 Outlie Backgroud Existig Solutios UH-MEM: Goal ad Key Idea Key Mechaisms of UH-MEM Evaluatio Coclusio 27

28 System Cofiguratio Cycle-accurate hybrid memory simulator Models memory system i details 8 cores, 2.67GHz LLC: 2MB, 8-way, 64B cache block We will release the simulator o GitHub: Baselie cofiguratio (DRAM-NVM) Memory latecies based o real products, prior studies Memory Timig Parameter DRAM NVM row activatio time (t RCD ) 15 s 67.5 s colum access latecy (t ij ) 15 s 15 s write recovery time (t WR ) 15 s 180 s precharge latecy (t lm ) 15 s 15 s Eergy umbers derived from prior works [Lee+ ISCA 2009] 28

29 Bechmarks Idividual applicatios From SPEC CPU 2006, YCSB bechmark suites Classify applicatios as memory-itesive or o-memory- itesive based o last-level cache MPKI (misses per kilo-istructio) Geerate 40 multiprogrammed workloads, each cosistig of 8 applicatios Workload memory itesity: the proportio of memory-itesive applicatios withi the workload 29

30 Results: System Performace ALL FREQ RBLA UH-MEM Normalized Weighted Speedup % 9% 3% 5% 0% 25% 50% 75% 100% Workload Memory Itesity Category UH-MEM improves system performace over the best state-of-the-art hybrid memory maager 30

31 Results: Sesitivity to Slow Memory Latecy We vary t li ad t ol of the slow memory Weighted Speedup t li : t ol : ALL FREQ RBLA UH-MEM 8% 6% 14% 13% 13% x3.0 x3.0 x4.0 x4.0 x4.5 x12 x6.0 x16 x7.5 x20 Slow Memory Latecy Multiplier UH-MEM improves system performace for a wide variety of hybrid memory systems 31

32 More Results i the Paper UH-MEM reduces eergy cosumptio, achieves similar or better fairess compared with prior proposals Sesitivity study o size of fast memory We vary the size of fast memory from 256MB to 2GB UH-MEM cosistetly outperforms prior proposals Hardware overhead: 42.9kB (~2% of last-level cache) Mai overhead: hardware structure to store access freuecy, row buffer locality, ad MLP 32

33 Outlie Backgroud Existig Solutios UH-MEM: Goal ad Key Idea Key Mechaisms of UH-MEM Evaluatio Coclusio 33

34 Coclusio DRAM faces sigificat techology scalig difficulties Emergig memory techologies overcome difficulties (e.g., high capacity, low idle power), but have other shortcomigs (e.g., slower tha DRAM) Hybrid memory system pairs DRAM with emergig memory techology Goal: combie beefits of both memories Processor i a cost-effective maer Problem: Which memory do we place each Memory A Memory B page i, to optimize system performace? Our approach: UH-MEM (Utility-based Hybrid MEmory Maagemet) Key Idea: for each page, estimate utility (i.e., performace impact) of migratig page, the use utility to guide page placemet Key Mechaism: a comprehesive model to estimate utility usig memory access characteristics, applicatio impact o system performace UH-MEM improves performace by 14% o average over the best of three state-of-the-art hybrid memory maagers 34

35 UH-MEM: Utility-Based Hybrid Memory Maagemet Yag Li, Saugata Ghose, Jogmoo Choi, Ji Su, Hui Wag, Our Mutlu 35

36 Results: Fairess ALL FREQ RBLA UH-MEM Normalized Ufairess % 25% 50% 75% 100% Workload Memory Itesity Category UH-MEM achieves similar or better fairess compared with prior proposals 36

37 Results: Total Stall Time ALL FREQ RBLA UH-MEM Average App Stall Time (x10^9 cycles) % 25% 50% 75% 100% Workload Memory Itesity Category UH-MEM reduces applicatio stall time compared with prior proposals 37

38 Results: Eergy Cosumptio ALL FREQ RBLA UH-MEM Memory Eergy (J) % 25% 50% 75% 100% Workload Memory Itesity Category UH-MEM reduces eergy cosumptio for data-itesive workloads 38

39 Results: Sesitivity to Fast Memory Capacity ALL FREQ RBLA UH-MEM 3.5 WSpeedup MB 512MB 1GB 2GB Fast Memory Capacity UH-MEM outperforms prior proposals uder differet fast memory capacities 39

40 MLP Distributio for All Pages Freuecy (%) Freuecy (%) Freuecy (%) MLP MLP MLP (a) soplex (b) xalacbmk (c) YCSB-B 40

41 Mai Hardware Cost 41

42 Baselie System Parameters 42

43 Impact of Differet Factors o Stall Time Correlatio coefficiets betwee the average stall time per page ad differet factors (AF: access freuecy; RBL: row buffer locality; MLP: memory level parallelism) 43

44 Migratio Threshold Determiatio Use hill climbig to determie the threshold At the ed of the curret period, compare Performace i667rg m76.s9 with Performace j8`g m76.s9 If the system performace improves, the threshold adjustmet at the ed of last period beefits system performace adjust the threshold i the same directio Otherwise, adjust the threshold i the opposite directio 44

45 Overall UH-MEM Mechaism Period-based approach - each period has a migratio threshold Actio 1: Recalculate the page s utility Actio 2: Compare the page s utility with the migratio threshold, ad decide whether to migrate Actio 3: Adjust the migratio threshold at the ed of a uatum Adjust the migratio threshold Memory Cotrollers Recalculate Page Utility Compare & Decide Memory A (Fast, Small)? Memory B (Large, Slow) 45

46 UH-MEM: Puttig It All Together 46