Emerging NVM Enabled Storage Architecture:

Transcription

1 Emerging NVM Enabled Storage Architecture: From Evolution to Revolution. Yiran Chen Electrical and Computer Engineering University of Pittsburgh Sponsors: NSF, DARPA, AFRL, and HP Labs 1

2 Outline Introduction Evolution with envm: On chip high speed storage; Off chip secondary storage; Revolution with envm: Memristor based neuromorphic accelerator Conclusion 2

3 Conventional Memory Scaling Aspect Ratio A/R Å Burj Khalifa A/R=6 9Å 8Å 7Å 3Å T OX Mb/Chip EDO 50 SDRAM 133 DDR Intrinsic difficulty of charge-based 5Å 10 2 DDR computing and storage! DDR Mbps Technology Node nm 32nm M: Stacked MIM P: Planar A: 6F 2,bWL G: poly/sio 2 C: Si V: 1.35V nm 22nm M: Stacked MIM P: Planar, HKMG A: 6F 2, bwl G: HKMG C: Si V: 1.2V nm 16nm M: Stacked MIM P: Planar A: 6F 2, bbl, LBL, 1T1C(VFET) G: HKMG C: Si V: 1.1V nm 14nm M: FBRAM, STT RAM, RRAM, PCRAM P: Planar A: 4F 2, 1T, 1T1R, 1TMTJ (VFET) G: HKMG C: Si V: ~1V Sources: ASML, ITRS, IMEC, Hynix, IBM 3

4 Emerging Nonvolatile Memory 4

5 Memory Technologies Comparison SRAM DRAM NAND FLASH STT RAM PCRAM ReRAM Data Retention N 4 ms 10y >10 y >10 y >10 y Memory Cell (F 2 ) <1 Read Time 0.2 ns 2 ns 0.1 ms 5 10 ns 5 10 ns 5 10 ns 12 ns 5 10 ns Write/Erase Time 70 ps 1 ns 1/0.1ms <10 ns <50 ns <10 ns Number of Rewrites Power Consumption Read/Write Low Low High Low Low Low Power Consumption other than R/W Leakage Current Refresh Power None None None None Source: ITRS ERD workshop presentation by Prof. Y. Chen 5

6 Challenges: Identifying the evolutional applications that can Easily and seamlessly integrated into the current memory hierarchy and computing platform; Fully leverage the advantages of emerging NVM; Not be easily replaced by other alternative technology or architecture. Inventing a revolutionary computing and storage architecture that can Offer a high performance, power efficient, and scalable computing model; Provide a truly seamless integration between computing and memory. 6

7 Outline Introduction Evolution with envm: On chip high speed storage; STT RAM based 3D cache for CPU. Racetrack based register file for GPU. Off chip secondary storage; Revolution with envm: Memristor based neuromorphic accelerator. Conclusion 7

8 STT RAM based 3D cache Spin Transfer Torque Random Access Memory Bit line Free Layer MgO Layer Reference Layer Word line Source line MTJ Writing 1 0 Magnetic tunneling junction 1T 1MTJ STT RAM Schematic A scalable technology 8

9 SRAM vs. MRAM (STT RAM) Area (65nm) 3.66mm 2 SRAM 3.30mm 2 MRAM Capacity/Bank 128KB 512KB Read latency 2.25ns 2.32ns Write latency 2.26ns 11.02ns Read energy 0.90nJ 0.86nJ Write energy 0.80nJ 5.00nJ Cache configurations Leakage power 2MB (16x128KB) SRAM cache 2.09W 8MB (16x512KB) MRAM cache 0.26W Pros: Low leakage power, high density. Cons: Long write latency and large write power 9

10 STT RAM based 3D cache Baseline 3D Architecture Core Layer + Cache Layers. NUCA caches with NOC connections. Cache Bank Data Migration Cache Bank Cache Bank Router TSV Layer 2 Cache Bank R Cache Bank R Vertical Hop R R Core Layer 1 Horizontal Hop Cache Controller G. Sun, X. Dong, Y. Xie, J. Li, Y. Chen, HPCA,

11 STT RAM based 3D cache Challenges: long write latency of STT RAM. Solution 1 (S1): Read Preemptive Write Buffer. Write Op. Write Buffer (FIFO) Read Op. Read Data Write is just almost begins. done. Write Req. Read Req. STT-RAM Caches Read Op. Cores Read Data 11

12 STT RAM based 3D cache Solution S2: SRAM MRAM Hybrid L2 Cache 31-Way STT-RAM & 32-Way STT-RAM 1-Way SRAM MRAM Bank TSV Core Core Core Core Core Core Core Core SRAM Bank 12

13 STT RAM based 3D cache Result (S1 & S2): Performance is improved by 4.91% compared with STT RAM baseline. Power consumption is reduced by 73.5%. IPC M-SRAM-DNUCA 8M Hybrid DNUCA 8M-MRAM-DNUCA Power

14 Outline Introduction Evolution with envm: On chip high speed storage; STT RAM based 3D cache for CPU. Racetrack based register file for GPU. Off chip secondary storage; Revolution with envm: Memristor based neuromorphic accelerator. Conclusion 14

15 Racetrack for GPU Racetrack cell: WWL BL RWL Racetrack SL Reference layer Free layer Pinning layer Pinning layer Two fixed pinning regions: free region, and fix region Write `0 Write `1 Read Racetrack magnetic track Inject current to move cell Access port 15

16 Racetrack for GPU Benefits from Racetrack: Extremely small cell size; Major challenges: Shifting caused delay/energy. Warp register remapping (WRR) Warp 0 Warp 0 Arbitrator Shift Controller Write/Read/Shifter Driver 60.0% RF are allocated during the execution Non optimal warp register mapping, max shift distance 8 cell WRR, interleaves the warp registers across the access ports, max shift distance 4 cell M. Mao, W. Wen, Y. Zhang, Y. Chen, H. Li, DAC 2014 Row Decoder BL SL BL SL WWL RWL... WWL RWL BL Column Mux Sense Amplifier Arrays SL BL SL 16

17 Racetrack for GPU Write buffer piggyback write to write back to RF from write buffer; 3 Rely on the track movement triggered by the read requests; Positive side effect: filter the redundant RF R/W by leveraging RAW and WAW To EXE/MEM 17

18 Racetrack for GPU Experiment results: Baseline: SRAM based register files. Energy reduction: 59%. Performance improvement: 4%. 18

19 Outline Introduction Evolution with envm: On chip high speed storage; Secondary storage; PCRAM and NAND hybrid SSD; Revolution with envm: Memristor based neuromorphic accelerator. Conclusion 19

20 Hybrid SSD Memory hierarchy On-chip memory 1~30 cycles Erase Unit Page mode PN=0, V Random access Off-chip memory 100~300 cycles PN=1, V X Solid State Disk (Flash) 25K~2M cycles PN=2, V PN=n, V erase-beforewrite (EBW) In-placeupdate (IPU) X Courtesy: Al Fazio (Intel) 20

21 PRAM (PCM) Cell One transistor/diode and one GST (GeSbTe). In place updating (IPU) Top Electrode GST Amorphous Crystalline Heater Bottom Electrode +N Substrate Low High resistance: 1 0 Top Electrode GST Heater Bottom Electrode +N Substrate 21

22 Hybrid SSD Conventional SSD: FLASH. Promising candidate: PRAM (Phase change). To combine benefits of both technologies: Hybrid SSD. Two usage: Performance; Reliability. 22

23 Hybrid SSD: performance enhancement Erase Unit 1 PN=0, V PN=1, V PN=2, V Erase Unit 2 PN=2, VI Erase Unit 3 PN=0, V PN=1, V PN=2, V PN=n, VI PN=n, V PN=n, V G.Sun, Y. Joo, Y. Chen, Y. Xie, Y.Chen, H. Li, HPCA, (Empty Pages) Merge Operation (time consuming) PN=Page Number; V=Valid; I=Invalid Erase Unit = 128/256KB, Page = 512Bytes ~ 8KB 23

24 Hybrid SSD: performance enhancement In-place updating Data Region Erase Unit NAND flash Data Buffer in Memory Log Region PRAM Physical View Hybrid Architecture Sector (512Bytes) Structural View 24

25 Different Log Assignments Erase Unit Erase Unit Data Region Data Region Fixed Assignment Log Region Organize log pages in group Log Region Data Region Dynamic Assignment Log Region Erase Unit Static log assignment Group log assignment Dynamic log assignment 25

26 Hybrid SSD: performance enhancement 26

27 Outline Introduction Evolution with envm: On chip high speed storage; Secondary storage; Revolution with envm: Memristor based neuromorphic accelerator. Conclusion 27

28 Computing: Present and Future Clock Frequency (MHz) Multi core Power Density (mw/mm 2 ) 1000 New Trend: - Multi core, advanced power management, large on chip storage. Future: - Heterogeneous system, Brain likecomputing. Nuclear Reactor Hot Plate Rocket Launch Source: CPU DB, Intel Neural Network 28

29 Brain The Most Efficient Computing Machine Brain: 15 30B neurons Extremely complex organ 4km/mm 3 35w Neuron: Process signals from other neurons. Gray matter White matter Neocortex 6 layers Signals travel within and between layers Synapse: Memory Weight signals Neural Network 29

30 Brain like Neuromorphic Circuits Slow progress in neuoromoprhic hardware implementation Lack of efficient synapse design Not supportive to mass connection Highly parallel Ultra power efficient Real world input Data friendly Human friendly output Flexible Extremely robust 30

31 Memristor Rebirth of Neuromorphic Circuits Memristor Synapse Two terminal, high density Non volatility Analog/multi level states Crossbar Network Natural matrix function A MIMO system Good combination with memristor TaN1+x EI lab, APL 13 HP lab, 2012 Resistance ( ) Voltage (V) i i Pulse number TiN-TaOx device, pulses grows linearly in amplitude EI lab & HP lab n j-1 j n-1 n EI lab, DAC 12 31

32 Conclusion Emerging nonvolatile memory technology (NVM) such as STT RAM, racetrack, PRAM delivers significant improvement for various applications. Challenges exist and can be solved by architecture level optimization. Innovation of revolutionary architecture which provides Multi order speedup, power efficiency improvement, and hardware cost reduction is promised. 32