Novel Nonvolatile Memory Hierarchies to Realize "Normally-Off Mobile Processors" ASP-DAC 2014

Transcription

1 Novel Nonvolatile Memory Hierarchies to Realize "Normally-Off Mobile Processors" ASP-DAC 2014 Shinobu Fujita, Kumiko Nomura, Hiroki Noguchi, Susumu Takeda, Keiko Abe Toshiba Corporation, R&D Center Advanced LSI technology laboratory Acknowledgement This work was partly supported by Normally-off Computing PJ (NEDO) in Japan. 1

2 OUTLINE Introduction: Normally-off (N-off) Processor (from ver.0 to ver.1. ) Key Point 1: Advanced STT-MRAM Key Point 2: Decrease in power for short CPU standby state by applying new memory cell design Key Point 3: Power Decrease for long CPU standby state by Ultra-Fast- Power Gating Conclusions Towards N-off ver 2. 2

3 History of Concept on Normally-Off Computer Normally-Off Computer Ver.0 (2001) Proposed by K. Ando, AIST, Japan (FED journal Japan, 2001) ALU/ FlipFlop Register files Cache (L1) Cache (L2) Volatile Memory volatile ALU/ FlipFlop Register files Cache (L1) Cache (L2) Nonvolatile Volatile Memory (MRAM) Main memory Main memory Storage non-volatile Storage non-volatile Memory Hierarchy Memory Hierarchy The same Ver.0 concept presented by T. Kawahara, ASP-DAC Nonvolatile memory and normally-off computing (based on MRAM) 3

4 Rethink Normally-off Concept Ver.0 Ver.0 (All Nonvolatile Memory Hierarchy) is not suitable for decreasing power.. Power Active power is dominant. ALU resistor L1 - cache L2 - cache Main memory Storage Standby power is dominant. Time Attention: -Active power (write power) of nonvolatile memory is so large! -Speed of NV-Memory is much slower than that of SRAM. 4 (CPU core power and performance is largely degraded by Ver.0!)

5 History of Concept on Normally-Off Computer (2) Normally-Off Computer Ver.0 (2001) K. Ando, AIST, Japan (FED journal Japan, 2001) Ver.1 (2010) K. Abe, S. Fujita et al., Toshiba, SSDM ALU/ FlipFlop Register files Cache (L1) Cache (L2) Main memory Volatile Memory volatile ALU/ FlipFlop Register files Cache (L1) Cache (L2) CPU core power and performance is largely degraded! Main memory Nonvolatile Volatile Memory (MRAM) ALU/ FlipFlop Register files Cache (L1) Cache (L2) Main memory Non- Volatile Memory (MRAM) Storage non-volatile Storage non-volatile Storage non-volatile Memory Hierarchy Memory Hierarchy Memory Hierarchy Ver.0 (2011) T. Kawahara, ASP-DAC CPU core power and performance is largely degraded! Ultra low power applications such as Sensor Networks etc. 5

6 MB Capacitance of Cache Memory in CPU is increasing, which increases standby power of processors! Server, W/S CPU Desk Top PC CPU Note PC CPU 64 Smart Phone CPU Why nonvolatile L2, L3, LL Cache? CPU Core Resistors Resistor file L1Cache L2, L3 Cache More cache, <Background> More Leakage.. Increase performance not by increasing clock frequency. Multi-core. 6

7 Especially for Mobile-Processor, not Standby Power but Leakage Power is Dominant! Energy consumed (%) 100% 80% 60% 40% 20% 0% Active state Clock gated state Power gated state (except L2 (retention)) (Evaluation from one-day use case.) Consumed Energy Caused by Leakage Power of Last Level Cache (L2$) 7

8 STT-MRAM is the best in NVM, but.. SRAM Access speed (ns) MRAM FeRAM RAM STT-MRAM ReRAM PCM limited endurance DRAM practically unlimited endurance Storage Its operation speed is slow and its power is high for cache memory NAND Flash HDD M 1G 1T Memory capacity (bit) 8

9 Standby power is low, but active energy is extremely higher than that of SRAM even using conventional STT-MRAM. Dilemma of Nonvolatile Memory! Energy consumed (%) 100% 80% 60% 40% 20% 0% Active state General STT-MRAM Active power Increases drastically! Clock gated state Power gated state ( L2 retention) General STT-MRAM Standby power Decreases largely! 9

10 OUTLINE Introduction: New Design Concept Normally-off (N-off) Processor (from ver.0 to ver.1. ) Key Point 1: Advanced STT-MRAM Key Point 2: Decrease in power for short CPU standby state (in CPU active state) by applying new memory cell design Key Point 3: Power Decrease for long CPU standby state by Ultra-Fast- Power Gating Conclusions Towards N-off ver 2. 10

11 Advanced STT-MRAM has been developed! SRAM Access speed (ns) Advanced MRAM FeRAM RAM p-stt-mram STT-MRAM ReRAM PCM limited endurance DRAM NAND Flash practically unlimited endurance Storage HDD M 1G 1T Memory capacity (bit) 11

12 Breakthrough by Toshiba s s advanced STT-MRAM Power down Programming current ( A) 1.0E E-03 [3] [4] [2] Reduction in 1.0E-04power of cache memory Power of cache memory is increased [6] Toshiba E-05 (Advanced STT-MRAM, 28-30nm) 1.E-10 1.E-09 1.E-08 1.E-07 Programming time (nsec) [1] [5] Higher speed Breakeven point for the replacement of hp-sram in power of cache memory [7] [1] Sony corp. IEDM (2005) [2] New York univ. APPLIED PHYSICS LETTERS 97, (2010) [3] Cornel Univ. APPLIED PHYSICS LETTERS 95, (2009) [4] Minnesota univ. J. Phys. D: Appl. Phys. 45, (2012). [5] NEC corp. Symposium on VLSI Circuits 7.3 (2012). [6] IBM corp. Appl Phys Lett 98, (2011). 12 [7] TDK-Headway Applied Physics Express (2012)

13 Embedded Memory Integration (by Toshiba N-off PJ) 1K WL μm μm XDEC STT-MRAM Test Chip μm μm 55.26μm μm 32.69μm READ & WRITE 1K BL Process 65-nm CMOS process Macro size mm 2 Organization 4K words x 256 bits = 1 Mb 256Rx256C 4:1MUX 256Rx256C 4:1MUX 256Rx256C 4:1MUX 256Rx256C 4:1MUX Driver SA Voltage (V) Access Time Measurements Read 3.3ns@1.2V 4ns@1.05V 1.2 Pass Fail Access time (ns) 2T-2MTJ H. Noguchi et al., VLSI circuit symposium, 2013 Access time < 4ns 13

14 High speed STT-MRAM is NOT for high CPU performance, but for lower power CPU! Average Time per Instruction (ns) < STT-MRAM SRAM Access Time (ns) Relative Average Power for L2 Cache Memory Higher Power Lower Power Access Time of STT-MRAM (ns) HP-SRAM=1 1MB 14

15 Development of STT-MRAM-top Integration Specific MRAM Integration Process STT-MRAM 配線層 CMOS Cross section image Pool -top construction (Marina Bay Sands Hotel) Conventional CMOS Process (in-house fab, foundry..) To be presented in VLSI-TSA

16 OUTLINE Introduction: New Design Concept Normally-off (N-off) Processor (from ver.0 to ver.1. ) Key Point 1: Advanced STT-MRAM Key Point 2: Decrease in power for short CPU standby state (in CPU active state) by applying new memory cell design (normally-off type design) Key Point 3: Power Decrease for long CPU standby state by Ultra-Fast- Power Gating Conclusions Towards N-off ver 2. 16

17 From Normally-On Type Memory with Power Gating to Normally-Off Type Memory without Power Gating Normally-On Type (1) SRAM and Nonvolatile SRAM without Power Gating Leakage path Leakage path Power BL /BL BL /BL Short standby Active WL WL Active Active 10~30ns MTJ F P P F Nonvolatile SRAM (2004 Toshiba) - MTJ NV-SRAM for High Speed! Power Short Standby Leakage power Power Gating? Short Standby Leakage power Time Active Active Active Time 17

18 From Normally-On Type Memory with Power Gating to Normally-Off Type Memory without Power Gating Normally-On Type Leakage path BL /BL WL WL BL MTJ F P MTJ P F (1) SRAM and Nonvolatile SRAM with Power Gating Power gating switch /BL Area SRAM NV-SRAM power gating switch Nonvolatile SRAM (2004 Toshiba) - Overhead of power gating switch is much large! (Delay and Power overhead also ) (2) Normally-off Type Memory without Power Gating New design: Normally-off Type (Next page) No Leakage path, No power gating switch. Area SRAM x2 ~x4 STT-MRAM STT-MRAM cell is much smaller than SRAM cell. 18

19 Various kinds of Normally-off Type Memory Cell designs using advanced p-stt-mram presented by Toshiba. WWL RWL SWL WBL MTJ SL RBL BL WL MTJ WWL SL /BL BL WL MTJ MTJ SL /SL /BL MTJ BL WL N1 M1 MTJ1 P F M2 SL N2 M3 MTJ2 P F M4 /BL (a) D-MRAM (b) 3T-2MTJ (c) 2T-2MTJ (d) 4T-2MTJ K. Abe et al. IEDM2012 (Toshiba) A. Kawasumi et al. IMW2013 (Toshiba) H. Noguchi et al. VLSI Circuit 2013 (Toshiba) C. Tanaka et al. SSDM 2013 (Toshiba) As there are No Leakage paths like SRAM, no power gating switch is needed in the memory arrays. 19

20 CPU-Simulation (ARM core, Linux-OS) Processors # of cores 1 Frequency 1GHz Issue width ISA 1(out of order) ARMv7 Execution Warm-up Execution Memory L1 cache 32+32kB, - 4 way, 64B line, Write-back, 1 read/write port, 1ns latency L2 cache 1MB, - 8 way, 64B line, Write-back,1 read/write port 1M inst. 10M inst. Cell Type SRAM (Reference) 2MTJ-6T 2MTJ-4T D-MRAM (1MTJ-3T, This work) MTJ device Write Time / Current -- 3ns / 50uA (Advanced p-mtj) 25ns / 120uA (Reference p-mtj) 3ns / 50uA (Advanced p-mtj) Processor benchmark sets: SPEC

21 Results of Power of Cache Memory (Short standby state) (case study: (a) D-MRAM) Normalized 1MB-8way L2$ power SRAM Worse than SRAM (Normally-on) Better than SRAM (Normally-Off) bzip2 gobmk h264ref xalancbmk gromacs namd calculix lbm Normally-On without PG 2MTJ-6T 2MTJ-4T 1MTJ-3T (advanced p-mtj this work) 1MTJ-3T (referenced p-mtj) Normally-Off Normally-Off memory using low-power and advanced p-stt-mram can reduce the cache power the most effectively. 21

22 Processor performance (Short standby state) (case study: Normally-off STT-MRAM) SRAM case = 1 Normalized instructions per cycles (IPC) (a. u.) bzip2 gobmk h264ref xalancbmk gromacs namd calculix lbm 2MTJ-6T 2MTJ-4T 1MTJ-3T (advanced p-mtj this work) 1MTJ-3T (referenced p-mtj) Normally-Off memory cell design using advanced p-mtj(stt-mram) has the best performance comparable to that of SRAM. 22

23 OUTLINE Introduction: New Design Concept Normally-off (N-off) Processor (from ver.0 to ver.1. ) Key Point 1: Advanced STT-MRAM Key Point 2: Decrease in power for short CPU standby state (in CPU active state) by applying new memory cell design Key Point 3: Power Decrease for long CPU standby state by Ultra-Fast- Power Gating Conclusions Towards N-off ver 2. 23

24 Conventional Power Gating Clock L1-$ L2-$ Others Recovery time to active-state Active state Clock gated state OFF ~1ns CPU core sleep (L2-Cache retention) CPU core Sleep (L2-Cache decay) Deep powerdown state (DPS) High-speed PG with NV-cache Active state Deeper powerdown state OFF OFF 10 s OFF OFF decay 20 s OFF OFF OFF OFF except State SRAM Clock OFF L1-$ OFF L2-logic +prepheral OFF OFF L2-memory Normally OFF Clock gated state OFF Normally OFF CPU core sleep; Deep power-down state (DPS) OFF Normally OFF Normally OFF 100 s Recovery time to active-state ~1ns ~10ns ~100ns 24

25 State transition policy for long time standby state: Conventional power gating (PG) vs. Ultra-fast PG with NV-L2-cache Power (%) Active state Conventional PG 100 Short standby state Clock gated Long time standby state CPU core sleep state 50 Clock gated L2 cache decay 0 1 2nsec 20 sec Deep power-down state (DPS) nsec 200nsec 200 sec Deep power-down state (DPS) 40 sec Idle time (ns) Ultra-fast PG + Nonvolatile-L2 cache 25

26 a) Conventional PG (Power Gating) Interrupt Interrupt Power Interrupt Deep sleep state Time b) Ultra-fast PG+NV-cache Interrupt Interrupt Interrupt Power Deep sleep state Deep sleep state Deep sleep state Time 26

27 Case studies: Decrease in average power of processor by ultra-fast PG with nonvolatile-l2 cache. Average Power of Processor (mw) % 66% Conventional NV-L2 cache Case 1 Case 2 Case 3 90% Case1:CPU active state dominant, Case2: Moderate, Case3: CPU idle state dominant. 27

28 Conclusion For HP-mobile processors, we proposed N-off processor ver.1; volatile L1- cache/ nonvolatile L2,LLC. To realize N-off processor ver.1, advanced STT-MRAM, normally-off type memory cell design, ultra-fast power gating are three key points. By applying new memory cell designs without leakage paths, not only CPU standby power but CPU active power has been effectively reduced. Average power reduction by 29 to 90% can be expected with little degradation of CPU performance. N-off processor concept shifting Ver.1 to Ver.2 has been in progress. Nonvolatile Volatile CPU core Registers Register file L1 Cache L2, L3 Cache Normally-off Processor (Concept Ver.1) 2010 Rethink! Nonvolatile/ Volatile Hybrid CPU core Volatile Nonvolatile Volatile Normally-off Processor (Concept Ver.2) from 2012 Ex. CPU Core (LITTLE) L2 (256kB) SRAM/ STT-MRAM L3 (16MB) High density STT-MRAM CPU Core (Big) L2 (1MB) STT-MRAM 28

29 Thank you! 29