Memory in Embedded Systems. Tajana Simunic Rosing Department of Computer Science and Engineering University of California, San Diego.

Transcription

1 Memory in Embedded Systems Tajana Simunic Rosing Department of Computer Science and Engineering University of California, San Diego.

2 Hardware platform architecture

3 Traditional Memory Hierarchies Why SRAM as Cache? Why DRAM as main memory? Why Flash as SSD? Why HDD as Secondary storage? Speed Density Non-volatility & speed Price & density 3

4 Embedded memory hierarchy Registers Very fast, next to ALU, power hungry Cache Small, expensive, fast memory stores a copy of likely accessed parts L1, L2, L3 Predictability Scratchpad memory Main memory Large, inexpensive, slower Permanence Non-volatile memories Processor Registers Cache Main memory SSD Disk & Backup (Black box)

5 Caches and CPUs Servers: L1,L2 &L3 cache on chip Embedded: L1, L2 on chip Intel Xeon Server Raspberry Pi 3

6 Cache Designed with SRAM, Usually on same chip as processor Cache operation: Request for main memory access (read or write) First, check cache for copy cache hit cache miss Design choices cache mapping Direct - each memory location maps onto exactly one cache entry Fully associative anywhere in memory, never implemented Set-associative - each memory location can go into one of n set write techniques Write-through - write to main memory at each update Write-back write only when dirty block replaced replacement policies Random LRU: least-recently used FIFO: first-in-first-out Tag Index Offset V T D = Tag Index Offset V T D = V T D = Valid Data Valid Data

7 Cache impact on system performance Most important parameters in terms of performance: Total size of cache (data and control info tags etc) Degree of associativity Data block size Larger caches -> lower miss rates, higher access cost Average memory access time (h1=l1 hit rate, h2=l2 hit rate) tav = h1tl1 + (h2-h1)tl2 + (1- h2-h1)tmain e.g., if miss cost = 20 2 Kbyte: miss rate = 20%, hit cost = 2 cycles, average access time 5.6 cycles 4 Kbyte: miss rate = 10%, hit cost = 3 cycles, access 4.7 cycles 8 Kbyte: miss rate = 8%, hit cost = 4 cycles, access 4.8 cycles Your project: Cash miss rate affects execution time note the relationship between the data & cache sizes and the execution time Study the relationship between the cash miss rate at top speed and at lower speed/power of execution

8 Predictability Embedded systems are often real-time: Have to guarantee meeting timing constraints. Pre-run-time scheduling - predictability Time-triggered, statically scheduled operating systems Predictable cache design?

9 Scratch pad memories (SPM) Hierarchy main Example SPM processor no tag memory Address 0 space scratch pad memory ARM7TDMI well-known for low power consumption FFF..

10 Why not just use a cache? Worst case execution time (WCET) may be large [P. Marwedel et al., ASPDAC, 2004]

11 ARMv8 memory hierarchy Tightly coupled memory (TCM) = Scratchpad!

12 Memory management unit (MMU) Memory management unit translates addresses: CPU logical address memory management unit physical address main memory Duties of MMU Handles DRAM refresh, bus interface & arbitration Takes care of memory sharing among multiple CPUs Translates logic memory addresses from processor to physical memory addresses of DRAM Modern CPUs often come with MMU built-in

13 Address translation Mapping logical to physical addresses Two basic schemes: Segmented memory footprint can change dynamically usually only a few segments per process; e.g. data and stack Paged size preassigned can be combined (x86) page 1 page 2 segment 1 memory segment 2 SEGMENTATION Involves programmer Separate compiling Separate protection Shared PAGING Transparent to programmer No separate compiling No separate protection No sharing

14 ARM memory management Memory region types: section: 1 MB block large (64KB) & small (4KB) pages Address is marked as section or page-mapped Two-level translation scheme

15 ARM address translation Translation table base register descriptor 1st level table 1st index 2nd index concatenate offset concatenate descriptor 2nd level table physical address

16 Register file Fastest But biggest size built from D-FFs Volatile Memory SRAM: Static RAM Memory cell uses flip-flop to store bit Requires 6 transistors Holds data as long as power supplied DRAM: Dynamic RAM Memory cell uses MOS transistor and capacitor to store a bit DRAM scaling limit is around 55 nm, at which point the charge stored in the capacitor is too small to be detected More compact than SRAM Refresh required due to capacitor leak word s cells refreshed when read Slower to access than SRAM CLK OUT1 OUT2 OUT3 OUT4 R S R S R S R S D Q D Q D Q D Q IN1 IN2 IN3 IN4 Data' W DRAM Data SRAM Data W

17 m words RAM organization Stores large number of bits m x n: m words of n bits each k = Log2(m) address input signals or m = 2^k words e.g., 4,096 x 8 memory: 32,768 bits 12 address input signals 8 input/output data signals Memory access r/w: selects read or write enable A 0 A 1 rd/wr 4 4 RAM 2 4 decoder enable: read or write only when asserted multiport: multiple accesses to different locations simultaneously To every cell internal view I 3 I 2 I 1 Q 3 Q 2 Q 1 I 0 Q 0 Memory cell r/w enable A 0 m n memory n bits per word memory external view 2 k n read and write memory A k-1 Q n-1 Q 0

18 BCM2837 SoC Raspberry Pi 3 Memory Architecture Broadcom BCM2837 SoC CPU: Quad-core Cortex-A53: L1 and L2 cache GPU: VideoCore IV Processor: exclusive memory system Main Memory: 1GB RAM : Shared by CPU and GPU ARM Cortex-A53 L1 Cache (per core) I Cache D Cache VideoCore IV Per slice I Cache Uniform cache Textual Memory Unit L2 Cache L2 Cache Used by CPU Used by GPU Main Memory

19 Cortex-A53 Memory: Snoop Control Unit Cache coherence: Consistency of shared data that can be stored in multiple caches BUS Snoop Control Unit (SCU): Cache coherence mechanism between L1 D-Cache and L2 Cache Keep track of allocated data in each processor s line When a write is observed at a processor, invalidate the cache line of other processors Broadcast mechanism is used. L2 Memory System Pros: faster if enough bandwidth is available Cons: not scalable due to the broadcast overhead on buses

20 Cortex-A53 Memory: L1 Cache 64KB L1 Cache Instruction Cache (I-Cache) 64-byte cache lines 2-way set-associative Data Cache (D-Cache) 64-byte cache lines 4-way set-associative Pseudo random cache replacement Data cache unit has a controller for MOESI full cache coherency protocol M (Modified): The line is only in this cache and is dirty. O (Owned): The line is possibly in more than one cache and is dirty. Avoids writing a dirty cache line back to main memory when another processor tries to read it. Instead, the owner will supply the modified data directly to the other processor E (Exclusive):The line is only in this cache and is clean. S (Shared): The line is possibly in more than one cache and is clean Any snoop request to that cache line will be filled from memory, rather than a cache -> slow share of clean lines I (Invalid): The line is not in this cache.

21 VideoCore 4 (GPU) Memory Specialized to accelerate 3D/multimedia data Vertex Pipe Memory(VPM): shared cache that is system-wide performs DMA from main memory to read/write vertex data L2 Cache: shared by all slices Per-slice memory Icache: instruction cache Uniforms Cache: Stores a stream of data Texture and Memory Lookup Unit (TMU): Stores general-purpose data and textures FIFO-based texture lookup

22 Raspberry Pi 3: Main Memory (RAM) Divided in ARM Cortex-53 and VideoCore 4 components Partially shared (e.g. frame buffer) Kernel is loaded from 0x8000 (32KB) Local peripherals: Reserved memory area for ARM timer: 64-bit timing signal IRQs (Interrupt Requests) to cores Mailbox (part of local peripherals) memory area to facilitate communication between ARM and VideoCore DMA, GPIO, PCM, etc SD RAM VC SD RAM ARM 1. Write data to the mailbox 2. Issue special Interrupts to either ARM core or VideoCore Local peripherals 0x4001_FFFF

23 Raspberry Pi 3: MMU In 32-bit mode, ARMv8 address translation resembles the ARMv7 Extended VMSAv7 MMU Virtual memory system architecture Security extension Large physical address extension Virtualization extension Uses 2 Level TLBs L1 MMU 2 micro TLBs: I-cache (IuTLB) and D-cache (DuTLB) 10 entry full-associative L2 MMU A unified TLB 256 entry 2-way set-assoc.

24 Non-volatile memory A new class of data storage/memory devices Emerging NVMs have exciting features: Non-volatile like Flash ( ~ 10 years) Fast access times (~ SRAM) High density (~ DRAM) NVM blurs the distinction between Memory (fast, expensive, volatile) & Storage (slow, cheap, non-volatile) Key issues: Slow writes, low endurance, costly and complex manufacturing

25 floating-gate transistor Older NVMs Mask or fuse programmed ROM Erasable Programmable ROM (EPROM) Uses floating-gate transistor in each cell Programmer uses higher-than-normal voltage so electrons tunnel into the gate Electrons become trapped in the gate Only done for cells that should store 0 Other cells will be 1 To erase, shine ultraviolet light onto chip Gives trapped electrons energy to escape Requires chip package to have window Electronically-Erasable Programmable ROM (EEPROM) Erasing one word at a time electronically vs. UV light Flash memory Like EEPROM, but large blocks of words can be erased simultaneously EEPROM & FLASH are in-system programmable word enable data line 1 cell data addr e Ð e Ð data line 0 trapped electrons en write busy 1024x32 EEPROM cell

26 1T-1C FeRAM Similar in construction to DRAM Both cell types include one capacitor and one access transistor DRAM cell capacitor use a linear dielectric; FeRAM includes ferroelectric material, typically lead zirconate titanate (PZT) Writing is accomplished by applying a field across the ferroelectric layer by charging the plates on either side of it, forcing the atoms inside into the "up /logic 1 or "down /logic "0 Advantage: No need for refresh 99% lower power than DRAM Similar performance to DRAM Disadvantage: It is unclear how it will scale as materials stop being ferroelectric at small sizes (now produced in 130nm) Reads are destructive data has to be rewritten

27 STT-RAM: Spin-Transfer Torque RAM The spin torque direction of electrons to flip a bit in a magnetic tunneling junction (MTJ) Advantages: High endurance & fast reads Disadvantages: Write energy: large current needed to reorient the magnetization for most commercial applications.; Write latency: low ON/OFF resistance ratio (~2) Asymmetric write: Writing a 1 needs much more time and energy than writing a 0 (a) Structure of MTJ (b) Parallel: bit 0 (low resistance) (c) Anti-Parallel: bit 1 (high resistance)

28 Domain Wall Memory (DWM) Similar to STT-RAM structure Advantage: needs only one tunneling barrier and fixed layer area savings Disadvantages: complexity of design, read/write delay due to sequential access Ferromagnetic tape Domain Wall Free Layer Domain Fixed Layer Extra Domains MTJ

29 Shift-based DWM Writes by shifting data of one of the two fixed layers with the desirable direction comp Advantage: faster writes than a traditional DWM Disadvantage: cost and manufacturing complexity Polarized direction 1-bit DWM is fast Multi-bit DWM is area efficient, but needs extra latency for shifting

30 PCM: Phase Change Memory Phase Change Memory (PCM) Flips a bit by changing the state of material Crystalline (SET) and amorphous (RESET) phase Advantages: better scalability than other emerging technologies Disadvantages: Slow in write (non-symmetric write operation) Low endurance (10 7 ) PCM Cell Phases PCM Operation Candidate for DRAM replacement

31 ReRAM: Resistive RAM Two types: Access-based (1T-1R) and crossbar ReRAM (1T-nR) Access-based ReRAM (1T-1R) A dielectric, which is normally insulating can be made to conduct through after application of a sufficiently high voltage Advantage: Very fast reads and writes ~ 20ns Very high density Disadvantage: Limited endurance (10 5 ) 31

32 Crossbar ReRAM Crossbar ReRAM (1T-nR) Advantage: Highly scalable Can be implemented at the top of the chip with in 3D architecture Very low energy consumption Low cost (possible replacement for Flash) Disadvantage: Much slower than 1T-1R ~us

33 Crossbar RRAM Applications Crossbar 1T-nR 1T-1R 33

34 Active vs Passive Power The blue area marks active power in the power equations The red area marks passive power in the power equations Passive power is unproductive. It just causes heat For memories it is leakage and refresh power, which is typically smaller than maximum active power For disks it is keeping the motor spinning and the standby power of the electronics, which is typically larger than the maximum active power For PCM it is the leakage and small standby power and is typically much smaller than the maximum active power. P DRAM P Disk P V dd I leak V dd I refresh CV d r I V I PCM I standby V dd i&c I active s& t V 2 dd V dd passive active is the portion of time that the device is active and productive is the normalized power of the disk motor f

35 NVRAM Comparison

36 NVMs Comparison STT-RAM: SRAM cache replacement PCRAM: DRAM main memory and storage ReRAM: NAND flash, embedded NOR flash

37 Summary Memory hierarchy Needs: speed, low power, predictable Cache design Mapping, replacement & write policies Memory types ROM vs RAM, types of ROM/RAM NVM Many new technologies that are still maturing Excellent target for big data and energy-efficient applications

38 Take a look at: Energy Efficient Computing The rate of data generation is beyond the capability of current computing systems Solutions for energy efficient computing: Approximation computing Near data computing Approximate computing Hindering computer systems efficiency by demanding too much accuracy from them E.g. Allow =4 Balance between energy and accuracy

39 Near Data Computing Data movement from main memory to caches is dominates energy consumption in many apps Near data computing: Preform analysis in the main memory and send only the result back to the processor Bring computation closer to data Often helpful for machine learning algorithms Contact Mohsen 39 Imani at:

40 Sources and References Frank Vahid, Tony Givargis, Embedded System Design, Wiley, Wayne Wolf, Computers as Components, Morgan Kaufmann, Peter Marwedel, Embedded Systems Design, 2004.

41 Phase-change RAM Bit-line PCRAM programmable resistor Word -line Access device (transistor, diode) Voltage temperature RESET pulse T melt Potential headache: High power/current affects scaling! SET pulse T cryst time Potential headache: If crystallization is slow affects performance!