MEMORY SYSTEM MEMORY TECHNOLOGY SUMMARY DESIGNING MEMORY SYSTEM. The goal in designing any memory system is to provide

Transcription

1 SUMMARY MEMORY SYSTEM ORGANIZATION Memory technology Hierarchical memory systems Characteristics of the storage-device Main memory organization SRAM DRAM Cache memory COMPUTER ARCHITECTURE 2 MEMORY TECHNOLOGY Every computer system contains a variety of devices to store the instructions and data The storage devices + the algorithms (HW or SW implemented) needed to control or manage the stored information constitute the memory system of the computer Desirable: processors should have immediate and uninterrupted access to memory maximum speed to transfer information between a processor and memory Memories that operate at speeds comparable to processor speeds are relatively costly It is not feasible to employ a single memory using just one type of technology The stored information is distributed over a variety of different memory units with very different physical characteristics DESIGNING MEMORY SYSTEM The goal in designing any memory system is to provide An adequate storage capacity An acceptable level of performance A reasonable cost COMPUTER ARCHITECTURE 3 COMPUTER ARCHITECTURE 4

2 DESIGNING MEMORY SYSTEM 1. The use of a number of different memory devices with different cost performance ratios organized to provide a high average performance at a low average cost per bit The individual memories form a hierarchy of storage devices 2. The development of automatic storage-allocation methods to make more efficient use of the available memory space. 4. The development of virtual-memory concepts to free the ordinary user from memory management to make programs largely independent of the physical memory configurations used. 5. The design of communication links to the memory system so that all processors connected to it can operate at or near their maximum rates increasing the effective memory processor bandwidth providing protection mechanisms to prevent programs from accessing or altering one another's storage areas. COMPUTER ARCHITECTURE 5 MAIN COMPONENTS OF THE MEMORY 1. Internal processor memory A small set of high-speed registers used for temporary storage of instructions and data. 2. Main memory (or primary memory) A relatively large & fast memory used for program and data storage during computer operation Locations in main memory can be accessed directly and rapidly by the CPU instruction set Semiconductor technology 3. Secondary memory (or auxiliary memory) Large memory capacity & much slower than main memory Store system programs, large data files, and the like which are not continually required by the CPU Information in secondary storage is accessed indirectly via input-output programs Representative technologies used for secondary memory are magnetic and optic disks COMPUTER ARCHITECTURE 6 BLOCK DIAGRAM OF A MEMORY HIERARCHY (main components) LOCALITY OF REFERENCES PRINCIPLES S,p,t A CPU Registers Cache Memory Direct Access Main Memory Indirect Access Auxiliary IO Memory (external) Programs commonly localize their memory references Only a small portion of the memory is likely to be accessed at a given time This characteristic is not a natural law; it may be violated But it is a behavior pattern exhibited by many programs at most times Exploiting this behavior is key to designing an effective memory hierarchy c COMPUTER ARCHITECTURE 7 COMPUTER ARCHITECTURE 8

3 TWO TYPES OF LOCALITY Temporal Locality (Locality in Time) The location of a memory reference is likely to be the same as another recent reference Programs tend to reference the same memory locations at a future point in time Due to loops and iteration, programs spending a lot of time in one section of code Spatial Locality (Locality in Space) The location of a memory reference is likely to be near another recent reference Programs tend to reference memory locations that are near other recently-referenced memory locations COMPUTER ARCHITECTURE 9 TWO TYPES OF LOCALITY Exploit Temporal Locality Keep recently accessed data close to processor Exploit Spatial Locality Move contiguous blocks of data between levels At any given time, data is copied only between two adjacent levels Upper Level (the one closer to the processor) Smaller, faster, and uses more expensive technology Lower Level (the one further from the processor) Bigger, slower, and uses less expensive technology COMPUTER ARCHITECTURE 10 MEMORY HIERARCHY PRINCIPLES Increasing distance from the CPU represents also an increasing for the access time CPU Memory Memory Memory Level 1 Level i Level i+1 Block A Block B COMPUTER ARCHITECTURE 11 MEMORY HIERARCHY PRINCIPLES The multi-level read-write memory system must satisfy two properties : Inclusion property: All information located on a upper memory level it is also stored on a lower memory level (ILi represents the information stored in memory level i): IL1 IL2... ILn word miss / word hit Coherence property: An information stored at a specific address in level i must be the same on all lower memory levels Maintain coherence: propagation of the modified value to the lower levels ("write-through") updating information on lower levels when replaced from level i ("write-back") COMPUTER ARCHITECTURE 12

4 HIERARCHY TERMINOLOGY Hit: Accessed data is found in upper level (for example Block A in level i) Hit Rate = fraction of accesses found in upper level Hit Time = Time to access the upper level = Memory access time + Time to determine hit/miss Miss: Accessed data found only in lower level (for example Block B in level i+1) Processor waits until data is fetched then restarts the access Miss Rate = 1 (Hit Rate) Miss Penalty Time to get block from lower level + time to replace in upper level + time to deliver the block the processor Hit Time << Miss Penalty COMPUTER ARCHITECTURE 13 Memory-Device Functional Characteristics 1. Storage capacity (S). Expressed in multiple of bits, or bytes bits = 1024 bits = 1 Kb; 1024 Kb = 1 Mb 8 bits = 1 Byte = 1 B 1024 Mb = 1 Gb; 1024 Gb = 1 Tb 2. Cost. The price include not only the cost of the information storage cells themselves but also the cost of the peripheral equipment or access circuitry essential to the operation of the memory Let C be the price in dollars of a complete memory system with S bits of storage capacity, and specific cost c of the memory: C c = dollars / bit S COMPUTER ARCHITECTURE 14 Memory-Device Characteristics 3. Access time The average time required to read a fixed amount of information, e.g., one word, from the memory - read access time or, more commonly, the access time of the memory (t A ) The write access time is defined similarly - it is typically, but not always, equal to the read access time Access time depends on the physical characteristics of the storage medium, and also on the type of access mechanism used t A is usually calculated from the time a read request is received by the memory unit to the time at which all the requested information has been made available at the memory output terminals The access rate b A of the memory defined as 1/t A and measured in words per second Memory-Device Characteristics 4. Access modes - the order of sequence in which information can be accessed Random-access memory (RAM) - if locations may be accessed in any order and access time is independent of the location being accessed Semiconductor memories Each storage location has a separate access (addressing) mechanism COMPUTER ARCHITECTURE 15 COMPUTER ARCHITECTURE 16

5 Memory-Device Characteristics Serial-access memories - storage locations can be accessed only in certain predetermined sequences Magnetic-tape, optical memories. The access mechanism is shared among different locations, and must be assigned to different locations at different times - moving the stored information, the read-write head, or both Serial access tends to be slower than random access Magnetic disks contain a large number of independent rotating tracks If each track has its own read-write head, the tracks may be accessed randomly, although access within each track is serial The access mode is sometimes called semi-random or, rather misleadingly, direct access Memory-Device Characteristics 5. Alterability of information contained ROMs - memories whose contents cannot be altered on-line ROMs whose contents can be changed (usually offline and with some difficulty) are called programmable read-only memories (PROMs). RWM - memories in which reading or writing can be done on-line COMPUTER ARCHITECTURE 17 COMPUTER ARCHITECTURE 18 Memory-Device Characteristics 6. Permanence of storage - the stored information may be lost over a period of time unless appropriate action is taken Destructive readout (reading the memory destroys the stored information) Each read operation must be followed by a write operation that restores the original state of the memory The restoration is usually carried out automatically, by write back into the location originally addressed from a buffer register Dynamic storage Over a period of time, a stored charge tends to leak away, causing a loss of information unless the charge is restored The process of restoring is called refreshing (dynamic memories) Volatility A memory is said to be volatile if the stored information can be destroyed by a power failure COMPUTER ARCHITECTURE 19 Memory-Device Characteristics 7. Cycle time and data transfer rate Cycle time (t M ) is the time needed to complete any read or write operation in the memory This means that the minimum time that must elapse between the initiation of two different accesses by the memory can be greater than t M Data transfer rate or bandwidth (b M ) - the maximum amount of information that can be transferred to or from the memory every second (= 1/t M ) 1 b = w [ words /sec] b [bits / M t t sec] M = M M w = memory bus width COMPUTER ARCHITECTURE 20

6 Memory-Device Characteristics Cycle time vs access time Memory cycle time Access time Seek Transfer time time Latency time Timp Memory-Device Characteristics 8. Power: specific power P p = S tot [ W / bit] t 1 t 2 t 3 t 4 In cases where t A t M both are used to measure memory speed 9. Geometry: only for semiconductor memories N lines M columns COMPUTER ARCHITECTURE 21 COMPUTER ARCHITECTURE 22 Main memory devices SRAM DRAM Cache memory MAIN MEMORY Main memory (primary or internal memory): Used for program and data storage during computer operation Directly accessed by the CPU (microprocessor) Semiconductor technology Volatile memory Alterable memory (RWM) A random accessed memory Every cell is connected to selection lines (bit line for RD / WR) Every cell is connected to data lines (column bit lines for RD / WR) COMPUTER ARCHITECTURE 23 COMPUTER ARCHITECTURE 24

7 RANDOM ACCESS MEMORIES ROM-MOS Example cell memory Static random-access memories (SRAM) Each cell contains 4-6 transistors Very fast, expensive, low degree of integration, high power consumption Operation characteristic cycles: Read + Write Dynamic random access memories (DRAM) Each cell contains 1 transistor + 1 MOS capacitor High integration, low power consumption, slow, cheap Operation characteristic cycles: Read + Write + Refresh Column select Line select V GG To output amplifier ROM cell Q 1 COMPUTER ARCHITECTURE 25 COMPUTER ARCHITECTURE 26 Example cell memory Static RAM (SRAM) cell Dynamic RAM Example cell memory Word Line Word Line Word Line V DD pmos pmos C Bit Line nmos nmos Transfer Gate Sense Amplifier Bit Line Bit Line Bit Line Bit Line COMPUTER ARCHITECTURE 27 COMPUTER ARCHITECTURE 28

8 Example: RAM selection cells Simple example: 4 x 4 bits RAM Data In Read/Write Line Bit Line Bit Line Bit Register 1 Line 0 0 Input Data pins Line 1 Line 2 Line Bit Line Bit Line Bit Line Bit Line Bit Line Bit A1 A0 Address Chip Select 2 to 4 Decoder CS Register 2 Register 3 Address input b 3 b 2 b 1 b 0 A 1 R/W A 0 CS 4 4 RAM b 3 b 2 b 1 b 0 Output Data pins Control signals Register 4 Data Out COMPUTER ARCHITECTURE 29 COMPUTER ARCHITECTURE 30 Static RAM (SRAM) Bipolar or unipolar technology Operating cycles: Read cycle Write cycle Every cell contain a latch (1 bit) Access circuitry: Decode the address word Control the cell selection lines Read and write cells Control internal logic SRAM A general approach to reducing the access circuitry cost in randomaccess memories is called matrix (array) organization The storage cells are physically arranged as rectangular arrays of cells This is primarily to facilitate layout of the connections between the cells and the access circuitry The memory address is partitioned into d components so that the address A i of cell C i becomes a d-dimensional vector (A i,1, A i,2,..., A i,d ) = A i Each of the d parts of an address word goes to a different address decoder and a different set of address drivers A particular cell is selected by simultaneously activating all d of its address lines A memory unit with this kind of addressing is said to be a d- dimensional memory COMPUTER ARCHITECTURE 31 COMPUTER ARCHITECTURE 32

9 SRAM Principle of 1-d addressing scheme Storage locations The simplest array organizations have d = 1 and are called one-dimensional, or 1-D, memories If the storage capacity of the unit is N w bits, then the access circuitry typically contains a one-out-of-n address decoder and N address drivers For example, for a 8K 8 RAM memory, for this 1-D organization: a=13, N = 2 13 and w = 8. Address Bus a Address Register a Internal Control Signals Timing and control circuits Address Decoder N w w Data drivers and registers Data Bus CS R / W OE COMPUTER ARCHITECTURE 33 COMPUTER ARCHITECTURE 34 Explanation N = 2 a The CS (chip select) input is used to enable the device This is an active low input, so it will only respond to its other inputs if the CS line is low. The OE (output enable) input is used to cause the memory device to place the contents of the location specified by the address inputs onto the data pins The WR (write enable) input is used to cause the memory device to write the internal location specified by the address inputs with the data currently appearing on the data pins This is an active low signal, and the actual write occurs on the rising edge when WR goes from low back to the high state COMPUTER ARCHITECTURE 35 Principle of 2-D addressing The address field is divided into two components, called X and Y, which consist of ax and ay bits, respectively Address Bus a x a y X Address Decoder N x Array of memory storage N y locations Y Address Decoder COMPUTER ARCHITECTURE 36

10 Principle of 2-D addressing Theoretical example of a 2-d organization for a 8K 8 bits memory The cells are arranged in a rectangular array of rows and columns The total number of cells is N = N x N y A cell is selected by the coincidence of signals on its X and Y address lines The 2-D organization requires substantially less access circuitry than the 1-D for a fixed amount of storage For example, if N x = N y = N the number of address drivers needed is 2 N For N >> 4 the difference between 1-D and 2-D organizations is significant If the 1-bit storage cells are replaced by w-bit registers, then an entire word can be accessed in each read or write cycle but the bits within a word are not individually addressable For the example of 8K 8 memory the 2-D organization, for 8-bits on a word can look theoretically like in next slide ADRESS 7 6 DCD X K celule 2 8K celule 1 8K cells 64 DCD Y K cells COMPUTER ARCHITECTURE 37 COMPUTER ARCHITECTURE 38 Real structure example of a 2-d organization for a 8K 8 bits memory Read Cycle Timing Diagram t CR ADRESS 8 DCD X 256 Array of cells cells ADR CS R/W t AC 5 MDX 0 MDX 1 MDX 7 DATA OUT HiZ t AA VALID DATA HiZ d 0 d 1 d 7 Read / Write amplifiers 8 Data Bus t CR = read cycle time t AA = access time (from address change) t AC = access time (from CS active) COMPUTER ARCHITECTURE 39 COMPUTER ARCHITECTURE 40

11 ADR CS R/W Data BUS Write Cycle Timing Diagram t CW = write cycle time t DH = data hold time t DS = data set-up time t W = write width t AW t CW t W t DW t DS t DH t DW = delay for input data to write t AW = delay between address change and write control COMPUTER ARCHITECTURE 41 DYNAMIC MEMORIES DRAMs are currently the primary memory device of choice because they provide the lowest cost per bit and greatest density among solid-state memory technologies. Every cell contain MOS capacitor(s) DRAM - dynamic since needs to be refreshed periodically ( 8 ms, 1% time) Addresses divided (address pins multiplexed) into 2 halves (Memory as a 2D matrix): RAS or Row Address Strobe CAS or Column Address Strobe E.g cells = 4 Mbit, memory array 2048 x 2048 cells, number of address IC pins = 11 Operating cycles: Read cycle Write cycle Refreshment cycle COMPUTER ARCHITECTURE 42 The Conventional DRAM (asynchronous interface) The Conventional DRAM The multiplexed address bus uses two control signals - the row and column address strobe signals, (RAS and CAS respectively) The row address causes a complete row in the memory array to propagate down the bit lines to the sense amps The column address selects the appropriate data subset from the sense amps and causes it to be driven to the output pins. Access transistors called sense amps are connected to the each column and provide the read and restore operations of the chip Since the cells are capacitors that discharge for each read operation, the sense amp must restore the data before the end of the access cycle. A refresh controller determines the time between refresh cycles, and a refresh counter ensures that the entire array (all rows) is refreshed COMPUTER ARCHITECTURE 43 COMPUTER ARCHITECTURE 44

12 The Conventional DRAM (typical memory access) Conventional DRAM read cycles 1. The row address bits are placed onto the address pins 2. After a period of time the /RAS signal falls, which activates the sense amps and causes the row address to be latched into the row address buffer 3. The column address bits are set up 4. The column address is latched into the column address buffer when /CAS falls, at which time the output buffer is also turned on 5. When /CAS stabilizes, the selected sense amp feeds its data onto the output buffer /WE = 0 write, = 1 read. If became active (LOW) before /CAS ( advance write cycle ) data outputs remains in HiZ state If /WE became active (LOW) after /CAS the cycle is a write-read cycle COMPUTER ARCHITECTURE 45 COMPUTER ARCHITECTURE 46 Key DRAM Timing Parameters t RAC : minimum time from RAS line falling to the valid data output (quoted as the speed of a DRAM when buy) A typical 4Mb DRAM t RAC = 60 ns t RC : minimum time from the start of one row access to the start of the next. t RC = 110 ns for a 4Mbit DRAM with a t RAC of 60 ns t CAC: minimum time from CAS line falling to valid data output. 15 ns for a 4Mbit DRAM with a t RAC of 60 ns t PC : minimum time from the start of one column access to the start of the next. 35 ns for a 4Mbit DRAM with a t RAC of 60 ns A 60 ns (t RAC ) DRAM can perform a row access only every 110 ns (t RC ) perform column access (t CAC ) in 15 ns, but time between column accesses is at least 35 ns (t PC ). DRAM - fast speed op. modes Modes: change the access to columns, to reduce the average access time Fast page mode Page = All bits on the same ROW (spatial locality) Row-address is held constant and data from multiple columns is read from the sense amplifiers Don t need to wait for word-line to recharge Toggle CAS with new column address Nibble mode Four successive column addresses generated internally Burst mode More successive column addresses (in the same page) generated internally COMPUTER ARCHITECTURE 47 COMPUTER ARCHITECTURE 48

13 Fast Page Mode DRAM (FPM DRAM) Extended Data Out DRAM (EDO DRAM) EDO DRAM adds a latch between the sense-amps and the output pins. This latch holds output pin state and permits the CAS to rapidly de-assert, allowing the memory array to begin precharging sooner an EDO DRAM would allow access to the memory to start before the previous one had entirely completed The latch in the output path also implies that the data on the outputs of the DRAM circuit remain valid longer into the next clock phase COMPUTER ARCHITECTURE 49 COMPUTER ARCHITECTURE 50 Extended Data Out DRAM Nibble mode (serial four) COMPUTER ARCHITECTURE 51 COMPUTER ARCHITECTURE 52

14 Burst mode Refreshment modes RAS only refresh (conventional refresh) CAS-before-RAS refresh (internal logic for refresh) Hidden refresh (refresh cycle is hidden in an read/write access cycle) COMPUTER ARCHITECTURE 53 COMPUTER ARCHITECTURE 54 RAS only refresh CAS-before-RAS refresh COMPUTER ARCHITECTURE 55 COMPUTER ARCHITECTURE 56

15 Hidden refresh COMPUTER ARCHITECTURE 57 Synchronous DRAM (SDRAM) SDRAM exchanges data with the processor synchronized to an external clock signal SDRAM latches information to and from the controller based on a clock signal SDRAM employs a burst mode In burst mode, a series of data bits can be clocked out rapidly after the first bit has been accessed This mode is useful when all the bits to be accessed are in sequence and in the same row of the array as the initial access SDRAM has a dual-bank internal architecture that improves opportunities for on-chip parallelism (interleaved memory banks) COMPUTER ARCHITECTURE 58 Synchronous DRAM (SDRAM) SDRAM contains a programmable mode register and associated control logic that provides a mechanism to customize the SDRAM to suit specific system needs The mode register specifies the burst length, which is the number of separate units of data synchronously fed onto the bus The register also allows the programmer to adjust the latency between receipt of a read request and the beginning of data transfer. For SDRAM, there are 5 important timings: The time required to switch internal banks The time required between /RAS and /CAS access The amount of time necessary to "prepare" for the next output in burst mode The column access time The time required to make data ready by the next clock cycle in burst mode (read cycle time) COMPUTER ARCHITECTURE 59 SDRAM Read Operation Clock Diagram COMPUTER ARCHITECTURE 60

16 DRAM vs SDRAM SDRAM tymings DRAM No clock RAS control by change level One bank (array) of memory A transfer for every column address (or CAS pulse) Read delay (latency) no programmable SDRAM Operate at the clock frequency RAS control at the clock impulse Two interleaved memory banks Sometimes static cache at the interface Burst transfer programmable (1, 2, 4, 8, or 256 transfers for a single provided column address, in the same row Read latency programmable t CAS or CL = CAS latency. The time between transmission of column address and beginning of (response by) output data t RCD = Row Address to Column Address Delay. The number of clock cycles required to open a memory line and accessing one column from that line t RP = Row Precharge Time. The number of clock cycles required between issuing the precharge command and the opening the next line from the same memory bank t RAS = Row Active Time. The number of clock cycles between a command for activating line and issuing the precharge command. This is the time internaly required to refresh the line and includes the delay t RCD COMPUTER ARCHITECTURE SDRAM tymings First the address line is transmittes, and after t RCD the line is open and can be accessed. Many columns from the same line can be accessed, each reading requiring the time t CAS. After finishing the column access, make precharge that return to the previous state after time t RP. t RAS, the time to finish line refreshment before close the line t WR, the time that must elapse after the last writing, before the line can be closed SDRAM tymings SDRAM specific times are mentioned using four parameters, clock cycles units: CL, trcd, trp, tras Example: Parameters specify latencies that affect the speed of memory devices Absolute performance of the system is given by the actual latency measured usually in nanoseconds For example if CL = 7 to a memory bus with 1 ns clock cycle (frequency 1000 MHz) result the absolute latency 7 ns

17 Packaging modules SDRAM From Wikipedia, the free encyclopedia DRAM IC Dual in-line Package (DIP) DRAM modules Single In-line Pin Package (SIPP) Single In-line Memory Module (SIMM) Dual In-line Memory Module (DIMM) Rambus In-line Memory Module (RIMM), technically DIMMs but called RIMMs due to their proprietary slot. Small outline DIMM (SO-DIMM), about half the size of regular DIMMs, are mostly used in notebooks, small footprint PCs Small outline RIMM (SO-RIMM). Smaller version of the RIMM, used in laptops. Technically SO-DIMMs but called SO-RIMMs due to their proprietary slot. 65 DDR? SDRAM with SDR - Single Data Rate Have a 64-bits data bus and transfer is done according to the frequency of the data bus, theoretically, for each clock pulse Powered to 3.3 V The modules are commonly called Pcaaa; aaa = maximum frequency at which the memory module can work For example: PC100 works at maximum100 MHz and has a maximum transfer rate of 800 MB / s (8 x 100) DDR (Double Data Rate) memory is based on SDRAM DDR can transfer two data words per memory bus clock cycle Speed performance achieved double the SDR at the same clock frequency of the memory bus The interface uses the so-called double pumping = transferring data on both sides of the clock signal 66 DDR (1) 2.5 V supply These memories are referred to twice the real maximum clock frequency they can work For example, DDR2-800 memory works at 400 MHz, DDR and DDR works at 533 MHz, and DDR memory works at MHz Compared with SDR, DDR SDRAM interface enables higher transfer rates, through a tighter control of timings for clock signals and for data transfer The data being transferred 64 bits at a time, DDR SDRAM gives a transfer rate f clock memory bus x 2 (DDR) x 64 (bits) / 8 (Byte) 67 DDR (1) With a bus frequency of 100MHz DDR-200 offers a maximum transfer rate of 1600 MB / s (memory module denoted as PC-1600) Frequency (MHz) Throughput (MBps) DDR-200 PC-1600 (100x2x8) DDR-226 PC-2100 DDR-333 PC-2700 DDR-400 PC

18 DDR2 In addition to double pumping DDR2 allows higher bus speeds and require less power for operation of the internal clock by halving the internal clock frequency compared to the data bus clock It follows a total of four data transfers per internal clock cycle 1.8 V supply DDR2 provides a transfer rate f clock memory bus x 2 (DDR) x 2 (multiply f clock bus ) x 64 (bits) / 8 (Byte) High latency: internaly it is required a 4-bit depth prefetch queue, but, without changing memory cells, DDR2 can operate effectively at twice the speed of the DDR bus 69 DDR2 DDR2 has a higher latency than DDR at the same clock frequency. However, it can operate at higher frequencies resulting a decrease in its cycle time Thus, with a clock frequency of 100 MHz, DDR2 provides a maximum transfer rate of 3200 MB / s Tag IC number: For example, DDR2-533 memories have a maximum theoretical transfer speed 4264 MB / s (8 x 533). Memory module name is PC Frequency (MHz) Throughput (MBps) DDR2-400 PC (100x2x2x8) DDR2-533 PC DDR2-667 PC DDR2-800 PC DDR PC DDR3 DDR4 DDR3 can transfer double data than DDR2 (8 times (2 3 ) internal memory speed). Clock frequency is 4 times higher than DDR. 1.5 V supply Frequency (MHz) Throughput (MBps) DDR3-800 PC (8x800) DDR PC DDR PC DDR PC DDR PC DDR PC V power supply DDR4 can transfer double data rate comparing to DDR3 (16 times the speed of internal memory array) Operating frequencies: MHz (compared to DDR3) DDR

19 DDR# Successive generations (SDR, DDR, DDR2, DDR3...) are not backward compatible Different voltages Different control protocol Different physical interfaces and different capsules Access rates (throughput) is the theoretical maximum that can be achieved For example, if you install DDR memory on a computer that can access the memory subsystem at 400 MHz (800 MHz DDR) memories will be accessed at 400 MHz (800 MHz DDR) and not at 533 MHz (1066 MHz DDR) DDR# The clock signal is provided by the memory controller, a circuit which is located outside of the memory DDR#-xxx denotes data transfer rate, and describes raw DDR chips PC#-xxxx denotes theoretical bandwidth (with the last two digits truncated), and is used to describe assembled DIMMs END with DDR! Modern DIMMs include a ROM chip (EEPROM) called SPD (Serial Presence Detect) SPD store recommended timings for memory and values are used to automatically configure DIMM The computer BIOS can allow the user to make adjustments to these recommended values in order to increase performance (possible risk of decreasing the stability). CACHE MEMORY 75 COMPUTER ARCHITECTURE 76

20 Cache principles Cache memory is an intermediate temporary memory unit positioned between the processor registers and main memory Large and slow main memory smaller and faster cache memory The cache contains a copy of portions of main memory When the processor attempts to read a word of memory, a check is made to determine if the word is in the cache If so, the word is delivered to the processor If not, a block of main memory, consisting of some fixed number of words, is read into the cache and then the word is delivered to the processor Because of the phenomenon of locality of reference, when a block of data is fetched into the cache to satisfy a single memory reference, it is likely that future references will be to other words in the block. Multiple cache memories Advances in chip process technology possible to implement multiple levels of cache memory Some of these levels will be a part of the microprocessor (on-chip), whereas other levels may be external to the chip. Names: level 1 (L1), level 2 (L2) etc. CPU Microprocessor Fastest Level 1 cache Fast Level 2 cache Slow Main Memory COMPUTER ARCHITECTURE 77 COMPUTER ARCHITECTURE 78 Cache and Main Memory STRUCTURE OF A CACHE/MAIN- MEMORY SYSTEM M = 2 a /K blocks m < < M COMPUTER ARCHITECTURE 79 COMPUTER ARCHITECTURE 80

21 HIT RATIO CACHE READ OPERATION The performance of cache memory can be given by a synthetic parameter, called hit ratio (HR) Hit ratio determined running benchmarks HR represents the ratio between the total numbers of hits in cache and the total number of memory accesses (hit + miss numbers) The value of HR must be greater then 0.9 From HR value we can compute the average memory access time. For example if HR = 0.9, the access time to main memory (for misses) is 100 nsec. and access time to cache memory is 10 ns, the average memory access time is: t = acm = 10 19ns COMPUTER ARCHITECTURE 81 COMPUTER ARCHITECTURE 82 Mapping Function Because there are fewer cache lines than main memory blocks is needed: An algorithm for mapping main memory blocks into cache lines A means for determining which main memory block currently occupies a cache line The choice of the mapping function dictates how the cache is organized Three techniques can be used: Direct mapping Associative mapping Set associative mapping Direct mapping Direct mapping is the simplest technique Direct mapping maps each block of main memory into only one possible cache line The mapping is expressed as: i = j modulo m where i = cache line number; j = main memory block number; m = number of lines in the cache For purposes of cache access, each main memory address can be viewed as consisting of three fields The least significant w bits identify a unique word or byte within a block of main memory (in most contemporary machines, the address is at the byte level) The remaining a-w bits specify one of the 2 a-w blocks of main memory The cache logic interprets these a-w bits as a tag of t bits (most significant portion) plus a cache line field of r bits (a = t+r+w) COMPUTER ARCHITECTURE 83 COMPUTER ARCHITECTURE 84

22 Direct mapping Sometime the word and block fields are called index field, because the index is used to address data in cache The use of a portion of the address as a line number provides a unique mapping of each block of main memory into the cache When a block is actually read into its assigned line, it is necessary to tag the data to distinguish it from other blocks that can fit into that line The effect of this mapping is that blocks of main memory are assigned to lines of the cache as follows: Direct- Mapping Cache Organization COMPUTER ARCHITECTURE 85 COMPUTER ARCHITECTURE 86 Example for address 1D00002E hex 32 bits memory address 8 bits tag 20 bits block address 4 bits word addr. Direct mapping Advantages for direct mapping: Simple and cheap The tag field is short Only those bits have to be stored which are not used to address the cache Access is very fast. Disadvantage: There is a fixed cache location for any given block If a program happens to reference words repeatedly from two different blocks that map into the same line, then the blocks will be continually swapped in the cache, and the hit ratio will be low. COMPUTER ARCHITECTURE 87 COMPUTER ARCHITECTURE 88

23 Associative mapping Each main memory block can be loaded into any line of the cache The cache control logic interprets a memory address simply as a tag and a word field In the cache is stored also data and the corresponding address The associative mapping is implemented with associative memories (content addressable memories) as cache memories more expensive than a random access memory each cell must have storage capacity as well comparison logic circuits for matching its content with an external argument Fully associative mapping COMPUTER ARCHITECTURE 89 COMPUTER ARCHITECTURE 90 Fully associative mapping Fully associative mapping Advantages: provides the highest flexibility concerning the line to be replaced when a new block is read into the cache i = j modulo 1 where i = cache line number; j = main memory block number; Disadvantages: complex the tag field is long fast access can be achieved only using high performance associative memories for the cache, which is difficult and expansive. COMPUTER ARCHITECTURE 91 COMPUTER ARCHITECTURE 92

24 Set associative mapping Set associative mapping is a compromise that exhibits the strengths of both the direct and associative approaches while reducing their disadvantages In this case, the cache is divided into v sets, each of which consists of k lines This is referred to as k-way set associative mapping With set associative mapping, block Bj can be mapped into any of the lines of set i In this case, the cache control logic interprets a memory address simply as three fields: tag, set, and word With fully associative mapping, the tag in a memory address is quite large and must be compared to the tag of every line in the cache With k-way set associative mapping, the tag in a memory address is much smaller and is only compared to the k tags within a single set Set associative mapping The mapping relationships are: i = j modulo v m = v k where i = cache set number; j = main memory block number; m = number of lines in the cache; v = number of sets in cache; k = number of cache lines per set In the extreme case of v = m, k = 1, the set associative technique reduces to direct mapping In the extreme case of v = 1, k = m, it reduces to associative mapping The use of two lines per set (v = m/2, k = 2) is the most common set associative organization (two-way set associative mapping) Four-way set associative (v = m/4, k = 4) makes a modest additional improvement for a relatively small additional cost Further increases in the number of lines per set have little effect. COMPUTER ARCHITECTURE 93 COMPUTER ARCHITECTURE 94 Example two-way set associative COMPUTER ARCHITECTURE 95 EXAMPLE Assume that: Main memory is divided in 32 blocks Cache cache has 8 block frames (lines) The set-associative organization has 4 sets with 2 blocks per set, called two-way set associative Where in cache main memory block 12 can be placed, for the three categories of cache organization? Fully associative: block 12 can go into any of the eight block frames of the cache Direct mapped: block 12 can only be placed into block frame 4 (12 modulo 8) Two-way set associative: allows block 12 to be placed anywhere in set 0 (12 modulo 4). Line 0 or line 1 COMPUTER ARCHITECTURE 96

25 EXAMPLE Replacement Algorithms When a new block is brought into the cache, one of the existing blocks must be replaced For direct mapping, there is only one possible line for any particular block, and no choice is possible For the associative and set associative techniques, a replacement algorithm is needed To achieve high speed, such an algorithm must be implemented in hardware COMPUTER ARCHITECTURE 97 COMPUTER ARCHITECTURE 98 Replacement Algorithms LRU (least recently used): Replace that block in the set that has been in the cache longest with no reference to it For two-way set associative each line includes a USE bit When a line is referenced, its USE bit is set to 1 and the USE bit of the other line in that set is set to 0 When a block is to be read into the set, the line whose USE bit is 0 is used Because we are assuming that more recently used memory locations are more likely to be referenced, LRU should give the best hit ratio. FIFO (first-in-first-out): Replace that block in the set that has been in the cache longest FIFO is easily implemented as a round-robin or circular buffer technique. LFU (least frequently used: Replace that block in the set that has experienced the fewest references LFU could be implemented by associating a counter with each line Random replacement: is the simplest to implement and results are surprisingly good. COMPUTER ARCHITECTURE 99 Write Policy Write through: all write operations are made to main memory as well as to the cache Any other processor-cache module can monitor traffic to main memory to maintain consistency within its own cache The main disadvantage of this technique is that it generates substantial memory traffic and may create a bottleneck Write back: updates are made only in the cache Minimizes memory writes When an update occurs, an UPDATE bit associated with the line is set When a block is replaced, it is written back to main memory if and only if the UPDATE bit is set The problem with write back is that portions of main memory are invalid, and hence accesses by I/O modules can be allowed only through the cache COMPUTER ARCHITECTURE 100

26 Write Policy Write-through with buffered write The same as write-through, but instead of slowing the processor down by writing directly to main memory, the write address and data are stored in a high-speed write buffer; the write buffer transfers data to main memory while the processor continues it s task Dirty bits: this status bit indicates whether the block is dirty (modified while in the cache) or clean (not modified) If it is clean, the block is not written on a miss, since the lower level has identical information to the cache. Both write back and write through have their advantages With write back, writes occur at the speed of the cache memory, and multiple writes within a block require only one write to the lower-level memory write back uses less memory bandwidth, making write back attractive in multiprocessors With write through, read misses never result in writes to the lower level, and write through is easier to implement than write back Write through also has the advantage that the next lower level has the most current copy of the data This is important for I/O and for multiprocessors COMPUTER ARCHITECTURE 101 Write Policy In a bus organization in which more than one device (typically a processor) has a cache and main memory is shared: If data in one cache is altered: This invalidates the corresponding word in main memory This invalidates the corresponding word in other caches (if any other cache happens to have that same word) Even if a write-through policy is used, the other caches may contain invalid data A system that prevents this problem is said to maintain cache coherency COMPUTER ARCHITECTURE 102 Write Policy Possible approaches to cache coherency include the following: Bus watching with write through: Each cache controller monitors the address lines to detect write operations to memory by other bus masters If another master writes to a location in shared memory that also resides in the cache memory, the cache controller invalidates that cache entry Hardware transparency: Additional hardware is used to ensure that all updates to main memory via cache are reflected in all caches Thus, if one processor modifies a word in its cache, this update is written to main memory. In addition, any matching words in other caches are similarly updated. Noncachable memory: Only a portion of main memory is shared by more than one processor, and this is designated as noncachable All accesses to shared memory are cache misses, because the shared memory is never copied into the cache The noncachable memory can be identified using chip-select logic or highaddress bits Write Policy for write miss Since the data are not needed on a write, there are two common options on a write miss: Write allocate (also called fetch on write) The block is loaded on a write miss, followed by the write-hit actions above This is similar to a read miss. No-write allocate (also called write around) The block is modified in the lower level and not loaded into the cache. Write-back caches generally use write allocate (hoping that subsequent writes to that block will be captured by the cache) Write-through caches often use no-write allocate (since subsequent writes to that block will still have to go to memory) COMPUTER ARCHITECTURE 103 COMPUTER ARCHITECTURE 104