CSC Memory System. A. A Hierarchy and Driving Forces

Size: px

Start display at page:

Download "CSC Memory System. A. A Hierarchy and Driving Forces"

Shonda Stevens
5 years ago
Views:

1 CSC System A. A Hierarchy and Driving Forces 1A_1 The Big Picture: The Five Classic Components of a Computer Processor Input Control Datapath Output Topics: Motivation for Hierarchy View of Hierarchy Locality Principles 1A_2

2 Hierarchy Technology Random Access: Random is good: expected access time is the same for all locations DRAM: Dynamic Random Access - High density, low power, cheap, slow - Dynamic: need to be refreshed regularly SRAM: Static Random Access - Low density, high power, expensive, fast - Static: content will last forever (until lose power) Not-so-random Access Technology: Examples: Disk, CDROM Access time is influenced by where the read/write head is at the time of access relative to the access point Sequential Access Technology: access time linear in location (e.g.,tape) This Section will concentrate on the use of random access technology The Main : DRAMs + Caches: SRAMs 1A_3 Technology Trends Capacity Speed (latency) DRAM: 4x in 3 years 2x in 10 years Disk: 4x in 3 years 2x in 10 years Processor : 2x in 3 years 2x in 3 years DRAM Year Size Cycle Time 1000:1! 2:1! Kb 250 ns Kb 220 ns Mb 190 ns Mb 165 ns Mb 145 ns Mb 120 ns 1A_4

3 Performance Who Cares About the Hierarchy? 1000 Processor-DRAM Gap (latency) Moore s Law µproc CPU 60%/yr. (2/1.5yr) Processor- Performance Gap: (grows 50% / year) DRAM DRAM 9%/yr. (2/10 yrs) Time 1A_5 The Goal: illusion of large, fast, cheap memory Fact: Large memories are slow, fast memories are small How do we create an illusion of a memory that is large, cheap and fast (most of the time)? Hierarchy Parallelism 1A_6

4 A Real-Life Example: Working in a library Imagine Alice (A) going to the library for writing an essay on Early British Computers. A begins by selecting one or more books on this topic from a shelf (slow access + large storage) A places them on a desk (limited storage) close to the shelf which has all the relevant books stacked together If A cannot find some information in the book she holds, she checks with other books on her desk (fast access) If none provides, she walks to the shelf finds an appropriate one and adds to her collection on the desk If too many books accumulate on her desk, she puts the least important one back in the shelf 1A_7 A Conceptual View and Observations Alice (Holding One book) Desk Shelf Shelves On the floor Multi- Floor Library Access Speed: Capacity or Size: Fastest Smallest Slowest Biggest Over time, there is a high probability that All information needed can be found in those books on the desk Alice s average access time to information is small This probability increases with How books are replaced when desk has no space for a new book Of course, the desk space itself 1A_8

5 Features for Reducing Access Time - Localities (1) Reducing average access time assumes two features 1. Temporal Locality A feature that promotes re-use of what has been used recently. An Example Alice works for 30 minutes in the library, taking one book at a time from the shelf and storing them on the desk until she leaves She notes down 10 points from the books she read during that time, and takes at least one point from each book she reads Consider the following two circumstances: C1: Each of these 10 points comes from different books: - She would walk to the shelf 10 times before leaving C2: All the points Alice noted come from just 3 books - She would walk to the shelf only 3 times before leaving C2 has temporal locality because book(s) recently referred to, was (were) referred again 1A_9 Features for Reducing Access Time - Localities (2) Second Feature: If a new book is needed, it can be found in the same shelf where she collected the books from (space locality) That is, books on a given topic are lumped together in the same shelf or nearby ones Imagine that the books in the library had been shelved as per the alphabetical order of their titles! Alice would be walking all over the library for each new book! There is no spatial locality here: choosing a desk close to one particular shelf is of no help Spatial Locality The feature that promotes the next wanted item to be found closer to where the current item was found Observation: Both Temporal and Spatial localities help Alice in keeping her average access time small. 1A_10

6 A Conceptual View of the System Processor Control Datapath Speed: Size: Cost: Fastest Smallest Highest Slowest Biggest Lowest 1A_11 Levels of the Hierarchy Capacity Access Time Cost CPU Registers 100s Bytes <10s ns SRAM K Bytes ns $ /bit DRAM M Bytes 100ns-1us $ Disk G Bytes ms cents Tape infinite sec-min 10-6 Registers Cache Main Secondary Tape Instructions, Operands Blocks Pages Files Staging fer Unit Control unit 1-8 bytes cache cntl bytes OS 512-4K bytes user/operator Mbytes faster Larger 1A_12

7 Why hierarchy works Program access a relatively small portion of the address space in any small period of time. That is, spatial + temporal localities normally hold Assume a program s address space has 2 n bytes, for some n, numbered from 0..2 n -1 (When n = 10, 2 10 = 1KB) The pattern of reference in a brief duration during execution: Probability of reference The Principle of Locality: 0 Address Space 2**n - 1 Design principles promoting temporal + spatial localities In programming terms, modularisation promotes locality 1A_13 Locality principles in detail Two Different Types of Locality: Temporal Locality (Locality in Time): If a data item is referenced, it will tend to be referenced again soon (e.g., array index) Spatial Locality (Locality in Space): If an item is referenced, items whose addresses are close by tend to be referenced soon (e.g., elements of an array or a list) By taking advantage of the principles of locality: Present the user with as much memory as is available in the cheapest technology. Provide access at the speed offered by the fastest technology. DRAM is slow but cheap and dense: Good choice for presenting the user with a BIG main memory SRAM is fast but expensive and not very dense: Good choice for cache that provides the user FAST access time. 1A_14

8 Levels of the Hierarchy Capacity access Time Cost CPU Registers 100s Bytes <10s ns SRAM K Bytes ns $ /bit DRAM M Bytes 100ns-1us $ Disk G Bytes ms cents Tape infinite sec-min 10-6 Registers Instr. Operands Cache Blocks Pages Disk Files Tape Staging fer Unit prog./compiler 1-8 bytes cache cntl bytes OS 512-4K bytes user/operator Mbytes faster Larger 1A_15 Hierarchy: How Does it Work? Between any two consecutive levels in the memory hierarchy (e.g., cache and memory), exploit: Temporal Locality (Locality in Time): Keep most recently accessed data items closer to the processor, as long as possible why only as long as possible, not forever? Spatial Locality (Locality in Space): If Byte is referenced and is to be moved to the upper level, move also the neighbouring bytes In anticipation that Byte (-1) or Byte (+1) will be referenced soon The size of neighbours of Byte that can be moved increases as we go down the level. Why? To Processor From Processor 1A_16

9 Terminology: Hit and Miss Hit and Miss are defined between any consecutive levels in the memory hierarchy (see 1A_15) Say, the processor requests for the contents of Byte (Recall is an address) Request for Byte is a Hit if the content of is already in the upper level at the time of request Hit Rate: the fraction of total requests found in the upper level Hit Time: Time to determine that an access request is a hit + access time to the upper level To Processor From Processor Between cache and memory: Hit means Byte is already in the cache; hit rate = the fraction of memory accesses found in the cache; hit time = time to determine a hit + time to access the contents of Byte from the cache 1A_17 Terminology: Miss Request for Byte is a Miss if the content of is NOT in the upper level at the time of request, and needs to be retrieved from the lower level Miss Rate = 1 - (Hit Rate) Miss Penalty: Time to place content of in the upper level + Access time to the upper level Before a Miss After a Miss Why it is very important that Hit Time << Miss Penalty, and/or Hit rate >> Miss rate How many hits/misses Alice had in C1 and C2 (1A_9)? 1A_18

10 Hierarchy of a Modern Computer System By taking advantage of the principle of locality: Present the user with as much memory as is available in the cheapest technology. Provide access at the speed offered by the fastest technology. Processor Datapath Control Registers On-Chip Cache Second Level Cache (SRAM) Main (DRAM) Secondary Storage (Disk) Tertiary Storage (Disk) Speed (ns): 1s 10s 100s Size (bytes): 100s Ks Ms 10,000,000s (10s ms) Gs 10,000,000,000s (10s sec) Ts 1A_19 How is the hierarchy managed? Registers <-> cache by control unit within processor cache <-> memory by cache control memory <-> disks by the hardware and operating system (virtual memory) 1A_20

11 Example: Pentium System Organization Processor/ Bus PCI Bus (peripheral component interconnect) I/O Busses 1A_21

ECE468 Computer Organization and Architecture. Memory Hierarchy

ECE468 Computer Organization and Architecture Hierarchy ECE468 memory.1 The Big Picture: Where are We Now? The Five Classic Components of a Computer Processor Control Input Datapath Output Today s Topic: