CIT 668: System Architecture. Computer Systems Architecture

Transcription

1 CIT 668: System Architecture Computer Systems Architecture

2 1. System Components Topics 2. Bandwidth and Latency 3. Processor 4. Memory 5. Storage 6. Network 7. Operating System 8. Performance Implications Images courtesy of Majd F. Sakr or from Wikipedia unless otherwise noted.

3 A Single CPU Computer Components

4 Von Neumann Architecture Single storage for both data and commands Storage separate from control unit RAM CPU The Brain von Neumann 4

5 The 5 Von Neumann Components Input/Output CPU and ALU Memory

6 Motherboard

7 High Level Machine Architecture

8 Bandwidth and Latency Bandwidth (aka throughput) Rate of data transfer (bit rate) How wide is your pipe. Latency (aka lag) Time to get data since request was sent. How long is your pipe.

9 Latency Examples

10 Latency lags Bandwidth Moore s Law More transistors help bandwidth, hurt latency. Faster transistors helps both. Latency helps bandwidth Faster disk rotation improves both. More devices in parallel improve bandwidth, hurt latency. Distance limits latency. Speed of light bound. Source: David A. Patterson, Latency lags bandwith, Communications of the ACM, Vol. 47 No. 10, Pages 71-75

11 The Processor The Brain: a functional unit that interprets and carries out instructions (mathematical operations) Also called a CPU (actually includes CPU + ALU) Consists of hundreds of millions of transistors.

12 Instruction Set Architecture (ISA) Every processor has its own set of operations Written in binary machine language (ML) Each instruction consists of operator + operand Assembly language is humanly-readable ML Category Arithmetic Logic Program Control Data Movement I/O Example Add, subtract, multiply, divide And, or, not, exclusive ov Branch (conditional), call, jump Move, load, store Read, write

13 Source Code to Machine Language

14 Program Execution Cycle Fetch/Execute Cycle Fetch: retrieve instruction from memory Execute: decode instruction, fetch operands, execute instruction, store results to memory (if any) Fetch Store Decode Execute

15 Synchronous Execution Each step waits for click tick to begin A 100 MHz processor uses a clock that ticks 100,000,000 times per second. Fetch Store Decode Cycle speed of 1 GHz: billionths of a second Execute

16 Processor Components

17 Processor Layout

18 From Sand to Silicon Wafers Purification and heating to make a pure silicon crystal. Sand, made up of 25% silicon, is the starting point 100kg of % pure silicon crystal Slicing the crystal into 300mm wafers for use in photolithography.

19 Photolithography

20 Packaging Wafer is cut into dies, each containing one processor. Substrate, die, and heatspreader assembled to make final CPU product for use in a PC.

21 Processor Components: Control Control Unit Processor s supervisor Fetch/execute cycle Retrieve instruction Decode instruction Execute actions Program Counter (PC): stores address of instruction to be fetched. Instruction Register (IR): has instruction most recently fetched.

22 Processor Components: Datapath Register File: General purpose storage locations inside processor that hold addresses or values. Arithmetic Logic Unit: Set of functional units that perform arithmetic and logic operations. A 32-bit processor has registers that are typically 32 bits in size.

23 Superscalar Architecture Instead of one ALU, use multiple execution units Some execution units are identical to others Others are different: Integer, FPU, multi-media

24 AMD Athlon Architecture

25 Computer System Layers

26 Moore s Law Number of transistors doubles every 18 months More transistors = Cheaper CPUs Higher speeds More features More cache 26

27 Improvements in CPU Clock Speed

28 Computation Produces Heat

29 Processor Power Density Sun s Surface Power Density (W/cm 2 ) Hot Plate Rocket Nozzle Nuclear Reactor Pentium Pentium IV Pentium III Pentium II Pentium Pro Year Source: Intel 29

30 The Single CPU is History What can you do with more transistors at same speed? Multicore CPUs Multiple processors (cores) on a single chip Run different programs on each core Some programs can be rewritten to run on multiple cores: PhotoShop, bzip2 Most of you are running multi-core

31 2014 Server CPUs Intel Xeon Up to 15 cores with 2 threads 2.2 to 4.0 GHz Up to 30 MB L3 cache AMD Opteron Up to to 3.5 GHz Up to 16 MB L3 cache IBM Power 7+ Up to 8 cores with 4 threads 3.0 to 4.4 GHz 10 MB L3 cache per core (up to 80MB for 8-core) Sun/Oracle SPARC T5 16 cores with 8 threads 3.6 GHz 8 MB L3 cache

32 End of Moore s Law Si 0.3 nm

33 Where do we go from here? Better Materials More Machines Better Algorithms GaAs Cloud Map Reduce Diamond Containers Fuzzy Logic

34 Main Memory

35 Memory Bandwidth and Latency Generation Type Peak Bandwidth Latency (1 st word) SDRAM (1990s) PC GB/s 20ns DDR (2000) DDR GB/s 20ns DDR DDR GB/s 15ns DDR2 (2003) DDR GB/s 15ns DDR2 DDR GB/s 15ns DDR3 (2007) DDR GB/s 13ns DDR3 DDR GB/s 11.25ns DDR4 (2014) DDR GB/s?

36 Processor-Memory Bottleneck Solution: Caches

37 Processor-Memory Gap

38 Principle of Locality Programs tend to reuse data and instructions near those they have used recently. Temporal locality: Recently referenced items are likely to be referenced in the near future. Spatial locality: Items with nearby addresses tend to be referenced close together in time.

39 Caches Cache: A smaller, faster storage device that transparently stores a subset of the data in a larger slower device so that future requests for that data can be served more quickly. Average Access Time = Time for a Cache Hit + Miss Rate Miss Penalty

40 Improving Cache Performance Reduce the time for a cache hit Reduce cache size Use direct mapped cache Reduce the miss rate Increase cache size More flexible placement Reduce the miss penalty Use multiple cache levels

41 Memory Hierarchy

42 Memory Design: Hierarchy Fundamental Idea: Faster, smaller storage devices at level k serve as a cache for the larger, slower device at level k+1. CPU cache hit rates typically above 90% If hit rate is high, gives illusion that large amount of slow memory is just as fast as fastest memory Why can t all of our memory be fastest type? Size and cost Speed of light

43 AMAT Computation Example #1 CPU speed is 1 GHz 2 cycles to access L1 cache 80 cycles to access RAM L1 (90%) RAM (100%) AMAT = 2 cycles + 10% 80 cycles = (2 cycles+8 cycles) 1 second / 10 9 cycles = 10-8 seconds = 10ns

44 AMAT Computation Example #2 CPU speed is 1 GHz AMAT = 2 cycles + 10% (6 cycles + 5% 80 cycles) L1 (90%) L2 (95%) RAM (100%) 2 cycles 6 cycles 80 cycles AMAT = 2 cycles + 10% (6 cycles + 4 cycles) = (2 cycles + 1 cycle) 1 second / 10 9 cycles = seconds = 3 ns

45 Cores may share Caches

46 Error Correcting Code (ECC) DRAM SECDED is most widely used ECC Single Error Correction Double Error Detection ECC DRAM costs more than non-ecc Need extra bit for ECC + Lower volume of ECC DRAM sold Memory errors per year

47 Disk

48 Disk Geometry Disks consist of platters, each with two surfaces Each surface consists of concentric rings called tracks Each track consists of sectors separated by gaps Aligned tracks form a cylinder.

49 Disk Operation Disk spins at a fixed rotational rate (rpm) Read/write heads are attached to end of arm Moving radially, arm positions head over desired track Arm waits for disk to spin desired sector under head

50 Hard Drive Components

51 Disk Performance Seek Time Time to move head to desired track (3-8 ms) Rotational Latency Average time to position head over desired sector ½ 1/RPMs 60sec/min (8ms for 7200rpm) Total Latency Seek Time + Rotational Delay Transfer Time Time to read bits in target sector 1/RPM 1/(average # sectors/track) 60sec/min

52 I/O Bus

53

54

55

56 Latency Across Memory Hierarchy

57 Solid State Drives (SSDs) Integrated circuit-based data storage devices. Usually flash memory. OS sees as a hard drive. Compared to HDs Low latency (10X) High throughput (2-10X) High cost (5-10X) Low capacity (5-10X) Reliability: No moving parts, but all SSDs except Intel shown to lose data on repeated power failures.

58 RAID: Redundant Array of Independent Disks RAID 0: Striping (non-redundant) RAID 1: Mirroring RAID 3: Bit Striping + Parity Disk RAID 4: Block Striping + Parity Disk RAID 5: Block Striping + Distributed Parity RAID 6: Block Striping + Multiple Parity Nested RAID: Combinations of above Files are striped across multiple disks (spindles) Throughput increases via use of many disks Reliability increases as data on a failed disk can be reconstructed from parity data

59 Network

60 The Bus: An Internal Computer Network A bus is a collection of parallel wires that carry address, data, and control signals. Buses typically connect multiple devices.

61 I/O Bus and Devices

62 Network Adapter connects Bus to Other PCs

63 Bus Speeds Bus Name Theoretical Throughput Purpose USB Mb/s Connect PCs to peripherals (high overhead) USB Gb/s Connect PCs to peripherals, incl external HDs Thunderbolt 2 20 Gb/s Fast external peripherals, like GPUs and SSDs Ethernet 10 Mb/s to 100 Gb/s Local area network (LAN) n 600 Mb/s Wireless local area network (WLAN) T Mb/s Basic wide area network (WAN) connection OC-12 to Mb/s to 160 Gb/s Optical carrier (OC) WAN connections Fiber Channel 2 Gb/s to 16 Gb/s Storage area networks SATA Gb/s Connect mass storage devices PCIe 3.0 (16x) 128 Gb/s Connect internal peripherals like GPUs, SSDs DDR Gb/s Connect DDR3 RAM to CPU QPI 256 Gb/s Connect CPUs to each other or to I/O bus

64 Network Network: collection of computers connected together through some communication channel (Ethernet, Fiber Channel, wireless, DSL, etc.) Internet

65 Bandwidth Data transmission rate Bits/second Latency Time to transfer data from end to end ms Network

66 OSI Model

67 Routing

68 Operating Systems

69 Process Control A process is an instance of a program in execution. Processes can be executed concurrently. The OS is responsible for: Process creation and termination Allocating resources (CPU, memory, I/O) for processes Interprocess communication

70 Memory Management Each process has its own virtual memory space. OS maintains a page table of virtual address to physical address translations CPU has a Memory Management Unit that performs translations automatically based on register pointing to OS page table

71 A filesystem is a way of storing and organizing data on a disk to ensure easy human access. Filesystems usually support nested sets of directories forming a tree structure. Filesystem

72 Key Points 1. Slow CPU speed increases focus on parallelism 1. Can t increase clock speeds much above 4GHz 2. Use increased transistor counts for multi-core CPUs 3. Moore s law will end, stopping transistor growth 2. Processor-storage bottleneck 1. Use caching to hide slowness of storage 2. Registers > CPU Cache >> Main memory >> Disk 3. Lots of cheap, slow memory + little expensive, fast memory leads to memory hierarchy architecture 3. To improve disk performance, we can use 1. Solid State Drives (SSDs) and/or 2. RAID combinations of disks.

73 References 1. Jeff Dean, Latency Numbers Every Programmer Should Know, 2. David A. Patterson. "Latency lags bandwidth." Communications of the ACM (2004): Wikipedia, List of Device Bit Rates, bit_rates