Why Parallel Computing?

Size: px

Start display at page:

Download "Why Parallel Computing?"

Julie Simmons
6 years ago
Views:

1 Why Parallel Computing? Jinkyu Jeong Computer Systems Laboratory Sungkyunkwan University SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong

performance Concurrent computing can either alternate multiple threads on a single processor or

2 What is Parallel Computing? Software with multiple threads Parallel vs. concurrent Parallel computing executes multiple threads at the same time on multiple processors performance Concurrent computing can either alternate multiple threads on a single processor or execute in parallel on multiple processors convenient design SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 2

3 What is Parallel Architecture? Machines with multiple processors Intel Quad Core i7 (Nehalem) Sony Playstation 3/4 IBM Blue Gene Parallel Computing vs. Distributed Computing Google Data Center Locations Facebook Data Center SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 3

4 What is Parallel Architecture? One definition A parallel computer is a collection of processing elements that cooperate to solve large problems fast Some general issues Resource allocation Data access, communication and synchronization Performance and scalability SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 4

5 Why Study Parallel Computing? 10 years ago Some important applications demand high performance With CPU clock frequency scaling, it is not enough Parallel computing provides higher performance Today Many interesting applications demand high performance CPU clock rates are no longer increasing! Instruction-level parallelism is not increasing either! Parallel computing is the only way to achieve higher performance in the foreseeable future SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 5

6 Why Study Parallel Architecture? Role of a computer architect To design and engineer the various levels of a computer system to maximize performance and programmability within limits of technology and cost Parallelism Provides alternative to faster clock for performance Applies at all levels of system design Bits, instructions, loops, threads, SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 6

7 Why Parallel Computing? Application demands Architectural Trends The impact of technology and power SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 7

8 Application Trends A positive feedback cycle between the two Delivered performance Applications demand for performance New Applications More Performance Example application domains Scientific computing: computational fluid dynamics, biology, meteorology, General purpose computing: video, graphics, Commercial computing: databases, SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 8

9 Speedup Goal of applications in using parallel machines Speedup (p processors) = Performance (p processors) Performance (1 processor) For a fixed problem size (input data set), performance = 1/time Speedup fixed problem (p processors) = Time (1 processor) Time (p processors) SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 9

10 Scientific Computing Demand SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong

11 Example: N-body Simulation Simulation of a dynamical system of particles N particles (location, mass, velocity) Influence of physical forces (e.g., gravity) Physics, astronomy, SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 11

Example: Graph Computation Google page-rank algorithm Determining the importance of a web page by counting the number and quality of likes to the page Relative

12 Example: Graph Computation Google page-rank algorithm Determining the importance of a web page by counting the number and quality of likes to the page Relative importance of a web page u Repeat until stabilized Trillion pages in Web Credit: wikipedia SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 12

13 Engineering Computing Large parallel machines are mainstays in many industries Petroleum reservoir analysis Automotive crash simulation, drag analysis, combustion efficiency Aeronautics airflow analysis, engine efficiency, structural mechanics, electromagnetism Computer-aided design VLSI layout Pharmaceuticals molecular modeling Visualization all the above, entertainment, architecture Financial modeling yield and derivative analysis SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 13

14 Commercial Computing Relies on parallelism for high end Computational power determines scale of business that can be handled e.g.) databases, online-transaction processing, decision support, data mining, data warehousing... TPC benchmarks TPC-C (order entry), TPC-D (decision support) Explicit scaling criteria provided Size of enterprise scales with size of system Problem size no longer fixed as p increases, so throughput is used as a performance measure, transactions per minute (tpm) SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 14

15 Commercial Computing Hardware threads in the Top-10 TPC-C machines SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 15

16 Why Parallel Computing? Application demands Architectural Trends The impact of technology and power SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 16

17 Architectural Trends Architecture translates technology s gifts into performance and capability Four generations of architectural history Tube, transistor, IC, VLSI Greatest delineation in VLSI has been in type of parallelism exploited SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 17

18 Moore s Law Gordon Moore Co-founder of Intel co. 1965: the number of transistors/inch 2 doubles every 1.5 years Credit: shigeru23 (CC-BY-SA-3.0) SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 18

Evolution of Intel Microprocessors Intel 8088, 1978 1 core, no

2.3B transistors Intel knight landing, 2016(?

ExtremeTech, The future of microprocessors, Communications of the

19 Evolution of Intel Microprocessors Intel 8088, core, no cache 29K transistors Intel Nehalem-EX, cores, 24MB cache 2.3B transistors Intel knight landing, 2016(?) 72 cores, 16GB DDR3 LLC(?) 7.1B transistors Figure credits: ExtremeTech, The future of microprocessors, Communications of the ACM, vol. 54, no.5 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 19

20 Transistors & Clock Frequency Credit: The future of computing performance: game over or next level?, The National Academies Press, 2011 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 20

21 Impact of Device Shrinkage What happens when the transistor size shrinks by a factor of x? Clock rate increases by x because wires are shorter Actually less than x, because of power consumption Transistors per unit area increases by x 2 Die size also tends to increase Typically another factor of ~x Raw computing power of the chip goes up by ~ x 4! Typically x 3 is devoted to either on-chip Parallelism: hidden parallelism such as ILP Locality: caches So most programs x 3 times faster, without changing them SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 21

22 Architectural Trends Greatest trend in VLSI generation is Increase in parallelism Up to 1985: bit level parallelism 4-bit 8 bit 16-bit, slows after 32 bit Adoption of 64-bit now under way, 128-bit far (not performance issue) Mid 80s to mid 90s: instruction level parallelism Pipelining and simple instruction sets + compiler advances (RISC) On-chip caches and functional units superscalar execution Greater sophistication: out of order execution, speculation, prediction To deal with control transfer and latency problems Now: thread level parallelism Multicore processors SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 22

23 Phases in VLSI Generation 100,000,000 Bit-level parallelism Instruction-level Thread-level (?) Transistors 10,000,000 1,000, ,000 4bits 8bits 16bits 32bits i80286 R10000 Pentium i80386 R2000 R3000 Multicore / manycore Heterogenous multicore GPGPU 10,000 i8086 i8080 i8008 i4004 Pipelining Superscalar Branch prediction Out-of-order execution VLIW 1, SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu)

24 Superscalar Execution Exploits instruction-level parallelism With N pipelines, N instructions can be issued (executed) in parallel 1. Load 2. Load 3. Add 4. Add 5. Add R1, R2 6. Store 2-way superscalar execution pipeline SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 24

25 Out-of-order Execution Exploits instruction-level parallelism With N pipelines, N instructions can be issued (executed) in parallel Reorder instructions 1. Load 2. Load 3. Add 4. Add 5. Add R1, R2 6. Store dependency 2-way superscalar execution pipeline SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 25

26 Architecture-driven Advances Credit: The Future of Microprocessors, Communications of the ACM, 2011 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 26

27 Limitation in ILP Fraction of total cycles (%) Speedup Number of instructions issued Instructions issued per cycle (Issue width) Speedup begins to saturate after issue width of 4 Assumption of ideal machine Infinite resources and fetch bandwidth, perfect branch prediction and renaming But with realistic memory Real caches and non-zero miss latencies SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 27

Instruction-level Parallelism Concerns Issue waste cycles Full issue slot Empty issue slot Horizontal waste = 9 slots Vertical waste = 12 slots Issue slots (4way) Contributing factors Instruction

28 Instruction-level Parallelism Concerns Issue waste cycles Full issue slot Empty issue slot Horizontal waste = 9 slots Vertical waste = 12 slots Issue slots (4way) Contributing factors Instruction dependencies Long-latency operations within a thread Credit: D. M. Tullsen, S. J. Eggers, and H. M. Levy, Simultaneous Multithreading: Maximizing On-Chip Parallelism, ISCA 1995 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 28

Some Sources of Wasted Issue Slots TLB miss Instruction cache miss Data cache miss Load delays (L1 hit latency) Branch misprediction Instruction

29 Some Sources of Wasted Issue Slots TLB miss Instruction cache miss Data cache miss Load delays (L1 hit latency) Branch misprediction Instruction dependency Memory conflict Memory hierarchy Control flow Instruction stream SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 29

Simulation of 8-issue Superscalar Processor Highly underutilized processor cycles In most of SPEC 92 applications On average < 1.5 IPC (19%) Dominant waste differs by application Credit: D.

30 Simulation of 8-issue Superscalar Processor Highly underutilized processor cycles In most of SPEC 92 applications On average < 1.5 IPC (19%) Dominant waste differs by application Credit: D. M. Tullsen, S. J. Eggers, and H. M. Levy, Simultaneous Multithreading: Maximizing On-Chip Parallelism, ISCA 1995 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 30

decreased [Computer Architectures Hennessy & Patterson]

31 Single-Thread Performance Curve Architecture -driven Technology -driven The rate of single-thread performance improvement has decreased [Computer Architectures Hennessy & Patterson] SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 31

32 Single-chip Multiprocessor (Multicore) Limited instruction-level parallelism à Exploits thread-level parallelism 30% area 6-way superscalar processor 4 2-way multiprocessor Credit: K. Olukotun, B. A. Nayfeh, L. Hammond, K. Wilson, and K. Chang, A case for single-chip multiprocessor, ASPLOS, 1996 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 32

33 Why Parallel Computing? Application demands Architectural Trends The impact of technology and power SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 33

34 Desperate Cooling? SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 34

(advanced power management) BIOS manages clock speed Triggered by thermal diodes ACPI (Advanced Configuration and Power

35 Cooling Power delivery, packaging, and cooling costs Increased cost of thermal packing $1 per watt for CPUs more than 35W [Tiwari, DAC98] At high-end, 1W of cooling for every 1W of power Cooling Physical solutions Heatsink, thermal paste, fan APM (advanced power management) BIOS manages clock speed Triggered by thermal diodes ACPI (Advanced Configuration and Power Interface) OS manages power for all devices Idle, nap, sleep modes SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 35

36 Power Density (W/ cm 2 ) Power Consumption Trend Source: Patrick Gelsinger, Shenkar Bokar, Intelâ Reason for power trends Smaller feature size Increased compaction High clock frequency More complicated processors Sun s Surface Rocket Nozzle Nuclear Reactor Hot Plate 8086 P Pentium Year Process Transistors Supply voltage Frequency Power nm 0.2B V 1GHz 100W 90nm 1.2B V 3GHz 175W 55nm 7.8B 0.7V 10GHz 300W 3nm 50B 0.5V 30GHz 540W source: Intel (2002) SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 36

37 Power Density Limits Serial Performance Single-core processor Dynamic power = V 2 fc V = supply voltage, f = frequency, C = capacitance Increasing frequency (f) increases supply voltage (V) à Cubic effect Multicore processor Dynamic power = V 2 fc Increasing # of cores increase capacitance (C) linearly More transistors by Moore s law à more cores for higher performance instead or, same performance at reduced power SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 37

38 Power Consumption (watts) Core2Quad 3.0GHz Core2duo 3.0GHz Core2duo 2.4GHz (mobile) Core2duo 1.2GHz (mobile) SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong Source: MIT IAP

39 Evolution of Microprocessors Chip density continues increase 2x every 2 years Clock speed is not # of cores increase Power no longer increases SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 39

Recent Multicore Processors 2H 15: Intel Knight s Landing 60+ cores; 4-way SMT; 16GB (on pkg) 2015: Oracle SPARC M7 32 cores; 8-way fine-grain MT; 64MB cache Fall 14: Intel Haswell 18 cores; 2-way

40 Recent Multicore Processors 2H 15: Intel Knight s Landing 60+ cores; 4-way SMT; 16GB (on pkg) 2015: Oracle SPARC M7 32 cores; 8-way fine-grain MT; 64MB cache Fall 14: Intel Haswell 18 cores; 2-way SMT; 45MB cache June 14: IBM Power8 12 cores; 8-way SMT; 96MB cache Sept 13: SPARC M6 12 cores; 8-way fine-grain MT; 48MB cache May 12: AMD Trinity 4 CPU cores; 384 graphics cores Q2 13: Intel Knight s Corner (coprocessor) 61 cores; 2-way SMT; 16MB cache Feb 12: Blue Gene/Q cores; 4-way SMT; 32MB cache Blue Gene/Q compute chip Credit: The IBM Blue Gene/Q Compute Chip, IEEE Micro 2012 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 40

Simultaneous Multithreading (SMT) Permit several independent threads to issue to multiple functional units each cycle SSE3054: Multicore Systems, Spring

41 Simultaneous Multithreading (SMT) Permit several independent threads to issue to multiple functional units each cycle SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 41 Credit: R. Kalla, B. Sinharoy, J. Tendler, Simultaneous Multi-threading implementation in Power5, HotChips, 2003

Simultaneous Multithreading (SMT) SMT are easily added to superscalar architecture Two set of PCs and registers PC shares I-fetch unit Register rename maps second set of registers

42 Simultaneous Multithreading (SMT) SMT are easily added to superscalar architecture Two set of PCs and registers PC shares I-fetch unit Register rename maps second set of registers Credit: R. Kalla, B. Sinharoy, J. Tendler, Simultaneous Multi-threading implementation in Power5, HotChips, 2003 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 42

43 Large-scale Parallel Hardware Top 7 sites in Top500 supercomputers (Nov. 2014) 148th, 169th, 170th Korea Meteorological administration Credit: top500.org SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 43

Cores/Socket & Cores in Top500 fef Systems 500 450 400 350 300 250 200 150 1 2 3-4 5-8 9-16 17-32 33-64 65-128 129-256 257-512 513-1024 1025-2048 2049-4096 4k-8k 100 8k-16k 50

44 Cores/Socket & Cores in Top500 fef Systems k-8k 100 8k-16k 50 16k-32k 32k-64k 0 64k-128k k- Credit: top500.org, CS267@UC Berkeley by J. Demmel SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 44

45 Blue Gene/Q Packing Hierarchy 1. Chip 16 cores 2. Module Single Chip 3. Compute Card One single chip module, 16 GB DDR3 Memory 4. Node Card 32 Compute Cards, Optical Modules, Link Chips, Torus 5b. I/O Drawer 8 I/O Cards 8 PCIe Gen2 slots 6. Rack 2 Midplanes 1, 2 or 4 I/O Drawers 7. System 20PF/s 5a. Midplane 16 Node Cards Credit: The Blue Gene/Q Compute Chip, HotChips 2011 SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 45

46 Summary: Why Parallel Computing? If you can t exploit parallelism, performance will be a zero sum game If you want to add a new feature and maintain performance, you will need to remove something else. The future is multicore (i.e. parallel) according to all major processor vendors We expect more and more cores will be delivered with each generation Starting now, everyone will need to know how to write parallel programs It is no longer a niche area: it is a mainstream. SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu) 46

47 References Some slides are adopted and adapted from CS194: Parallel Programming by Prof. Katherine Yelick at UC Berkeley CS267: Applications of Parallel Computers by Prof. Jim Demmel at UC Berkeley CS258: Parallel Computer Architecture by Prof. David E. Culler at UC Berkeley COMP422: Parallel Computing by Prof. John Mellor-Crummey at Rice Univ. SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong 47

Why Parallel Computing? Jinkyu Jeong Computer Systems Laboratory Sungkyunkwan University

Why Parallel Computing? Jinkyu Jeong (jinkyu@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu What is Parallel Computing? Software with multiple threads Parallel vs. concurrent