The Many-Core Revolution Understanding Change. Alejandro Cabrera January 29, 2009

Transcription

1 The Many-Core Revolution Understanding Change Alejandro Cabrera January 29, 2009

2 Disclaimer This presentation currently contains several claims requiring proper citations and a few images that may or may not be licensed under the Creative Commons. The dwarf twins later on are CC-compatible. In short, it is not ready for production use. You've been warned.

3 Acknowledgements Berkeley View: The bulk of the presentation is based off of this paper. NVIDIA: Their GPUs and spec-sheets provide some exciting numbers. Google: Image searching made easy. Tilera: Many-core CPU.

4 Overview Exciting Pictures, Exciting Numbers State of the Core Why Many-Core? Common Wisdoms Refuted Dwarves and Applications Programming Many-Core Discussion

5 Many-Core CPU: Tilera Pictures

6 Many-Core CPU: Tilera Numbers Memory Installed 2.5GB DDR2 Processor Clock 700 MHz

7 Many-Core CPU: Tilera Numbers Not many numbers available yet... Memory Installed 2.5GB DDR2 Processor Clock 700 MHz

8 Many-Core GPU: Nvidia GTX 295 Pictures

9 Many-Core GPU: Nvidia GTX 295 Numbers GPU Engine Specs Cores 480 Graphics Clock 576 MHz Processor Clock 1242 MHz Texture Fill Rate 92.2 billion pixels/sec Memory Specs Memory Clock 999 MHz Memory Config 1792 MB GDDR3 Memory Interface Width 896-bit Memory Bandwidth GB/s

10 Many-Core GPU: Nvidia GTX 295 GPU Engine Specs Cores 480 Graphics Clock 576 MHz Processor Clock 1242 MHz Texture Fill Rate 92.2 billion pixels/sec Typical CPU has no more than 4 cores!

11 Many-Core GPU: Nvidia GTX 295 GPU Engine Specs Cores 480 Graphics Clock 576 MHz Processor Clock 1242 MHz Texture Fill Rate 92.2 billion pixels/sec 92.2 billion pixels per second...

12 Many-Core GPU: Nvidia GTX 295 GPU Engine Specs Cores 480 Graphics Clock 576 MHz Processor Clock 1242 MHz Texture Fill Rate 92.2 billion pixels/sec 92.2 billion pixels per second... A high-end monitor has a resolution of 2560 x 1600

13 Many-Core GPU: Nvidia GTX 295 GPU Engine Specs Cores 480 Graphics Clock 576 MHz Processor Clock 1242 MHz Texture Fill Rate 92.2 billion pixels/sec 92.2 billion pixels per second... A high-end monitor has a resolution of 2560 x 1600 That's...

14 Many-Core GPU: Nvidia GTX 295 GPU Engine Specs Cores 480 Graphics Clock 576 MHz Processor Clock 1242 MHz Texture Fill Rate 92.2 billion pixels/sec 92.2 billion pixels per second... A high-end monitor has a resolution of 2560 x 1600 That's million pixels

15 Many-Core GPU: Nvidia GTX 295 GPU Engine Specs Cores 480 Graphics Clock 576 MHz Processor Clock 1242 MHz Texture Fill Rate 92.2 billion pixels/sec 4.1 million << 92.2 billion You could re-draw an entire scene about 22,500 times per second!

16 Many-Core GPU: Nvidia GTX 295 GPU Engine Specs Cores 480 Graphics Clock 576 MHz Processor Clock 1242 MHz Texture Fill Rate 92.2 billion pixels/sec 4.1 million << 92.2 billion You could re-draw an entire scene about 22,500 times per second! (assumes trivial, flat scene)

17 Many-Core GPU: Nvidia GTX 295 Numbers GB/s Memory Specs Memory Clock 999 MHz Memory Config 1792 MB GDDR3 Memory Interface Width 896-bit Memory Bandwidth GB/s

18 Many-Core GPU: Nvidia GTX 295 Numbers GB/s ~1TB / 4 seconds Memory Specs Memory Clock 999 MHz Memory Config 1792 MB GDDR3 Memory Interface Width 896-bit Memory Bandwidth GB/s

19 Many-Core GPU: Nvidia GTX 295 Numbers GB/s ~1TB / 4 seconds Let's picture it. Memory Specs Memory Clock 999 MHz Memory Config 1792 MB GDDR3 Memory Interface Width 896-bit Memory Bandwidth GB/s

20 Many-Core GPU: Nvidia GTX 295 Numbers WARNING The following slides feature assumptions that have no basis in reality. *A disk cannot store data at 223.8GB/s (yet)

21 Many-Core GPU: Nvidia GTX seconds of data processing ~1 TB

22 Many-Core GPU: Nvidia GTX minute of data processing

23 Many-Core GPU: Nvidia GTX hour of data processing 3.5 form factor = 4 x 5.75 x 1 = 23in 3 15 x 60 disks filled in one hour 23in 3 x 15 x 60 = 20,700in 3 = 1,725ft 3 Height of world's tallest building: 1,730ft To fit all the data, would require a cube as wide, long, and tall as Sears (Willis) Tower!

24 State of the Core Where Are We Now? Sequential processors aren't getting any faster No free lunch via Moore's Law voucher Quad-core = commodity Parallel applications few and far between in consumer markets Look to scientific and enterprise computing Multi-process Google Chrome browser Vendors want more performance and they want it yesterday Often a selling point

25 State of the Core Where Are We Now? Graphics processors no longer take the back seat GPGPU (2002) CUDA v1.1 (2007) CUDA v3.0b (2010) Powerful accelerators mingling with CPU IBM Cell

26 Biggest problem: State of the Core Where Are We Now? How do we develop efficient, correct, scalable parallel components? How do we develop highly-parallel applications composed of those components? Components: Data structures Algorithms

27 Why Many-Core? Beyond Exciting Numbers It's not a victory parade towards a bright, new idea. It's a retreat from an even greater challenge. We can't make sequential processors faster without melting them (or exploding our energy bills). We still want to get faster as quickly as possible, so we pursue the most immediate solution towards that end. As a result, many of our conventional wisdoms acquired over previous decades of computing have been overturned.

28 Common Wisdoms (Refuted) Power vs. Transistors Old Wisdom Power is free, but transistors are expensive. New Wisdom Power is expensive, but transistors are free. We can fit as many transistors on a chip as we have power to turn them on!

29 Common Wisdoms (Refuted) Dynamic vs. Static Power Old Wisdom You should only worry about dynamic power (voltage scaling). New Wisdom For desktops and servers, static power leakage can be 40% of total power.

30 Common Wisdoms (Refuted) Hardware Errors Old Wisdom Uniprocessors are reliable internally. New Wisdom With transistor designs falling below 65nm scale, errors occur at quantum level.

31 Common Wisdoms (Refuted) Scaling Designs Old Wisdom Old successes guide future successes, so we need only build upon prior designs. New Wisdom As design size (nanometer) drops, a multitude of factors will stretch development time.

32 Common Wisdoms (Refuted) Architecture Research Old Wisdom Let academia evaluate experimental designs they can build their own chips. New Wisdom Academia can no longer afford tools required to build believable chips.

33 Common Wisdoms (Refuted) Bandwidth vs. Latency Old Wisdom Performance improvements latency drops and bandwidth increases New Wisdom Bandwidth improves exponentially compared to latency (memory wall)

34 Common Wisdoms (Refuted) Computation vs. Memory Access Old Wisdom Store common computations in tables arithmetic is slow. New Wisdom Re-compute needed results data storage is slow.

35 Common Wisdoms (Refuted) Instruction Level Parallelism Old Wisdom There's an abundance of ILP waiting to be found by compilers, architectures designs, VLIW... New Wisdom Diminishing returns on ILP.

36 Common Wisdoms (Refuted) Moore's Law Old Wisdom Performance doubles every 18 months on a uniprocessor. New Wisdom Power Wall + Memory Wall + ILP Wall greater than 60 months to maintain Moore's Law for uniprocessors

37 Common Wisdoms (Refuted) Why Parallelize? Old Wisdom Don't bother parallelizing Moore's Law promises it'll run faster in a couple of years, unmodified. New Wisdom It'll be a long time before an unmodified program gets faster.

38 Common Wisdoms (Refuted) Parallel Performance Value Old Wisdom If it doesn't scale linearly, trash it. New Wisdom Any performance scaling is better than none- it's the only way to get faster now!

39 Common Wisdoms Moore's Law II New Wisdom The number of cores available on a chip doubles every 18 months.

40 Classifying Parallelism Dwarves To better understand parallel applications, a series of areas where parallelism is commonly exploited were analyzed. These areas are dwarves, patterns of communication and computation that identify a category of application.

41 Classifying Parallelism The Seven Dwarves

42 The Seven Dwarves Dense Linear Algebra (BLAS) Sparse Linear Algebra (conjugate gradient) Spectral Methods (FFT, DSP) N-Body simulation Structured Grid (PDE solver) Unstructured Grid Monte Carlo (embarrassingly parallel why?) Independent events

43 The Six (Other) Dwarves Why more dwarves? Up and coming algorithmic techniques and application domains require parallelization Observed domains: Combinatorial logic (SHA, MD5, AES, cryptography) Graph traversal (BFS, A*, maximum network flow) Finite state machines Bayesian networks, Hidden Markov Models Machine learning Dynamic programming Backtrack, branch-and-bound

44 The Six (Other) Dwarves In particular, no approach is currently known for parallel evaluation of a finite state machine. How can a system be in multiple states at once? Thought to be embarrassingly sequential. This brings us to our crux how do we develop new parallel applications?

45 Programming Many-Core We have exciting equipment. GPUs, CPUs, accelerators... We have clear applications areas. Linear algebra, graph traversal, dynamic programming... How do we use those exciting machines to satisfy those application requirements?

46 Programming Many-Core #include <???> import??? from

47 Programming Many-Core #include <???> import??? from How did I get here...?

48 Programming Many-Core #include <???> import??? from and who is that behind me?

49 Parallel Programming Models There are many (more) details that may be necessary to manage in order to produce an efficient parallel application versus a sequential application. Programming models seek to simplify the management of one or more of the following: Task identification Task mapping Data distribution Communication mapping Synchronization

50 MPI Parallel Programming Models A Few Existing Examples Pthreads MapReduce OpenMP CUDA OpenCL

51 Parallel Programming Models A Few Existing Examples Model Task ID Task Mapping Data Distrib. Comm. Mapping Sync MPI explicit explicit explicit implicit implicit Pthreads explicit explicit implicit implicit explicit MapReduce explicit implicit implicit implicit explicit OpenMP implicit implicit implicit implicit implicit CUDA explicit explicit explicit implicit explicit OpenCL N/A N/A N/A N/A N/A

52 Parallel Programming Models Message passing: Pros: Harder to make mistakes Much upfront planning Highly verifiable model Cons: Difficult to learn Hardware not yet thought of widely as networked components Shared Memory Pros: Friendly learning curve Matches hardware layout Cons: May not scale Cache coherence... Easy to make mistakes

53 Looking to the Many-Core Future We need: Better parallel programming models Better compilers A set of primitives to build APIs from Thoroughly tested parallel components Libraries Tools

54 Looking to the Many-Core Future Many challenges await. However, thanks to the revised Moore's Law, we have much to look forward to. Now, more than ever before, we'll be able to make great advances in sciences depending on extensive computation. All we have to do is add another core.

55

56 We'll yet get to the core of this issue.