Parallel Computer Architecture Concepts

Size: px

Start display at page:

Download "Parallel Computer Architecture Concepts"

August Bates
5 years ago
Views:

1 Outline This image cannot currently be displayed. arallel Computer Architecture Concepts TDDD93 Lecture 1 Christoph Kessler ELAB / IDA Linköping university Sweden 2015 Lecture 1: arallel Computer Architecture Concepts arallel computer, multiprocessor, multicomputer SIMD vs. MIMD execution Shared memory vs. Distributed memory architecture Interconnection networks arallel architecture design concepts Instruction-level parallelism Hardware multithreading Multi-core and many-core Accelerators and heterogeneous systems Clusters Implications for programming and algorithm design 2 arallel Computer Traditional Use of arallel Computing: in HC 3 4 Example: Weather Forecast (very simplified ) arallel Computing for HC Applications 3D Space discretization (cells) Time discretization (steps) Start from current observations (sent from weather stations) Simulation step by evaluating weather model equations cell Air pressure Temperature Humidity Sun radiation Wind direction Wind velocity High erformance Computing Much computational work (in FLOs, floatingpoint operations) Often, large data sets E.g. climate simulations, particle physics, engineering, sequence matching or proteine docking in bioinformatics, Single-CU computers and even today s multicore processors cannot provide such massive computation power Aggregate LOTS of computers Clusters Need scalable algorithms Need to exploit multiple levels of parallelism NSC Triolith 5 6 1

Shared memory by interconnection network topology op op 1 op 2 op op 3 4 7 8 Classification by Memory Organization

2 arallel Computer Architecture Concepts Classification by Control Structure Classification of parallel computer architectures: by control structure by memory organization in particular, Distributed memory vs. Shared memory by interconnection network topology op op 1 op 2 op op Classification by Memory Organization Interconnection Networks (1) R 9 10 Interconnection Networks (2): Simple Topologies fully connected Interconnection Networks (3): Hypercube Inductive definition:

More about Interconnection Networks Fat-Tree, Butterfly, See TDDC78 Instruction Level arallelism (1): ipelined Execution Units Switching and routing algorithms Discussion of interconnection network

, vector supercomputers Cray (1970s, 1980s), Fujitsu, Instruction-Level arallelism (2): VLIW and Superscalar Multiple functional units in parallel 2 main paradigms: VLIW (very large instruction word)

/compiler-managed (hard) Superscalar architecture 16 Sequential instruction stream Hardware-managed dispatch power + area overhead IL in applications is limited typ. < 3.

3 More about Interconnection Networks Fat-Tree, Butterfly, See TDDC78 Instruction Level arallelism (1): ipelined Execution Units Switching and routing algorithms Discussion of interconnection network properties Cost (#switches, #lines) Scalability (asymptotically, cost grows not much faster than #nodes) Node degree Longest path ( latency) Accumulated bandwidth Fault tolerance (node or switch failure) SIMD computing with ipelined Vector Units 15 e.g., vector supercomputers Cray (1970s, 1980s), Fujitsu, Instruction-Level arallelism (2): VLIW and Superscalar Multiple functional units in parallel 2 main paradigms: VLIW (very large instruction word) architecture ^ arallelism is explicit, progr./compiler-managed (hard) Superscalar architecture 16 Sequential instruction stream Hardware-managed dispatch power + area overhead IL in applications is limited typ. < instructions can be issued simultaneously Due to control and data dependences in applications Solution: Multithread the application and the processor Hardware Multithreading SIMD Instructions vector register E.g., data dependence 17 Single Instruction stream, Multiple Data streams single thread of control flow op SIMD unit restricted form of data parallelism apply the same primitive operation (a single instruction) in parallel to multiple data elements stored contiguously SIMD units use long vector registers each holding multiple data elements Common today MMX, SSE, SSE2, SSE3, Altivec, VMX, SU, erformance boost for operations on shorter data types Area- and energy-efficient Code to be rewritten (SIMDized) by programmer or compiler Does not help (much) for memory bandwidth 18 3

The Memory Wall erformance gap CU Memory Memory hierarchy Increasing cache sizes shows diminishing returns Costs power and chip area GUs spend the area instead on many simple cores with little memory

.. Solution: Spread out / overlap memory access delay rogrammer/compiler: refetching, on-chip pipelining, SW-managed on-chip buffers Generally: Hardware multithreading, again!

4 The Memory Wall erformance gap CU Memory Memory hierarchy Increasing cache sizes shows diminishing returns Costs power and chip area GUs spend the area instead on many simple cores with little memory Relies on good data locality in the application What if there is no / little data locality? Irregular applications, e.g. sorting, searching, optimization... Solution: Spread out / overlap memory access delay rogrammer/compiler: refetching, on-chip pipelining, SW-managed on-chip buffers Generally: Hardware multithreading, again! 19 Moore s Law (since 1965) Exponential increase in transistor density Data from Kunle Olukotun, Lance Hammond, Herb Sutter, 20Burton Smith, Chris Batten, and Krste Asanoviç The ower Issue ower = Static (leakage) power + Dynamic (switching) power Dynamic power ~ Voltage 2 * Clock frequency where Clock frequency approx. ~ voltage Dynamic power ~ Frequency 3 Total power ~ #processors Moore s Law vs. Clock Frequency ~3GHz #Transistors / mm 2 still growing exponentially according to Moore s Law Clock speed flattening out More transistors + Limited frequency More cores Conclusion: Moore s Law Continues, But... Single-processor erformance Scaling 16,0 Log2 Speedup 14,0 12,0 10,0 8,0 6,0 Throughput incr. 55%/year arallelism Assumed increase 17%/year possible Device speed Limit: Clock rate 4,0 ipelining Data from Kunle Olukotun, Lance Hammond, Herb Sutter, Burton Smith, Chris Batten, and Krste Asanoviç 23 2,0 0,0 Source: Doug Burger, UT Austin 2005 Limit: RISC IL RISC/CISC CI 90 nm 65 nm 45 nm 32nm 22nm 24 4

5 Solution for CU Design: Multicore + Multithreading Single-thread performance does not improve any more IL wall Memory wall ower wall but we can put more cores on a chip And hardware-multithread the cores to hide memory latency All major chip manufacturers produce multicore CUs today Main features of a multicore system There are multiple computational cores on the same chip. The cores might have (small) private on-chip memory modules and/or access to on-chip memory shared by several cores. The cores have access to a common off-chip main memory There is a way by which these cores communicate with each other and/or with the environment Standard CU Multicore Designs Some early dual-core CUs (2004/2005) Standard desktop/server CUs have a few cores with shared off-chip main memory core core core core On-chip cache (typ., 2 levels) L1-cache mostly core-private Interconnect / Memory interface L2-cache often shared by groups of cores main memory (DRAM) Memory access interface shared by all or groups of cores Caching multiple copies of the same data item Writing to one copy (only) causes inconsistency L1$ L1$ L1$ L1$ Shared memory coherence mechanism to enforce automatic updating or invalidation of all copies around 0 SMT L1$ D1$ L1$ 1 Memory Ctrl Main memory IBM ower5 (2004) D1$ 0 1 Memory Ctrl Main memory AMD Opteron Dualcore (2005) 0 1 Memory Ctrl Main memory Intel Xeon Dualcore(2005) $ = cache L1$ = level-1 instruction cache D1$ = level-1 data cache = level-2 cache (uniform) More about shared-memory architecture, caches, data locality, consistency issues and coherence protocols in TDDC78/TDDD SUN/Oracle SARC T Niagara (8 cores) SUN / Oracle SARC-T5 (2012) Sun UltraSARC Niagara Niagara T1 (2005): 8 cores, 32 HW threads Niagara T2 (2008): 8 cores, 64 HW threads Niagara T3 (2010): 16 cores, 128 HW threads T5 (2012): 16 cores, 128 HW threads Niagara T1 (2005): 8 cores, 32 HW threads Memory Ctrl Memory Ctrl Memory Ctrl Memory Ctrl Main memory Main memory Main memory Main memory 29 28nm process, 16 cores x 8 HW threads, L3 cache on-chip, On-die accelerators for common encryption algorithms 30 5

Scaling Up: Network-On-Chip Cache-coherent shared memory (hardware-controlled) does not scale well to many cores power- and area-hungry signal latency across whole chip not well predictable access

but software responsible for maintaining coherence Ex

6 Scaling Up: Network-On-Chip Cache-coherent shared memory (hardware-controlled) does not scale well to many cores power- and area-hungry signal latency across whole chip not well predictable access times Idea: NCC-NUMA non-cache-coherent, non-uniform memory access hysically distributed on-chip [cache] memory, on-chip network, connecting Es or coherent tiles of Es global shared address space, but software responsible for maintaining coherence Examples: STI Cell/B.E., Tilera TILE64, Example: Cell/B.E. (IBM/Sony/Toshiba 2006) An on-chip network (four parallel unidirectional rings) interconnect the master core, the slave cores and the main memory interface LS = local on-chip memory, E = master, SE = slave Intel SCC, Kalray MA Towards Many-Core CUs... Intel, second-generation many-core research processor: 48-core (in-order x86) SCC Single-chip cloud computer, 2010 No longer fully cache coherent over the entire chip MI-like message passing over 2D mesh network on chip Towards Many-Core Architectures Tilera TILE64 (2007): 64 cores, 8x8 2D-mesh on-chip network Mem-controller C I/O I/O R 1 tile: VLIW-processor + cache + router (Image simplified) 33 Source: Intel 34 Kalray MA tiles with 16 VLIW compute cores each plus 1 control core per tile Message passing network on chip Virtually unlimited array extension by clustering several chips 28 nm CMOS technology Low power dissipation, typ. 5 W Intel Xeon HI (since late 2012) Up to 61 cores, 244 HW threads, 1.2 Tflops peak performance Simpler x86 (entium) cores (x 4 HW threads), with 512 bit wide SIMD vector registers Can also be used as a coprocessor, instead of a GU

General-purpose GUs Main GU providers for laptop/desktop Nvidia, AMD(ATI), Intel Example: NVIDIA s 10-series GU (Tesla, 2008) has 240 cores Each core has a Floating point / integer unit Logic unit

Nvidia Fermi (2010): 512 cores 1 Fermi C2050 GU SM 1 Streaming rocessor (S) L2 FU 1 shared-memory multiprocessor (SM) IntU 38 I-cache Scheduler Dispatch Register file 32 Streaming processors (cores)

7 General-purpose GUs Main GU providers for laptop/desktop Nvidia, AMD(ATI), Intel Example: NVIDIA s 10-series GU (Tesla, 2008) has 240 cores Each core has a Floating point / integer unit Logic unit Move, compare unit Branch unit Cores managed by thread manager Thread manager can spawn and manage 30,000+ threads Zero overhead thread switching 37 (Images removed) Nvidia Tesla C1060: 933 GFlops Nvidia Fermi (2010): 512 cores 1 Fermi C2050 GU SM 1 Streaming rocessor (S) L2 FU 1 shared-memory multiprocessor (SM) IntU 38 I-cache Scheduler Dispatch Register file 32 Streaming processors (cores) Load/Store units Special function units 64K configurable L1cache/ shared memory GU Architecture aradigm The future will be heterogeneous! Optimized for high throughput In theory, ~10x to ~100x higher throughput than CU is possible Massive hardware-multithreading hides memory access latency Massive parallelism GUs are good at data-parallel computations multiple threads executing the same instruction on different data, preferably located adjacently in memory 39 Need 2 kinds of cores often on same chip: For non-parallelizable code: arallelism only from running several serial applications simultaneously on different cores (e.g. on desktop: word processor, , virus scanner, not much more) Few (ca. 4-8) fat cores (power-hungry, area-costly, caches, out-of-order issue, ) for high single-thread performance For well-parallelizable code: (or on cloud servers) on hundreds of simple cores (power + area efficient) (GU-/SCC-like) 40 Heterogeneous / Hybrid Multi-/Manycore Heterogeneous / Hybrid Multi-/Manycore Systems Key concept: Master-slave parallelism, offloading General-purpose CU (master) processor controls execution of slave processors by submitting tasks to them and transfering operand data to the slaves local memory Master offloads computation to the slaves Slaves often optimized for heavy throughput computing Master could do something else while waiting for the result, or switch to a power-saving mode Master and slave cores might reside on the same chip (e.g., Cell/B.E.) or on different chips (e.g., most GU-based systems today) Slaves might have access to off-chip main memory (e.g., Cell) or not (e.g., today s GUs) 41 Cell/B.E. GU-based system: CU Main memory Offload heavy computation Data transfer 42 Device memory GU 7

Multi-GU Systems Connect one or few general-purpose (CU) multicore processors with shared off-chip memory to several GUs Increasingly popular in high-performance computing Cost and (quite) energy

off-the-shelf CUs (Xeon, Opteron, ) Cluster Example: Triolith (NSC, 2012 / 2013) Capability cluster (fast network for parallel applications) Final configuration: 1200 H SL230 compute nodes, each

8 Multi-GU Systems Connect one or few general-purpose (CU) multicore processors with shared off-chip memory to several GUs Increasingly popular in high-performance computing Cost and (quite) energy effective if offloaded computation fits GU architecture well Reconfigurable Computing Units FGA Field rogrammable Gate Array L2 L2 Main Memory (DRAM) Example: Beowulf-class C Clusters with off-the-shelf CUs (Xeon, Opteron, ) Cluster Example: Triolith (NSC, 2012 / 2013) Capability cluster (fast network for parallel applications) Final configuration: 1200 H SL230 compute nodes, each equipped with 2 Intel E (2.2 GHz Sandybridge) processors with 8 cores each cores in total Theoretical peak performance of 338 Tflops/s Mellanox Infiniband network NSC Triolith (Source: NSC) The Challenge Today, basically all computersare parallel computers! Single-thread performance stagnating Dozens, hundreds of cores Hundreds, thousands of hardware threads Heterogeneous (core types, accelerators) Data locality matters Clusters for HC, require message passing Utilizing more than one CU core requires thread-level parallelism One of the biggest software challenges: Exploiting parallelism Every programmer will eventually have to deal with it All application areas, not only traditional HC General-purpose, graphics, games, embedded, DS, Affects HW/SW system architecture, programming languages, algorithms, data structures arallel programming is more error-prone (deadlocks, races, further sources of inefficiencies) And thus more expensive and time-consuming 47 Can t the compiler fix it for us? Automatic parallelization? at compile time: Requires static analysis not effective for pointer-based languages needs programmer hints / rewriting... ok for few benign special cases: (Fortran) loop SIMDization, extraction of instruction-level parallelism, at run time (e.g. speculative multithreading) High overheads, not scalable More about parallelizing compilers in TDDD56 + TDDC

9 And worse yet, A lot of variations/choices in hardware Many will have performance implications No standard parallel programming model portability issue Understanding the hardware will make it easier to make programs get high performance erformance-aware programming gets more important also for single-threaded code Adaptation leads to portability issue again How to write future-proof parallel programs? The Challenge Bad news 1: Many programmers (also less skilled ones) need to use parallel programming in the future Bad news 2: There will be no single uniform parallel programming model as we were used to in the old sequential times Several competing general-purpose and domain-specific languages and their concepts will co-exist What we learned in the past Sequential von-neumann model programming, algorithms, data structures, complexity Sequential / few-threaded languages: C/C++, Java,... not designed for exploiting massive parallelism time T(n) = O ( n log n ) and what we need now arallel programming! arallel algorithms and data structures Analysis / cost model: parallel time, work, cost; scalability; erformance-awareness: data locality, load balancing, communication time number of processing units used T(n,p) = O ( (n log n)/p + log p ) 51 problem size 52 problem size This image cannot currently be displayed. Questions? 9

Parallel Computer Architecture Concepts

Outline Parallel Computer Architecture Concepts TDDD93 Lecture 1 Christoph Kessler PELAB / IDA Linköping university Sweden 2017 Lecture 1: Parallel Computer Architecture Concepts Parallel computer, multiprocessor,