Test on Wednesday! Material covered since Monday, Feb 8 (no Linux, Git, C, MD, or compiling programs)

Transcription

1 Test on Wednesday! 50 minutes Closed notes, closed computer, closed everything Material covered since Monday, Feb 8 (no Linux, Git, C, MD, or compiling programs) Study notes and readings posted on course website (e.g. LLNL documentation) Two programming problems: One MPI One OpenMP Do NOT need to memorize arguments and order of arguments for MPI or OpenMP, these will be provided in an Appendix

2 Class this Friday In-depth discussion of homework 3 Review of homework 2

3 SC 3260/5260 Jeopardy!

4 Moore s Law predicts that the number of these devices on a chip double roughly every 18 months.

5 Source: Wikipedia

6 The development of multi-core processors was spurred primarily by this problem.

7

8 The central processing unit is comprised of these two main units.

9 The von Neumann Architecture (CPUs)

10 Flynn s taxonomy defines these four computing paradigms.

11 Flynn s taxonomy SISD Single instruction stream Single data stream (SIMD) Single instruction stream Multiple data stream MISD Multiple instruction stream Single data stream (MIMD) Multiple instruction stream Multiple data stream

12 This is an example of compiler-aided parallelization in which multiple loop iterations are performed on a single clock cycle.

13 These are defined as the smallest unit of programmed instruction that can be scheduled by a computer s operating system.

14

15 These enable a programmer to access and process shared memory space in a parallel fashion.

16 Captain Jean Luc Picard of the Enterprise was abducted and assimilated by this life form in Star Trek Next Generation.

17

18 These are units of programmed instruction that exist in an entirely private memory space.

19 For data transfers, this is the time that passes between a request being made and the first byte of data arriving at its destination.

20 For data transfers, this is the rate at which the destination receives data after it has received the first byte.

21 Message transmission time = l + n / b! latency (seconds) length of message (bytes) bandwidth (bytes per second)

22 This law states that the speedup of a parallel algorithm is limited by the fraction of the program that cannot be parallelized.

23 Speedup = S Number of cores = p Serial run-time = T serial Parallel run-time = T parallel

24 Amdahl s Law The possible speedup of a program is limited by the fraction that cannot be parallelized, regardless of the number of cores available. For example, if 50% of a program is serial, the max speedup is 2. p is the number of CPU cores T is the execution time B is the serial fraction of the program p p

25 This is the ability of a program to continue running efficiently as the number of processes/threads is increased.

26 Efficiency of a parallel program

27 Multi-threaded programs make use of this mechanism for limiting access to a section of code to a single thread.

28 This type of bug in multi-threaded programs occurs when the correctness of the program depends on the order of thread execution.

29 This type of bug in multi-threaded programs occurs when a thread is waiting for a signal that will never come.

30 This programming standard provides an API for running programs across multiple computers.

31 Distributed Memory Programming Model

32 This type of MPI communication is used for sending data from a single process to another single process.

33 Message Matching Process rank MPI_COMM_ WORLD Process rank

34 MPI_send() generally blocks until this occurs.

35 Buffering

36 Creating a distributed array out of a serial one can be accomplished with this MPI function.

37 MPI_Gather, MPI_Scatter data data processes A 0 A 1 A 2 A 3 A 4 A 5 Scatter Gather processes A 0 A 1 A 2 A 3 A 4 A 5

38 Compared to a CPU, a GPU has a higher proportion of transistors devoted to this processor unit.

39

40 In CUDA, threads within what programming element have access to a shared memory space and can synchronize their execution?

41 Many blocks of threads... SMEM SMEM SMEM SMEM Global Memory

42 Lieutenant Worf belongs to this species.

43 Good luck, humans! You must fight the exam with great honor.