Parallel Programming (Part 2)

Transcription

1 Parallel Programming (Part 2)

2 Dining Philosophers as Scheduling Tasks Scheduling the eating schedule for philosophers can be thought of as scheduling CPU cores to perform tasks. Dining Philosophers Problem: Input: n philosophers arriving to a dinner party and the times that they arrive. Output: A schedule of philosophers eating/thinking so that all eat as much as possible. In order to eat, a philosopher must hold the chopstick on each side.

3 Difficult Problems to Parallelize Think: What kinds of problems would be difficult to parallelize? 1. Compute n-th Fibonacci number F n (depends on computing F n-1 and F n-2 ). 2. Given initial configuration of a Game of Life Board, a cell, and an integer T, is the cell alive after T steps? (Depends on computing the board at T-1 steps, which depends on T-2 steps, etc.)

4 Example of Partial Parallelization Think: How could we parallelize the problem of updating a GoL board after a single iteration?

5 Parallelizing One Game of Life Iteration 1 processor

6 Parallelizing One Game of Life Iteration 2 processors

7 Parallelizing One Game of Life Iteration 4 processors

8 Parallelizing One Game of Life Iteration 16 processors

9 Parallelizing One Game of Life Iteration Think: What is the difficulty of this approach?

10 Parallelizing One Game of Life Iteration Think: What is the difficulty of this approach? Answer: Cores must communicate across the boundaries.

11 Parallelizing One Game of Life Iteration Think: What is the difficulty of this approach? Answer: Cores must communicate across the boundaries. The more cores, the more boundary cells, which can slow us down.

12 Quantifying Limits of Parallel Programming Exercise: If we have 4 cores, and 40% of our algorithm can be parallelized, how much quicker will a parallel algorithm run than a serial algorithm?

13 Quantifying Limits of Parallel Programming Exercise: If we have 4 cores, and 40% of our algorithm can be parallelized, how much quicker will a parallel algorithm run than a serial algorithm? Answer: 0.4/ = 0.7, so it is 30% faster.

14 Quantifying Limits of Parallel Programming Exercise: If we have N cores, and p percent of our algorithm can be parallelized, how much quicker will a parallel algorithm run than a serial algorithm?

15 Quantifying Limits of Parallel Programming Exercise: If we have N cores, and p percent of our algorithm can be parallelized, how much quicker will a parallel algorithm run than a serial algorithm? Answer: It will run in fraction p/n + (1-p) time compared to the serial algorithm. This is called Amdahl s law.

16 Quantifying Limits of Parallel Programming Exercise: If we have 3 million cores, and 99% of our algorithm can be parallelized, how much quicker will a parallel algorithm run than a serial algorithm?

17 Quantifying Limits of Parallel Programming Exercise: If we have 3 million cores, and 99% of our algorithm can be parallelized, how much quicker will a parallel algorithm run than a serial algorithm? Answer: 0.99/(3 million) ~ So the algorithm is only about 100 times faster.

18 Quantifying Limits of Parallel Programming Exercise: If we have 3 million cores, and 99% of our algorithm can be parallelized, how much quicker will a parallel algorithm run than a serial algorithm? Answer: 0.99/(3 million) ~ So the algorithm is only about 100 times faster. General Rule: as N à, p/n + (1-p) à 1-p

19 You ve Learned to Think Serially about Go Think: What is the flow of the following program? func f(n int) { for i := 0; i < 10; i++ { fmt.println(n, ":", i) func main() { f(0)

20 Goroutines Represent Parallelism in Go Goroutine: When go f(0) is called, f(0) is called in the background and the program continues running. func f(n int) { for i := 0; i < 10; i++ { fmt.println(n, ":", i) func main() { go f(0)

21 Goroutines Represent Parallelism in Go Goroutine: When go f(0) is called, f(0) is called in the background and the program continues running. func f(n int) { for i := 0; i < 10; i++ { fmt.println(n, ":", i) func main() { go f(0) Think: What do you think will be printed?

22 Goroutines Represent Parallelism in Go Goroutine: When go f(0) is called, f(0) is called in the background and the program continues running. func f(n int) { for i := 0; i < 10; i++ { fmt.println(n, ":", i) func main() { go f(0) Answer: Nothing! Program exits before it has any time...

23 Goroutines Represent Parallelism in Go We add a Scanln() line so the program doesn t quit early. func f(n int) { for i := 0; i < 10; i++ { fmt.println(n, ":", i) func main() { go f(0) var input string fmt.scanln(&input)

24 Running Multiple Goroutines func f(n int) { for i := 0; i < 10; i++ { fmt.println(n, ":", i) func main() { for i := 0; i < 10; i++ { go f(i) var input string fmt.scanln(&input) Think: What do you think will be printed?

25 Goroutines Really are Running Concurrently! func f(n int) { for i := 0; i < 10; i++ { fmt.println(n, ":", i) time.sleep(time.millisecond * 100) func main() { for i := 0; i < 10; i++ { go f(i) var input string fmt.scanln(&input) Now we can see that calls to f() are running concurrently.

26 Timing Programs Think: What do you think the speedup is? for i := 0; i < 10; i++ { go f(i) var input string fmt.scanln(&input) for i := 0; i < 10; i++ { f(i) var input string fmt.scanln(&input)

27 Timing Programs Think: What do you think the speedup is? for i := 0; i < 10; i++ { go f(i) var input string fmt.scanln(&input) for i := 0; i < 10; i++ { f(i) var input string fmt.scanln(&input) One way of timing functions is with the following code. start := time.now() // insert code you want to time here... elapsed := time.since(start) log.printf("g took %s", elapsed)

28 Recall: Peculiar Stats of #G - #C Forward half-strand (single-stranded life): shortage of C, normal G Reverse half-strand (double-stranded life): shortage of G, normal C #C #G #G - #C Reverse half-strand Forward half-strand Difference

29 Recall: #G - #C is DECREASING #G - #C is INCREASING ori 5 C high G low You walk along the genome and see that #G - #C has been decreasing and then suddenly starts increasing. Where are you in the genome? C low G high terc C high/g low #G - #C is DECREASING as we walk along the REVERSE half-strand C low/g high #G - #C is INCREASING as we walk along the FORWARD half-strand

30 Defining the Skew Array Skew[k]: #G - #C for the first k nucleotides of Genome. Skew diagram: Plot Skew[k] against k 2 1 skew position C ATGGGC ATC GGCCATAC G CC

31 Skew Array of E. coli Identifies ori ori! You walk along the genome and see that #G - #C have been decreasing and then suddenly starts increasing. Where are you in the genome?

32 Generating Skew Arrays for Many Genomes genomefilenames := []string{"bacillus_anthracis.txt", "chlamydia_muridarum.txt", //etc. skews := make(map[string][]int) for i := range genomefilenames { currentname := genomefilenames[i] currentgenome := go readfasta(currentname) skews[currentname] = go SkewArray(currentGenome)

33 Generating Skew Arrays for Many Genomes genomefilenames := []string{"bacillus_anthracis.txt", "chlamydia_muridarum.txt", //etc. skews := make(map[string][]int) for i := range genomefilenames { currentname := genomefilenames[i] currentgenome := go readfasta(currentname) skews[currentname] = go SkewArray(currentGenome) Error! A goroutine can t return a value (when functions are run concurrently, return values may be overwritten).

34 Channels Allow Goroutines to Communicate func pinger(c chan string) { for i := 0; ; i++ { c <- "ping" func printer(c chan string) { for { msg := <- c fmt.println(msg) time.sleep(time.second) func main() { var c chan string = make(chan string) go pinger(c) go printer(c) var input string fmt.scanln(&input)

35 Adding a Ponger to Illustrate Blocking Blocking: When pinger() tries to send a message, it waits until printer() is ready to receive it. func ponger(c chan string) { for i := 0; ; i++ { c <- "pong" func main() { var c chan string = make(chan string) go pinger(c) go ponger(c) go printer(c) var input string fmt.scanln(&input) Think: What do you think will be printed?

36 Returning to Skew Arrays Next Time: Using channels to generate multiple skew arrays.