Introduction to Multiprocessor Synchronization
|
|
|
- Kellie Osborne
- 5 years ago
- Views:
Transcription
1 Introduction to Multiprocessor Synchronization Maurice Herlihy
2 Moore's Law Transistor count still rising Clock speed flattening sharply Art of Multiprocessor Programming 2
3 Once roamed the Earth: the Uniprocesor cpu memory Art of Multiprocessor Programming 3
4 Endangered: The Shared Memory Multiprocessor (SMP) cache cache Bus cache Bus shared memory Art of Multiprocessor Programming 4
5 Meet he New Boss: The Multicore Processor (CMP) All on the same chip cache cache cache Bus Bus shared memory Oracle Niagara Chip Art of Multiprocessor Programming 5
6 CDS.IISc.ac.in Department of Computational and Data Sciences Turing Cluster 24 Compute Nodes in two 12 node 3U blades. Each node has one 8-core AMD Opteron GHz, 32GB RAM, 2TB HDD, Gigabit Ethernet port 1 Head Node with one 6-core Intel Xeon E v3 2.40GHz, 48GB RAM, 1+4TB HDD, Gigabit Ethernet ports One 24 port L2 Gigabit Ethernet switch Running CentOS, MPI, PBS and Apache Hadoop/Yarn Mounted on a 24U Rack Art of Multiprocessor Programming
7 CDS.IISc.ac.in Department of Computational and Data Sciences Turing Cluster: Xeon E v3
8 Traditional Scaling Process Speedup 1.8x 3.6x 7x User code Traditional Uniprocessor Time: Moore's law Art of Multiprocessor Programming 9
9 Ideal Multicore Scaling Process Speedup 1.8x 3.6x 7x User code Multicore Unfortunately, not so simple Art of Multiprocessor Programming 10
10 Actual Multicore Scaling Process Speedup 1.8x 2x 2.9x User code Multicore Parallelization and Synchronization require great care Art of Multiprocessor Programming 11
11 Sequential Computation thread memory object object Art of Multiprocessor Programming 12
12 Concurrent Computation memory object object Art of Multiprocessor Programming 13
13 Asynchrony Sudden unpredictable delays Cache misses (short) Page faults (long) Scheduling quantum used up (really long) Art of Multiprocessor Programming 14
14 Model Summary Multiple threads Single shared memory Objects live in memory Unpredictable asynchronous delays Art of Multiprocessor Programming 15
15 Concurrency Jargon Hardware Processors Software Threads, processes Sometimes OK to confuse them, sometimes not. Art of Multiprocessor Programming 17
16 Parallel Primality Testing Challenge Print primes from 1 to Given Ten-processor multiprocessor One thread per processor Goal Get ten-fold speedup (or close) Art of Multiprocessor Programming 18
17 Load Balancing P 0 P 1 P 9 Split the work evenly Each thread tests range of 10 9 Art of Multiprocessor Programming 19
18 Procedure for Thread i void primeprint { int i = ThreadID.get(); // IDs in {0..9} for (j = i* , j<(i+1)*10 9 ; j++) { if (isprime(j)) print(j); } } Art of Multiprocessor Programming 20
19 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads Uneven Hard to predict Art of Multiprocessor Programming 21
20 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads Uneven Hard to predict Need dynamic load balancing Art of Multiprocessor Programming 22
21 Shared Counter each thread takes a number 17 Art of Multiprocessor Programming 23
22 Procedure for Thread i int counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getandincrement(); if (isprime(j)) print(j); } } Art of Multiprocessor Programming 24
23 Procedure for Thread i Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getandincrement(); } } if (isprime(j)) print(j); Shared counter object Art of Multiprocessor Programming 25
24 Where Things Reside void primeprint { int i = ThreadID.get(); // IDs in {0..9} for (j = i* , j<(i+1)*10 9 ; j++) { if (isprime(j)) print(j); } } code Local variables cache cache Bus cache Bus 1 shared memory shared counter Art of Multiprocessor Programming 26
25 Procedure for Thread i Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getandincrement(); if (isprime(j)) print(j); } } Stop when every value taken Art of Multiprocessor Programming 27
26 Procedure for Thread i Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getandincrement(); if (isprime(j)) print(j); } } Increment & return each new value Art of Multiprocessor Programming 28
27 Counter Implementation public class Counter { private long value; } public long getandincrement() { return value++; } Art of Multiprocessor Programming 29
28 Counter Implementation public class Counter { private long value; } public long getandincrement() { return value++; } Art of Multiprocessor Programming 30
29 What It Means public class Counter { private long value; } public long getandincrement() { return value++; } Art of Multiprocessor Programming 31
30 What It Means public class Counter { private long value; } public long getandincrement() { return value++; temp = value; } value = temp + 1; return temp; Art of Multiprocessor Programming 32
31 Not so good Value read 1 write 2 read 2 write 3 read 1 write 2 time Art of Multiprocessor Programming 33
32 Challenge public class Counter { private long value; } public long getandincrement() { temp = value; value = temp + 1; return temp; } Art of Multiprocessor Programming 35
33 Challenge public class Counter { private long value; } public long getandincrement() { temp = value; value = temp + 1; return temp; } Make these steps atomic (indivisible) Art of Multiprocessor Programming 36
34 Hardware Solution public class Counter { private long value; public long getandincrement() { temp = value; value = temp + 1; return temp; } } ReadModifyWrite() instruction Art of Multiprocessor Programming 37
35 An Aside: Java public class Counter { private long value; } public long getandincrement() { synchronized { temp = value; value = temp + 1; } return temp; } Art of Multiprocessor Programming 38
36 An Aside: Java public class Counter { private long value; } public long getandincrement() { synchronized { temp = value; value = temp + 1; } return temp; } Synchronized block Art of Multiprocessor Programming 39
37 An Aside: Java public class Counter { private long value; } Mutual Exclusion public long getandincrement() { synchronized { temp = value; value = temp + 1; } return temp; } Art of Multiprocessor Programming 40
38 Mutual Exclusion, or Alice & Bob share a pond A B Art of Multiprocessor Programming 41
39 Alice has a pet A B Art of Multiprocessor Programming 42
40 Bob has a pet A B Art of Multiprocessor Programming 43
41 The Problem A B The pets don't get along Art of Multiprocessor Programming 44
42 Formalizing the Problem Two types of formal properties in asynchronous computation: Safety Properties Nothing bad happens ever Liveness Properties Something good happens eventually Art of Multiprocessor Programming 45
43 Formalizing our Problem Mutual Exclusion Both pets never in pond simultaneously This is a safety property No Deadlock if only one wants in, it gets in if both want in, one gets in This is a liveness property Art of Multiprocessor Programming 46
44 Simple Protocol Idea Just look at the pond Gotcha Not atomic Trees obscure the view Art of Multiprocessor Programming 47
45 Interpretation Threads can't see what other threads are doing Explicit communication required for coordination Art of Multiprocessor Programming 48
46 Cell Phone Protocol Idea Bob calls Alice (or vice-versa) Gotcha Bob takes shower Alice recharges battery Bob out shopping for pet food Art of Multiprocessor Programming 49
47 Interpretation Message-passing doesn't work Recipient might not be Listening There at all Communication must be Persistent (like writing) Not transient (like speaking) Art of Multiprocessor Programming 50
48 Flag Protocol A B Art of Multiprocessor Programming 56
49 Alice's Protocol (sort of) A B Art of Multiprocessor Programming 57
50 Bob's Protocol (sort of) A B Art of Multiprocessor Programming 58
51 Alice's Protocol Raise flag Wait until Bob's flag is down Unleash pet Lower flag when pet returns Art of Multiprocessor Programming 59
52 Bob's Protocol Raise flag Wait until Alice's flag is down Unleash pet Lower flag when pet returns Art of Multiprocessor Programming 60
53 Bob's Protocol (2 nd try) Raise flag While Alice's flag is up Lower flag Wait for Alice's flag to go down Raise flag Unleash pet Lower flag when pet returns Art of Multiprocessor Programming 61
54 Bob's Protocol Bob defers Raise flag to Alice While Alice's flag is up Lower flag Wait for Alice's flag to go down Raise flag Unleash pet Lower flag when pet returns Art of Multiprocessor Programming 62
55 The Flag Principle Raise the flag Look at other's flag Flag Principle: If each raises and looks, then Last to look must see both flags up Art of Multiprocessor Programming 63
56 Remarks Protocol is unfair Bob's pet might never get in Protocol uses waiting If Bob is eaten by his pet, Alice's pet might never get in Art of Multiprocessor Programming 69
57 CDS.IISc.ac.in Department of Computational and Data Sciences The Fable Continues Bob falls ill, cannot tend to the pets She gets the pets Pets get along fine But Bob has to feed them Producer-Consumer Problem Art of Multiprocessor Programming 73
58 Bob Puts Food in the Pond A Art of Multiprocessor Programming 75
59 Alice releases her pets to Feed mmm mmm B Art of Multiprocessor Programming 76
60 CDS.IISc.ac.in Department of Computational and Data Sciences Producer/Consumer Alice and Bob can't meet Bob s disease is contagious So he puts food in the pond And later, she releases the pets Avoid Releasing pets when there's no food Putting out food if uneaten food remains Art of Multiprocessor Programming 77
61 Producer/Consumer Need a mechanism so that Bob lets Alice know when food has been put out Alice lets Bob know when to put out more food Art of Multiprocessor Programming 78
62 cola Can Solution A B Art of Multiprocessor Programming 79
63 cola Bob puts food in Pond A B Art of Multiprocessor Programming 80
64 Bob knocks over Can A B Art of Multiprocessor Programming 81
65 Alice Releases Pets A yum yum B Art of Multiprocessor Programming 82
66 cola Alice Resets Can when Pets are Fed A B Art of Multiprocessor Programming 83
67 Pseudocode while (true) { while (can.isup()){}; pet.release(); pet.recapture(); can.reset(); } Alice's code Art of Multiprocessor Programming 84
68 Pseudocode while (true) { while (can.isup()){}; pet.release(); pet.recapture(); can.reset(); while (true) { } while (can.isdown()){}; pond.stockwithfood(); can.knockover(); } Alice's code Bob's code Art of Multiprocessor Programming 85
69 Mutual Exclusion Correctness Pets and Bob never together in pond Art of Multiprocessor Programming 86
70 Mutual Exclusion Correctness Pets and Bob never together in pond No Starvation if Bob always willing to feed, and pets always famished, then pets eat infinitely often. Art of Multiprocessor Programming 87
71 Mutual Exclusion Correctness Pets and Bob never together in pond No Starvation if Bob always willing to feed, and pets always famished, then pets eat infinitely often. safety liveness safety Producer/Consumer The pets never enter pond unless there is food, and Bob never provides food if there is unconsumed food. Art of Multiprocessor Programming 88
72 CDS.IISc.ac.in Department of Computational and Data Sciences Spin Locks Aside Art of Multiprocessor Programming
73 Pseudocode while (true) { while (can.isup()){}; pet.release(); pet.recapture(); can.reset(); } Spin Lock! Has to be protected while (true) { while (can.isdown()){}; pond.stockwithfood(); can.knockover(); } Art of Multiprocessor Programming 90
74 What Should you do if you can t get a lock? Keep trying spin or busy-wait Good if delays are short Give up the processor Good if delays are long Always good on uniprocessor Art of Multiprocessor Programming 91 (1)
75 What Should you do if you can t get a lock? Keep trying spin or busy-wait Good if delays are short Give up the processor Good if delays are long Always good on uniprocessor our focus Art of Multiprocessor Programming 92
76 Basic Spin-Lock CS. spin lock critical section Resets lock upon exit Art of Multiprocessor Programming 93
77 Basic Spin-Lock lock introduces sequential bottleneck CS. spin lock critical section Resets lock upon exit Art of Multiprocessor Programming 94
78 Basic Spin-Lock lock suffers from contention CS. spin lock critical section Resets lock upon exit Art of Multiprocessor Programming 95
79 Basic Spin-Lock lock suffers from contention CS. spin lock critical section Resets lock upon exit Notice: these are distinct phenomena Art of Multiprocessor Programming 96
80 Basic Spin-Lock lock suffers from contention CS. spin lock critical section Resets lock upon exit Seq Bottleneck no parallelism Art of Multiprocessor Programming 97
81 Basic Spin-Lock lock suffers from contention CS. spin lock critical section Resets lock upon exit Contention??? Art of Multiprocessor Programming 98
82 Review: Test-and-Set Boolean value Test-and-set (TAS) Swap true with current value Return value tells if prior value was true or false Can reset just by writing false TAS aka getandset Art of Multiprocessor Programming 99
83 Review: Test-and-Set public class AtomicBoolean { boolean value; public synchronized boolean getandset(boolean newvalue) { boolean prior = value; value = newvalue; return prior; } } Art of Multiprocessor Programming 100 (5)
84 Review: Test-and-Set public class AtomicBoolean { boolean value; public synchronized boolean getandset(boolean newvalue) { boolean prior = value; value = newvalue; return prior; } } Package java.util.concurrent.atomic Art of Multiprocessor Programming 101
85 Review: Test-and-Set public class AtomicBoolean { boolean value; public synchronized boolean getandset(boolean newvalue) { boolean prior = value; value = newvalue; return prior; } } Swap old and new values Art of Multiprocessor Programming 102
86 Review: Test-and-Set AtomicBoolean lock = new AtomicBoolean(false) boolean prior = lock.getandset(true) Art of Multiprocessor Programming 103
87 Review: Test-and-Set AtomicBoolean lock = new AtomicBoolean(false) boolean prior = lock.getandset(true) Swapping in true is called test-and-set or TAS Art of Multiprocessor Programming 104 (5)
88 Test-and-Set Locks Locking Lock is free: value is false Lock is taken: value is true Acquire lock by calling TAS If result is false, you win If result is true, you lose Release lock by writing false Art of Multiprocessor Programming 105
89 Test-and-set Lock class TASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (state.getandset(true)) {} } void unlock() { state.set(false); }} Art of Multiprocessor Programming 106
90 Test-and-set Lock class TASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (state.getandset(true)) {} } void unlock() { state.set(false); }} Lock state is AtomicBoolean Art of Multiprocessor Programming 107
91 Test-and-set Lock class TASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (state.getandset(true)) {} } void unlock() { state.set(false); }} Keep trying until lock acquired Art of Multiprocessor Programming 108
92 Test-and-set Lock class TASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (state.getandset(true)) {} } void unlock() { state.set(false); }} Release lock by resetting state to false Art of Multiprocessor Programming 109
93 Space Complexity TAS spin-lock has small footprint N thread spin-lock uses O(1) space As opposed to O(n) Peterson/Bakery How did we overcome the W(n) lower bound? We used a RMW operation Art of Multiprocessor Programming 110
94 Performance Experiment n threads Increment shared counter 1 million times How long should it take? How long does it take? Art of Multiprocessor Programming 111
95 time Graph no speedup because of sequential bottleneck ideal threads Art of Multiprocessor Programming 112
96 time Mystery #1 TAS lock threads Ideal What is going on? Art of Multiprocessor Programming 113
97 Test-and-Test-and-Set Locks Lurking stage Wait until lock looks free Spin while read returns true (lock taken) Pouncing state As soon as lock looks available Read returns false (lock free) Call TAS to acquire lock If TAS loses, back to lurking Art of Multiprocessor Programming 114
98 Test-and-test-and-set Lock class TTASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (true) { while (state.get()) {} if (!state.getandset(true)) return; } } Art of Multiprocessor Programming 115
99 Test-and-test-and-set Lock class TTASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (true) { while (state.get()) {} if (!state.getandset(true)) return; } } Wait until lock looks free Art of Multiprocessor Programming 116
100 Test-and-test-and-set Lock class TTASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (true) { while (state.get()) {} if (!state.getandset(true)) return; } } Then try to acquire it Art of Multiprocessor Programming 117
101 time Mystery #2 TAS lock TTAS lock Ideal threads Art of Multiprocessor Programming 118
102 Mystery Both TAS and TTAS Do the same thing (in our model) Except that TTAS performs much better than TAS Neither approaches ideal Art of Multiprocessor Programming 119
103 Opinion Our memory abstraction is broken TAS & TTAS methods Are provably the same (in our model) Except they aren t (in field tests) Need a more detailed model Art of Multiprocessor Programming 120
104 Simple TASLock TAS invalidates cache lines Spinners Miss in cache Go to bus Thread wants to release lock delayed behind spinners Art of Multiprocessor Programming 121
105 Test-and-test-and-set Wait until lock looks free Spin on local cache No bus use while lock busy Problem: when lock is released Invalidation storm Art of Multiprocessor Programming 122
106 Local Spinning while Lock is Busy busy busy Bus busy memory busy Art of Multiprocessor Programming 123
107 On Release invalid invalid Bus free memory free Art of Multiprocessor Programming 124
108 Everyone misses, rereads On Release miss invalid invalid miss Bus free memory free Art of Multiprocessor Programming 125 (1)
109 Everyone tries TAS On Release TAS( ) invalid TAS( ) invalid Bus free memory free Art of Multiprocessor Programming 126 (1)
110 Problems Everyone misses Reads satisfied sequentially Everyone does TAS Invalidates others caches Eventually quiesces after lock acquired How long does this take? Art of Multiprocessor Programming 127
111 time Quiescence Time Increses linearly with the number of processors for bus architecture threads Art of Multiprocessor Programming 128
112 time Mystery Explained TAS lock TTAS lock Ideal threads Better than TAS but still not as good as ideal Art of Multiprocessor Programming 129
113 Solution: Introduce Delay If the lock looks free But I fail to get it There must be contention Better to back off than to collide again time r 2 d r 1 d d spin lock Art of Multiprocessor Programming 130
114 Dynamic Example: Exponential Backoff If I fail to get lock Wait random duration before retry Each subsequent failure doubles expected wait time 4d 2d d spin lock Art of Multiprocessor Programming 131
115 CDS.IISc.ac.in Department of Computational and Data Sciences Concurrent Data Structures Art of Multiprocessor Programming
116 CDS.IISc.ac.in Department of Computational and Data Sciences What if you had multiple producers, consumers? while (true) { while (a.islocked()){}; while (can.isup()){}; pet.release(); pet.recapture(); can.reset(); } while (true) { while (b.islocked()){}; while (can.isdown()){}; pond.stockwithfood(); can.knockover(); } Alice & Co. Bob & Co.
117 CDS.IISc.ac.in Department of Computational and Data Sciences Does this improve performance? Sequential bottleneck! Art of Multiprocessor Programming
118 Why do we care About Sequential Bottlenecks? We want as much of the code as possible to execute in parallel A larger sequential part implies reduced performance Amdahl's law: this relation is not linear Eugene Amdahl Art of Multiprocessor Programming 152
119 Amdahl's Law Speedup = 1 thread execution time N thread execution time Art of Multiprocessor Programming 153
120 Amdahl's Law 1 Speedup = (1- p) + p n Art of Multiprocessor Programming 154
121 Amdahl's Law 1 Parallel fraction Speedup = (1- p) + p n Art of Multiprocessor Programming 155
122 Amdahl's Law Sequential fraction 1 Parallel fraction Speedup = (1- p) + p n Art of Multiprocessor Programming 156
123 Amdahl's Law Sequential fraction 1 Parallel fraction Speedup = Number of threads (1- p) + p n Art of Multiprocessor Programming 157
124 Amdahl's Law (in practice) Art of Multiprocessor Programming 158
125 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup? Art of Multiprocessor Programming 159
126 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup? Speedup = 2.17 = Art of Multiprocessor Programming 160
127 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup? Art of Multiprocessor Programming 161
128 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup? Speedup = 3.57 = Art of Multiprocessor Programming 162
129 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup? Art of Multiprocessor Programming 163
130 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup? Speedup = 5.26 = Art of Multiprocessor Programming 164
131 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup? Art of Multiprocessor Programming 165
132 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup? Speedup = 9.17 = Art of Multiprocessor Programming 166
133 Back to Real-World Multicore Scaling Speedup 1.8x 2x 2.9x User code Multicore Not reducing sequential % of code Art of Multiprocessor Programming 167
134 Shared Data Structures Coarse Grained Fine Grained 25% Shared 25% Shared 75% Unshared 75% Unshared
135 Shared Data Structures Honk! Honk! Honk! Why only 2.9 speedup Coarse Grained Fine Grained 25% Shared 25% Shared 75% Unshared 75% Unshared
136 Shared Data Structures Honk! Honk! Honk! Why fine-grained parallelism maters Coarse Grained Fine Grained 25% Shared 25% Shared 75% Unshared 75% Unshared
137 CDS.IISc.ac.in Department of Computational and Data Sciences Need for Concurrent Queues Avoid sequential bottleneck by introducing a buffer between the producers and consumers Producers add item to queue Consumers consume from queue Neither wait as long as queue is not full or empty Art of Multiprocessor Programming
138 Concurrent Objects Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
139 Concurrent Computation memory object object Art of Multiprocessor Programming 173
140 Objectivism What is a concurrent object? How do we describe one? How do we implement one? How do we tell if we re right? Art of Multiprocessor Programming 174
141 Objectivism What is a concurrent object? How do we describe one? How do we tell if we re right? Art of Multiprocessor Programming 175
142 FIFO Queue: Enqueue Method q.enq( ) Art of Multiprocessor Programming 176
143 FIFO Queue: Dequeue Method q.deq()/ Art of Multiprocessor Programming 177
144 Lock-Based Queue head 0 x y 1 tail capacity = 8 Art of Multiprocessor Programming 178
145 Lock-Based Queue head 0 x y 1 tail Fields protected by single shared lock 5 4 capacity = 8 Art of Multiprocessor Programming 179
146 A Lock-Based Queue head 0 1 capacity-1 class LockBasedQueue<T> { y z 2 int head, tail; T[] items; Lock lock; public LockBasedQueue(int capacity) { head = 0; tail = 0; lock = new ReentrantLock(); items = (T[]) new Object[capacity]; } tail Fields protected by single shared lock Art of Multiprocessor Programming 180
147 Lock-Based Queue Initially: head = tail 0 1 head tail Art of Multiprocessor Programming
148 A Lock-Based Queue head 0 1 capacity-1 class LockBasedQueue<T> { y z 2 int head, tail; T[] items; Lock lock; public LockBasedQueue(int capacity) { head = 0; tail = 0; lock = new ReentrantLock(); items = (T[]) new Object[capacity]; } tail Initially head = tail Art of Multiprocessor Programming 182
149 Lock-Based deq() head 0 x y 1 tail Art of Multiprocessor Programming 183
150 Acquire Lock My turn head 7 0 x x y y 1 2 tail Waiting to enqueue Art of Multiprocessor Programming 184
151 Implementation: deq() public T deq() throws EmptyException { lock.lock(); try { if (tail == head) } throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } Acquire lock at method start head capacity y z 2 tail Art of Multiprocessor Programming 185
152 Check if Non-Empty head 0 1 tail Not equal? 7 x y 2 Waiting to enqueue Art of Multiprocessor Programming 186
153 Implementation: deq() public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } } If queue empty throw exception head capacity y z 2 tail Art of Multiprocessor Programming 187
154 Modify the Queue head head 0 x y 1 tail 7 2 Waiting to enqueue Art of Multiprocessor Programming 188
155 Implementation: deq() public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } } finally { lock.unlock(); } Queue not empty? Remove item and update head head capacity y z 2 tail Art of Multiprocessor Programming 189
156 Implementation: deq() public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } } Return result head capacity y z 2 tail Art of Multiprocessor Programming 190
157 Release the Lock head 0 1 tail y 7 2 Waiting 6 3 x 5 4 Art of Multiprocessor Programming 191
158 Release the Lock head 0 1 tail y 7 2 My turn! 6 3 x 5 4 Art of Multiprocessor Programming 192
159 Implementation: deq() public T deq() throws EmptyException { lock.lock(); try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } } Release lock no matter what! head capacity y z 2 tail Art of Multiprocessor Programming 193
160 Implementation: enq() public void enq(item ) throws EmptyException { lock.lock(); try { if (tail-head == capacity) throw new FullException(); items[tail % capacity] = x; tail++; head } finally { 0 1 capacity-1 lock.unlock(); y z } } 2 tail Art of Multiprocessor Programming
161 Wait-free Queue? public class WaitFreeQueue { int head = 0, tail = 0; items = (T[]) new Object[capacity]; head capacity-1 capacity y z 2 tail }} public void enq(item x) { if (tail-head == capacity) throw new FullException(); items[tail % capacity] = x; tail++; } public Item deq() { if (tail == head) throw new EmptyException(); Item item = items[head % capacity]; head++; return item; Art of Multiprocessor Programming 195
162 Linearizability Each method should take effect Instantaneously Between invocation and response events Object is correct if this sequential behavior is correct Any such concurrent object is Linearizable A linearizable object: one all of whose possible executions are linearizable Art of Multiprocessor Programming 196
![fields modified int head = 0, tail = 0; items = (T[]) new](/docs-images/84/91202532/images/163-2.jpg "Object[capacity]; public void enq(item x) { if (tail-head == capacity)")
![throw new FullException(); items[tail % capacity] = x; tail++; } public](/docs-images/84/91202532/images/163-3.jpg "Item deq() { if (tail == head) throw new EmptyException(); Item item =")
163 Wait-free Queue? public class WaitFreeQueue { Linearization order is order head and tail fields modified int head = 0, tail = 0; items = (T[]) new Object[capacity]; public void enq(item x) { if (tail-head == capacity) throw new FullException(); items[tail % capacity] = x; tail++; } public Item deq() { if (tail == head) throw new EmptyException(); Item item = items[head % capacity]; head++; return item; }} Art of Multiprocessor Programming 197
164 Reasoning About Linearizability: Locking public T deq() throws EmptyException { lock.lock(); } try { if (tail == head) throw new EmptyException(); T x = items[head % items.length]; head++; return x; } finally { lock.unlock(); } head capacity y z Linearization points are when locks are released 2 tail Art of Multiprocessor Programming 198
Introduction. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Introduction Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Moore s Law Transistor count still rising Clock speed flattening sharply Art of Multiprocessor Programming
Introduction. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit. Art of Multiprocessor Programming
Introduction Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Art of Multiprocessor Programming Moore s Law Transistor count still rising Clock speed flattening
Introduction to Concurrent Programming
Introduction to Concurrent Programming Based on the companion slides for the book The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit, 2008 From the New York Times SAN FRANCISCO, 7 May
Programming Paradigms for Concurrency Introduction
Programming Paradigms for Concurrency Introduction Based on companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified by Thomas Wies New York University Moore
Spin Locks and Contention. Companion slides for Chapter 7 The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Spin Locks and Contention Companion slides for Chapter 7 The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Focus so far: Correctness and Progress Models Accurate (we never lied to you)
Spin Locks and Contention. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Spin Locks and Contention Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Focus so far: Correctness and Progress Models Accurate (we never lied to you) But idealized
Spin Locks and Contention. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Spin Locks and Contention Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Real world concurrency Understanding hardware architecture What is contention How to
Spin Locks and Contention
Spin Locks and Contention Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified for Software1 students by Lior Wolf and Mati Shomrat Kinds of Architectures
Introduction to Concurrent Programming
Σχολή Ηλεκτρολόγων Μηχανικών και Μηχανικών Υπολογιστών Τομέας Τεχνολογίας Πληροφορικής και Υπολογιστών Εθνικό Μετσόβιο Πολυτεχνείο Γλώσσες Προγραμματισμού ΙΙ Διδάσκοντες: Νικόλαος Παπασπύρου, Κωστής Σαγώνας
Locks Chapter 4. Overview. Introduction Spin Locks. Queue locks. Roger Wattenhofer. Test-and-Set & Test-and-Test-and-Set Backoff lock
Locks Chapter 4 Roger Wattenhofer ETH Zurich Distributed Computing www.disco.ethz.ch Overview Introduction Spin Locks Test-and-Set & Test-and-Test-and-Set Backoff lock Queue locks 4/2 1 Introduction: From
COMP 322: Principles of Parallel Programming Lecture 11: Parallel Programming Issues (Chapter 6) Fall 2009
COMP 322: Principles of Parallel Programming Lecture 11: Parallel Programming Issues (Chapter 6) Fall 2009 Vivek Sarkar Department of Computer Science Rice University vsarkar@rice.edu COMP 322 Lecture
Agenda. Lecture. Next discussion papers. Bottom-up motivation Shared memory primitives Shared memory synchronization Barriers and locks
Agenda Lecture Bottom-up motivation Shared memory primitives Shared memory synchronization Barriers and locks Next discussion papers Selecting Locking Primitives for Parallel Programming Selecting Locking
Programming Paradigms for Concurrency Lecture 3 Concurrent Objects
Programming Paradigms for Concurrency Lecture 3 Concurrent Objects Based on companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified by Thomas Wies New York University
Introduction to Concurrency and Multicore Programming. Slides adapted from Art of Multicore Programming by Herlihy and Shavit
Introduction to Concurrency and Multicore Programming Slides adapted from Art of Multicore Programming by Herlihy and Shavit Overview Introduction Mutual Exclusion Linearizability Concurrent Data Structure
Locking. Part 2, Chapter 11. Roger Wattenhofer. ETH Zurich Distributed Computing
Locking Part 2, Chapter 11 Roger Wattenhofer ETH Zurich Distributed Computing www.disco.ethz.ch Overview Introduction Spin Locks Test-and-Set & Test-and-Test-and-Set Backoff lock Queue locks 11/2 Introduction:
Introduction. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Introduction Companion slides for The by Maurice Herlihy & Nir Shavit Moore s Law Transistor count still rising Clock speed flattening sharply 2 Moore s Law (in practice) 3 Nearly Extinct: the Uniprocesor
Concurrent Objects. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Concurrent Objects Companion slides for The by Maurice Herlihy & Nir Shavit Concurrent Computation memory object object 2 Objectivism What is a concurrent object? How do we describe one? How do we implement
Spin Locks and Contention. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Spin Locks and Contention Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit MIMD Architectures memory Shared Bus Memory Contention Communication Contention Communication
Modern High-Performance Locking
Modern High-Performance Locking Nir Shavit Slides based in part on The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Locks (Mutual Exclusion) public interface Lock { public void lock();
Spin Locks and Contention. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Spin Locks and Contention Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Focus so far: Correctness and Progress Models Accurate (we never lied to you) But idealized
Spin Locks and Contention. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Spin Locks and Contention Companion slides for The Programming by Maurice Herlihy & Nir Shavit Focus so far: Correctness Models Accurate (we never lied to you) But idealized (so we forgot to mention a
9/28/2014. CS341: Operating System. Synchronization. High Level Construct: Monitor Classical Problems of Synchronizations FAQ: Mid Semester
CS341: Operating System Lect22: 18 th Sept 2014 Dr. A. Sahu Dept of Comp. Sc. & Engg. Indian Institute of Technology Guwahati Synchronization Hardware Support TAS, CAS, TTAS, LL LC, XCHG, Compare, FAI
9/23/2014. Concurrent Programming. Book. Process. Exchange of data between threads/processes. Between Process. Thread.
Dr A Sahu Dept of Computer Science & Engineering IIT Guwahati Course Structure & Book Basic of thread and process Coordination and synchronization Example of Parallel Programming Shared memory : C/C++
Models of concurrency & synchronization algorithms
Models of concurrency & synchronization algorithms Lecture 3 of TDA383/DIT390 (Concurrent Programming) Carlo A. Furia Chalmers University of Technology University of Gothenburg SP3 2016/2017 Today s menu
Locking Granularity. CS 475, Spring 2019 Concurrent & Distributed Systems. With material from Herlihy & Shavit, Art of Multiprocessor Programming
Locking Granularity CS 475, Spring 2019 Concurrent & Distributed Systems With material from Herlihy & Shavit, Art of Multiprocessor Programming Discussion: HW1 Part 4 addtolist(key1, newvalue) Thread 1
10/27/11. Is The Queue Correct? Concurrent Computaton. Objects. A Concurrent FIFO Queue head. A Concurrent FIFO Queue head
DD2451 Parallel and Distributed Computing --- FDD3008 Distributed Algorithms Lecture 2 Concurrent Objects memory Concurrent Computaton Mads Dam Autumn/Winter 2011 object object Art of Mul/processor 2 Objects
Review of last lecture. Goals of this lecture. DPHPC Overview. Lock-based queue. Lock-based queue
Review of last lecture Design of Parallel and High-Performance Computing Fall 2013 Lecture: Linearizability Instructor: Torsten Hoefler & Markus Püschel TA: Timo Schneider Cache-coherence is not enough!
Solution: a lock (a/k/a mutex) public: virtual void unlock() =0;
1 Solution: a lock (a/k/a mutex) class BasicLock { public: virtual void lock() =0; virtual void unlock() =0; ; 2 Using a lock class Counter { public: int get_and_inc() { lock_.lock(); int old = count_;
DD2451 Parallel and Distributed Computing --- FDD3008 Distributed Algorithms
DD2451 Parallel and Distributed Computing --- FDD3008 Distributed Algorithms Lecture 2 Concurrent Objects Mads Dam Autumn/Winter 2011 Concurrent Computaton memory object object Art of Mul*processor 2 Objects
Lecture 7: Mutual Exclusion 2/16/12. slides adapted from The Art of Multiprocessor Programming, Herlihy and Shavit
Principles of Concurrency and Parallelism Lecture 7: Mutual Exclusion 2/16/12 slides adapted from The Art of Multiprocessor Programming, Herlihy and Shavit Time Absolute, true and mathematical time, of
Concurrent Objects and Linearizability
Chapter 3 Concurrent Objects and Linearizability 3.1 Specifying Objects An object in languages such as Java and C++ is a container for data. Each object provides a set of methods that are the only way
Sequen&al Consistency and Linearizability
Sequen&al Consistency and Linearizability (Or, Reasoning About Concurrent Objects) Acknowledgement: Slides par&ally adopted from the companion slides for the book "The Art of Mul&processor Programming"
Transactional Memory: Architectural Support for Lock-Free Data Structures Maurice Herlihy and J. Eliot B. Moss ISCA 93
Transactional Memory: Architectural Support for Lock-Free Data Structures Maurice Herlihy and J. Eliot B. Moss ISCA 93 What are lock-free data structures A shared data structure is lock-free if its operations
Spin Locks and Contention Management
Chapter 7 Spin Locks and Contention Management 7.1 Introduction We now turn our attention to the performance of mutual exclusion protocols on realistic architectures. Any mutual exclusion protocol poses
CMSC 330: Organization of Programming Languages. Threads Classic Concurrency Problems
: Organization of Programming Languages Threads Classic Concurrency Problems The Dining Philosophers Problem Philosophers either eat or think They must have two forks to eat Can only use forks on either
Programming at Scale: Concurrency
Programming at Scale: Concurrency 1 Goal: Building Fast, Scalable Software How do we speed up software? 2 What is scalability? A system is scalable if it can easily adapt to increased (or reduced) demand
CMSC 330: Organization of Programming Languages. The Dining Philosophers Problem
CMSC 330: Organization of Programming Languages Threads Classic Concurrency Problems The Dining Philosophers Problem Philosophers either eat or think They must have two forks to eat Can only use forks
CSE 613: Parallel Programming. Lecture 17 ( Concurrent Data Structures: Queues and Stacks )
CSE 613: Parallel Programming Lecture 17 ( Concurrent Data Structures: Queues and Stacks ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2012 Desirable Properties of Concurrent
Universality of Consensus. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Universality of Consensus Companion slides for The by Maurice Herlihy & Nir Shavit Turing Computability 1 0 1 1 0 1 0 A mathematical model of computation Computable = Computable on a T-Machine 2 Shared-Memory
Computer Engineering II Solution to Exercise Sheet Chapter 10
Distributed Computing FS 2017 Prof. R. Wattenhofer Computer Engineering II Solution to Exercise Sheet Chapter 10 Quiz 1 Quiz a) The AtomicBoolean utilizes an atomic version of getandset() implemented in
Concurrent Preliminaries
Concurrent Preliminaries Sagi Katorza Tel Aviv University 09/12/2014 1 Outline Hardware infrastructure Hardware primitives Mutual exclusion Work sharing and termination detection Concurrent data structures
Concurrent Queues, Monitors, and the ABA problem. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Concurrent Queues, Monitors, and the ABA problem Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Queues Often used as buffers between producers and consumers
THREADS & CONCURRENCY
27/04/2018 Sorry for the delay in getting slides for today 2 Another reason for the delay: Yesterday: 63 posts on the course Piazza yesterday. A7: If you received 100 for correctness (perhaps minus a late
Unit 6: Indeterminate Computation
Unit 6: Indeterminate Computation Martha A. Kim October 6, 2013 Introduction Until now, we have considered parallelizations of sequential programs. The parallelizations were deemed safe if the parallel
CONCURRENT LIBRARIES. Correctness Criteria, Verification
CONCURRENT LIBRARIES Correctness Criteria, Verification Verification Ingredients Specifying a Library: φ Implementing a Library: L Verifying a Library implementation: L φ The History of an Object Object
Introduction. Chapter 1
Chapter 1 Introduction Every year, processors get faster and cheaper: processor speeds double roughly every two years. This remarkable rate of improvement will probably continue for a while, but eventually,
Computer Architecture
Jens Teubner Computer Architecture Summer 2016 1 Computer Architecture Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de Summer 2016 Jens Teubner Computer Architecture Summer 2016 83 Part III Multi-Core
Distributed Computing
HELLENIC REPUBLIC UNIVERSITY OF CRETE Distributed Computing Graduate Course Section 3: Spin Locks and Contention Panagiota Fatourou Department of Computer Science Spin Locks and Contention In contrast
Linked Lists: Locking, Lock-Free, and Beyond. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Linked Lists: Locking, Lock-Free, and Beyond Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Concurrent Objects Adding threads should not lower throughput Contention
Operating Systems. Designed and Presented by Dr. Ayman Elshenawy Elsefy
Operating Systems Designed and Presented by Dr. Ayman Elshenawy Elsefy Dept. of Systems & Computer Eng.. AL-AZHAR University Website : eaymanelshenawy.wordpress.com Email : eaymanelshenawy@yahoo.com Reference
COMP 3430 Robert Guderian
Operating Systems COMP 3430 Robert Guderian file:///users/robg/dropbox/teaching/3430-2018/slides/06_concurrency/index.html?print-pdf#/ 1/76 1 Concurrency file:///users/robg/dropbox/teaching/3430-2018/slides/06_concurrency/index.html?print-pdf#/
Other consistency models
Last time: Symmetric multiprocessing (SMP) Lecture 25: Synchronization primitives Computer Architecture and Systems Programming (252-0061-00) CPU 0 CPU 1 CPU 2 CPU 3 Timothy Roscoe Herbstsemester 2012
Mutual Exclusion. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Mutual Exclusion Companion slides for The by Maurice Herlihy & Nir Shavit Mutual Exclusion Today we will try to formalize our understanding of mutual exclusion We will also use the opportunity to show
Cache-Aware Lock-Free Queues for Multiple Producers/Consumers and Weak Memory Consistency
Cache-Aware Lock-Free Queues for Multiple Producers/Consumers and Weak Memory Consistency Anders Gidenstam Håkan Sundell Philippas Tsigas School of business and informatics University of Borås Distributed
Multiprocessor Synchronization
Multiprocessor Synchronization Material in this lecture in Henessey and Patterson, Chapter 8 pgs. 694-708 Some material from David Patterson s slides for CS 252 at Berkeley 1 Multiprogramming and Multiprocessing
Synchronization. CS61, Lecture 18. Prof. Stephen Chong November 3, 2011
Synchronization CS61, Lecture 18 Prof. Stephen Chong November 3, 2011 Announcements Assignment 5 Tell us your group by Sunday Nov 6 Due Thursday Nov 17 Talks of interest in next two days Towards Predictable,
Kernel Synchronization I. Changwoo Min
1 Kernel Synchronization I Changwoo Min 2 Summary of last lectures Tools: building, exploring, and debugging Linux kernel Core kernel infrastructure syscall, module, kernel data structures Process management
Multiprocessors and Locking
Types of Multiprocessors (MPs) Uniform memory-access (UMA) MP Access to all memory occurs at the same speed for all processors. Multiprocessors and Locking COMP9242 2008/S2 Week 12 Part 1 Non-uniform memory-access
The Dining Philosophers Problem CMSC 330: Organization of Programming Languages
The Dining Philosophers Problem CMSC 0: Organization of Programming Languages Threads Classic Concurrency Problems Philosophers either eat or think They must have two forks to eat Can only use forks on
Review of last lecture. Peer Quiz. DPHPC Overview. Goals of this lecture. Lock-based queue
Review of last lecture Design of Parallel and High-Performance Computing Fall 2016 Lecture: Linearizability Motivational video: https://www.youtube.com/watch?v=qx2driqxnbs Instructor: Torsten Hoefler &
Concurrent Queues and Stacks. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit The Five-Fold Path Coarse-grained locking Fine-grained locking Optimistic synchronization
CMSC 330: Organization of Programming Languages
CMSC 330: Organization of Programming Languages Multithreading Multiprocessors Description Multiple processing units (multiprocessor) From single microprocessor to large compute clusters Can perform multiple
Chapter 6: Process Synchronization
Chapter 6: Process Synchronization Objectives Introduce Concept of Critical-Section Problem Hardware and Software Solutions of Critical-Section Problem Concept of Atomic Transaction Operating Systems CS
CSE 451: Operating Systems Winter Lecture 7 Synchronization. Steve Gribble. Synchronization. Threads cooperate in multithreaded programs
CSE 451: Operating Systems Winter 2005 Lecture 7 Synchronization Steve Gribble Synchronization Threads cooperate in multithreaded programs to share resources, access shared data structures e.g., threads
10/17/ Gribble, Lazowska, Levy, Zahorjan 2. 10/17/ Gribble, Lazowska, Levy, Zahorjan 4
Temporal relations CSE 451: Operating Systems Autumn 2010 Module 7 Synchronization Instructions executed by a single thread are totally ordered A < B < C < Absent synchronization, instructions executed
Multiprocessor Systems. Chapter 8, 8.1
Multiprocessor Systems Chapter 8, 8.1 1 Learning Outcomes An understanding of the structure and limits of multiprocessor hardware. An appreciation of approaches to operating system support for multiprocessor
Concurrent Computing
Concurrent Computing Introduction SE205, P1, 2017 Administrivia Language: (fr)anglais? Lectures: Fridays (15.09-03.11), 13:30-16:45, Amphi Grenat Web page: https://se205.wp.imt.fr/ Exam: 03.11, 15:15-16:45
Lecture 26: Multiprocessing continued Computer Architecture and Systems Programming ( )
Systems Group Department of Computer Science ETH Zürich Lecture 26: Multiprocessing continued Computer Architecture and Systems Programming (252-0061-00) Timothy Roscoe Herbstsemester 2012 Today Non-Uniform
Recap. Contents. Reenterancy of synchronized. Explicit Locks: ReentrantLock. Reenterancy of synchronise (ctd) Advanced Thread programming.
Lecture 07: Advanced Thread programming Software System Components 2 Behzad Bordbar School of Computer Science, University of Birmingham, UK Recap How to deal with race condition in Java Using synchronised
Today. SMP architecture. SMP architecture. Lecture 26: Multiprocessing continued Computer Architecture and Systems Programming ( )
Lecture 26: Multiprocessing continued Computer Architecture and Systems Programming (252-0061-00) Timothy Roscoe Herbstsemester 2012 Systems Group Department of Computer Science ETH Zürich SMP architecture
Blocking Non-blocking Caveat:
Overview of Lecture 5 1 Progress Properties 2 Blocking The Art of Multiprocessor Programming. Maurice Herlihy and Nir Shavit. Morgan Kaufmann, 2008. Deadlock-free: some thread trying to get the lock eventually
Chapter 6: Synchronization. Operating System Concepts 8 th Edition,
Chapter 6: Synchronization, Silberschatz, Galvin and Gagne 2009 Outline Background The Critical-Section Problem Peterson s Solution Synchronization Hardware Semaphores Classic Problems of Synchronization
Operating Systems. Synchronization
Operating Systems Fall 2014 Synchronization Myungjin Lee myungjin.lee@ed.ac.uk 1 Temporal relations Instructions executed by a single thread are totally ordered A < B < C < Absent synchronization, instructions
Computer Science 61 Scribe Notes Tuesday, November 25, 2014 (aka the day before Thanksgiving Break)
Computer Science 61 Scribe Notes Tuesday, November 25, 2014 (aka the day before Thanksgiving Break) Problem Set 6 Released! People have fun with it Make Games Snake Game Hack JavaScript Due Wed., last
Lecture 21: Transactional Memory. Topics: Hardware TM basics, different implementations
Lecture 21: Transactional Memory Topics: Hardware TM basics, different implementations 1 Transactions New paradigm to simplify programming instead of lock-unlock, use transaction begin-end locks are blocking,
Chapter 5. Multiprocessors and Thread-Level Parallelism
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 5 Multiprocessors and Thread-Level Parallelism 1 Introduction Thread-Level parallelism Have multiple program counters Uses MIMD model
PERFORMANCE ANALYSIS AND OPTIMIZATION OF SKIP LISTS FOR MODERN MULTI-CORE ARCHITECTURES
PERFORMANCE ANALYSIS AND OPTIMIZATION OF SKIP LISTS FOR MODERN MULTI-CORE ARCHITECTURES Anish Athalye and Patrick Long Mentors: Austin Clements and Stephen Tu 3 rd annual MIT PRIMES Conference Sequential
Stanford University Computer Science Department CS 140 Midterm Exam Dawson Engler Winter 1999
Stanford University Computer Science Department CS 140 Midterm Exam Dawson Engler Winter 1999 Name: Please initial the bottom left corner of each page. This is an open-book exam. You have 50 minutes to
Lecture 9: Multiprocessor OSs & Synchronization. CSC 469H1F Fall 2006 Angela Demke Brown
Lecture 9: Multiprocessor OSs & Synchronization CSC 469H1F Fall 2006 Angela Demke Brown The Problem Coordinated management of shared resources Resources may be accessed by multiple threads Need to control
Serial. Parallel. CIT 668: System Architecture 2/14/2011. Topics. Serial and Parallel Computation. Parallel Computing
CIT 668: System Architecture Parallel Computing Topics 1. What is Parallel Computing? 2. Why use Parallel Computing? 3. Types of Parallelism 4. Amdahl s Law 5. Flynn s Taxonomy of Parallel Computers 6.
Chapter 6: Process [& Thread] Synchronization. CSCI [4 6] 730 Operating Systems. Why does cooperation require synchronization?
![Chapter 6: Process [& Thread] Synchronization. CSCI [4 6] 730 Operating Systems. Why does cooperation require synchronization?](/thumbs/88/114609946.jpg "Chapter 6: Process [& Thread] Synchronization. CSCI [4 6] 730 Operating Systems. Why does cooperation require synchronization?")
Chapter 6: Process [& Thread] Synchronization CSCI [4 6] 730 Operating Systems Synchronization Part 1 : The Basics Why is synchronization needed? Synchronization Language/Definitions:» What are race conditions?»
CS140 Operating Systems and Systems Programming Midterm Exam
CS140 Operating Systems and Systems Programming Midterm Exam October 31 st, 2003 (Total time = 50 minutes, Total Points = 50) Name: (please print) In recognition of and in the spirit of the Stanford University
Programming Paradigms for Concurrency Lecture 2 - Mutual Exclusion
Programming Paradigms for Concurrency Lecture 2 - Mutual Exclusion Based on companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified by Thomas Wies New York University
The Multicore Transformation
Ubiquity Symposium The Multicore Transformation The Future of Synchronization on Multicores by Maurice Herlihy Editor s Introduction Synchronization bugs such as data races and deadlocks make every programmer
Parallel Algorithm Engineering
Parallel Algorithm Engineering Kenneth S. Bøgh PhD Fellow Based on slides by Darius Sidlauskas Outline Background Current multicore architectures UMA vs NUMA The openmp framework and numa control Examples
Overview of Lecture 4. Memory Models, Atomicity & Performance. Ben-Ari Concurrency Model. Dekker s Algorithm 4
Concurrent and Distributed Programming http://fmt.cs.utwente.nl/courses/cdp/ Overview of Lecture 4 2 Memory Models, tomicity & Performance HC 4 - Tuesday, 6 December 2011 http://fmt.cs.utwente.nl/~marieke/
An Introduction to Parallel Programming
An Introduction to Parallel Programming Ing. Andrea Marongiu (a.marongiu@unibo.it) Includes slides from Multicore Programming Primer course at Massachusetts Institute of Technology (MIT) by Prof. SamanAmarasinghe
Multiprocessor System. Multiprocessor Systems. Bus Based UMA. Types of Multiprocessors (MPs) Cache Consistency. Bus Based UMA. Chapter 8, 8.
Multiprocessor System Multiprocessor Systems Chapter 8, 8.1 We will look at shared-memory multiprocessors More than one processor sharing the same memory A single CPU can only go so fast Use more than
CSE 451: Operating Systems Winter Lecture 7 Synchronization. Hank Levy 412 Sieg Hall
CSE 451: Operating Systems Winter 2003 Lecture 7 Synchronization Hank Levy Levy@cs.washington.edu 412 Sieg Hall Synchronization Threads cooperate in multithreaded programs to share resources, access shared
Locking. Part 2, Chapter 7. Roger Wattenhofer. ETH Zurich Distributed Computing
Locking Part 2, Chapter 7 Roger Wattenhofer ETH Zurich Distributed Computing www.disco.ethz.ch Overview Introduction Spin Locks Test-and-Set & Test-and-Test-and-Set Backoff lock Queue locks Concurrent
NON-BLOCKING DATA STRUCTURES AND TRANSACTIONAL MEMORY. Tim Harris, 28 November 2014
NON-BLOCKING DATA STRUCTURES AND TRANSACTIONAL MEMORY Tim Harris, 28 November 2014 Lecture 8 Problems with locks Atomic blocks and composition Hardware transactional memory Software transactional memory
Multi-core Architectures. Dr. Yingwu Zhu
Multi-core Architectures Dr. Yingwu Zhu What is parallel computing? Using multiple processors in parallel to solve problems more quickly than with a single processor Examples of parallel computing A cluster
Synchronization. CS 475, Spring 2018 Concurrent & Distributed Systems
Synchronization CS 475, Spring 2018 Concurrent & Distributed Systems Review: Threads: Memory View code heap data files code heap data files stack stack stack stack m1 m1 a1 b1 m2 m2 a2 b2 m3 m3 a3 m4 m4
Chapter 5 Asynchronous Concurrent Execution
Chapter 5 Asynchronous Concurrent Execution Outline 5.1 Introduction 5.2 Mutual Exclusion 5.2.1 Java Multithreading Case Study 5.2.2 Critical Sections 5.2.3 Mutual Exclusion Primitives 5.3 Implementing
Multiprocessor Systems. COMP s1
Multiprocessor Systems 1 Multiprocessor System We will look at shared-memory multiprocessors More than one processor sharing the same memory A single CPU can only go so fast Use more than one CPU to improve
Linked Lists: Locking, Lock- Free, and Beyond. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Linked Lists: Locking, Lock- Free, and Beyond Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Coarse-Grained Synchronization Each method locks the object Avoid
More thread safety and Concurrent collections. Nasser Giacaman CompSci 230: Software Design and Construction
More thread safety and Concurrent collections Nasser Giacaman CompSci 230: Software Design and Construction 1 Today's lecture More on Java's mutual exclusion tools Intrinsic locks Developing our own thread-safe
MULTIPROCESSORS AND THREAD LEVEL PARALLELISM
UNIT III MULTIPROCESSORS AND THREAD LEVEL PARALLELISM 1. Symmetric Shared Memory Architectures: The Symmetric Shared Memory Architecture consists of several processors with a single physical memory shared
COL 380: Introduc1on to Parallel & Distributed Programming. Lecture 1 Course Overview + Introduc1on to Concurrency. Subodh Sharma
COL 380: Introduc1on to Parallel & Distributed Programming Lecture 1 Course Overview + Introduc1on to Concurrency Subodh Sharma Indian Ins1tute of Technology Delhi Credits Material derived from Peter Pacheco:
wait with priority An enhanced version of the wait operation accepts an optional priority argument:
wait with priority An enhanced version of the wait operation accepts an optional priority argument: syntax: .wait the smaller the value of the parameter, the highest the priority