Concurrency: Signaling and Condition Variables

Transcription

1 Concurrency: Signaling and Condition Variables Questions Answered in this Lecture: How do we make fair locks perform better? How do we notify threads of that some condition has been met? Hale CS450 Thanks to Remzi & Andrea Arapci-Dusseau for slide material 1

2 Recall: Ticket Lock Implementation typedef struct lock_t { int ticket; int turn; void lock_init(lock_t *lock) { lock->ticket = 0; lock->turn = 0; void acquire(lock_t *lock) { int myturn = FAA(&lock->ticket); while (lock->turn!= myturn); // spin void release (lock_t *lock) { FAA(&lock->turn); Hale CS450 2

3 Spinlock Performance Fast when - many CPUs - locks held a short time - advantage: avoid context switch Slow when - one CPU - locks held a long time - disadvantage: spinning is wasteful Hale CS450 3

4 CPU Scheduler is Ignorant lock unlock lock spin spin spin spin spin A B C D A B C D CPU scheduler may run B instead of A even though B is waiting for A Hale CS450 4

5 Ticket Lock with yield() typedef struct lock_t { int ticket; int turn; void lock_init(lock_t *lock) { lock->ticket = 0; lock->turn = 0; void acquire(lock_t *lock) { int myturn = FAA(&lock->ticket); while (lock->turn!= myturn) yield(); void release (lock_t *lock) { FAA(&lock->turn); Hale CS450 5

6 yield() Instead of spin lock unlock lock spin spin spin spin spin no yield: A B C D A B C D lock unlock lock yield: A A B Hale CS450 6

7 Spinlock Performance Waste Without yield: O(threads * time_slice) With yield: O(threads * context_switch) So even with yield, spinning is slow with high thread contention Next improvement: Block and put thread on waiting queue instead of spinning Hale CS450 7

8 How do we evaluate lock performance? Fairness: do some processes wait more than others? another way: do processes get lock in order requested? Performance: it depends Is the lockcontended? (many threads attempting to acquire lock) Or is it uncontended? (likely to get the lock if we request it) Hale CS450 8

9 Why is yield() useful? How can it affect performance? lock unlock lock spin spin spin spin spin no yield: A B C D A B C D lock unlock lock yield: A A B Hale CS450 9

10 Lock implementation: block when waiting Lock implementation removes waiting threads from the ready queue (e.g. with park() and unpark()) Scheduler only runs ready threads Separates concerns (programmer doesn t need to deal with yield vs not/spinning issue) Quiz: where should locks be implemented? Hale CS450 10

11 Example Ready: {A, B, C, D Waiting: { Running: { Hale CS450 11

12 Example Ready: {B, C, D Waiting: { Running: {A lock() A Hale CS450 12

13 Example Ready: {A, C, D Waiting: { Running: {B lock() A B Hale CS450 13

14 Example Ready: {A, C, D Waiting: {B Running: { lock() lock() A B Hale CS450 14

15 Example Ready: {A, D Waiting: {B Running: {C lock() lock() A B C Hale CS450 15

16 Example Ready: {A, C Waiting: {B Running: {D lock() lock() A B C D Hale CS450 16

17 Example Ready: {A, C Waiting: {B, D Running: { lock() lock() lock() A B C D Hale CS450 17

18 Example Ready: {C Waiting: {B, D Running: {A lock() lock() lock() A B C D A Hale CS450 18

19 Example Ready: {A Waiting: {B, D Running: {C lock() lock() lock() A B C D A C Hale CS450 19

20 Example Ready: {C Waiting: {B, D Running: {A lock() lock() lock() A B C D A C A Hale CS450 20

21 Example lock() lock() lock() Ready: {C, B, D Waiting: { Running: {A unlock() A B C D A C A Hale CS450 21

22 Example lock() lock() lock() Ready: {C, B, D Waiting: { Running: {A unlock() A B C D A C A Hale CS450 22

23 Example lock() lock() lock() Ready: {B, D, A Waiting: { Running: {C unlock() A B C D A C A C Hale CS450 23

24 Example Ready: {D, A, C Waiting: { Running: {B lock() lock() unlock() lock() A B C D A C A B Hale CS450 24

25 What do we get? Threads aren t contending on the lock when they shouldn t be (mostly) Fewer context switches Hale CS450 25

26 Lock: block when waiting typedef struct { bool lock = false; bool guard = false; queue_t q; lock_t; 1. Why do we need guard? 2. Why is it OK to spin on guard? 3. In release(), why not set lock=false? 4. What is the race condition? void acquire(lock_t *l) { while (TAS(&l->guard, true)); if (l->lock) { enqueue(l->q, tid); l->guard = false; park(); // blocked else { l->lock = true; l->guard = false; void release(lock_t *l) { while (TAS(&l->guard, true)); if (qempty(l->q)) l->lock=false; else unpark(dequeue(l->q)); l->guard = false; Hale CS450 26

27 time Race! void acquire(lock_t *l) { while (TAS(&l->guard, true)); if (l->lock) { enqueue(l->q, tid); l->guard = false; Thread 1 Thread 2 park(); // blocked else { l->lock = true; l->guard = false; void release(lock_t *l) { while (TAS(&l->guard, true)); if (qempty(l->q)) false! l->lock=false; else unpark(dequeue(l->q)); l->guard = false; bad! Hale CS450 27

28 Lock: the fix typedef struct { bool lock = false; bool guard = false; queue_t q; lock_t; 1. Why does this fix the race? 2. What does the scheduler need to do here? 3. Why can t we just hold the guard when we park()? void acquire(lock_t *l) { while (TAS(&l->guard, true)); if (l->lock) { enqueue(l->q, tid); setpark(); // notify of plan l->guard = false; park(); // unless unpark() else { l->lock = true; l->guard = false; void release(lock_t *l) { while (TAS(&l->guard, true)); if (qempty(l->q)) l->lock=false; else unpark(dequeue(l->q)); l->guard = false; Hale CS450 28

29 Spin-waiting vs. Blocking Each approach has tradeoffs: Uniprocessor Waiting process is scheduled -> process holding lock isn t Waiting process should always be descheduled Associate queue of waiters with each lock Multiprocessor Waiting process is scheduled -> process holding lock might also be Spin or block depends on how long (!) before lock is released Lockreleased quickly? spin() Lock taken for long time? block() we ll define this relative to the context switch time Hale CS450 29

30 Oracle Suppose we know how long (!) a lock will be held And suppose we make our decision to block or spin based on ", the cost of a context switch #$%&'( = * %,./&( % >, 12'$3 BUT: we have to know the future! Hale CS450 30

31 Applying Bounds The theory: bound our worst-case performance using actual wait time over optimal (oracle) Consider:! # What would the oracle do? What can we do (without knowing future)?! > # What would the oracle do? What can we do? We pay 2C, because we wait an extra C. 2C/C = 2 => 2-competitive algorithm! Hale CS450 31

32 Concurrency Goals Mutual Exclusion Keep two threads fromexecuting in a critical section concurrently We solved this with locks Dependent Events We want a thread to wait until some particular event has occurred Or some condition has been met Solved with condition variables and semaphores Hale CS450 32

33 I just dropped in to see what condition my condition was in Hale CS450 33

34 Example: join() pthread_t p1, p2; pthread_create(&p1, NULL, mythread, A ); pthread_create(&p2, NULL, mythread, B ); // join waits for the threads to finish pthread_join(p1, NULL); pthread_join(p2, NULL); printf( Main: done\n [balance: %d]\n, balance); return 0; Hale CS450 34

35 Condition Variables CV: queue of waiting threads B waits for a signal on CV before running wait(cv, ); A sends signal() on CV when time for B to run signal(cv, ); Hale CS450 35

36 API wait(cond_t * cv, mutex_t * lock) assumes lock is held when wait() is called (why?) puts caller to sleep + releases the lock (atomically) when awoken, reacquires lock before returning signal(cond_t * cv) wake a single waiting thread (if >= 1 thread is waiting) if there is no waiting thread, NOP Hale CS450 36

37 Join Implementation: Attempt #1 parent void thread_join() { // x cond_wait(&c, &m); // y // z child void thread_exit() { cond_signal(&c); // a // b // c Hale CS450 37

38 Join Implementation: Attempt #1 parent void thread_join() { // x cond_wait(&c, &m); // y // z child void thread_exit() { cond_signal(&c); // a // b // c Example schedule: parent x y z child a b c Hale CS450 38

39 Join Implementation: Attempt #1 parent void thread_join() { // x cond_wait(&c, &m); // y // z child void thread_exit() { cond_signal(&c); // a // b // c Example (bad) schedule: parent child a b c x y parent will wait forever! Hale CS450 39

40 Rule of Thumb #1 Always associate some state with the CV CVs are used to signal() when state changes If state is as required, don t need to wait for a signal() Hale CS450 40

41 Join Implementation: Attempt #2 parent void thread_join() { // w if (!done) // x cond_wait(&c, &m);// y // z child void thread_exit() { done = 1 // a cond_signal(&c); // b Hale CS450 41

42 Join Implementation: Attempt #2 parent void thread_join() { // w if (!done) // x cond_wait(&c, &m);// y // z Example schedule: parent child a b w x child void thread_exit() { done = 1 // a cond_signal(&c); // b fixed! y z Hale CS450 42

43 Join Implementation: Attempt #2 parent void thread_join() { // w if (!done) // x cond_wait(&c, &m);// y // z Example (bad) schedule: parent child w x a b child void thread_exit() { done = 1 // a cond_signal(&c); // b sleeps forever! y Hale CS450 43

44 Join Implementation: Attempt #3 (correct) parent void thread_join() { // w if (!done) // x cond_wait(&c, &m);// y // z child void thread_exit() { // a done = 1 // b cond_signal(&c); // c // d Hale CS450 44

45 Join Implementation: Attempt #3 (correct) parent void thread_join() { // w if (!done) // x cond_wait(&c, &m);// y // z Example chedule: parent child w x y a b child void thread_exit() { // a done = 1 // b cond_signal(&c); // c // d c d z fixed! Hale CS450 45

46 Signaling between threads Bounded Buffer (producer-consumer queue) Hale CS450 46

47 Signaling between threads Bounded Buffer (producer-consumer queue) t1: put() Hale CS450 47

48 Signaling between threads Bounded Buffer (producer-consumer queue) t1: put() Hale CS450 48

49 Signaling between threads Bounded Buffer (producer-consumer queue) t2: take() Hale CS450 49

50 Signaling between threads Bounded Buffer (producer-consumer queue) t2: take() Hale CS450 50

51 Signaling between threads Bounded Buffer (producer-consumer queue) Hale CS450 51

52 Signaling between threads Bounded Buffer (producer-consumer queue) t2: take()??? Hale CS450 52

53 Signaling between threads Bounded Buffer (producer-consumer queue) wait queue t2 What condition are we waiting on? Hale CS450 53

54 Signaling between threads Bounded Buffer (producer-consumer queue) wait queue t2 t1: put() Hale CS450 54

55 Signaling between threads Bounded Buffer (producer-consumer queue) wait queue t2 t1: put() What should happen? Hale CS450 55

56 Signaling between threads Bounded Buffer (producer-consumer queue) wait queue t2 t1: signal() Hale CS450 56

57 Signaling between threads Bounded Buffer (producer-consumer queue) wait queue t2: take() Hale CS450 57

58 Signaling between threads Bounded Buffer (producer-consumer queue) wait queue t2: take() Hale CS450 58

59 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) Hale CS450 59

60 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) most recent put Hale CS450 60

61 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) We re full Hale CS450 61

62 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t2:put() Hale CS450 62

63 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t2 Hale CS450 63

64 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t2 t1: take() Hale CS450 64

65 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t2 Hale CS450 65

66 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t2 t1: signal() Hale CS450 66

67 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t1: take() Hale CS450 67

68 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t2:put() Hale CS450 68

69 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t2:put() Hale CS450 69

70 The queue is a circular queue Also sometimes called a ring buffer Bounded Buffer (producer-consumer queue) wait queue t2:put() wraparound property Hale CS450 70

71 Bounded Buffer Reads and writes to buffer require locking when buffers are full, writers wait() when buffers are empty, readers wait() Many programming languages expose this abstraction directly chan in Go channel in Julia Python requires a queue() with a lock() Rust might use CircularBuffer() and CondVar() maybe crossbeam_channel() Hale CS450 71

72 Bounded Buffer Producers generate data Consumers take data and process it Very frequent situation encountered in systems programming General strategy: use CVs to notify: waiting readers when data is available waiting writers when slots are free Hale CS450 72

73 Example Easy case: 1 consumer thread 1 producer thread 1 shared buffer (max slots=1) numfill is buffer slots currently filled let s start with some code that doesn t quite work Hale CS450 73

74 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 74

75 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 75

76 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 76

77 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 77

78 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 78

79 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 79

80 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 80

81 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 81

82 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 82

83 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNABLE] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 83

84 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNABLE] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 84

85 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNABLE] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 85

86 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNABLE] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 86

87 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNABLE] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 87

88 [SLEEPING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNABLE] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 88

89 [SLEEPING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 89

90 [SLEEPING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 90

91 [SLEEPING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 91

92 [SLEEPING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 92

93 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 93

94 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 94

95 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 95

96 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 96

97 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 97

98 [RUNNABLE] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [RUNNING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 98

99 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS450 99

100 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS

101 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 0 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS

102 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [SLEEPING] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS

103 [RUNNING] void *producer (void *arg) { for (int i = 0; i < loops; i++) { while (numfull == max) do_fill(i); numfull = 1 [RUNNABLE] void *consumer (void *arg) { while (1) { while (numfull == 0) int tmp = do_get(); printf( %d\n, tmp); Hale CS

104 What about two consumers? Can you find a bad schedule? Hale CS

105 void *producer (void *arg) { for (int i = 0; i < loops; i++) { // p1 while (numfull == max) //p2 //p3 do_fill(i); // p4 // p5 // p6 Producer: Consumer1: [RUNNING] wait() wait() c1 c2 c3 signal() p1 p2 p3 p4 p5 p6 p1 p2 p3 void *consumer (void *arg) { while (1) { // c1 wait() Consumer2: c1 c2 c3 c2 c4 c5 [RUNNABLE] while (numfull == 0) // c2 int tmp = do_get(); // c4 // c5 // c6 printf( %d\n, tmp); // c7 signal() does last signal wake producer or consumer1?? Hale CS

106 How to wake the right thread? One solution: wake allthreads! Hale CS

107 API wait(cond_t * cv, mutex_t * lock) assumes lock is held when wait() is called (why?) puts caller to sleep + releases the lock (atomically) when awoken, reacquires lock before returning signal(cond_t * cv) wake a single waiting thread (if >= 1 thread is waiting) if there is no waiting thread, NOP broadcast(cond_t * cv) wake all waiters if no waiting threads, NOP Hale CS

108 Summary Rules of thumb Keep state in addition to CVs Always wait() or signal() with lock held Recheck state assumptions when waking up from waiting Hale CS