Example. Example. Threading Language and Support. CS528 Multithreading: Programming with Threads. The boss/worker model. Thread Programming models

Transcription

1 Threading Language and Support CS528 Multithreading: Programming with Threads A Sahu Dept of CSE, IIT Guwahati Pthread: POSIX thread Popular, Initial and Basic one Improved Constructs for threading c++ thread : available in c++11, c++14 Java thread : very good memory model Atomic function, Mutex Thread Pooling and higher level management OpenMP (loop based) Cilk (dynamic DAG based) A Sahu 1 Thread Programming models The boss/worker model The boss/worker model The peer model A thread pipeline pp Input (Stream) Program Boss main ( ) Workers taskx tasky Files Resources Database s taskz Disks Special Devices Example main(){ /* the boss */ do(1) { get a request; switch( request ) case X: pthread_create(...,taskx); case Y: pthread_create(...,tasky); taskx(){/* worker */ perform the task, sync if accessing shared resources tasky(){/* worker */ perform the task, sync if accessing shared resources Example Above runtime overhead of creating thread Can be solved by thread pool the boss thread creates all worker thread at program Initialization and each worker thread suspends itself immediately for a wakeup call from boss 1

2 Input (static) The peer model Program Workers taskx tasky taskz Files Disks Resources Database s Special Devices Example main(){ do(1){ thread_create(...,thread1...task1); thread_create(...,thread2...task2);... signal all workers to start wait for all workers to finish do any cleanup A thread pipeline Program Filter Threads Stage 1 Stage 2 Stage 3 Input (Stream) Resources Files Files Files Databases Disks Special Devices Databases Disks Special Devices Databases Disks Special Devices Example main() { pthread_create(...,stage1); pthread_create(...,stage2);... wait for all pipeline threads to finish do any cleanup stage1() { get next input for the program do stage 1 processing of the input pass result to next thread in pipeline Example stage2(){ get input from previous thread in pipeline do stage 2 processing of the input pass result to next thread in pipeline stagen() { get input from previous thread in pipeline do stage N processing of the input pass result to program output. A Process With Multiple Threads Thread 1 (main thread) stack 1 Thread 1 context: Data registers Condition codes SP1 PC1 0 Shared code and data shared libraries run time heap read/write data read only code/data Kernel context: VM structures Descriptor table brk pointer Thread 2 (peer thread) stack 2 Thread 2 context: Data registers Condition codes SP2 PC2 2

3 Shared Variables in Threaded C Programs Question: Which variables in a threaded C program are shared variables? Answer not as simple as global variables are shared and stack variables are private Requires answers to the following questions: What is the memory model for threads? How are variables mapped to memory instances? How many threads reference each of these instances? Threads Memory Model Conceptual model: Each thread runs in the context of a process Each thread has its own separate thread context Thread ID, stack, stack pointer, program counter, condition o codes, and dgeneral e purpose pose registers e s All threads share remaining process context Code, data, heap, and shared library segments of process virtual address space Open files and installed handlers Threads Memory Model Operationally, this model is not strictly enforced: Register values are truly separate and protected But any thread can read and write the stack of any other thread Mismatch between conceptual and operational model is a source of confusion and errors Threads Accessing other Thrd s Stack char **ptr; /* global */ int main(){ int i; pthread_t tid; char *msgs[n] = { "Hello1, "Hello2 ; ptr = msgs; for (i = 0; i < 2; i++) pthread_create(&tid,null,thread,(void )i); pthread_exit(null); void *thread(void *vargp) { int myid = (int)vargp; printf("[%d]: %s \n", myid, ptr[myid]); Peer threads access main thread s stack indirectly through global ptr variable Shared Variable Analysis Variable Ref by Ref by Ref by instance Whvariables main are thread? shared? peer thread 0? peer thread 1? ptr yes yes yes i.m yes no no msgs.m yes yes yes myid.p0 no yes no myid.p1 no no yes Answer: A variable x is shared iff multiple threads reference at least one instance of x. Thus: ptr and msgs are shared. i and myid are NOT shared. Parallel Counter: without Lock int cnt = 0; /* shared */ int main() { pthread_t tid1, tid2; pthread_create(&tid1, NULL, count, NULL); pthread_create(&tid2, NULL, count, NULL); pthread_join(tid1, NULL); pthread_join(tid2, NULL); if(cnt!=niters*2)! printf("boom! cnt=%d,cnt); else printf("ok cnt=%d\n, cnt); void *count(void *arg) { for(int i=0;i<niters;i++) cnt++; cnt should be 200 What went wrong?! $./badcnt BOOM! cnt=196 $./badcnt BOOM! cnt=184 3

4 Parallel Counter: with Lock pthread_mutex M; int cnt = 0; /* shared */ int main() { pthread_t tid1, tid2; pthread_create(&tid1, NULL, count, NULL); pthread_create(&tid2, NULL, count, NULL); pthread_join(tid1, NULL); pthread_join(tid2, NULL); if(cnt!=niters*2) printf("boom! cnt=%d,cnt); else printf("ok cnt=%d\n, cnt); void *count(void *arg) { for(int i=0;i<niters;i++) { $./goodcnt pthread_mutex_lock(&m); cnt=200 cnt++; $./goodcnt pthread_mutex_unlock(&m); cnt=200 Thread Safe and Reentrant Thread safe function (Use Mutex) Can be called simultaneously from multiple threads Even when the invocations use shared data Because all references to the shared data are serialized Reentrant function Can also be called simultaneously from multiple threads But only if each invocation uses its own data. 20 Thread Safety Functions called from a thread must be threadsafe We identify four (non disjoint) classes of thread unsafe functions: Class 1: Failing to protect shared variables Class 2: Relying on persistent state across invocations Class 3: Returning pointer to static variable Class 4: Calling thread unsafe functions Thread Unsafe Functions Class 1: Failing to protect shared variables Fix: Use Lock and unlock semaphore operations Issue: Synchronization operations will slow down code Example: goodcnt.c Thread Unsafe Functions (cont) Class 2: Relying on persistent state across multiple function invocations Random number generator relies on static state Fix: Rewrite function so that caller passes in all necessary state, Maintain Thread Specific State int rand() { static unsigned int next = 1; next = next* ; return (unsigned int)(next/65536)% 32768; void srand(unsigned int seed) { next = seed; Thread Unsafe Functions (cont) Class 3: Returning pointer to static variable Fixes: 1. Rewrite code so caller passes pointer to struct,issue: Requires changes in caller and callee Lock and copy : Issue: Requires only simple changes in caller (and none in callee), However, caller must free memory struct hostent *gethostbyname(char* name){ static struct hostent h; <contact DNS and fill in h> return &h; 4

5 Thread Unsafe Functions (cont) Class 3: Returning pointer to static variable struct hostent *gethostbyname_ts(char *p) { struct hostent *q = malloc(...); pthread_mutex_lock(&mutex); /* lock */ p = gethostbyname(name); *q = *p; /* copy */ pthread_mutex_unlock(&mutex); return q; Thread Unsafe Functions Class 4: Calling thread unsafe functions Calling one thread unsafe function makes an entire function thread unsafe Fix: Modify thefunction soit calls only thread safe functions hostp = malloc(...)); gethostbyname_r(name, hostp); Reentrant Functions A function is reentrant iff it accesses NO shared variables when called from multiple threads Reentrant functions are a proper subset of the set of threadsafe functions NOTE: The fixes to Class 2 and 3 thread unsafe functions require modifying the function to make it reentrant (only first fix for Class 3 is reentrant) All functions Thread safe functions Reentrantt functions Thread unsafe functions Thread Safe Library Functions Most functions in the Standard C Library (at the back of your K&R text) are thread safe Examples: malloc, free, printf, scanf All Unix system calls are thread safe Library calls that aren t thread safe: Thread unsafe function Class Reentrant version asctime 3 asctime_r ctime 3 ctime_r Gethostbyaddr 3 gethostbyaddr_r Gethostbyname 3 gethostbyname_r inet_ntoa 3 (none) localtime 3 localtime_r rand 2 rand_r. Many threads trying to acquire Mutex LOCK spin lock CS critical section Resets lock upon exit Synchronization Primitives int pthread_mutex_init( pthread_mutex_t *mutex_lock, const pthread_mutexattr_t *lock_attr); int pthread_mutex_lock( pthread_mutex_t *mutex_lock); int pthread_mutex_unlock( pthread_mutex_t *mutex_lock); int pthread_mutex_trylock( pthread_mutex_t *mutex_lock); 5

6 Locking Overhead Serialization points Minimize the size of critical sections Be careful Rather than wait, check if lock is available pthread _ mutex_ trylock If already locked, will return EBUSY Will require restructuring of code Suspend self by pthread_yeild() Give chance to others Suspend self by doing a timed wait.. {1, 1, 1,.., {1, 2, 3, 4,.., {1,2,4,8,.., Performance of Locking Spinning/busy wait waste time Recall MAC Protocol Non Persistence CSMA protocol Wait random time if medium if busy, then send Spin lock with exponential back off reduces contention Wait k amount of time for 1 st attempt Wait k*c i amount of time for i th attempt time Spin/Continuous Lock threads Backoff lock A Sahu slide 32 Example: simple backup void ExpBackup(int _ival){ int i=0,ival=_ival; while (i<100) { if (m.try_lock()) { ival=_ival; BigNonSharedWork();i++;m.unlock(); else { ival=ival*2; chrono::milliseconds interval(ival); this_thread::sleep_for(interval); //End Else //End while //EndExpBackup Example :simple backup int main(int argc, char *argv[]){ thread th[10]; for(int i=0;i<10;i++) th[i]=thread(expbackup, atoi(argv[1])); for(int i=0;i<10;i++) th[i].join(); return 0; Condition Variables for Synchronization Condition variable allows a thread To block itself until specified data reaches a predefined state. Condition variable Associated with a predicate (P) When the predicate becomes true, the condition variable is used to signal one or more threads waiting on the condition. Single condition variable May be associated with more than one predicate Ex: P= X OR Y AND (Z OR K) Condition Variables for Synchronization A condition variable always has a mutex associated with it. A thread locks this mutex and tests the predicate defined on the shared variable. If the predicate is not true The thread waits on the condition variable associated with the predicate Using the function pthread_cond_wait. 6

7 Condition Variables for Synchronization Pthreads provides the following functions for condition variables int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); int pthread_cond_signal(pthread_cond_t*cond); _ //Take out top thread of queue, signal that thread int pthread_cond_broadcast(pthread_cond_t *cond); int pthread_cond_init(pthread_cond_t *cond, const pthread_condattr_t *attr); int pthread_cond_destroy(pthread_cond_t *cond); Barriers Synchronization Barrier holds a thread until all threads participating in the barrier have reached it Barriers can be implemented Using a counter, a mutex and a condition variable. Counter is used to keep track of the number of threads that have reached the barrier. If the count is less than the total number of threads, the threads execute a condition wait. The last thread entering (and setting the count to the number of threads) wakes up all the threads using a condition broadcast. DoWork(){ Do Phase I work Barrier(); Do Phase II work Barrier(); Do Phase III work() Barrier(); for (i=0;i<numproc;i++){ P[i].Start(DoWork); Print result(); Initialize Phase I N Phase II N 1 Phase III 2 3 N Print Result A Sahu slide 40 w1 w4 w3 w4 Example: suppose control will not pass the point before all the thread/child finishes Work barrier: phase wise work pthread_barrier typedef struct{ pthread_mutex_t pthread_cond_t int count; mylib_barrier_t; count_lock; ok_to_proceed; void mylib_init_barrier(mylib_barrier_t *b){ b > count = 0; pthread_mutex_init(&(b >count_lock), NULL); pthread_cond_init(&(b > ok_to_proceed), NULL); 41 7

8 void mylib_barrier (mylib_barrier_t *b, int num_threads) { pthread_mutex_lock(&(b > count_lock)); b > count ++; if (b > count == num_threads) { b > count = 0; pthread_cond_broadcast(&(b > ok_to_proceed)); else while (pthread_cond_wait(&(b > ok_to_proceed), &(b > count_lock))!= 0); pthread_mutex_unlock(&(b > count_lock)); Sum of all finished processor Lock P1 P2 P3 PN Every one accessing to same lock O(n) Lock1 P1 P1 PM Lock Lock2 P1 P1 PM O(lg n) Lock contention is distributed in a Tree Fashion A Sahu slide 44 Synch. Primitives pthread_mutex_init, lock, unlock, trylock pthread_attr_setdetachstate, guardsize_np, stacksize, inheritsched, schedpolicy, schedparam pthread_cond_wait, signal, broadcast, init, destroy Our main concern Lock, unlock, trylock, condsignal, condbroadcast Synchronization Hardware Many systems provide hardware support for Sync. All solutions below based on idea of locking Protecting critical regions via locks Uniprocessors could disable interrupts Currently running code would execute without preemption Generally too inefficient on multiprocessor systems Operating systems using this not broadly scalable Atomic Sync Instructions Multiprocessor disable interrupts Generally too inefficient on multiprocessor systems Operating systems using this not broadly scalable Modern machines provide special atomic hardware instructions Atomic = non interruptible Either test memory word and set value Or swap contents of two memory words Test and Set (TAS) Compare and Swap (CAS) Exchange (XCHG) Fetch and Increment (FAI) How to provide these Load Locked and Store Conditional 8

9 test_and_set Instruction : TAS //Definition: bool test_and_set(bool *target){ bool rv = *target; *target = TRUE; return rv: Executed atomically Returns the original value of passed parameter Set the new value of passed parameter to TRUE. Shared Boolean variable lock, initialized to FALSE Solution: do { while (test _ and_ set(&lock)) ; /* do nothing */ /* critical section */ lock = false; /* remainder section */ while (true); int CAS(int *value, int expected, int new_value){ int temp = *value; if (*value == expected) *value = new_value; return temp; Executed atomically Returns the original value of passed parameter value Set the variable value the value of the passed parameter new_value but only if value == expected. That is, the swap takes place only under this condition. Sync Solution using CAS Shared integer lock initialized to 0; do { while (CAS(&lock, 0, 1)!= 0) ; /* do nothing */ /* critical section */ lock = 0; /* remainder section */ while (true); Synchronization Hierarchy One == (used by )== > other LL+SC ==> TAS/CAS/FAI/XCHG==>Lock/Unlock All TAS/CAS/GAS/FAI/XCHG do the same work Lock/Unlock == > Mutex //Mutex use L/UL Mutex == > Semaphore // Semaphore uses Mutex Wait() and Signal() Semaphore == > Monitor //Monitor uses Semaphore Many wait/many Signal, Processes in Queue Monitor : Another Abstract Type which use semaphore, mutex, conditions Semaphore Semaphore: Synchronization tool Provides more sophisticated ways (than Mutex) For process to synchronize their activities. Semaphore: Abstract data type Used for controlling access, by multiple processes Can be access by two atomic Wait() and Signal() Edsger Wybe Dijkstra (SSSP Algo) Proberen (to test) Verhogen (to increment) In short P() and V() 9

10 Semaphore: Wait(), Signal() void synchronized wait(s) { while (S <= 0) ; // busy wait S ; void synchronized signal(s) { S++; synchronized function run atomically import java.util.concurrent.atomic; import java.util.concurrent.*; Binary Semaphore: Wait(), Signal() S=1; // Initialized to 1 // S =0 Locked; S=1 Available void synchronized wait(s) { while (S <= 0) ; // busy wait S ; void synchronized signal(s) { S++; A high level abstraction that provides A convenient and effective mechanism for process synchronization Monitor : Abstract data type internal variables only accessible by code within the procedure Only one process may be active within the monitor at a time Remember synchronized function of java.util Counting Semaphore: Wait(), Signal() S=50; // Initialized to 1 // S =0 Locked; S>=1 Available void synchronized wait(s) { while (S <= 0) ; // busy wait S ; void synchronized signal(s) { S++; C++ Thread Asynchronous tasks and threads Promises and tasks Mutexes and condition variables Atomics C++ Thread:atomic atomic_uint AtomicCount; void DoCount(){ int j, timesperthrd; timesperthrd=(times/num_threads); for(j=0;j<timesperthrd;j++) AtomicCount++; main(){ thread T[NUM_THREADS]; int i; for(i=0;i<num_threads;i++) T[i]=thread(DoCount); for(i=0;i<num_threads;i++) T[i].join(); 10