Transactional Memory. Yaohua Li and Siming Chen. Yaohua Li and Siming Chen Transactional Memory 1 / 41

Size: px

Start display at page:

Download "Transactional Memory. Yaohua Li and Siming Chen. Yaohua Li and Siming Chen Transactional Memory 1 / 41"

Aileen Lamb
5 years ago
Views:

1 Transactional Memory Yaohua Li and Siming Chen Yaohua Li and Siming Chen Transactional Memory 1 / 41

2 Background Processor hits physical limit on transistor density Cannot simply put more transistors to get more performance Multi-processor and multi-core systems are now normal Full utilization via threaded execution Synchronization is needed to access shared memory Otherwise program state may become corrupted due to interleaving accesses Mutual exclusion locks have been widely used for synchronization Yaohua Li and Siming Chen Transactional Memory 2 / 41

3 Mutual Exclusion Locks Popular for its simplicity and support by general-purpose computers Only the thread owning lock can access shared object Difficult to use and easily causes many problems, if number of locks becomes large Deadlock/Livelock if two or more threads need to obtain multiple locks, they may block each other indefinitely or come into a situation that never exits Priority Inversion if low-priority task obtains a lock, it could block a high-priority task which also wants this lock Convoy if a thread owning a lock is de-scheduled from running, then other threads are prevented from making progress Effort for perfect locking can be huge Race condition can raise, if system run without certain locks, leading to errors and vulnerabilities Yaohua Li and Siming Chen Transactional Memory 3 / 41

4 Transactional Memory What is it? A mechanism to tackle parallel-programming by abstracting concurrent access to shared memory. With TM, Multiple threads can simultaneously try to access shared memory in an atomic way. 1 atomic { Atomic transaction: all the accesses of a thread succeed or none 2 hist[array[i][j]]++; 3 } Code of atomic region that computes the histogram of an array Yaohua Li and Siming Chen Transactional Memory 4 / 41

5 Blocking For highly scalable network servers, non-blocking I/O APIs are desired Server is notified when a channel is ready for more input or output operations Fire an API call, do something else, and check its status later Demand-driven: input and output processing only occur when input data or output space are available Non-blocking synchronization algorithms Does not require mutual exclusion locks to achieve safe concurrency Try to keep threads (CPUs) from busy waiting for locks Transactional memory is one member Yaohua Li and Siming Chen Transactional Memory 5 / 41

6 Transaction Transaction Transaction is composed of one or more revocable, temporary steps, which are committed on the whole with a single atomic command, and no step has visible impact until the commit phase completes. Yaohua Li and Siming Chen Transactional Memory 6 / 41

7 Transaction ACID Properties Atomicity the transaction will complete in its entirety or not at all Consistency all modifications preserve the underlying structure of the object being modified in an consistent way Isolation operations may appear to have executed in isolation, in their entirety. No partial modification is perceived. Durability once committed the changes persist, or once aborted no residue of partial transaction will remain In regard to transactional memory, we are more interested in amoticity and isolation. Yaohua Li and Siming Chen Transactional Memory 7 / 41

8 Comparison Conflict violation of a temporal order, when performing memory access Transactional Memory Optimistic against conflict Abandon work of one of conflicting transactions Mutual Exclusion Lock Pessimistic against conflict Prevent conflicts from happening Since actual conflicts are rare in many programs, the optimistic transactional memory makes more sense. Yaohua Li and Siming Chen Transactional Memory 8 / 41

9 Benefits of Transactional Memory TM provides Higher-level abstraction Better trade-off between scaling and implementation effort Inherently deadlock free since no lock used Failure atomicity all or none Yaohua Li and Siming Chen Transactional Memory 9 / 41

10 Synchronization Levels Hierarchy of synchronization levels within non-blocking algorithms Obstruction-freedom guarantees that a thread operating on a shared data structure will complete, if eventually executed in isolation. It means deadlock will not occur, but no guard against livelock. Lock-freedom denies livelock, i.e. guarantees that all threads of execution will make progress, are lock-free, and all lock-free algorithms are obstruction-free. It does not guarantee that individual threads may not starve. Wait-freedom guarantees that any thread can complete any operation in a finite number of steps, and all wait-free algorithms are also lock-free. Yaohua Li and Siming Chen Transactional Memory 10 / 41

11 Hardware Support for TM No intrinsic support for transactional algorithms in most machines Support instruction for basic atomic operations CompareAndSwap is one such instruction commonly found in modern processors Yaohua Li and Siming Chen Transactional Memory 11 / 41

12 CompareAndSwap CompareAndSwap, CAS Set a memory location to a new value, if it currently contains a specified expected value CompareAndSwap (a:wordaddress, old:word, new:word): Bool if *a = old then *a new return True else return False Executed in one machine instruction, providing atomicity Building block for create arbitrarily complicated non-blocking algorithms Yaohua Li and Siming Chen Transactional Memory 12 / 41

13 More Instructions LoadLinked Load a value from memory and lock it LoadLinked (r :Register, a :WordAddress) r *a Linked(a ) True StoreConditional Store succeeds only if the memory location read in the LoadLinked step has not been modified by some other memory operation StoreConditional (r :Register, a :WordAddress) if Linked(a ) = True then *a r r 1 else r 0 Yaohua Li and Siming Chen Transactional Memory 13 / 41

14 Example CompareAndSwap Obstruction-free double-ended queue implementation. Push and Pop operations from both sides Look into only operations on right side Implemented with CompareAndSwap Yaohua Li and Siming Chen Transactional Memory 14 / 41

15 RightPush Procedure 1 type element = val: valtype; ctr: int 2 RightPush(A, value ) 3 while True 4 do k Oracle(A, right ) 5 prev A k 1 // author cited algorithm wrong here 6 cur A k 7 if prev.val End r cur.val = End r 8 then if k = Max then return Full 10 if CAS(&A k 1, prev, prev.val, prev.ctr + 1 ) 11 then if CAS(&A k,cur, value, cur.ctr + 1 ) 12 then return OK The Oracle procedure guesses where in the queue the left or right end of the queue is, and returns that index. Yaohua Li and Siming Chen Transactional Memory 15 / 41

16 RightPop Procedure 1 type element = val: valtype; ctr: int 2 RightPop(A) 3 while True 4 do k Oracle(A, right ) 5 cur A k 1 6 next A k 7 if cur.val End r next.val = End r 8 then if cur.val = End l A k 1 = cur 9 then return Empty 10 if CAS(&A k, next, End r, next.ctr + 1 ) 11 then if CAS(&A k 1,cur, End r, cur.ctr + 1 ) 12 then return cur.val The Oracle procedure guesses where in the queue the let or right end of the queue is, and returns that index. Yaohua Li and Siming Chen Transactional Memory 16 / 41

17 Implementations Hardware Transactional Memory (HTM) Software Transactional Memory (STM) Hybrid Transactional Memory (HyTM) Hardware-assisted STM (HaSTM) HyTM Supports HTM but falls back on STM transactions when hardware resources are exceeded. HaSTM Combines STM with new architectural support to accelerate parts of the STM s implementation. Yaohua Li and Siming Chen Transactional Memory 17 / 41

18 Hardware transactional memory Minimalist approaches ISA additions Complement the instruction set architecture (ISA) with a small set of new instructions. Cache/Buffer modifications Modify the cache consistency protocols Alternates Require minimal or no ISA support or cache modifications. Yaohua Li and Siming Chen Transactional Memory 18 / 41

19 API design: ISA additions STR/ETR: Start/end transaction TLD/TST: Transactional read/write ABR/VLD: Abort/validation of a transaction ABR Select a victim transaction for aborting under program control VLD Can validate a transaction and catch a conflict early Yaohua Li and Siming Chen Transactional Memory 19 / 41

20 Data versioning and conflicts: cache/buffer modifications Work at the word or cache line level Transactional loads and stores in a separate transactional cache or in conventional data caches with transactional support Keep transactions speculative state in the data cache or in a hardware buffer area Rely on extending cache coherence protocols, such as MESI (modified, exclusive, shared, invalid) to detect conflicts and enforce atomicity Yaohua Li and Siming Chen Transactional Memory 20 / 41

21 One design of cache modifications by Herlihy and Moss Keeps the transactions read set and write set in the data cache Hardware extension: two extra bits needed per cache line Two bits Indicate whether the line is to be discarded on commit (for lines holding unmodified data) or on abort (for speculatively modified lines). Protocol extension: whether the version is to be kept or dropped. Conflict A load has read invalid data and the associated transaction must abort On abort, the write set of the aborting transaction (associated with the tentative store instructions) is dropped On commit, the version of the original values before the store instructions are dropped, and the transactions speculative stores are committed to memory Yaohua Li and Siming Chen Transactional Memory 21 / 41

22 One variant of the design The system keeps the original state in main memory Keeps the speculative state in the data cache One bit added per cache line for this design The bit is set when a transaction accesses the line Upon commit and abort, the bit is cleared On abort, the modified lines with this bit set are also invalidated Yaohua Li and Siming Chen Transactional Memory 22 / 41

23 Software transactional memory First use By Shavit and Touitou in 1995 Basic case Non-nesting transactions that make updates to shared memory within a single multithreaded process Main problems an STM must tackle Must provide separate per-thread views of the heap Must provide a mechanism for detecting and resolving conflicts between transactions Yaohua Li and Siming Chen Transactional Memory 23 / 41

24 STM API design: Managing transactional state Mechanism Allows a transaction to see its own writes as it continues to run and allows memory updates to be discarded if the transaction ultimately aborts Data organization in memory One approach separates transactional data and ordinary data, introducing a distinct memory format for transactional objects; An alternative approach allows data to retain its ordinary structure in memory, and the STM uses separate structures to maintain its own metadata. Object-based STM (OSTM) Uses the object header to track which transactions are currently accessing the object. The programming API provides operations to open an object header, returning a reference to a copy of the objects body for the transaction to use Bartok STM Makes no low-level distinction between ordinary and transactional data Yaohua Li and Siming Chen Transactional Memory 24 / 41

25 OSTM API design o body OpenForReading(tx tx, o header o); o body OpenForWriting(tx tx, o header o); Open-ForWriting Returns a private shadow copy of the object body Open-ForReading only Must not update object bodies Pointer-based data structures Must always add an extra level of indirection by storing references to object headers rather than direct references to object bodies Yaohua Li and Siming Chen Transactional Memory 25 / 41

26 OSTM API design Read/Write set Maintained by the OSTM runtime system Abort Discards any shadow copies that have been created for transaction Commit 1) Atomically checks that no conflicting transaction has updated objects in the read set or the write set 2) Updates the object headers for objects in the write set, thus publishes the private shadow copies as the objects new contents. Yaohua Li and Siming Chen Transactional Memory 26 / 41

27 Bartok STM API design Metadata Holded by STM in separate structures for concurrency control TMW Transactional Metadata Word, mapped by STM from a word s address to manage that data TMW API Includes functions to open the TMWs for the data that it will read from or write to Read/Write API void OpenForReading(tx *tx, tmw *t); void OpenForWriting(tx *tx, tmw *t); word STMRead(tx *tx, word *a); void STMWrite(tx *tx, word *a, word d); Yaohua Li and Siming Chen Transactional Memory 27 / 41

28 Bartok STM API design: Compiler s Translation 1 atomic { 2 hist[index]++; 3 } After compiler s translation 1 atomic { OpenForReading(tr, TMW_FOR(index)); 4 OpenForWriting(tr, TMW_FOR(hist)); 5 int *addr = &hist[stmread(tr, &index)]; 6 STMWrite(tr, addr, STMRead(tr, addr) + 1); } The atomic block itself must be translated into calls to library functions to start and commit transactions and to reexecute the transaction if the commit fails. Yaohua Li and Siming Chen Transactional Memory 28 / 41

29 Bartok STM API implementation Broadly, there are two ways of managing tentative updates. Buffered updates Keeps a private shadow copy of all the memory words it updates. Calls to STMRead must consult the shadow copies so that they will see earlier writes by the same transaction. Hashing can accelerate this look-up (mapping an address to a slot in the current transactions shadow table to avoid searching it). In-place updates STMWrite directly updates the heap so that calls to STMRead will see earlier updates without needing to search a table. In this case, STMWrite must maintain an undo log of all values that it overwrites, so that the transaction can roll back its changes if it aborts. Yaohua Li and Siming Chen Transactional Memory 29 / 41

30 Detecting and resolving conflicts in STM Blocking Objects locking, which acquires locks as a transaction executes and holds them until it commits. (lock in shared-read mode) Disadvantage Scales poorly on multiprocessor hardware because it introduces contention in the memory hierarchy. Nonblocking OSTM performs an atomic multiword update across the header contents of the objects in the read and write sets. It checks that there has been no update to objects in the read set and updates the object headers in the transactions write set to publish the transactions shadow copies. Advantage This design avoids read locks; the transactions read set is validated purely by memory read operations. Disadvantage Complicated in terms of both software engineering and the number of atomic compare-and-swap operations used at runtime. Yaohua Li and Siming Chen Transactional Memory 30 / 41

31 Hybrid approach in detecting and resolving conflicts Hybrid approach Combines optimistic and pessimistic schemes by using versioned mutual-exclusion locks, which support normal mutual-exclusion semantics and provide access to a version number counting the number of times the lock has been acquired and released. The design uses mutual exclusion for pessimistic concurrency control to grant write access to the data the lock protects. The version number allows optimistic concurrency control for read access. That is, a transaction records the version number before it first reads from an object and then, at commit time, checks that the version number is unchanged, meaning that no one has updated the object concurrently with the transaction. Yaohua Li and Siming Chen Transactional Memory 31 / 41

32 Procedure of Hybrid approach 1 write { 2 acquire lock to write the object; 3 update the lock version number; 4 write access; 5 release write lock; 6 } 1 read { 2 record the object version number as v1; 3 read access; 4 record the object version number as v2; 5 if (v1 == v2) { 6 commit; 7 } 8 else 9 abort; 10 } Yaohua Li and Siming Chen Transactional Memory 32 / 41

33 Challenges Open/Closed nesting transactions I/O TM mixed with programming models, OpenMP or MPI Yaohua Li and Siming Chen Transactional Memory 33 / 41

34 Open and closed nesting Closed nesting Either all or none of the transactions in a nested region commit Open nesting When an inner transaction commits, its effects become visible for all threads in the system. Flattening model Includes all nested transactions in the outermost transaction HTM systems must use counters to implement flattening (increment for STR, decrement for ETR, commit when zero) On conflict, must roll back to the outermost transaction Alternate Allow each nested transaction to have its own read and write sets Open-nested transactions increase the programmers burden. Compensating actions Complex and the programmer must have an expert grasp of the codes semantics. Yaohua Li and Siming Chen Transactional Memory 34 / 41

35 Open and closed nesting challenges Hardware complexity for flattening model while limiting concurrency and performance Proposed open-nested transaction increase complexity for programmers (commit/abort handler) Yaohua Li and Siming Chen Transactional Memory 35 / 41

36 Original code without nesting 1 atomic { 2 someprocess(shared_data); 3 work=getwork(workqueue); 4 process(work, shared_data); 5 } 6 7 node *getwork(workqueue_t &workqueue) 8 { 9 node *work; 10 work = workqueue.head; 11 if (work!= NULL) 12 workqueue.head = work->next; 13 return work; 14 } Yaohua Li and Siming Chen Transactional Memory 36 / 41

37 Code with closed-nested transactions 1 atomic { 2 someprocess(shared_data); 3 atomic { 4 work=getwork(workqueue); 5 } 6 process(work, shared_data); 7 } Yaohua Li and Siming Chen Transactional Memory 37 / 41

38 Code with open-nested transactions 1 atomic { 2 someprocess(shared_data); 3 atomic_open { 4 work = getwork(workqueue); 5 commit { 6 free(work); 7 } 8 abort { 9 insertwork(workqueue, work); 10 } 11 } 12 process(work, shared_data); 13 } Commit executes when the outermost atomic block commits, free(work) Abort executes when a surrounding transaction aborts, Yaohua Li and Siming Chen Transactional Memory 38 / 41

39 I/O challenges Example Suppose that inside a transaction, a system call attempts to output a character to the terminal Solution 1 Execute the system call immediately This transaction aborted later? Solution 2 Defer the I/O operation until commit Real-time systems? Solution 3 Try to forbid I/O within transactions Yaohua Li and Siming Chen Transactional Memory 39 / 41

40 Final approach to I/O challenges A final approach to solving the I/O problem is to categorize inputs and outputs according to their abortive properties. Undoable If its effects can be rolled back Challenge Programmers must be aware of types of I/O operations that don t make sense to transparently perform as part of a single atomic transaction, for instance, prompting the user for input and then receiving and acting on the input. Yaohua Li and Siming Chen Transactional Memory 40 / 41

41 Thank You Thank You! Yaohua Li and Siming Chen Transactional Memory 41 / 41

6.852: Distributed Algorithms Fall, Class 20

6.852: Distributed Algorithms Fall, 2009 Class 20 Today s plan z z z Transactional Memory Reading: Herlihy-Shavit, Chapter 18 Guerraoui, Kapalka, Chapters 1-4 Next: z z z Asynchronous networks vs asynchronous