Memory Consistency. Minsoo Ryu. Department of Computer Science and Engineering. Hanyang University. Real-Time Computing and Communications Lab.

Transcription

1 Memory Consistency Minsoo Ryu Department of Computer Science and Engineering

2 2 Distributed Shared Memory Two types of memory organization in parallel and distributed systems Shared memory (shared address space) Distributed memory (message passing) The shared memory or single address space abstraction provides several advantages over the message passing (or private memory) abstraction It gives the impression of a single monolithic memory, as in traditional von Neumann architecture Programmers access the data across the network using only read and write primitives, as they would in a uniprocessor system Programmers do not have to deal with send and receive communication primitives 2

3 3 Distributed Shared Memory All computers share a single virtual address space Pages can be physically located on any computer When process accesses data in shared address space a memory manager handles the request to the physical page If the page is remote block the process and fetch it 3

4 4 Memory Coherence and Consistency Memory coherence is the ability of the system to execute memory operations correctly While a traditional definition of correctness says that a correct memory execution is one that returns to each Read operation, the value stored by the most recent Write operation The very definition of most recent becomes ambiguous in the presence of concurrent access and multiple replicas of the data item possible permutations or interleavings of the operations issued by the processes The problem of ensuring memory coherence then becomes the problem of identifying which of these interleavings are correct, Memory consistency defines the set of allowable memory access orderings The objective is to disallow the interleavings that make no semantic sense, while not being overly restrictive so as to permit a high degree of concurrency 4

5 5 Memory Consistency Consistency models with a strong global agreement Strict Consistency Sequential Consistency Linearizable Consistency Consistency models with a weak global agreement Causal Consistency FIFO Consistency (Pipelined RAM) Processor Consistency (PRAM+) Consistency models with synchronization Weak Consistency Release Consistency Entry Consistency 5

6 6 Strict Consistency Behavior A write to a variable by any processor needs to be seen instantaneously by all processors A read should return the most recent value All processors see the same ordering of events Hard to implement (actually may not be possible) Examples W i (x)a a write by process i to item x with a value of a R i (x)b a read by process i from item x producing b <Strictly Consistent> <Not Strictly Consistent> 6

7 7 Example Strict Consistency Not Strict Consistency P1 P2 P3 P1 P2 P3 W2(x, a) W2(x, a) a R1(x) nil R1(x) a R2(x) a R3(x) a R2(x) a R3(x) Sequential event ordering W2(x) R1(x) R3(x) R2(x) 7

8 8 Sequential Consistency Behavior The result of any execution is the same as if the operations of all the processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program - Lamport in 1979 Interleaving of operations does not matter, but all processes must see the same ordering <Sequentially Consistent> <Not Sequentially Consistent> 8

9 9 Linearizability Behavior Sequential consistency (program order) + linearization points (real-time order) Comparison with other consistency models Linearization is weaker than strict consistency It does not require true event order when events overlap strict consistency linearizability linearizability strict consistency Linearization is stronger than sequential consistency linearizability sequential consistency sequential consistency linearizability 9

10 10 Linearizability vs. Sequential Consistency Linearlizability P1 P2 P3 P4 Sequential Consistency P1 P2 P3 P4 a R1(x) W2(x, a) W3(x, b) b R1(x) W2(x, a) W3(x, b) b R2(x) b R3(x) b R4(x) a R2(x) b R3(x) a R4(x) Sequential event ordering (linearizability) W2 R1 W3 R3 R4 R2 Sequential event ordering (sequential consistency) W3 R1 W2 R3 R4 R2 10

11 11 Causal Consistency Behavior Events that are potentially causally related must be seen by all processes in the same order Concurrent writes may be seen by different processes in different orders Potential causality (= happened before) Program order within a process: i j Send-receive order between processes: j k For shared memory: write read on the same data item Transitivity: if i j and j k, then i k Why potential? Program order may not imply causality Lamport used potential causality (= happened before relation) in an informal explanation 11

12 12 Potential Causality Program order P 1 a = 5; b = 25; c = a + b; not causal P 2 (but potentially causal) a = 5; b = a*a; c = a + b; causal Causal write read write vs. concurrent writes W(a, 5) W(a, 5) P 1 P 1 causal not causal P 2 5 R(a) W(a, 10) P 2 W(a, 10) read write? read read? 12

13 13 Example Example 1 (Causally Consistent) W 1 (X)a, R 2 (X)a, and W 2 (X)b are causally related W 2 (X)b and W 1 (X)c are not causally related but concurrent Example 2 W 1 (X)a, R 2 (X)a, and W 2 (X)b are causally related W 1 (X)a and W 2 (X)b are concurrent (A) Not Causally Consistent (B) Causally Consistent 13

14 14 Example Causal Consistency P1 P2 P3 W1(x, a) a R1(x) a R4(x) W2(x, c) W3(x, b) c R2(x) b R3(x) P4 a R5(x) b R6(x) c R7(x) Not Causal Consistency P1 P2 P3 P4 W1(x, a) a R3(x) W2(x, b) a R1(x) b R4(x) b R2(x) a R5(x) Event ordering (causal consistency) W1 W2 W3 W1 W3 W2 Event ordering (not causal consistency) W1 W2 14

15 15 Causal vs. Sequential Consistency Causal consistency is a weakening model of sequential consistency sequential consistency causal consistency 15

16 16 FIFO (PRAM) Consistency Behavior Writes done by a single process are seen by all other processes in the order in which they were issued, but writes from different processes may be seen in a different order by different processes - Lipton and Sandberg in 1988 PRAM = pipelined RAM Easy to implement With local timestamps (process ID, seq. #) <FIFO Consistent, but not causally consistent> 16

17 17 Processor Consistency Behavior All processors need to be consistent in the order in which they see writes done by one processor and in the way they see writes by different processors to the same location (coherence is maintained) However, they do not need to be consistent when the writes are by different processors to different locations (aka PRAM+) 17

18 18 FIFO vs. Processor Consistency FIFO Consistency Processor Consistency P1 P2 P3 W2(x, a) W2(x, b) W3(x, 0) a R1(x) 0 R1(x) W3(x, 1) 1 R1(x) W2(y, a) W3(z, a) b R1(x) a R1(y) a R1(z) P4 a R4(x) 0 R4(x) b R4(x) 1 R4(x) a R4(z) a R4(y) a R1(x) 0 R1(x) 1 R1(x) b R1(x) a R1(y) a R1(z) P1 P2 P3 W2(x, a) W2(x, b) W2(y, a) W3(x, 0) W3(x, 1) W3(z, a) P4 a R4(x) 0 R4(x) 1 R4(x) b R4(x) a R4(z) a R4(y) FIFO consistency W2(x,a) W2(x,b) W2(y,a) is observed by all machines W3(x,0) W3(x,1) W3(z,a) is observed by all machines Processor consistency W2(x,a) W3(x,0) W3(x,1) W2(x,b) is observed by all machines 18

19 19 Weak Consistency The consistency conditions apply only to a set of distinguished synchronization instructions Other program statements between synchronization statements may be executed by different processors without any conditions The weak consistency models enforce consistency on a group of operations, as opposed to individual reads and writes (as is the case with strict, sequential, causal and FIFO consistency) The three properties of Weak Consistency Accesses to synchronization variables are sequentially consistent No access to a synchronization variable is allowed to be performed until all previous writes have completed everywhere No data access (either Read or Write) is allowed to be performed until all previous accesses to synchronization variables have been performed 19

20 20 Weak Consistency By doing a sync., a (writer) process can force the just written value out to all the other replicas also, a (reader) process can be sure it s getting the most recently written value before it reads (a) A valid sequence of events for weak consistency This is because P2 and P3 have yet to synchronize, so there s no guarantees about the value in x (b) An invalid sequence for weak consistency P2 has synchronized, so it cannot read a from x it should be getting b 20

21 21 Release Consistency The drawback of weak consistency is that when a synchronization variable is accessed, the memory does not know whether this is being done because the process is finished writing the shared variables (exiting the CS) or about to begin reading them (entering the CS) To overcome the drawback, two sync operations can be used, acquire and release, and their use leads to the Release Consistency model Definition When a process does an acquire, the data-store will ensure that all the local copies of the protected data are brought up to date to be consistent with the remote ones if needs be Tell the memory system that a critical section is about to be entered When a release is done, protected data that have been changed are propagated out to the local copies of the data-store Tell the memory system that a critical section has just been exited 21

22 22 Release Consistency A valid event sequence for release consistency Process P3 has not performed an acquire, so there are no guarantees that the read of x is consistent. The datastore is simply not obligated to provide the correct answer P2 does perform an acquire, so its read of x is consistent 22

23 23 Eager Release vs. Lazy Release Eager Release Consistency Make modifications globally visible at the time of a release Lazy Release Consistency Only a processor that acquires a lock will see all modifications that precede the lock acquire 23

24 24 Eager Release vs. Lazy Release 24

25 25 Entry Consistency A different twist on things is Entry Consistency Acquire and release are still used Each operation is required to be associated with some type of synchronization variable such as a lock or a barrier Essentially the synchronization is individualized When an acquire is done on a synchronization variable, only those ordinary shared variables guarded by that synchronization variable are made consistent Entry consistency differs from lazy release consistency in that the latter does not associate shared variable with locks or barriers and at acquire time has to determine empirically which variables it needs 25

26 26 Entry Consistency Locks associated with individual data items, as opposed to the entire data-store Note: P2 s read on y returns NIL as no locks have been requested 26

27 27 27