DISTRIBUTED SHARED MEMORY

Transcription

1 DISTRIBUTED SHARED MEMORY COMP 512 Spring 2018 Slide material adapted from Distributed Systems (Couloris, et. al), and Distr Op Systems and Algs (Chow and Johnson) 1

2 Outline What is DSM DSM Design and Implementation Memory Consistency Models 2

3 Multiple Processor Systems Multiprocessors Tightly coupled, share common memory Difficult to design hardware/easy to design software Multicomputers Loosely coupled, private memory Easy to build hardware/difficult to design software 3

4 Shared Memory By examining the entire spectrum of shared memory, we can get a better feel for where DSM fits in; can share physically or logically On chip memory Bus-based Ring-based Switched NUMA Page-based Shared variable DSM Object-based DSM 4

5 Bus-Based Multiprocessors Snooping caches Cache consistency protocols Write through 5

6 Bus-Based: Write Once 6

7 Ring-Based Multiprocessors No global memory Each CPU has private and shared memory Shared memory divided into blocks Each block has home machine (space always reserved) Block may reside in cache on other machine Caches are all there is to form global memory 7

8 Switched Multiprocessors UMA (uniform memory access) Must address problem of limited bandwidth Slow Costly 8

9 Shared Memory: UMA Uniform memory access 9

10 Shared Memory: NUMA Non-uniform memory access Single virtual memory address space visible to all CPUs Access to remote memory is much slower than access to local memory Complex software (must decide best place to put page/block to maximize performance) 10

11 Page-based DSM Single paged virtual address space Simulates multiprocessor memory model by giving each process a linear, paged memory Each page present on exactly one machine Remote page sent to local machine on fault Issues: finding owner, finding copies, page replacement 11

12 Shared Variable DSM Machines share only a portion of address space More structured approach than page-based Share only certain variables and data structures Problem changes from how do we do paging over the network to how do we maintain a potentially replicated distributed database of shared variables 12

13 Object-based DSM Object defined by programmer Processes share abstract space filled with shared objects Modular, flexible Can t be used with old multiprocessor programs that assume the existence of a shared linear address space All accesses are through methods 13

14 Comparison of 6 Kinds of Shared Memory Systems 14

15 Comparison of Shared Memory Systems SMP= single bus multiprocessor Hardware access to all of memory, hardware caching of blocks NUMA Hardware access to all of memory, software caching of pages Distributed Shared Memory (DSM) Software access to remote memory, software caching of pages 15

16 Distributed Shared Memory Allows programs running on separate computers to share data without the programmer having to deal with sending messages Underlying technology does all the message passing to keep DSM consistent 16

17 DSM 17

18 Multiprocessor vs. DSM Architecture 18

19 Advantages/Disadvantages of DSM What are the advantages of DSM? What are the disadvantages? 19

20 Approaches to DSM Hardware based Use specialized h/w to handle load and store instructions applied to addresses in DSM Page based Implement DSM as a region of virtual memory by dividing memory up into pages spread over all processors in system Library based Language/language extension provides support for DSM 20

21 Design and Implementation Issues Structure of data held in DSM Synchronization model Consistency Model Granularity Page replacement How to keep track of location of remote data How to overcome the communication delays and high overhead How to make shared data concurrently accessible How to perform updates 21

22 Algorithms for Implementing DSM Categories Non-migrating Migrating Non-replicated Replicated 22

23 Memory Coherence Informally, a memory is coherentif the value returned by a read operation is always the value that the programmer expects The set of allowable memory access orderings forms the memory consistency model A consistency model is essentially a contract between the s/w and memory 23

24 Contracts for Consistency Strict Sequential Causal PRAM Processor Weak Release 24

25 Strict Consistency Also known as linearizabilityor atomic consistency Strictest of all Used as base against which others are measured Read must return the most recently written value Global concept of time Not always necessary; not always possible 25

26 Sequential Consistency The result of any execution is the same as if the operations of all the processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program. -- Lamport Equivalent to one copy serializability All processors must agree on the order of observed effects (writes); temporal order does not matter 26

27 Causal Consistency Lamport defined the notion of potential causality to capture the flow of info in a DS Notion can be applied to memory system by interpreting a write as a message send and a read as a message read Makes a distinction between events that are potentially causally related and those that aren t All processors must agree on the order of potentially casually related writes 27

28 PRAM Consistency Pipelined RAM In causal consistency, legal for concurrent writes to be seen in different order on different machines; causally related ones must be seen in same order by ALL machines With PRAM, all processors observe the writes from a single processor in the same order while they may disagree on the order of writes by different processors 28

29 Processor Consistency Refinement of PRAM; PRAM plus For every memory location x there must be global agreement about the order of writes to X; writes to different locations need not be viewed in the same order by different processors 29

30 Quick Summary so Far Strict: absolute time ordering Sequential: agree on order of all writes Causal: agree on order of causally-related writes PRAM: agree on same-process writes Processor: PRAM plus writes to same location are seen in same order 30

31 Weak Consistency Uses synchronization variables to propagate write to/from a machine at appropriate points Accesses to synchronization variables are sequentially consistent No access to a synchronization variable is issued in a processor before all previous data accesses have been performed No access is issued by a processor before a previous access to a synchronization variable has been performed Previous refers to program order Accessing synch variable flushes the pipeline and propagates writes 31

32 Release Consistency Refinement of weak consistency Includes acquire and release (like lock/unlock) Competing accesses are divided into acquire, release, and non-synchronizing accesses (accesses that do not serve a synch purpose) 32

33 Entry Consistency Variant of release Weaker than release consistency but imposes more restrictions on the programming model Like release consistency except every shared variable needs to be associated with a synchronization variable 33

34 Summary: Memory Consistency Strict: absolute time ordering Sequential: agree on order of all writes Causal: agree on order of causally-related writes PRAM: agree on same-process writes Processor: PRAM plus write to same location are seen in same order Weak: consistent only after synch operation Release: consistent within critical section Entry: specific data consistent within critical section 34