Distributed Data Management

Transcription

1 Lecture Distributed Data Management Chapter 7: Lazy Replication Erik Buchmann IPD, Forschungsbereich Systeme der Informationsverwaltung

2 Synchronous vs. Asynchronous Updates Synchronous (eager replication) within transaction Asynchronous (lazy replication) separate transaction to update replicas no communication to remote sites (e.g., for locking or concurrency control) before commit is processed updates for remote sites are independent transactions that are processed after commiting and releasing all locks Erik Buchmann DDM: Lazy Replication 2

3 Synchronous Updates of Replicas Bad scalability w.r.t. the number of nodes each transaction locks quorum or primary copy With synchronous updates, all TA must wait for the slowest node Failure of nodes similar problem. Transactions: T 1 : w[a] w[b] c T 2 : w[b] c T 3 : r[a] r[b] c a Possible global execution plan: w 1 [a X ] w 1 [a Y ] w 1 [a Z ] w 1 [b Y ] w 1 [b Z ] c 1 w 2 [b Y ] w 2 [b Z ] c 2 r 3 [a Z ] r 3 [b Z ] c 3 X Erik Buchmann DDM: Lazy Replication 3 T2 Y b a Z T2 T3

4 Asynchronous Updates of Replicas Lazy replication underlying idea: if update has been successful on one node, it will be successful on all other nodes as well transaction may already commit without having updated all replicas Transactions: T 1 : w[a] w[b] c T 2 : w[b] c T 3 : r[a] r[b] c a Possible global execution plan: w 1 [a X ] w 1 [b Y ] c 1 w 2 [b Y ] c 2 w 2 [b Z ] w 1 [a Y ] w 1 [a Z ] w 1 [b Z ] r 3 [a Z ] r 3 [b Z ] c 3 X Erik Buchmann DDM: Lazy Replication 4 T2 Y b a Z T2 T3

5 Outline Lazy Updates performance tends to be better than eager (but not by orders of magnitude, depending on the distribution scheme of replicas) serializability typically not guaranteed Our topic in what follows: lazy protocols guaranteeing serializability. a)protocols making assumptions regarding the arrangement of replicas. b)hybrid protocol without such assumptions that is lazy whenever possible. Erik Buchmann DDM: Lazy Replication 5

6 System Model/Assumptions Each data object has a primary site. primary copy, secondary copies/replicas. Transaction has originating site. Transaction can only modify data objects whose primary site = originating site; but may read any data object. Nodes (sites) use 2PL. Network is reliable; delivery of messages in FIFO order. Primary subtransaction, secondary subtransaction. Erik Buchmann DDM: Lazy Replication 6

7 Copy Graph: nodes sites. Copy Graph Edge from si to s j iff primary copy of a data object is on s i, and secondary copy on s j. Example: three sites, two data objects a and b. a s1 s3 s2 Backedges := set of edges s.t. deletion of such edges makes the graph cycle-free. Is it possible to construct a replication scenario where an acyclic graph cannot be constructed? b a Erik Buchmann DDM: Lazy Replication 7

8 Non-serializable Execution a s1 T2 s2 b a s3 T2 T3 Three transactions: on Site 1 updates a. T 2 on Site 2 reads a and writes b. T 3 on Site 3 reads a and b. Lazy propagation of updates possible sequence of executions: on Site 2: w 1 [a] r 2 [a] w 2 [b] on Site 3: w 2 [b] r 3 [a] r 3 [b] w 1 [a] Erik Buchmann DDM: Lazy Replication 8

9 and IPD, Forschungsbereich Systeme der Informationsverwaltung

10 Protocol (1) Directed Acyclic Graph (Without Timestamps) Copy graph is cycle-free and has no duplicate paths from primary to secondary copies (cycle-free when deleting edge directions) Generate Tree T from copy graph: s i child of s j. s i successor of s j in T s1 a s3 s1 a s3 s2 b a s2 b a Erik Buchmann DDM: Lazy Replication 10

11 Protocol (2) Node forwards update transactions to children in T. Commit order: Order in which transactions arrive at node = order in which transactions are forwarded to children Commit order + cycle-free tree without duplicate paths ensure serializability! a Now impossible: w 1 [a X ] w 1 [b Y ] c 1 w 2 [b Y ] c 2 w 2 [b Z ] w 1 [a Y ] w 1 [a Z ] w 1 [b Z ] r 3 [a Z ] r 3 [b Z ] c 3 X Erik Buchmann DDM: Lazy Replication 11 T2 Y b a Z T2 T3

12 Local Deadlocks with Local deadlocks feasible, i.e., transaction does not necessarily commit. Local deadlock feasible because of conflict of T with transaction Tx that has not occurred at other sites. remember: read operations are local! interaction with transaction that has already occurred at other sites. no handshaking, only commit order is given. But local transaction must commit. Thus, victim selection policy should be fair, e.g.: last transaction. Erik Buchmann DDM: Lazy Replication 12

13 Local Deadlock Example Replication scheme that might cause deadlocks: a c s1 s3 s2 : r 1 (c) w 1 (c) w 1 (a) T 2 : r 2 (a) r 2 (c) w 2 (b) Possible execution at s2: w 1 (c) r 2 (a) r 2 (c) w 1 (a) b a c (remember: uses Strict 2PL only) Erik Buchmann DDM: Lazy Replication 13

14 Further Problem With Transaction might be routed to nodes that are not relevant. delays, message overhead, processing costs Example: := w[a] w[c] a d s1 s3 c b s2 b a Erik Buchmann DDM: Lazy Replication 14

15 Protocol - - Timestamps - Protocol DAG with Timestamps Propagate update transactions along the edges of the copy graph unlike, duplicate paths are allowed results can be sent directly to all copies Primary subtransactions have timestamp that specify execution order. challenge: decentralized generation of timestamps that are globally correct Erik Buchmann DDM: Lazy Replication 15

16 Timestamps Acyclicity results in total order of sites; si < s i+k - - Timestamps - Protocol In the following: three notations for timestamps that build on each other local timestamp counter, timestamp of a site, timestamp of a transaction. Erik Buchmann DDM: Lazy Replication 16

17 Local Timestamp Counter tuple corresponding to Node si : (s i, LTS i ) LTS = Local Timestamp Counter; counts the primary subtransactions committed there. a s1 T2 s2 b a (s1,0),(s2,0),(s3,0) (s1,1) (s2,1) (s3,1)< independentfrom the orderthe updates arrive;localtim estam ps are notsuficcientforserializability s3 T3 T2 Erik Buchmann DDM: Lazy Replication 17

18 Site Timestamps Timestamp of a site si : TS(s i ) - - Timestamps - Protocol vector of tuples (s i, LTS i ), a tuple for s i and further tuples for predecessors of s i in copy graph. Important: tuple in vector ordered by sites. Timestamp of a site reflects how many primary subtransactions and how many secondary subtransactions have committed there Erik Buchmann DDM: Lazy Replication 18

19 Example for Site Timestamps a - - Timestamps - Protocol s1 1 Site Timestamp 2 Site Timestamp s1 <(s1, 0)> s1 <(s1,1)> s2 <(s1, 0), (s2, 0)> s2 <(s1,0), (s2,0)> 3 Tx Site Timestamp s1 <(s1, 1)> s2 <(s1, 1), (s2, 0)> 4 T2 s2 b a s3 Site Timestamp s1 <(s1, 1)> s2 <(s1, 1), (s2, 1)> Erik Buchmann DDM: Lazy Replication 19

20 Transaction Timestamps Timestamp of a transaction: TS(Ti ) - - Timestamps - Protocol timestamp of the site of the primary transaction immediately after commit. Example: a s1 s3 T3 : <(s 1, 1)> T2: <(s 1, 1), (s 2, 1)> T3: <(s 1, 1), (s 3, 1)> T2 s2 b a Note: tuples where LTS = 0 are omitted Erik Buchmann DDM: Lazy Replication 20

21 Order of Transaction Timestamps - - Timestamps - Protocol X is a common prefix Global serializability: Order of timestamps must reflect commit order. Lexicographic ordering < of timestamps: TS 1 <TS 2 : TS 1 is prefix of TS 2, or TS 1 =X (s i, LTS i ) Y 1, TS 2 =X (s j, LTS j ) Y 2, and a) s i >s j, or b) s i =s j, and LTS i <LTS j. Erik Buchmann DDM: Lazy Replication 21

22 Timestamp Order Explanation Order of timestamps shall reflect commit order. - - Timestamps - Protocol TS1 <TS 2 TS 1 is prefix of TS 2. Example: <(s 1, 1)> describes state before <(s 1, 1), (s 2, 1)>. TS1 <TS 2 TS 1 =X (s i, LTS i ) Y 1, TS 2 =X (s j, LTS j ) Y 2, and s i =s j, und LTS i <LTS j. Example: <(s 1, 1)> describes state before <(s 1, 2)>. Hence, <(s 1,1), (s 3,1)> < <(s 1,1), (s 2,1)>. Erik Buchmann DDM: Lazy Replication 22

23 Data Structures - - Timestamps - Protocol Data structure for each node: timestamp vector of the node timestamp of the last committed secondary subtransaction + tuple of the node, waiting queues one for each predecessor. Erik Buchmann DDM: Lazy Replication 23

24 Protocol for Primary Transaction - - Timestamps - Protocol Primary transaction commits at Node s i : 1. increment LTS i (Local Timestamp Counter), 2. TS(T i ) := TS(s i ) 3. Send secondary (sub-)transaction to children of s i. Timestamp of a site reflects how many primary subtransactions and how many secondary subtransactions have committed there. Erik Buchmann DDM: Lazy Replication 24

25 Protocol for Secondary Transaction - - Timestamps - Protocol One waiting queue for each direct predecessor of the site in the copy graph. Requirements: only one secondary transaction at a time at one site, but any number of primary transactions at least one transaction in each queue Why is this important? Choose transaction from queues with minimal timestamp Idle nodes should commit dummy transactions from time to time to fill the queues Erik Buchmann DDM: Lazy Replication 25

26 - - Timestamps - Protocol Continuation of Example has Timestamp (s 1, 1), a s1 T2 b a T2 has Timestamp (s 1, 1), (s 2, 1). (When T 1 commits on s 2, the site timestamp is set to (s 1, 1), (s 2, 0).) Site s3 : timestamp of T 1 is prefix of the one of T 2. T 1 is executed there before T 2. s2 s3 T2 Erik Buchmann DDM: Lazy Replication 26

27 Extension to Ensure Progress - - Timestamps - Protocol We can construct graphs where some TAs are never executed s1 s2 s3 (s 2, i) < (s 1, 1) for any i; thus (s 1, 1) would be never processed Solution: add a global epoch number to the timestamp, process TA with smaller epoch first Erik Buchmann DDM: Lazy Replication 27

28 Back-Edge IPD, Forschungsbereich Systeme der Informationsverwaltung

29 Protocol Motivation Copy graph must be acyclic for and protocols. Example: two sites s 1 and s 2. s 1 holds primary copy of a and copy of b, s 2 vice versa. T 1 at Node s 1 reads b and updates a, T 2 at Node s 2 reads a and updates b. Both transactions execute concurrently and commit. No serializability. Erik Buchmann DDM: Lazy Replication 29

30 Protocol Overview Hybrid: for some replicas eager updates, for other ones lazy. Can be described both as extension of and. Gdag results from G by removing backedges. s1 s3 s2 Erik Buchmann DDM: Lazy Replication 30

31 Terminology Backedge from si to s j. s j is predecessor of s i in G dag. Ti primary subtransaction with node s i. Backedge subtransactions : transactions S 1,..., S j at predecessor s i1,..., s ij of s i in T. predecessorofs3 s1 s3 Tx writing b s2 Erik Buchmann DDM: Lazy Replication 31

32 Protocol 1. After execution of T i, S 1 is sent to s 1 no commit, locks are not released. 2. S 2,..., S j as before, but without commit and without releasing locks. 3. 2PC for T i, S 1,..., S j. 4. Remaining secondary subtransactions: lazy, as in. lock b write b unlock b s1 lock b write b unlock b s2 Erik Buchmann DDM: Lazy Replication 32 s3 lock b write b unlock b

33 Protocol Discussion No magic here. Better than primary-copy schemes, according to simulations. upto Factor 5, if there are many readers and few backedges. Implementation on top of commercial DBMSs relatively easy. Erik Buchmann DDM: Lazy Replication 33

34 Literature Yuri Breitbart et al.: Update Propagation Protocols for Replicated Databases. Proceedings SIGMOD 99. Erik Buchmann DDM: Lazy Replication 34