Advanced Databases 2006 Based on slides from Kifer et al., Shasha et al. LOCKING

Size: px

Start display at page:

Download "Advanced Databases 2006 Based on slides from Kifer et al., Shasha et al. LOCKING"

Imogene Gregory
5 years ago
Views:

1 Advanced Databases 2006 Based on slides from Kifer et al., Shasha et al. LOCKING 1

2 Agenda Correctness and performance Concurrency control principles Serialization graph 2 Phase locking Lock Implementation Lock Granularity The Phantom Problem Lock Tuning Isolation levels Transaction Chopping 2

3 Conflicts in Relational Databases Audit: SELECT SUM (balance) FROM Accounts WHERE nam e = M ary ; SELECT totbal FROM Depositors WHERE nam e = M ary NewAccount: INSERT INTO Accounts V A LU ES ( 123, M ary,100); UPDATE Depositors SET totbal = totbal WHERE nam e = M ary Operations on Accounts and Depositors conflict Interleaved execution is not correct 3

4 Schedule T1 T2 Arriving schedule (merge of transaction schedules) Concurrency Control Schedule in which requests are serviced (to preserve isolation) database T3 transaction schedules Database server 4

5 Isolation Serial execution: Since each transaction is consistent and isolated from all others, schedule is guaranteed to be correct for all applications Inadequate performance Since system has multiple asynchronous resources and transaction uses only one at a time Concurrent execution: Improved performance (multiprogramming) Some interleavings produce correct result, others do not We are interested in concurrent schedules that are equivalent to serial schedules. These are referred to as serializable schedules. 5

6 Concurrency Control Transforms arriving interleaved schedule into a correct interleaved schedule to be submitted to the DBMS Delays servicing a request (reordering) - causes a transaction to wait Refuses to service a request - causes transaction to abort Actions taken by concurrency control have performance costs Goal is to avoid delaying or refusing to service a request 6

7 Commutativity of Read and Write Operations p 1 commutes with p 2 if They operate on different data items w 1 (x) commutes with w 2 (y) and r 2 (y) Both are reads r 1 (x) commutes with r 2 (x) Operations that do not commute conflict w 1 (x) conflicts with w 2 (x) w 1 (x) conflicts with r 2 (x) 7

8 Example of Equivalence conflict S 1 : r 1 (x) r 2 (x) w 2 (x) r 1 (y) w 1 (y) S 2 : r 1 (x) r 2 (x) r 1 (y) w 2 (x) w 1 (y) S 3 : r 1 (x) r 1 (y) r 2 (x) w 2 (x) w 1 (y) S 4 : r 1 (x) r 1 (y) r 2 (x) w 1 (y) w 2 (x) S 5 : r 1 (x) r 1 (y) w 1 (y) r 2 (x) w 2 (x) conflicting operations ordered in same way S 1 is equivalent to S 5 S 5 is the serial schedule T 1, T 2 S 1 is serializable S 1 is not equivalent to the serial schedule T 2, T 1 8

9 Example of Equivalence T 1 : begin transaction read (x, X); X = X+4; write (x, X); commit; initial state T 2 : begin transaction read (x,y); write (y,y); commit; final state x=1, y=3 r 1 (x) r 2 (x) w 2 (y) w 1 (x) x=5, y=1 Interchange commuting operations Interchange conflicting operations x=1, y=3 r 2 (x) w 2 (y) r 1 (x) w 1 (x) x=5, y=1 T 2 T 1 x=1, y=3 r 1 (x) w 1 (x) r 2 (x) w 2 (y) x=5, y=5 T 1 T 2 9

10 Serializable Schedules S is serializable if it is equivalent to a serial schedule Transactions are totally isolated in a serializable schedule A schedule is correct for any application if it is a serializable schedule of consistent transactions The schedule : r 1 (x) r 2 (y) w 2 (x) w 1 (y) is not serializable 10

11 Isolation Levels Serializability provides a conservative definition of correctness For a particular application there might be many acceptable non-serializable schedules Requiring serializability might degrade performance DBMSs offer a variety of isolation levels: SERIALIZABLE is the most stringent Lower levels of isolation give better performance Might allow incorrect schedules Might be adequate for some applications 11

12 Agenda Correctness and performance Concurrency control principles Serialization graph 2 Phase locking Lock Implementation Lock Granularity The Phantom Problem Lock Tuning Isolation levels Transaction Chopping 12

13 Conflict Equivalence Definition- Two schedules, S 1 and S 2, of the same set of operations are conflict equivalent if conflicting operations are ordered in the same way in both Or if one can be obtained from the other by a series of commutative interchanges 13

14 Conflict Equivalence Result- A schedule is serializable if it is conflict equivalent to a serial schedule r 1 (x) w 2 (x) w 1 (y) r 2 (y) r 1 (x) w 1 (y) w 2 (x) r 2 (y) conflict conflict If in S transactions T 1 and T 2 have several pairs of conflicting operations (p 1,1 conflicts with p 2,1 and p 1,2 conflicts with p 2,2 ) then p 1,1 must precede p 2,1 and p 1,2 must precede p 2,2 (or vice versa) in order for S to be serializable. 14

15 Conflict Equivalence and Serializability Serializability is a conservative notion of correctness and conflict equivalence provides a conservative technique for determining serializability However, a concurrency control that guarantees conflict equivalence to serial schedules ensures correctness and is easily implemented 15

16 Serialization Graph of a Schedule, S Nodes represent transactions There is a directed edge from node T i to node T j if T i has an operation p i,k that conflicts with an operation p j,r of T j and p i,k precedes p j,r in S Theorem - A schedule is conflict serializable if and only if its serialization graph has no cycles 16

17 Example Conflict (*) S : p 1,i,, p 2,j,... * T 2 T 4 S is serializable in order T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 1 T 5 T 6 T 7 T 3 T 2 T 4 S is not serializable due to cycle T 2 T 6 T 7 T 2 T 1 T 5 T 6 T 7 T 3 17

18 Intuition: Serializability and Nonserializability Consider the nonserializable schedule r 1 (x) w 2 (x) r 2 (y) w 1 (y) Two ways to think about it: Because of the conflicts, the operations of T 1 and T 2 cannot be interchanged to make an equivalent serial schedule Because T 1 read x before T 2 wrote it, T 1 must precede T 1 T 2 T 2 in any ordering, and because T 1 wrote y after T 2 read it, T 1 must follow T 2 in any ordering --- clearly an impossibility 18

19 Agenda Correctness and performance Concurrency control principles Serialization graph 2 Phase locking Lock Implementation Lock Granularity The Phantom Problem Lock Tuning Isolation levels Transaction Chopping 19

20 Concurrency Control Arriving schedule (from transactions) Concurrency Control serializable schedule (to processing engine) Concurrency control cannot see entire schedule: It sees one request at a time and must decide whether to allow it to be serviced Pessimistic strategy: Make conflicts explicit Delay operations that are conflicting with non committed operations 20

21 Locking A transaction can read a database item if it holds a read (shared) lock on the item It can read or update the item if it holds a write (exclusive) lock If the transaction does not already hold the required lock, a lock request is automatically made as part of the (read or write) request 21

22 Locking Request for read lock on an item is granted if no transaction currently holds write lock on the item Cannot read an item written by an active transaction Request for write lock granted if no transaction holds any lock on item Cannot write an item read/written by an active transaction Transaction is delayed if request cannot be granted Granted mode Requested mode read write read x write x x 22

23 Locking Protocol Locking does not guarantee serializability! Ex: release lock on item when finished accessing the item T 1 : l(x) r(x) l(y) r(y) u(x) w(y) u(y) T 2 : l(x) l(z) w(x) w(z) u(x) u(z) can lead to non-serializable schedules T 1 : l(x) r(x) u(x) T 2 : l(x) l(y) w(x) w(y) u(x) u(y) commit l(y) r(y) u(y) 23

24 Two-Phase Locking Transaction does not release a lock until it has all the locks it will ever require. Transaction has a locking phase followed by an unlocking phase Number of locks held by T T s first unlock T commits time 24

25 Two-Phase Locking Theorem: A concurrency control that uses two phase locking produces only serializable schedules. Proof (sketch): Consider two transactions T 1 and T 2 in schedule S produced by a two-phase locking control and assume T 1 s first unlock, t 1, precedes T 2 s first unlock, t 2. If they do not access common data items, then all operations commute. Suppose they do. All of T 1 s accesses to com m on item s precede all of T 2 s. If this w ere not so, T 2 s first unlock m ust precede a lock request of T 1. Since both transactions are two-phase, this implies that T 2 s first unlock precedes T 1 s first unlock, contradicting the assum ption. Hence, all conflicts between T 1 and T 2 are in the same direction. It follows that the serialization graph is cycle-free since if there is a cycle T 1, T 2, T n then it must be the case that t 1 < t 2 < < t n < t 1 25

26 Agenda Correctness and performance Concurrency control principles Serialization graph 2 Phase locking Lock Implementation Lock Granularity The Phantom Problem Lock Tuning Isolation levels Transaction Chopping 26

27 Locking Implementation Associate a lock set, L(x), and a wait set, W(x), with each active database item, x L(x) contains an entry for each granted lock on x W(x) contains an entry for each pending request on x When an entry is removed from L(x) (due to transaction termination), promote (non-conflicting) entries from W(x) using some scheduling policy (e.g., FCFS) Associate a lock list, L i, with each transaction, T i. L i links T i s elem ents in all lock and w ait sets Used to release locks on termination 27

28 Locking Implementation L r r x W w T i holds an r lock on x and waits for a w lock on y y L W w r w L i 28

29 Locking in SQL Server 7 syslockinfo spid dbid objid lock granularity lock mode RID X PAG IX lock owner LO1 LO1 lock waiter LW1, LW TAB IX LO1 LW RID S LO2, LO3 LW2 Lock 32 bytes Lock owner block 32 bytes Lock waiter block 32 bytes Dennis Shasha, Philippe Bonnet 2001

30 Locking in Oracle 8i Process H Enqueue resource structure (fixed array default 4 entries per transaction) T1 T1 Interested transaction list (fixed array - INITRANS MAXTRANS) T2 lock Enqueued locks array T1 lock T3 lock row Data page Enqueue wait (time out ~ 3sec) Deadlock detection Dennis Shasha, Philippe Bonnet 2001

31 Latches and Locks Locks are used for concurrency control Requests for locks are queued Priority queue Lock data structure Locking mode, lock granularity, transaction id. Lock table Latches are used for mutual exclusion Requests for latch succeeds or fails Active wait (spinning) on latches on multiple CPU. Single location in memory Test and set for latch manipulation Dennis Shasha, Philippe Bonnet 2001

32 Throughput (statements/sec) Throughput (statements/sec) Avoid Bottlenecks: Counters SQLServer system ad-hoc Number of concurrent insertion threads Oracle system ad-hoc System generated counter (system) much better than a counter managed as an attribute value within a table (ad hoc). The Oracle counter can become a bottleneck if every update is logged to disk, but caching many counter numbers is possible. Counters may miss ids Number of concurrent insertion threads Dennis Shasha, Philippe Bonnet 2001

33 Agenda Correctness and performance Concurrency control principles Serialization graph 2 Phase locking Lock Implementation Lock Granularity The Phantom Problem Lock Tuning Isolation levels Transaction Chopping 33

34 What to Lock? Lock tables: Execution is serializable but... performance suffers because lock granularity is coarse Lock rows: Performance improves because lock granularity is fine but... execution might not be serializable (we will get back to that) 34

35 Conflicts in Relational Databases Audit: SELECT SUM (balance) FROM Accounts WHERE nam e = M ary ; SELECT totbal FROM Depositors WHERE nam e = M ary NewAccount: INSERT INTO Accounts V A LU ES ( 123, M ary,100); UPDATE Depositors SET totbal = totbal WHERE nam e = M ary 35

36 Problem with Row Locking time Audit (1) L ocks and reads M ary s row s in A ccounts NewAccount (2) Inserts and locks new row, t, in Accounts (3) L ocks and updates M ary s row in D epositors (4) Commits and releases all locks Audit (5) L ocks and reads M ary s row in D epositors 36

37 Row Locking The two SELECT statements in Audit see inconsistent data The second sees the effect of NewAccount; the first does not Problem: A udit s SELECT and N ew A ccount s INSERT do not commute, but the row locks held by Audit did not delay the INSERT The inserted row is referred to as a phantom 37

38 Phantoms Phantoms occur when row locking is used and T 1 SELECTs, UPDATEs, or DELETEs using a predicate, P T 2 creates a row (using INSERT or UPDATE) satisfying P Example: T 1 : UPDATE Table T 2 : INSERT INTO Table SET A ttr =. VALUES ( satisfies P ) WHERE P 38

39 Phantoms INSERT and UPDATE cause phantoms with row locking. Question: Why does DELETE not cause a similar problem with row locking? Answer: A row that has been read cannot be deleted because it is locked 39

40 Preventing Phantoms Table locking prevents phantoms; row locking does not Predicate locking prevents phantoms A predicate describes a set of rows, some are in a table and some are not; e.g. nam e = M ary A subset of the rows satisfying nam e = M ary are in Accounts Every SQL statement has an associated predicate When executing a statement, acquire a (read or write) lock on the associated predicate Two predicate locks conflict if one is a write and there exists a row (not necessarily in the table) that is contained in both 40

41 Phantoms rows in R satisfying P (rows that can be locked) rows in table R rows satisfying predicate P rows satisfying P that do not exist in R all rows that can possibly be in table R 41

42 Preventing Phantoms With Predicate Locks Audit: SELECT SUM (balance) FROM Accounts WHERE nam e = M ary NewAccount: INSERT INTO Accounts V A LU ES ( 123, M ary,100) Audit gets read lock on predicate n a m e= M a ry. NewAccount requests a write lock on predicate (acctnum= 123 name= M ary bal=100) Request denied since predicates overlap 42

43 Conflicts And Predicate Locks Example 1 SELECT SUM (balance) DELETE FROM Accounts FROM Accounts WHERE nam e = M ary WHERE bal < 100 Statements conflict since predicates overlap and one is a write T here m ight be an account w ith bal < 100 and nam e = M ary Locking is conservative: there might be no rows in Accounts satisfying both predicates No phantom involved in this (DELETE) case 43

44 Conflicts And Predicate Locks Example 2 SELECT SUM (balance) FROM Accounts WHERE nam e = M ary DELETE FROM Accounts WHERE nam e = John Statements commute since predicates are disjoint. There can be no rows (in or not in Accounts) that satisfy both predicates No phantom involved in this (DELETE) case 44

45 Agenda Correctness and performance Concurrency control principles Serialization graph 2 Phase locking Lock Implementation Lock Granularity The Phantom Problem Lock Tuning Isolation levels Transaction Chopping 45

46 Concurrency Control Goals Performance goals Reduce blocking One transaction waits for another to release its locks Avoid deadlocks Transactions are waiting for each other to release their locks Correctness goals Serializability: each transaction appears to execute in isolation The programmer ensures that serial execution is correct. Trade-off between correctness and concurrency Dennis Shasha, Philippe Bonnet 2001

47 Ideal Transaction Acquires few locks and favors shared locks over exclusive locks Reduce the number of conflicts -- conflicts are due to exclusive locks Acquires locks with fine granularity Reduce the scope of each conflict Holds locks for a short time Reduce waiting Dennis Shasha, Philippe Bonnet 2001

48 Sacrificing Isolation for Performance A transaction that holds locks during a screen interaction is an invitation to bottlenecks Airline Reservation 1. Retrieve list of seats available 2. Talk with customer regarding availability 3. Secure seat Single transaction is intolerable, because each customer would hold lock on seats available. Keep user interaction outside a transactional context Problem: ask for a seat but then find it s unavailable. M ore tolerable. Dennis Shasha, Philippe Bonnet 2001

49 Isolation Levels Read Uncomitted (No dirty writes, No lost update) Exclusive locks for write operations are held for the duration of the transactions No locks for read Read Committed (No dirty reads) Shared locks are released as soon as the read operation terminates. Repeatable Read (no unrepeatable reads) Strict two phase locking Serializable (no phantoms) Table locking or index locking to avoid phantoms Dennis Shasha, Philippe Bonnet 2001

50 Snapshot isolation Each transaction executes against the version of the data items that was committed when the transaction started: No locks for read Locks for writes Costs space (old copy of data must be kept) Almost serializable level: T1: x:=y T2: y:= x Initially x=3 and y =17 Serial execution: x,y=17 or x,y=3 Snapshot isolation: x=17, y=3 if both transactions start at the same time. T2 T1 X=Y=Z=0 T3 Dennis Shasha, Philippe Bonnet 2001 TIME

51 Ratio of correct answers Ratio of correct answers Value of Serializability -- results SQLServer 1 0,8 read committed 0,6 0,4 0,2 serializable Concurrent update threads Oracle 1 0,8 0,6 0,4 read committed 0, serializable 8 10 Concurrent update threads With SQL Server and DB2 the scan returns incorrect answers if the read committed isolation level is used (default setting) With Oracle correct answers are returned (snapshot isolation), but beware of swapping 2 - Tuning the Guts Dennis Shasha, Philippe Bonnet

52 Throughput (trans/sec) Throughput (trans/sec) Cost of Serializability SQLServer read committed serializable Concurrent Update Threads Oracle read committed Because the update conflicts with the scan, correct answers are obtained at the cost of decreased concurrency and thus decreased throughput serializable Concurrent Update Threads 2 - Tuning the Guts Dennis Shasha, Philippe Bonnet

53 Agenda Correctness and performance Concurrency control principles Serialization graph 2 Phase locking Lock Implementation Lock Granularity The Phantom Problem Lock Tuning Isolation levels Transaction Chopping 53

54 Example: Simple Purchases Purchase item I for price P 1. If cash < P then roll back transaction (constraint) 2. Inventory(I) := inventory(i)+p 3. Cash := Cash P Two purchase transaction P1 and P2 P1 has item I for price 50 P2 has item I for price 75 Cash is 100 Dennis Shasha, Philippe Bonnet 2001

55 Example: Simple Purchases If as one transaction then one of P1, P2 rolls back. If 1, 2, 3 as three distinct transactions: P1 checks that cash > 50. It is. P2 checks that cash > 75. It is. P1 completes. Cash = 50. P2 completes. Cash = Dennis Shasha, Philippe Bonnet 2001

56 Example: Simple Purchases Orthodox solution Make whole program a single transaction Cash becomes a bottleneck! Chopping solution Find a way to rearrange and then chop up the transactions without violating serializable isolation level. Dennis Shasha, Philippe Bonnet 2001

57 Example: Simple Purchases Chopping solution: 1. If Cash < P then roll back. Cash := Cash P. 2. Inventory(I) := inventory(i) + P Chopping execution: P11: 100 > 50. Cash := 50. P21: 75 > 50. Rollback. P12: inventory := inventory Dennis Shasha, Philippe Bonnet 2001

58 Transaction Chopping Execution rules: When pieces execute, they follow the partial order defined by the transactions. If a piece is aborted because of a conflict, it will be resubmitted until it commits If a piece is aborted because of an abort, no other pieces for that transaction will execute. Dennis Shasha, Philippe Bonnet 2001

59 Lock Tuning Transaction Chopping Rewriting applications to obtain best locking performance Isolation Levels Relaxing correctness to improve performance Latches and Locks Using system features to circumvent bottlenecks Dennis Shasha, Philippe Bonnet 2001

Lock Tuning. Concurrency Control Goals. Trade-off between correctness and performance. Correctness goals. Performance goals.

Lock Tuning. Concurrency Control Goals. Trade-off between correctness and performance. Correctness goals. Performance goals. Lock Tuning Concurrency Control Goals Performance goals Reduce blocking One transaction waits for another to release its locks Avoid deadlocks Transactions are waiting for each other to release their locks