Database Management Systems CSEP 544. Lecture 9: Transactions and Recovery

Transcription

1 Database Management Systems CSEP 544 Lecture 9: Transactions and Recovery CSEP Fall

2 HW8 released Announcements OH tomorrow Always check the class schedule page for up to date info Last lecture today Finals on 12/9-10 Covers everything (lectures, HWs, readings) 2

3 Homework 8 A flight reservation transactional application in Java based on HW3 and Azure 2 weeks assignment Use your Azure credits to run and test 3

4 Homework 8 Throughput contest (completely optional): We will generate a random number of transactions and measure the time taken to execute them Fastest implementation wins 1 st place: 2% extra credit on HW 2 nd place: 1% extra credit on HW 3 rd place: 0.5% extra credit on HW You can create any extra tables, indexes, classes, etc in your implementation Need to pass all grading test cases to be eligible for prizes 4

5 Class overview Data models Relational: SQL, RA, and Datalog NoSQL: SQL++ RDBMS internals Query processing and optimization Physical design Parallel query processing Spark and Hadoop Conceptual design E/R diagrams Schema normalization Transactions Locking and schedules Writing DB applications CSEP Fall 2017 Data models Query Processing Using DBMS 5

6 Class Recap Data models Elements of a data model Relational data model SQL, RA, and Datalog Non-relational data model SQL++ RDBMS internals Relational algebra and basics of query processing Algorithms for relational operators Physical design and indexes Query optimization CSEP Fall

7 Class Recap Parallel query processing Different algorithms for relational operators MapReduce and Spark programming models Conceptual design E/R diagrams Normal forms and schema normalization Transactions and recovery Schedules and locking-based scheduler Recovery from failures CSEP Fall

8 Data Management Pipeline Application programmer Schema designer Conceptual Schema name price product Database administrator Physical Schema 8

9 Transactions We use database transactions everyday Bank $$$ transfers Online shopping Signing up for classes For this class, a transaction is a series of DB queries Read / Write / Update / Delete / Insert Unit of work issued by a user that is independent from others CSEP Fall

10 What s the big deal? CSEP Fall

11 Challenges Want to execute many apps concurrently All these apps read and write data to the same DB Simple solution: only serve one app at a time What s the problem? Want: multiple operations to be executed atomically over the same DBMS CSEP Fall

12 What can go wrong? Manager: balance budgets among projects Remove $10k from project A Add $7k to project B Add $3k to project C CEO: check company s total balance SELECT SUM(money) FROM budget; This is called a dirty / inconsistent read aka a WRITE-READ conflict CSEP Fall

13 What can go wrong? App 1: SELECT inventory FROM products WHERE pid = 1 App 2: UPDATE products SET inventory = 0 WHERE pid = 1 App 1: SELECT inventory * price FROM products WHERE pid = 1 This is known as an unrepeatable read aka READ-WRITE conflict CSEP Fall

14 App 1: What can go wrong? Set Account 1 = $200 Set Account 2 = $0 Account 1 = $100 Account 2 = $100 Total = $200 App 1: Set Account 1 = $200 App 2: Set Account 2 = $200 App 2: Set Account 2 = $200 Set Account 1 = $0 App 1: Set Account 2 = $0 App 2: Set Account 1 = $0 At the end: Total = $200 At the end: Total = $0 This is called the lost update aka WRITE-WRITE conflict CSEP Fall

15 What can go wrong? Buying tickets to the next Bieber / Swift concert: Fill up form with your mailing address Put in debit card number Click submit Screen shows money deducted from your account [Your browser crashes] Lesson: Changes to the database should be ALL or NOTHING CSEP Fall

16 Transactions Collection of statements that are executed atomically (logically speaking) BEGIN TRANSACTION [SQL statements] COMMIT or ROLLBACK (=ABORT) [single SQL statement] CSEP Fall 2017 If BEGIN missing, then TXN consists of a single instruction 16

17 Know your chemistry transactions: ACID Atomic State shows either all the effects of txn, or none of them Consistent Txn moves from a DBMS state where integrity holds, to another where integrity holds remember integrity constraints? Isolated Effect of txns is the same as txns running one after another (i.e., looks like batch mode) Durable Once a txn has committed, its effects remain in the database CSEP Fall

18 Atomic Definition: A transaction is ATOMIC if all its updates must happen or not at all. Example: move $100 from A to B UPDATE accounts SET bal = bal 100 WHERE acct = A; UPDATE accounts SET bal = bal WHERE acct = B; BEGIN TRANSACTION; UPDATE accounts SET bal = bal 100 WHERE acct = A; UPDATE accounts SET bal = bal WHERE acct = B; COMMIT; CSEP Fall

19 Isolated Definition An execution ensures that txns are isolated, if the effect of each txn is as if it were the only txn running on the system. CSEP Fall

20 Consistent Recall: integrity constraints govern how values in tables are related to each other Can be enforced by the DBMS, or ensured by the app How consistency is achieved by the app: App programmer ensures that txns only takes a consistent DB state to another consistent state DB makes sure that txns are executed atomically Can defer checking the validity of constraints until the end of a transaction CSEP Fall

21 Durable A transaction is durable if its effects continue to exist after the transaction and even after the program has terminated How? By writing to disk! (more later) CSEP Fall

22 Rollback transactions If the app gets to a state where it cannot complete the transaction successfully, execute ROLLBACK The DB returns to the state prior to the transaction What are examples of such program states? CSEP Fall

23 ACID Atomic Consistent Isolated Durable Enjoy this in HW8! Again: by default each statement is its own txn Unless auto-commit is off then each statement starts a new txn CSEP Fall

24 Transaction Schedules CSEP Fall

25 Schedules A schedule is a sequence of interleaved actions from all transactions CSEP Fall

26 Serial Schedule A serial schedule is one in which transactions are executed one after the other, in some sequential order Fact: nothing can go wrong if the system executes transactions serially (up to what we have learned so far) But DBMS don t do that because we want better overall system performance CSEP Fall

27 Example A and B are elements in the database t and s are variables in txn source code T1 T2 READ(A, t) READ(A, s) t := t+100 s := s*2 WRITE(A, t) WRITE(A,s) READ(B, t) READ(B,s) t := t+100 s := s*2 WRITE(B,t) WRITE(B,s) CSEP Fall

28 Example of a (Serial) Schedule Time T1 READ(A, t) t := t+100 WRITE(A, t) READ(B, t) t := t+100 WRITE(B,t) T2 READ(A,s) s := s*2 WRITE(A,s) READ(B,s) s := s*2 WRITE(B,s) CSEP Fall

29 Another Serial Schedule Time T1 T2 READ(A,s) s := s*2 WRITE(A,s) READ(B,s) s := s*2 WRITE(B,s) READ(A, t) t := t+100 WRITE(A, t) READ(B, t) t := t+100 WRITE(B,t) CSEP Fall

30 Serializable Schedule A schedule is serializable if it is equivalent to a serial schedule CSEP Fall

31 A Serializable Schedule T1 READ(A, t) t := t+100 WRITE(A, t) READ(B, t) t := t+100 WRITE(B,t) This is a serializable schedule. This is NOT a serial schedule T2 READ(A,s) s := s*2 WRITE(A,s) READ(B,s) s := s*2 WRITE(B,s) CSEP Fall

32 A Non-Serializable Schedule T1 READ(A, t) t := t+100 WRITE(A, t) READ(B, t) t := t+100 WRITE(B,t) T2 READ(A,s) s := s*2 WRITE(A,s) READ(B,s) s := s*2 WRITE(B,s) CSEP Fall

33 How do We Know if a Schedule is Serializable? Notation: T 1 : r 1 (A); w 1 (A); r 1 (B); w 1 (B) T 2 : r 2 (A); w 2 (A); r 2 (B); w 2 (B) Key Idea: Focus on conflicting operations CSEP Fall

34 Conflicts Write-Read WR Read-Write RW Write-Write WW Read-Read? CSEP Fall

35 Conflict Serializability Conflicts: (i.e., swapping will change program behavior) Two actions by same transaction T i : r i (X); w i (Y) Two writes by T i, T j to same element w i (X); w j (X) w i (X); r j (X) Read/write by T i, T j to same element r i (X); w j (X) CSEP Fall

36 Conflict Serializability A schedule is conflict serializable if it can be transformed into a serial schedule by a series of swappings of adjacent non-conflicting actions Every conflict-serializable schedule is serializable CSEP Fall

37 Conflict Serializability Example: r 1 (A); w 1 (A); r 2 (A); w 2 (A); r 1 (B); w 1 (B); r 2 (B); w 2 (B) CSEP Fall

38 Example: Conflict Serializability r 1 (A); w 1 (A); r 2 (A); w 2 (A); r 1 (B); w 1 (B); r 2 (B); w 2 (B) r 1 (A); w 1 (A); r 1 (B); w 1 (B); r 2 (A); w 2 (A); r 2 (B); w 2 (B) CSEP Fall

39 Example: Conflict Serializability r 1 (A); w 1 (A); r 2 (A); w 2 (A); r 1 (B); w 1 (B); r 2 (B); w 2 (B) r 1 (A); w 1 (A); r 1 (B); w 1 (B); r 2 (A); w 2 (A); r 2 (B); w 2 (B) CSEP Fall

40 Example: Conflict Serializability r 1 (A); w 1 (A); r 2 (A); w 2 (A); r 1 (B); w 1 (B); r 2 (B); w 2 (B) r 1 (A); w 1 (A); r 2 (A); r 1 (B); w 2 (A); w 1 (B); r 2 (B); w 2 (B) r 1 (A); w 1 (A); r 1 (B); w 1 (B); r 2 (A); w 2 (A); r 2 (B); w 2 (B) CSEP Fall

41 Example: Conflict Serializability r 1 (A); w 1 (A); r 2 (A); w 2 (A); r 1 (B); w 1 (B); r 2 (B); w 2 (B) r 1 (A); w 1 (A); r 2 (A); r 1 (B); w 2 (A); w 1 (B); r 2 (B); w 2 (B) r 1 (A); w 1 (A); r 1 (B); r 2 (A); w 2 (A); w 1 (B); r 2 (B); w 2 (B). r 1 (A); w 1 (A); r 1 (B); w 1 (B); r 2 (A); w 2 (A); r 2 (B); w 2 (B) CSEP Fall

42 Testing for Conflict-Serializability Precedence graph: A node for each transaction T i, An edge from T i to T j whenever an action in T i conflicts with, and comes before an action in T j The schedule is conflict-serializable iff the precedence graph is acyclic CSEP Fall

43 Example 1 r 2 (A); r 1 (B); w 2 (A); r 3 (A); w 1 (B); w 3 (A); r 2 (B); w 2 (B) CSEP Fall

44 Example 1 r 2 (A); r 1 (B); w 2 (A); r 3 (A); w 1 (B); w 3 (A); r 2 (B); w 2 (B) B A This schedule is conflict-serializable CSEP Fall

45 Example 2 r 2 (A); r 1 (B); w 2 (A); r 2 (B); r 3 (A); w 1 (B); w 3 (A); w 2 (B) CSEP Fall

46 Example 2 r 2 (A); r 1 (B); w 2 (A); r 2 (B); r 3 (A); w 1 (B); w 3 (A); w 2 (B) B 1 A 2 B 3 This schedule is NOT conflict-serializable CSEP Fall

47 Course Eval CSEP Fall

48 Implementing Transactions CSEP Fall

49 Scheduler Scheduler = the module that schedules the transaction s actions, ensuring serializability Also called Concurrency Control Manager We discuss next how a scheduler may be implemented CSEP Fall

50 Implementing a Scheduler Major differences between database vendors Locking Scheduler Aka pessimistic concurrency control SQLite, SQL Server, DB2 Multiversion Concurrency Control (MVCC) Aka optimistic concurrency control Postgres, Oracle We discuss only locking schedulers in this class CSEP Fall

51 Locking Scheduler Simple idea: Each element has a unique lock Each transaction must first acquire the lock before reading/writing that element If the lock is taken by another transaction, then wait The transaction must release the lock(s) By using locks scheduler CSEP 544 ensures - Fall 2017 conflict-serializability 51

52 What Data Elements are Locked? Major differences between vendors: Lock on the entire database SQLite Lock on individual records SQL Server, DB2, etc CSEP Fall

53 More Notations L i (A) = transaction T i acquires lock for element A U i (A) = transaction T i releases lock for element A CSEP Fall

54 A Non-Serializable Schedule T1 READ(A) A := A+100 WRITE(A) READ(B) B := B+100 WRITE(B) T2 READ(A) A := A*2 WRITE(A) READ(B) B := B*2 WRITE(B) CSEP Fall

55 A Serializable Schedule T1 READ(A, t) A := A+100 WRITE(A) READ(B) B := B+100 WRITE(B) T2 READ(A) A := A*2 WRITE(A) READ(B) B := B*2 WRITE(B) CSEP Fall

56 Enforcing Conflict-Serializability T1 L 1 (A); READ(A) A := A+100 WRITE(A); U 1 (A); L 1 (B) READ(B) B := B+100 WRITE(B); U 1 (B); with Locks T2 L 2 (A); READ(A) A := A*2 WRITE(A); U 2 (A); L 2 (B); BLOCKED Scheduler has ensured a conflict-serializable schedule CSEP Fall 2017 GRANTED; READ(B) B := B*2 WRITE(B); U 2 (B); 56

57 But T1 L 1 (A); READ(A) A := A+100 WRITE(A); U 1 (A); L 1 (B); READ(B) B := B+100 WRITE(B); U 1 (B); T2 L 2 (A); READ(A) A := A*2 WRITE(A); U 2 (A); L 2 (B); READ(B) B := B*2 WRITE(B); U 2 (B); Locks did not enforce conflict-serializability!!! What s wrong? 57

58 Two Phase Locking (2PL) The 2PL rule: In every transaction, all lock requests must precede all unlock requests CSEP Fall

59 Example: 2PL transactions T1 L 1 (A); L 1 (B); READ(A) A := A+100 WRITE(A); U 1 (A) READ(B) B := B+100 WRITE(B); U 1 (B); Now it is conflict-serializable T2 CSEP Fall 2017 L 2 (A); READ(A) A := A*2 WRITE(A); L 2 (B); BLOCKED GRANTED; READ(B) B := B*2 WRITE(B); U 2 (A); U 2 (B); 59

60 A New Problem: Non-recoverable Schedule T1 L 1 (A); L 1 (B); READ(A) A :=A+100 WRITE(A); U 1 (A) READ(B) B :=B+100 WRITE(B); U 1 (B); Rollback T2 L 2 (A); READ(A) A := A*2 WRITE(A); L 2 (B); BLOCKED GRANTED; READ(B) B := B*2 WRITE(B); U 2 (A); U 2 (B); Commit CSEP Fall

61 Strict 2PL The Strict 2PL rule: All locks are held until the transaction commits or aborts. With strict 2PL, we will get schedules that are both conflict-serializable and recoverable CSEP Fall

62 T1 L 1 (A); READ(A) A :=A+100 WRITE(A); L 1 (B); READ(B) B :=B+100 WRITE(B); Rollback U 1 (A);U 1 (B); Strict 2PL T2 L 2 (A); BLOCKED GRANTED; READ(A) A := A*2 WRITE(A); L 2 (B); READ(B) B := B*2 WRITE(B); Commit U 2 (A); U 2 (B); 62

63 Another problem: Deadlocks T 1 waits for a lock held by T 2 ; T 2 waits for a lock held by T 3 ; T 3 waits for T n waits for a lock held by T 1 SQL Lite: there is only one exclusive lock; thus, never deadlocks SQL Server: checks periodically for deadlocks and aborts one TXN CSEP Fall

64 Lock Modes S = shared lock (for READ) X = exclusive lock (for WRITE) Lock compatibility matrix: None S X None S X CSEP Fall

65 Lock Modes S = shared lock (for READ) X = exclusive lock (for WRITE) Lock compatibility matrix: None S X None S X CSEP Fall

66 Lock Granularity Fine granularity locking (e.g., tuples) High concurrency High overhead in managing locks E.g., SQL Server Coarse grain locking (e.g., tables, entire database) Many false conflicts Less overhead in managing locks E.g., SQL Lite Solution: lock escalation changes granularity as needed CSEP Fall

67 Lock Performance Throughput (TPS) To avoid, use admission control thrashing Why? TPS = Transactions per second # Active Transactions CSEP Fall

68 Phantom Problem So far we have assumed the database to be a static collection of elements (=tuples) If tuples are inserted/deleted then the phantom problem appears CSEP Fall

69 Suppose there are two blue products, A1, A2: Phantom Problem T1 SELECT * FROM Product WHERE color= blue SELECT * FROM Product WHERE color= blue T2 INSERT INTO Product(name, color) VALUES ( A3, blue ) Is this schedule serializable? CSEP Fall

70 Suppose there are two blue products, A1, A2: Phantom Problem T1 SELECT * FROM Product WHERE color= blue SELECT * FROM Product WHERE color= blue T2 INSERT INTO Product(name, color) VALUES ( A3, blue ) R 1 (A1);R 1 (A2);W 2 (A3);R 1 (A1);R 1 (A2);R 1 (A3) CSEP Fall

71 Suppose there are two blue products, A1, A2: Phantom Problem T1 SELECT * FROM Product WHERE color= blue SELECT * FROM Product WHERE color= blue T2 INSERT INTO Product(name, color) VALUES ( A3, blue ) R 1 (A1);R 1 (A2);W 2 (A3);R 1 (A1);R 1 (A2);R 1 (A3) W 2 (A3);R 1 (A1);R 1 (A2);R 1 (A1);R 1 (A2);R 1 (A3)

72 Phantom Problem A phantom is a tuple that is invisible during part of a transaction execution but not invisible during the entire execution In our example: T1: reads list of products T2: inserts a new product T1: re-reads: a new product appears! CSEP Fall

73 Dealing With Phantoms Lock the entire table Lock the index entry for blue If index is available Or use predicate locks A lock on an arbitrary predicate Dealing with phantoms is expensive! CSEP Fall

74 Isolation Levels in SQL 1. Dirty reads SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED 2. Committed reads SET TRANSACTION ISOLATION LEVEL READ COMMITTED 3. Repeatable reads SET TRANSACTION ISOLATION LEVEL REPEATABLE READ 4. Serializable transactions SET TRANSACTION ISOLATION LEVEL SERIALIZABLE ACID CSEP Fall

75 1. Isolation Level: Dirty Reads Long duration WRITE locks Strict 2PL No READ locks Read-only transactions are never delayed Possible problems: dirty and inconsistent reads CSEP Fall

76 2. Isolation Level: Read Committed Long duration WRITE locks Strict 2PL Short duration READ locks Only acquire lock while reading (not 2PL) Unrepeatable reads: When reading same element twice, may get two different values CSEP Fall

77 3. Isolation Level: Repeatable Read Long duration WRITE locks Strict 2PL Long duration READ locks Strict 2PL Why? This is not serializable yet!!! CSEP Fall

78 4. Isolation Level Serializable Long duration WRITE locks Strict 2PL Long duration READ locks Strict 2PL Predicate locking To deal with phantoms CSEP Fall

79 Beware! In commercial DBMSs: Default level is often NOT serializable Default level differs between DBMSs Some engines support subset of levels! Serializable may not be exactly ACID Locking ensures isolation, not atomicity Also, some DBMSs do NOT use locking and different isolation levels can lead to different pbs Bottom line: Read the doc for your DBMS! CSEP Fall

80 Recovery CSEP Fall

81 Log-based Recovery Basics (based on textbook Ch ) Undo logging Redo logging CSEP Fall

82 Transaction Abstraction Database is composed of elements. 1 element can be either: 1 page = physical logging 1 record = logical logging CSEP Fall

83 Primitive Operations of Transactions READ(X,t) copy element X to transaction local variable t WRITE(X,t) copy transaction local variable t to element X INPUT(X) read element X to memory buffer OUTPUT(X) write element X to disk CSEP Fall

84 Running Example BEGIN TRANSACTION READ(A,t); t := t*2; WRITE(A,t); READ(B,t); t := t*2; WRITE(B,t) COMMIT; CSEP Fall 2017 Initially, A=B=8. Atomicity requires that either (1) T commits and A=B=16, or (2) T does not commit and A=B=8. 84

85 READ(A,t); t := t*2; WRITE(A,t); READ(B,t); t := t*2; WRITE(B,t) Transaction Main memory Disk Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) OUTPUT(A) OUTPUT(B) COMMIT

86 Is this bad? Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) OUTPUT(A) OUTPUT(B) COMMIT Crash!

87 Is this bad? Yes it s bad: A=16, B=8. Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) OUTPUT(A) OUTPUT(B) COMMIT Crash!

88 Is this bad? Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) OUTPUT(A) OUTPUT(B) COMMIT Crash!

89 Is this bad? Yes it s bad: A=B=16, but not committed Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) OUTPUT(A) OUTPUT(B) COMMIT Crash!

90 Is this bad? Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) OUTPUT(A) OUTPUT(B) COMMIT Crash!

91 Is this bad? No: that s OK Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) OUTPUT(A) OUTPUT(B) COMMIT Crash!

92 Typically, OUTPUT is after COMMIT (why?) Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) COMMIT OUTPUT(A) OUTPUT(B)

93 Typically, OUTPUT is after COMMIT (why?) Action t Mem A Mem B Disk A Disk B INPUT(A) READ(A,t) t:=t* WRITE(A,t) INPUT(B) READ(B,t) t:=t* WRITE(B,t) COMMIT OUTPUT(A) OUTPUT(B) Crash!

94 Atomic Transactions FORCE or NO-FORCE Should all updates of a transaction be forced to disk before the transaction commits? STEAL or NO-STEAL Can an update made by an uncommitted transaction overwrite the most recent committed value of a data item on disk? CSEP Fall

95 Force/No-steal FORCE: Pages of committed transactions must be forced to disk before commit NO-STEAL: Pages of uncommitted transactions cannot be written to disk Easy to implement (how?) and ensures atomicity CSEP Fall

96 No-Force/Steal NO-FORCE: Pages of committed transactions need not be written to disk STEAL: Pages of uncommitted transactions may be written to disk In either case, Atomicity is violated; need WAL CSEP Fall

97 Write-Ahead Log The Log: append-only file containing log records Records every single action of every TXN Force log entry to disk After a system crash, use log to recover Three types: UNDO, REDO, UNDO-REDO CSEP Fall

98 UNDO Log FORCE and STEAL CSEP Fall

99 Undo Logging Log records <START T> transaction T has begun <COMMIT T> T has committed <ABORT T> T has aborted <T,X,v> T has updated element X, and its old value was v CSEP Fall

100 Action t Mem A Mem B Disk A Disk B UNDO Log <START T> INPUT(A) READ(A,t) t:=t* WRITE(A,t) <T,A,8> INPUT(B) READ(B,t) t:=t* WRITE(B,t) <T,B,8> OUTPUT(A) OUTPUT(B) COMMIT <COMMIT T> 100

101 Action t Mem A Mem B Disk A Disk B UNDO Log <START T> INPUT(A) READ(A,t) t:=t* WRITE(A,t) <T,A,8> INPUT(B) READ(B,t) t:=t* WRITE(B,t) <T,B,8> OUTPUT(A) OUTPUT(B) Crash! COMMIT <COMMIT T> WHAT DO WE DO? 101

102 Action t Mem A Mem B Disk A Disk B UNDO Log <START T> INPUT(A) READ(A,t) t:=t* WRITE(A,t) <T,A,8> INPUT(B) READ(B,t) t:=t* WRITE(B,t) <T,B,8> OUTPUT(A) OUTPUT(B) Crash! COMMIT <COMMIT T> WHAT DO WE DO? 102 We UNDO by setting B=8 and A=8

103 Action t Mem A Mem B Disk A Disk B UNDO Log <START T> INPUT(A) READ(A,t) t:=t* WRITE(A,t) <T,A,8> INPUT(B) READ(B,t) t:=t* WRITE(B,t) <T,B,8> OUTPUT(A) OUTPUT(B) COMMIT <COMMIT T> What do we do now? 103 Crash!

104 Action t Mem A Mem B Disk A Disk B UNDO Log INPUT(A) READ(A,t) t:=t* <START T> WRITE(A,t) <T,A,8> INPUT(B) READ(B,t) t:=t* WRITE(B,t) <T,B,8> OUTPUT(A) OUTPUT(B) COMMIT <COMMIT T> What do we do now? Crash! Nothing: log contains COMMIT

105 Recovery with Undo Log <T6,X6,v6> <START T5> <START T4> <T1,X1,v1> <T5,X5,v5> <T4,X4,v4> <COMMIT T5> <T3,X3,v3> <T2,X2,v2> Crash! Question1: Which updates are undone? Question 2: How far back do we need to read in the log? Question 3: What happens if there is a second crash, during recovery? 105

106 Action t Mem A Mem B Disk A Disk B UNDO Log When must we force pages to disk? INPUT(A) READ(A,t) t:=t* <START T> WRITE(A,t) <T,A,8> INPUT(B) READ(B,t) t:=t* WRITE(B,t) <T,B,8> OUTPUT(A) OUTPUT(B) COMMIT <COMMIT T>

107 Action t Mem A Mem B Disk A Disk B UNDO Log <START T> INPUT(A) READ(A,t) t:=t* WRITE(A,t) <T,A,8> INPUT(B) READ(B,t) t:=t* WRITE(B,t) <T,B,8> OUTPUT(A) OUTPUT(B) FORCE COMMIT <COMMIT T> RULES: log entry before OUTPUT before COMMIT 107

108 Undo-Logging Rules U1: If T modifies X, then <T,X,v> must be written to disk before OUTPUT(X) U2: If T commits, then OUTPUT(X) must be written to disk before <COMMIT T> Hence: OUTPUTs are done early, before the transaction commits FORCE CSEP Fall

109 REDO Log NO-FORCE and NO-STEAL CSEP Fall

110 Is this bad? Action t Mem A Mem B Disk A Disk B READ(A,t) t:=t* WRITE(A,t) READ(B,t) t:=t* WRITE(B,t) COMMIT OUTPUT(A) OUTPUT(B) Crash! 110

111 Is this bad? Yes, it s bad: A=16, B=8 Action t Mem A Mem B Disk A Disk B READ(A,t) t:=t* WRITE(A,t) READ(B,t) t:=t* WRITE(B,t) COMMIT OUTPUT(A) OUTPUT(B) Crash! 111

112 Is this bad? Action t Mem A Mem B Disk A Disk B READ(A,t) t:=t* WRITE(A,t) READ(B,t) t:=t* WRITE(B,t) COMMIT OUTPUT(A) Crash! OUTPUT(B)

113 Is this bad? Yes, it s bad: lost update Action t Mem A Mem B Disk A Disk B READ(A,t) t:=t* WRITE(A,t) READ(B,t) t:=t* WRITE(B,t) COMMIT OUTPUT(A) Crash! OUTPUT(B)

114 Is this bad? Action t Mem A Mem B Disk A Disk B READ(A,t) t:=t* WRITE(A,t) READ(B,t) t:=t* WRITE(B,t) COMMIT Crash! OUTPUT(A) OUTPUT(B)

115 Is this bad? No: that s OK. Action t Mem A Mem B Disk A Disk B READ(A,t) t:=t* WRITE(A,t) READ(B,t) t:=t* WRITE(B,t) COMMIT Crash! OUTPUT(A) OUTPUT(B)

116 Redo Logging One minor change to the undo log: <T,X,v>= T has updated element X, and its new value is v CSEP Fall

117 Action t Mem A Mem B Disk A Disk B REDO Log <START T> READ(A,t) t:=t* WRITE(A,t) <T,A,16> READ(B,t) t:=t* WRITE(B,t) <T,B,16> COMMIT <COMMIT T> OUTPUT(A) OUTPUT(B)

118 Action t Mem A Mem B Disk A Disk B REDO Log <START T> READ(A,t) t:=t* WRITE(A,t) <T,A,16> READ(B,t) t:=t* WRITE(B,t) <T,B,16> COMMIT <COMMIT T> OUTPUT(A) OUTPUT(B) Crash! How do we recover? 118

119 Action t Mem A Mem B Disk A Disk B REDO Log <START T> READ(A,t) t:=t* WRITE(A,t) <T,A,16> READ(B,t) t:=t* WRITE(B,t) <T,B,16> COMMIT <COMMIT T> OUTPUT(A) OUTPUT(B) Crash! How do we recover? We REDO by setting A=16 and 119 B=16

120 Recovery with Redo Log <START T1> <T1,X1,v1> <START T2> <T2, X2, v2> <START T3> <T1,X3,v3> <COMMIT T2> <T3,X4,v4> <T1,X5,v5> Crash! Show actions during recovery CSEP Fall

121 Action t Mem A Mem B Disk A Disk B REDO Log When must we force pages to disk? READ(A,t) t:=t* <START T> WRITE(A,t) <T,A,16> READ(B,t) t:=t* WRITE(B,t) <T,B,16> COMMIT <COMMIT T> OUTPUT(A) OUTPUT(B)

122 Action t Mem A Mem B Disk A Disk B REDO Log <START T> READ(A,t) t:=t* WRITE(A,t) <T,A,16> READ(B,t) t:=t* WRITE(B,t) <T,B,16> NO-STEAL COMMIT <COMMIT T> OUTPUT(A) OUTPUT(B) RULE: OUTPUT after COMMIT 122

123 Redo-Logging Rules R1: If T modifies X, then both <T,X,v> and <COMMIT T> must be written to disk before OUTPUT(X) NO-STEAL Hence: OUTPUTs are done late CSEP Fall

124 Comparison Undo/Redo Undo logging: OUTPUT must be done early: Inefficient Redo logging: OUTPUT must be done late: Inflexible Compromise: ARIES (see textbook) CSEP Fall

125 End of CSEP 544 Big data is here to stay Requires unique techniques / abstractions Logic (SQL) Algorithms (query processing) Conceptual modeling (FD s) Transactions Technology evolving rapidly, but Techniques/abstracts persist over may years, e.g. What goes around comes around CSEP Fall