Atomicity: All actions in the Xact happen, or none happen. Consistency: If each Xact is consistent, and the DB starts consistent, it ends up

Transcription

1 CRASH RECOVERY 1

2 REVIEW: THE ACID PROPERTIES Atomicity: All actions in the Xact happen, or none happen. Consistency: If each Xact is consistent, and the DB starts consistent, it ends up consistent. Isolation: Execution of one Xact is isolated from that of other Xacts. Durability: If a Xact commits, its effects persist. Which properties does Recovery Manager deal with?

3 MOTIVATION Atomicity: Transactions may abort ( Rollback ). Durability: What if DBMS stops running? 3. 12

4 MOTIVATION Desired state after system restarts? T1 & T3 should be durable. T2, T4 & T5 should be aborted (effects not seen).

5 ASSUMPTIONS Concurrency control is in effect. Strict 2PL, in particular. Updates are happening in place. i.e. data is overwritten on (deleted from) the actual page copies (not private copies). No harddisk failures. 3. 2

6 A SIMPLE NON-LOGGING SCHEME Can you come up with a non-logging scheme to guarantee Atomicity & Durability? Hint: after a crash, all the data in the buffer lost, data on the disk persists. Question is when to ush buffer pages to disks. What to do with each read/write operation? What to do when a transaction commits? What to do when a transaction aborts? What is the minimum lock granularity? 3. 3

7 BUFFER MANAGEMENT PLAYS A KEY ROLE Force policy make sure that every update is on disk before commit. Provides durability without REDO logging. But, can cause poor performance. No Steal policy no UNCOMMITED updates are written to disk. Useful for ensuring atomicity without UNDO logging. But can cause poor performance. Of course, there are some nasty details for getting Force/NoSteal to work 3. 4

8 PREFERRED POLICY: STEAL/NO-FORCE This combination is most complicated but allows for the highest performance. NO FORCE (complicates enforcing Durability) What if system crashes before a page updated by a committed Xact is ushed to disk? Write down the information about the updates performed by the committed Xact, to support REDOing modi cations. 3. 5

9 PREFERRED POLICY: STEAL/NO-FORCE STEAL (complicates enforcing Atomicity) What if the Xact that performed updates aborts? What if system crashes before Xact is nished? Must remember the old value of P (to support UNDOing the write to page P)

10 BUFFER MANAGEMENT SUMMARY

11 BASIC IDEA: LOGGING To enable REDO/UNDO: record each update in a log. Log: An ordered list of REDO/UNDO info What should a log record contain? a unique identi er LSN (log sequence number) Who makes the action? XID (Transaction ID: ) Where does the action happen? pageid, offset, length What is changed? old data, new data and some control info (we will see later) Only those log records written on disk could be used for recovery after crash! 3. 8

12 WHEN MUST A LOG RECORD BE FLUSHED TO DISK? What if the system crashes 1. after an updated page is written to disk but the Xact who makes the updates hasn t committed? Atomicity: After restart, we have to undo the updates. So log record must go to disk before the changed page! 2. after a Xact commit but its updates have not been written to disk? Durability: After restart, we have to redo the Xact All log records for a Xact must be written to disk before the Xact is considered Committed. The above is the so-called Write Ahead Logging (WAL) protocol 4. 1

13 HOW TO RECOVER AFTER A CRASH? At the time of crash, there are two categories of Xacts : Committed Xacts: their actions should be redone (durability) Incomplete Xacts: their actions should be undone (atomicity) Which part of the log should we process? 4. 2

14 SIMPLE RECOVERY ALGORITHM 1. From the beginning of the Log to the end, REDO to recover to the status at crash Including those actions of incomplete Xacts 2. From the end of the Log to the beginning, UNDO all the actions of the incomplete Xacts

15 THE LOG COULD BE VERY LONG! Can we avoid always examining the whole log? Why examine the whole log? REDO: we are not sure if the updates of committed records have been written to disk UNDO: we are not sure if there is a incomplete Xact starts at a very early time Solution: periodically record the system state, Checkpointing

16 A SIMPLE CHECKPOINTING SCHEME Periodically start checkpointing write a log record begin_checkpoint: and store the XIDs of the current incomplete Xacts Flush all the dirty buffer pages to disk (incl. updates by incomplete Xacts) Write a log record end_checkpoint and ush the log Redo just starts at the latest checkpoint Undo need not go beyong the earliest start point of the incomplete Xacts 5. 2

17 A SIMPLE CHECKPOINTING SCHEME What is the problem of this approach? 5. 3

18 CAN WE AVOID FLUSHING THE DIRTY PAGES? Idea: we already have the log records of the actions that make the pages dirty, so we could redo them! Maintain an in-memory table: Dirty Page Table: One entry per dirty page currently in buffer pool. Contains reclsn the LSN of the log record which rst caused the page to be dirty. 5. 4

19 A BETTER CHECKPOINTING SCHEME Periodically, write to log: begin_checkpoint record: Indicates when chkpt began. end_checkpoint record: Transaction Table: current incomplete Xact Dirty Page Table Other Xacts continue to run; so the information is accurate only as of the time of the begin_checkpoint record. REDO: use the reclsns in the dirty page table to redo the actions to restore to the state at crash UNDO: undo the actions of the incomplete Xacts 5. 5

20 CRASH WHILE ABORTING A XACT The actions of aborting a Xact should also be redone during recovery So we have to put those actions in the log. Compensation Log Records (CLRs): for UNDO actions 5. 67

21 COMPENSATION LOG RECORDS describes the action to undo an update action the contents is similar to an update log record, except it doesn t contain the new data value. (Why?) undonextlsn: the next log record to be undone to abort the Xact After restart from a crash the REDO phase will reapplied the undo actions described by the CLRs the UNDO phase will continue the abort by undoing undonextlsn and the earlier actions of the same Xact

22 WHAT IF THE SYSTEM CRASHES DURING RECOVERY The recovery algorithm should be run again, however, it could be run more ef ciently

23 CRASH DURING THE REDO PHASE: Some updates need not be redone if they are written to disk at the last REDO phase. Note: pagelsn stored at each page indicates the most recent log record for an update to that page 5. 10

24 CRASH DURING THE UNDO PHASE: Similar to crash during Xact aborts All the actions of the last UNDO phase should be reapplied the REDO phase will redo the CLRs then UNDO phase will continue with the undo tasks left 5. 11

25 SUMMARY/RECAP Transactions support the ACID properties. Recovery Manager guarantees Atomicity & Durability. NO FORCE & STEAL offers the best performance Need both REDO and UNDO Log is used to record the actions to REDO and UNDO

26 SUMMARY/RECAP WAL ensures the correctness of logging write log record to disk before writing the update page write all the log records of a Xact to disk before it commits Checkpointing avoids always processing the whole log UNDO actions should also be logged (CLR) 5. 12

27 PUTTING THEM TOGETHER IBM S ARIES Now we will see how to implement these ideas in a real system

28 LOG Each log record has a unique Log Sequence Number (LSN). LSNs always increasing. Log records are written to disk through an output buffer page System keeps track of ushedlsn > The max LSN ushed so far. Each data page contains a pagelsn. > The LSN of the most recent log record for an update to that page.

29 LOG 6. 2

30 LOG RECORDS (1) LogRecord elds: LSN prevlsn prevlsn is the LSN of the previous log record of the same Xact Transaction ID type 6. 3

31 LOG RECORDS (2) For update records only: pageid length offset before-image after-image 6. 4

32 LOG RECORD TYPES Update Commit Abort Checkpoint (for log maintainence) Compensation Log Records (CLRs, for UNDO actions) End (end of commit or abort) 6. 5

33 OTHER LOG-RELATED STATE Two in-memory tables: Transaction Table Dirty Page Table 6. 67

34 TRANSACTION TABLE One entry per currently incomplete Xact. entry removed when Xact commits or aborts Contains XID, status (running/committing/aborting), and lastlsn (most recent LSN written by Xact). 6. 8

35 DIRTY PAGE TABLE One entry per dirty page currently in buffer pool Contains reclsn the LSN of the log record which to be dirty. rst caused the page 6. 9

36 NORMAL EXECUTION OF AN XACT Series of reads & writes, followed by commit or abort. We will assume that disk write is atomic. (In practice, additional details to deal with non-atomic writes). Strict 2PL STEAL, NO-FORCE buffer management, with Write-Ahead Logging. WAL Rule #1: Before page i is written to DB, log must satisfy: pagelsni ushedlsn

37 NORMAL EXECUTION OF AN XACT 6. 10

38 TRANSACTION COMMIT Write commit record to log. Flush all log records up to Xact s commit record to log disk. WAL Rule #2: Ensure that ushedlsn >= lastlsn. Force log out up to lastlsn if necessary Note that log ushes are sequential, synchronous writes to disk and many log records per log page. so, cheaper than forcing out the updated data pages. Commit() returns. Write end record to log

39 SIMPLE TRANSACTION ABORT For now, consider an explicit abort of a Xact > No crash involved. We want to "play back" the log in reverse order, UNDOing updates. Write an Abort log record before starting to rollback operations. Get lastlsn of Xact from Xact table. Follow chain of log records backward via the prevlsn eld. For each update encountered: Write a "CLR" (compensation log record) for each undone operation. Undo the operation (using before image from log record)

40 ABORT 6. 13

41 ABORT To perform UNDO, must have a lock on data! > No problem (we re doing Strict 2PL)! Before restoring old value of a page, write a CLR: You continue logging while you UNDO!! CLR has one extra eld: undonextlsn Points to the next LSN to undo (i.e. the prevlsn of the record we re currently undoing). CLRs never Undone (but they might be Redone when repeating history: guarantees Atomicity!) At end of UNDO, write an "end" log record

42 CHECKPOINTING Periodically, the DBMS creates a checkpoint by writing to log: begin_checkpoint record: Indicates when chkpt began. end_checkpoint record: Contains current Xact table and dirty page table. This is a `fuzzy checkpoint : Other Xacts continue to run; so these tables accurate only as of the time of the begin_checkpoint record. No attempt to force dirty pages to disk; effectiveness of checkpoint limited by oldest unwritten change to a dirty page. Store LSN of most recent chkpt record in a safe place (master record)

43 THE BIG PICTURE: WHAT S STORED WHERE - LOG LogRecords: prevlsn XID type pageid length offset before-image after-image 6. 16

44 THE BIG PICTURE: WHAT S STORED WHERE - DB Data pages each with a pagelsn master record LSN of most recent checkpoint

45 THE BIG PICTURE: WHAT S STORED WHERE - RAM Transaction Table lastlsn status Dirty Page Table reclsn ushedlsn

46 EXAMPLE

47 SCENARIO

48 SCENARIO 7. 2

49 STATE WHEN SYSTEM UP AGAIN 7. 3

50 MISSION 7. 4

51 RECAP - ARIES 7. 5

52 Analysis - update structures: RECAP - ARIES Start from the latest checkpoint Three phases. Need to: Transaction Table: which Xacts were active at time of crash. Dirty Page Table: which pages might have been dirty in the buffer pool at time of crash. REDO all actions. (repeat history) UNDO effects of failed Xacts 7. 6

53 RECOVERY: THE ANALYSIS PHASE Re-establish knowledge of state at checkpoint. via transaction table and dirty page table stored in the checkpoint Restore to the state at crash. Scan log forward from checkpoint. End record: Remove Xact from Xact table. All Other records: Add Xact to Xact table, set lastlsn=lsn, change Xact status on commit. also, for Update records: If page P not in Dirty Page Table, Add P to DPT, set its reclsn=lsn. 7. 7

54 RECOVERY: THE ANALYSIS PHASE At the end of Analysis Transaction table says which xacts were active at time of crash. DPT says which dirty pages might not have made it to disk 7. 89

55 ANALYSIS - TRANSACTION TABLE

56 ANALYSIS - DIRTY PAGE TABLE 7. 10

57 PHASE 2: THE REDO PHASE We repeat History to reconstruct state at crash: Reapply all updates (even of aborted Xacts!), redo CLRs. Scan forward from log rec containing smallest reclsn in DPT. QUESTION: why start here? For each update log record or CLR with a given LSN, REDO the action unless: Affected page is not in the Dirty Page Table, or Affected page is in D.P.T., but has reclsn > LSN, or pagelsn (in DB) >= LSN. (this last case requires I/O) 7. 11

58 PHASE 2: THE REDO PHASE Reapply logged action. To REDO an action: Set pagelsn to LSN. No additional logging, no forcing!

59 REDO

60 REDO 7. 14

61 REDO 7. 15

62 REDO 7. 16

63 REDO 7. 17

64 PHASE 3: THE UNDO PHASE ToUndo = {lastlsns of all Xacts in the Trans Table} Repeat: Choose (and remove) largest LSN among ToUndo. If this LSN is a CLR and undonextlsn==null Write an End record for this Xact. If this LSN is a CLR, and undonextlsn!= NULL Add undonextlsn to ToUndo Else this LSN is an update. Undo the update, write a CLR, add prevlsn to ToUndo. Until ToUndo is empty

65 UNDO 7. 19

66 UNDO 7. 20

67 ADDITIONAL CRASH ISSUES How do you limit the amount of work in REDO? Flush data pages to disk asynchronously in the background (during normal operation and recovery). Watch hot spots! How do you limit the amount of work in UNDO? Avoid long-running Xacts

68 SUMMARY 9. 18

69 SUMMARY OF LOGGING/RECOVERY Use Write Ahead Logging (WAL) to allow STEAL/NO-FORCE buffer manager without sacri cing correctness. LSNs identify log records; linked into backwards chains per transaction (via prevlsn). pagelsn allows to check if updates have been written to disk. 9. 2

70 SUMMARY Checkpointing: A quick way to limit the amount of log to scan on recovery. ARIES recovery works in 3 phases: Analysis: Forward from checkpoint. Rebuild transaction and dirty page tables. Redo: Forward from oldest reclsn, repeating history for all transactions. Undo: Backward from end to rst LSN of oldest Xact alive at crash. Rollback all transactions not completed as of the time of the crash. Upon Undo, write CLRs. Nesting structure of CLRS avoids having to undo undo operations. Redo "repeats history": Simpli es the logic!

71 QUESTIONS?