Chapter 9: Log Manager

Transcription

1 Handout #14 Chapter 9: Log Manager Overview Reading & Writing the Log Performance Optimizations Making the Log Stable Recovery CS346 - Transaction Processing Markus Breunig - 9 / 1 -

2 Reading & Writing the Log log record LSN timestamp LSN {file name, relative byte address } log anchor files that make up log latest LSN CS346 - Transaction Processing Markus Breunig - 9 / 2 -

3 Log Anchor (in detail) LSN of most recently written record LSN of next record max LSN in stable storage checkpoint information (later!) prefix of "left" files prefix of "right" files current file prefix e.g.: /sys/log.l.001, /sys/log.l.002,... /sys/log.r.001, /sys/log.r.002,... array of open files 002 OS_Handle 003 OS_Handle CS346 - Transaction Processing Markus Breunig - 9 / 3 -

4 Read/Write Functions log open log close log read(lsn) log insert (returns LSN) Performance Optimizations flushing log (daemon, group commit) WADS multiple logs archiving log CS346 - Transaction Processing Markus Breunig - 9 / 4 -

5 Flushing the Log: the simple way log insert: allocate new record at tail fill in record write out tail block to log memory: tail disk: CS346 - Transaction Processing Markus Breunig - 9 / 5 -

6 Improvements only write when necessary move IO to daemon group commit Example: Flushing [on board - fill in] Group Commit (also called batching or boxcarrying) amortize I/O cost vs. delay in commit early lock releases? CS346 - Transaction Processing Markus Breunig - 9 / 6 -

7 WADS - Write Ahead Data Set Problem: Rotational Delay Solution: WADS assign a whole cylinder to log tail each track in this cylinder will hold one log tail block on flush write to closest sector on next free track when WADS track full, write log tail to end of log wrote log tail blocks twice! Does this improve performance? CS346 - Transaction Processing Markus Breunig - 9 / 7 -

8 Time to Write Log b = time to write block (or go over block) (approx. 0.1 ms) r = expected rotational delay (approx. 20 ms) n = number of wads tracks (approx. 10) For each of the n writes in the WADS track: t b = + b ms For the 1 write of all n+1 blocks at the log tail: t = r + ( n + 1) b 20 ms Standard scheme (per block): t = r + b 20 ms BIG savings! Note: ignores track switches when writing log at recovery, need to read WADS, reconstruct log with RAM disks, WADS not necessary CS346 - Transaction Processing Markus Breunig - 9 / 8 -

9 Multiple Logs (a) multiple disks - logically single log disk disk disk when is block i really flushed? block flushed when all blocks logical prior to it are flushed! CS346 - Transaction Processing Markus Breunig - 9 / 9 -

10 (b) separate logs - partition data or resources actions logged actions logged actions logged disk disk disk data separate logs similar to distributed system: CS346 - Transaction Processing Markus Breunig - 9 / 10 -

11 Archiving the Log writes (duplexed) archieve (tape) CS346 - Transaction Processing Markus Breunig - 9 / 11 -

12 Question: Can we Archive the Log Immediately in memory log tail disk disk archive - need dedicated tape drive (log volume may be low) - harder to compact log (e.g. remove undo entries for committed transactions) + do not need extra disk as in previous scheme CS346 - Transaction Processing Markus Breunig - 9 / 12 -

13 Making Log & Anchor Stable writing anchor to stable store log writes: serial vs. ping pong Making Anchor Stable can not write log anchor to stable storage after every update: too slow write it to a fixed location from time to time (using careful_put, stable_put, ; see Chapter 3) on crash: log anchor (especially pointer to log tail) lost retrieve log anchor from stable storage; use it as a hint to were the end of the log approximately is CS346 - Transaction Processing Markus Breunig - 9 / 13 -

14 Making Log Stable also see Chapter 3 (e.g. duplex) in class: careful writes (serial) slow but highly reliable in book: shotgun writes (parallel) quick but unsafe need to find better scheme! CS346 - Transaction Processing Markus Breunig - 9 / 14 -

15 Bad Scenario for Parallel Writes crash memory disk L both IOs fail R ok to lose log entry 74 but log entry 73 must not be lost! CS346 - Transaction Processing Markus Breunig - 9 / 15 -

16 Solution #1: Only write full pages + safe - all transactions committing in log entry 73 have to wait until 74 and 75 full Solution #2: ping pong writes memory write to two pages alternately only one lost in bad scenario L IOs i R disk i CS346 - Transaction Processing Markus Breunig - 9 / 16 -

17 Recovery finding anchor finding log tail low water marks long transactions Finding Anchor where you left it! Finding Log Tail read anchor. Anchor points to (possibly stale) log tail walk down log from (stale) pointer until you find junk identify junk by (wrong) LSN CS346 - Transaction Processing Markus Breunig - 9 / 17 -

18 Low Water Marks what part of log to keep on-line? what part to delete? log time 0 delete archive online tail keep online that part of the log needed to recover from system crashes and to undo transactions keep on archive the part of the log needed to recover from media failures CS346 - Transaction Processing Markus Breunig - 9 / 18 -

19 Three Reasons for Keeping a Log Record (1) it belongs to uncommitted/unfinished transaction transaction low water mark = def min LSN of all active transactions (2) it is a checkpoint record for a resource manager RM low water mark = def min LSN of all RMs Example of RM checkpoint: RM keeps hot pages in buffer, writes at checkpoint log RM i REDO chkpt 3 T end RM i 1 T 1 chkpt 4 low water mark RM i cannot throw away! CS346 - Transaction Processing Markus Breunig - 9 / 19 -

20 (3) media recovery (archive recovery) archive recovery low water mark = def min LSN of each object s archive recovery low water mark log Archive object x REDO x latest archive of object x delete log records with LSN j: j < archive LWM and j < RM LWM and j < transaction LWM keep on-line records with LSN k: k >= RM LWM or k >= transaction LWM CS346 - Transaction Processing Markus Breunig - 9 / 20 -

21 Long Transactions Example: T1 wrote DB, made log entry on Feb 29, 1947 and went to sleep log T 1 T x Time=Feb 29, 1947 = trans. LWM Time=1996 Solutions to Long Transactions Problem (a) delete transactions like T1 (b) recover from archive (c) copy aside log (dynamic log in IMS) move old but active log records to special log (carefully) (d) copy forwarding move old but active records to active log tail (carefully) CS346 - Transaction Processing Markus Breunig - 9 / 21 -

22 Summary Reading & Writing the Log Performance Optimizations Making the Log Stable Recovery CS346 - Transaction Processing Markus Breunig - 9 / 22 -