Ecient Redo Processing in. Jun-Lin Lin. Xi Li. Southern Methodist University

Transcription

1 Technical Report 96-CSE-13 Ecient Redo Processing in Main Memory Databases by Jun-Lin Lin Margaret H. Dunham Xi Li Department of Computer Science and Engineering Southern Methodist University Dallas, Texas , USA November 1996 This research is based upon work supported by the National Science Foundation under Grant IRI , and by a Massive Digital Data System (MDDS) eort sponsored by the Advanced Research and Development Committee of the Community Management Sta.

2 Ecient Redo Processing in Main Memory Databases y Jun-Lin Lin, Margaret H. Dunham, and Xi Li Department of Computer Science and Engineering Southern Methodist University Dallas, Texas , USA fjun, mhd, xilig@seas.smu.edu Abstract In a Main Memory Database system, ecient recovery after a system failure is crucial. As part of the recovery procedure, the correct location in the log from which to perform redo processing must be determined. In this paper we discuss the Logging After Writing (LAW) logging protocol and its extension, Relaxed LAW, which make the identication of the redo-point simple. Keywords: Databases; Main memory database recovery; Logging; Checkpointing 1 Introduction Main Memory Databases (MMDB) place all or a major portion of the database in main memory to achieve high throughput and short response time [2, 3, 4]. Due to the volatility of main memory, recovery with MMDB is especially crucial. The recovery activities should interfere with normal transactions as little as possible to avoid degrading the system performance. Fuzzy checkpointing has been shown to be one of the best checkpointing approaches for MMDB due to its low interference with normal transaction activities [6, 10]. With fuzzy checkpointing, the checkpointer rst writes a begin-chkpt-record to the log, y This research is based upon work supported by the National Science Foundation under Grant IRI , and by a Massive Digital Data System (MDDS) eort sponsored by the Advanced Research and Development Committee of the Community Management Sta. 1

3 then dumps out the main memory content to nonvolatile storage (e.g., disks) without quiescing normal transaction activities, and nally writes an end-chkpt-record to the log. A begin-chkpt-record and end-chkpt-record pair denes a complete checkpoint interval. Since all of the active transactions are still in progress when the dump is taken, the resulting dump is a fuzzy dump. A transaction-consistent state of the database can be reconstructed by applying the log to the fuzzy dump in case of a system crash. Applying the log to the fuzzy dump usually involves two steps: backward undo processing and forward redo processing. Backward undo processing is to remove the eect of aborted transactions by applying their undo-log-records backward to the fuzzy dump. Forward redo processing is to reconstruct the eect of committed transactions by applying their redo-logrecords forward to the dump. Here, undo-log-records and redo-log-records are generated by normal transactions during run time to record information about how to undo and redo a transaction. Since the checkpointer ushes dirty data to nonvolatile storage regularly, the eect of most committed transactions has been propagated to nonvolatile storage. Thus, there is no need to redo all the redo-log-records for all committed transactions after a system failure. We denote the redo-point as the position in the log from which to start forward redo processing. Logging After Writing (LAW) [7] is a logging protocol that facilitates the restart operation in determining a proper redo-point. LAW states that the redo-log-records of an operation must be written to nonvolatile storage after the operation updates the main memory database. When fuzzy checkpointing with LAW is used in a main memory database system the redo-point is at the begin-chkpt-record of the most recent complete checkpoint interval [7]. Notice that this may not be true if LAW is violated; Figure 1 gives an example. In Figure 1, a transaction T updates the content of a data item X, then commits. The checkpointing process is running in parallel with the execution of T. For clarity, the undo logging activity is not shown in Figure 1, since it does not aect where the redopoint is. In this example, X was checkpointed before it was updated, thus the updated result has not been ushed to nonvolatile storage. If the system crashes after taking a complete checkpoint and committing T, but before ushing the update of T to nonvolatile 2

4 begin-chkpt checkpoint X (clean, no flush) end-chkpt system crash time begin-transaction(t) redo-log(t,x) update X in main memory end-transaction(t) violate the LAW protocol Redo Rule: flush log buffer to nonvolatile log Figure 1: A logging example without LAW. storage, then the redo-point is not at the begin-chkpt-record of the most recent complete checkpoint; otherwise, the update of the T will be lost, since the redo-log-record of T precedes the begin-chkpt-record in the log. Without the LAW protocol, we cannot simply use the begin-chkpt-record of the most recent complete checkpoint interval as the redo-point. One way to decide a proper redopoint without enforcing LAW protocol is a backward scan on the log starting from the begin-chkpt-record of the most recent complete checkpoint interval. This approach certainly reduces the performance of recovery in the MMDB. A second approach requires the checkpointing process to store some information in the chkpt-records during run time in order to determine a proper redo-point if a system failure occurs. ARIES [9] falls into this category. With ARIES, each main memory page maintains a Recovery Log Sequence Number (denoted as REC-LSN). When a page is about to be updated, the REC-LSN of that page is assigned to be the current end-of-log log sequence number. Each chkpt-record contains minimum(rec-lsns of all dirty pages), and the redo-point is determined as minimum(lsn of begin-chkpt record, minimum(rec-lsns of all dirty pages)). With LAW, there is no need to maintain REC-LSNs during normal transaction processing. Also, it is not necessary to include minimum(rec-lsns of all dirty pages) in a chkpt-record. Thus, using LAW we can avoid the overhead caused by the REC-LSNs themselves and by the overhead of keeping track of the minimum(rec-lsns of all dirty pages). To make LAW work eciently, nonvolatile log buer is required, otherwise LAW will incur much overhead. In this paper, we relax the LAW protocol and prove that the Relaxed 3

5 LAW (R-LAW) protocol can achieve the same purpose as LAW, but without the need for special hardware, such as nonvolatile log buer, in order to work eciently. The rest of this paper is organized as follows. Section 2 discusses the assumption of this paper. Section 3 revises the LAW protocol and shows its correctness. Section 4 proposes and proves the R-LAW protocol. Section 5 concludes this paper and gives future directions. 2 Assumptions In this paper we assume that the underlying MMDB is composed of one or more CPUs, volatile main memory, log buer, archived disks, and log disks. The primary copy of the database resides in the main memory, while the nonvolatile version of the database is stored in the archived disks. Log records are rst written to the log buer, and then ushed to the log disks. The log buer can be volatile or nonvolatile. To ensure durability of committed transactions, a transaction cannot commit until all of its redo-log-records have been ushed to the nonvolatile log. This is called the Redo Rule [1]. We assume the fuzzy checkpointing process works as follows: 1. write a begin-chkpt-record into log buer, and record its address in the restart le; 2. check all pages in the database, ush dirty ones, and update dirty-page-bitmap accordingly; 3. write an end-chkpt-record into log buer; 4. ush log buer if log buer is volatile; In Step 2, the checkpointing process checks every page in main memory at least once to nd dirty ones, and ushes them to the archived disks. However, due to lack of synchronization between the checkpointing process and the active transactions, not all dirty pages are ushed to the archived disks. Basically, dirty pages can be categorized into two groups: old-dirty-pages and young-dirty-pages, depending upon whether they are made dirty before or after the most recent begin-chkpt-record is written to the log buer, respectively. Consequently, all old-dirty-pages will be checked and ushed to archived disks, while youngdirty-pages might be or might not be ushed to the archived disks, depending on whether 4

6 they are dirty or clean when they are checkpointed. As a result, after a complete checkpoint interval, all the old-dirty-pages and probably some young-dirty-pages are ushed to the archived disks. Those young-dirty-pages that have not been ushed during the current checkpoint interval will become old-dirty-pages for the next checkpoint interval and be ushed then. 3 LAW Protocol The LAW protocol proposed in [7] assumes the use of a nonvolatile log buer. As a result, all the log records survive a system failure. Thus, we can state the relationship between the LAW protocol and redo-point as follows: Lemma 1 (LAW Protocol and redo-point) If every redo-log-record is generated after its corresponding update is propagated to the database in main memory, and all the log records survive a system failure, then we can use the begin-chkpt-record of the most recent complete checkpoint interval as the redo-point. Proof: We use < to indicate the order of occurrence. If A < B, then A occurs earlier than B. We use redo-log-x to represent one of the redo-log-records for an operation update X. Thus with LAW, we have the following: update X < redo-log-x We denote the begin-chkpt-record of the most recent complete checkpoint interval as r-begin-chkpt-record. Since there is no synchronization between the checkpointer and normal transactions, there are three possible orderings among update X, redo-log-x, and r-begin-chkpt-record. In what follows, we show we can use r-begin-chkpt-record as redo-point in each of the three cases. Case 1: update X < redo-log-x < r-begin-chkpt-record Since X is made dirty before the checkpoint starts, X will be ushed by the checkpointer. Thus, if a system failure occurs, there is no need to apply redo-log-x during forward redo processing. Therefore, we can use r-begin-chkpt-record as redo-point. [8] uses the log sequence number to control that only old-dirty-pages are ushed. 5

7 Case 2: update X < r-begin-chkpt-record < redo-log-x Similar to Case 1, there is no need to apply redo-log-x during forward redo processing. Although r-begin-chkpt-record < redo-log-x, we can still use r-beginchkpt-record as redo-point. Case 3: r-begin-chkpt-record < update X < redo-log-x Since X is made dirty after the checkpoint starts, X might be or might not be ushed by the checkpointer. Thus, redo-log-x must be included in the forward redo processing. Since r-begin-chkpt-record < redo-log-x, using r-begin-chkpt-record as redo-point satises this requirement. 2 With a nonvolatile log buer, LAW can be implemented easily, and Lemma 1 can be used at recovery time to determine a proper redo-point. Each transaction rst updates the data item in main memory, then writes the corresponding redo-log-records to the nonvolatile log buer. But, without a nonvolatile log buer, it is more expensive to control when to ush those redo-log-records. The problem becomes more complicated when the Write Ahead Logging protocol [5] and the Redo Rule are enforced since both need to control when to ush the log buer. In the next section, we propose a Relaxed LAW (R-LAW) protocol, which can be implemented easily without a nonvolatile log buer, and still achieve the same eect as LAW does. 4 Relaxed LAW Protocol With Relaxed LAW Protocol (R-LAW), we do not assume that all the log records survive a system failure. Instead, we allow the log buer to be volatile. As a result, after a system crash, the log records in the log buer are lost, while the log records in the log disks are still intact. Lemma 2 shows that we can still use the begin-chkpt-record of the most recent complete checkpoint interval as the redo-point. Lemma 2 (R-LAW Protocol and redo-point) If every redo-log-record is generated after its corresponding update is propagated to the database in main memory, then we can use the begin-chkpt-record of the most recent complete checkpoint interval as the redo-point. 6

8 Proof: We denote the begin-chkpt-record of the most recent complete checkpoint interval as r-begin-chkpt-record. If all the log records survive a system failure (e.g., by using a nonvolatile log buer), then R-LAW is the same as LAW. However, if the log buer is volatile, then the log records in the log buer are lost after a system failure. Let redolog-x be one of the redo-log-records for an operation update X. All possible scenarios are listed below: 1. with nonvolatile log buer: R-LAW is the same as LAW. 2. without nonvolatile log buer: Case 1: update X < redo-log-x < r-begin-chkpt-record! X will be ushed by the checkpointer! there is no need to process redo-log-x during forward redo processing! the location of redo-log-x in the log does not aect where the redo-point should be.! we can use r-begin-chkpt-record as redo-point. Case 2: update X < r-begin-chkpt-record < redo-log-x! same as Case 1. Case 3: r-begin-chkpt-record < update X < redo-log-x Since update X occurs after r-begin-chkpt-record, its updated result might be or might not be ushed by the checkpointer. There are three possible scenarios as follows: (a) update X is ushed! same as Case 1. (b) update X is not ushed and redo-log-x is not lost! same as LAW. (c) update X is not ushed and redo-log-x is lost! according to Redo-Rule, this transaction is not committed yet.! there is no need to process redo-log-x during forward redo processing.! we can use r-begin-chkpt-record as redo-point. 2 7

9 R-LAW can be implemented easily with or without nonvolatile log buer. With nonvolatile log buer, when the nonvolatile log buer is full, LAW needs to wait for some log I/O in order to append new redo-log-records to the log buer. This is not needed with R-LAW; log records can be rst written to some volatile log buer if the nonvolatile log buer is full. Without nonvolatile log buer, R-LAW works ne with WAL and Redo-Rule, since it does not force log I/O. Thus, R-LAW is more ecient than LAW. 5 Conclusions and Future Directions This paper discusses two redo logging protocols: LAW and R-LAW. When fuzzy checkpointing is used in an Main Memory Database system, both protocols provide an easy way to decide the redo-point. We prove that, by using either LAW or R-LAW, we can use the begin-chkpt-record of the most recent complete checkpoint interval as redo-point. We also show that R-LAW is more ecient and easier to implement than LAW. Future directions include how to apply R-LAW to various fuzzy checkpointing techniques, and extend R-LAW to Disk Resident Database systems. References [1] P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. Addison-Wesley (Reading MA), [2] D. J. DeWitt, R. H. Katz, F. Ohlken, L. D. Shapiro, M. R. Stonebraker, and D. Wood. Implementation techniques for main memory databases. In Proceedings of ACM- SIGMOD 1984 International Conference on Management of Data, pages 1{8, July [3] M. H. Eich. A classication and comparison of main memory database recovery techniques. In Proc. IEEE CS Intl. Conf. No. 3 on Data Engineering, Los Angeles, pages 332{339, February [4] H. GarciaMolina and K. Salem. Main memory database systems: An overview. IEEE Transactions on Knowledge and Data Engineering, 4(6):509{516, December

10 [5] J. Gray and A. Reuter. Transaction Processing: Concepts and Techniques. Morgan Kaufmann Publishers, San Mateo, California, [6] R. B. Hagmann. A crash recovery scheme for a memory-resident database system. IEEE Transactions on Computers, C-35(9):839{843, September [7] X. Li and M. H. Eich. A new logging protocol: LAW. Technical Report CSE9221, Southern Methodist University, Dept. of Computer Science and Engineering, Dallas ( TX), October [8] B. G. Lindsay, P. G. Selinger, C. Galtieri, J. N. Gray, R. A. Lorie, T. G. Price, F. Putzolu, I. L. Traiger, and B. W. Wade. Notes on distributed databases. Technical report, IBM Research Report No. RJ2571, San Jose, Research.Laboratory (CA), July [9] C. Mohan, D. Haderle, B. Lindsay, H. Pirahesh, and P. Schwarz. ARIES: A transaction recovery method supporting ne-granularity locking and partial rollbacks using writeahead logging. ACM Transactions on Database Systems, 17(1), March Also published in/as: IBM Almaden Res.Ctr, Res.R. No.RJ-6649, Jan.1989, 45pp. [10] K. Salem and H. GarciaMolina. Checkpointing memory-resident databases. In Proc. IEEE CS Intl. Conf. No. 5 on Data Engineering, Los Angeles, pages 452{462, February