ECE 569 Transaction Processing Fall 2004 (Due last week of classes)

Size: px

Start display at page:

Download "ECE 569 Transaction Processing Fall 2004 (Due last week of classes)"

Kimberly Reed
6 years ago
Views:

1 ECE 569 Transaction Processing Fall 2004 (Due last week of classes) Project Overview In this part of the project we will add support for distributed transactions to the resource manager previously completed. The structure of the system is shown below: Client Transaction Program Client Transaction Program Resource (RM) Data Transaction (TM) Data Resource (RM) Lock (LM) Buffer (BM) Buffer (BM) Lock (LM) Log The client contacts the transaction manager (TM) to initiate and terminate transactions. The transaction manager is also responsible for coordinating commitment of distributed transactions and initiating recovery. The log manager (LGM) writes uninterpreted UNDO/REDO records to the log on behalf of RMs, writes commit and abort records generated by the TMs and controls the flushing of log records to disk. It also supports the retrieval of log records for the purpose of recovery. The RM must be modified to generate log records and support the REDO and UNDO of the operations that are logged. In addition, a lock manager (LM) is to be added. The LM provides locks in one of several possible modes upon request and delays transactions until the requested lock becomes available. It also supports the release of a transactions locks upon its commit. In this part of the project you will implement the LM, TM and LGM.

2 Buffer I will provide a version of BM that follows the write-ahead logging discipline, i.e., the committed version of a page is never modified until the log records needed to reconstruct it are on disk. Lock Implementation We will use rigorous two-phase locking as our method of concurrency control. A transaction will acquire a shared lock on every element it reads and an exclusive lock on every element it writes. It is required to hold all of its locks until it has committed. This results in a strict schedule and simplifies recovery. The database will have a large collection of lockable elements. Thus, the lock table will be large and sparse. To reduce the size of the lock table we only record information on locks currently granted or requested. If the lock is not found in the table we can assume it is free. To find locks quickly in the lock table, hashing is used. The interface to the lock manager is simple. The prototypes are given below: enum lock_reply {lock_ok, lock_deadlock}; lock_reply lock(trid trid, lock_name name, lock_mode mode lock_reply unlock(trid trid, lock_name name Deadlocks will be detected using timeout. This can be augmented with other methods if you choose. To support this, each lock request includes a timeout and returns lock_deadlock to indicate that the lock could not be allocated in time. If this occurs, the transaction must be aborted. A lock request also includes a transaction identifier and a lock mode (e.g., exclusive or shared). Modifications to the Resource The resource manager must be modified to support transactions. The changes required are: 1. Acquire a lock on each element that is accessed. Release all locks on commit. 2. Define a set of UNDO and REDO actions and a corresponding logging scheme for use with recovery. Every operation that modifies a page must be logged and must have an inverse operation. 3. The following functions are added to the interface. bool rm_prepare( trid_t trid, /* Transaction being prepared */

3 Flush log records generated by this transaction. Return TRUE if you vote to commit and FALSE otherwise. status_t rm_commit( trid_t trid, /* Transaction to terminate */ bool decision /* True if transaction was committed */ This is called by the transaction manager to tell each RM the commit decision. The decision argument is TRUE if the transaction was committed, FALSE if aborted. All locks should be released in either case. status_t rm_undo( log_record_t log_record /* Log record to undo */ Undo the effect of operation described in the log record. status_t rm_redo( log_record_t log_record /* Log record to redo */ Redo the effect of the operation described in the log record. status_t rm_recovered( Indicates that recovery is complete and that RM can begin servicing new transactions. Log Protocol The log manager is a server used by all resource managers to make their updates recoverable and also by the transaction manager. The transaction manager (TM) controls the recovery process and thus must also work with the log manager. The protocol is described below: 1. lsn_t log_insert( log_record_t log_record /* Log record to insert */ This function adds a new record to the end of the log. The Log Sequence Record (lsn) of the record is returned. This value must be stored in the lsn field of the frame holding the modified page before the frame is unfixed. The LSN is simply the byte offset of the record within the log file. typedef long int lsn_t; A LSN can be used to easily find records in the log file. The declaration of log records is given below: typedef struct { lsn_t lsn; /* Log sequence number of this record*/ lsn_t next_lsn; /* LSN of next record in log */

4 lsn_t prev_lsn; /* LSN of previous record in log */ lsn_t tran_prev_lsn; /* Prev.log record of same transaction*/ rmid_t rmid; /* Resource manager that wrote record*/ trid_t trid; /* Transaction that wrote record */ trans_status_t status; /* Indicates transaction status to log mgr */ opaque body<>; /* Body of record interpreted by rmid*/ } log_record_t; Only the last four fields must be completed by the client. The log manager determines the other information itself. The field prev_lsn makes backward scans through the log possible. The rmid is the name of the RM which wrote this record. The trid identifies the transaction which caused the log record to be written. The tran_prev_lsn is the LSN of the last record written by this transaction. The body contains a variable length string of bytes that is understood only by the RM rmid. 2. lsn_t log_flush( lsn_t lsn /* Forces all records up to and including lsn */ Flush all log records up to and including the specified LSN to persistent storage. The value returned is the LSN of the last record on persistent storage. This value is always greater or equal to the argument provided. Because log records are generally smaller than one disk page it is important not to force a log record to disk immediately. Instead maintain a one or two page buffer to collect log records. The log record will only be forced when the buffer is full or a flush request is made. 3. log_record_t log_read_lsn( lsn_t lsn, /* LSN of record to read */ Retrieve the log record with the given LSN. 4. lsn_t log_max_lsn( Retrieve the LSN of the last log record in the file. 5. status_t log_shutdown( Clean way to shutdown the log manager. This interface is not visible to clients, thus, you may make changes to this protocol to suit your design. Transaction Protocol The TM controls recovery for all resource managers in the network and coordinating the commitment of distributed transactions. The first three functions are used by clients to delimit transactions. 1. trid_t tm_begin_work(

5 Called when a transaction begins. The transaction identifier assigned to this transaction is returned. 2. status_t tm_abort_work( trid_t trid /* Transaction issuing request */ Called to indicate that the transaction failed. The TM undoes the updates of the transaction, and writes a log record indicating the abort, then informs the RM that it is finished using rm_commit. 3. bool tm_commit_work( trid_t trid, /* Transaction issuing request */ This is the way the client requests its transaction be committed. If the value returned is TRUE, the transaction was committed. If the call returns FALSE, the transaction is aborted. If the call does not return, the log must be consulted. This call triggers the first phase of the 2PC commitment protocol. 4. status_t tm_identify( mid_t rmid /* Resource manager wishing to recover */ This is called by an RM when it wishes to recover. This will trigger the recovery algorithm for the specified RM. 5. status_t tm_join_work( trid_t trid, /* Transaction to vote on */ mid_t rmid /* Resource manager wishing to vote */ This is called by an RM when it first sees a request from a new transaction. This informs the TM that the RM intends to vote in the commitment of this transaction. 6. status_t tm_shutdown( This performs a controlled shutdown of the transaction manager. Remember, this is not the only way the system can be terminated. Requirements You are not allowed to change the interface to the client. You are allowed to add new calls if you would like. You have more freedom to change the RPC interfaces that are not seen by the client. However, you should probably not make major changes without talking to me first. The other interfaces and data structures are simply a suggestion. Your system should adhere to the following: 1. Use strict two-phase locking with read and write locks. A write operation conflicts with a read or write operation of another distinct transaction. Access to the data dictionary must

6 preserve serializability as well. (Remember the updates to the data dictionary when DDL commands are executed.) 2. Implement recovery using write-ahead logging (WAL). 3. Coordinate commitment of distributed transactions using two-phase commit. You may assume that the TM never fails when a transaction is in doubt. Design Documentation The documentation for this stage of the project is similar to that required for the first part. Please incorporate your documentation from the first project into this report so that the report you submit describes the work done in both phases of the project. You should document your design. Describe those features of your system that you feel are unique to your design (i.e., those features that are not determined by the project requirements) and noteworthy. Please incorporate this with the documentation you generated for the first project. Testing Documentation Please run your system with the TPC test cases I will provide. Your report should include output from runs showing multiple clients running concurrently and a run demonstrating recovery. You should do your own testing to further convince yourselves that your system works. Please generally describe how you tested your system. You should discuss the kind of testing you did and then include a few examples of test cases and results. I assume that, if asked, you can repeat any sample test case you include in your document. Commented Listing Also include a commented listing. Templates The directory ~yyzhang/teaching/ece56/phse2/include will contain the protocol files for the log manager, resource manager and transaction manager. The directories ~yyzhang/teaching/ece56/phse2/tm, ~yyzhang/teaching/ece56/phse2/rm, ~yyzhang/teaching/ece56/phse2/lgm will contain a Makefile and sample server file for the transaction manager, resource manager and log manager, respectively.

Log Manager. Introduction

Log Manager. Introduction Introduction Log may be considered as the temporal database the log knows everything. The log contains the complete history of all durable objects of the system (tables, queues, data items). It is possible