Outlines Chapter 2 Storage Structure Instructor: Churee Techawut 1) Structure of a DBMS 2) The memory hierarchy 3) Magnetic tapes 4) Magnetic disks 5) RAID 6) Disk space management 7) Buffer management 8) formats 9) Record formats CS (204)321 Database System I Structure of a DBMS (with some simplification) Structure of a DBMS (with some simplification) Query optimizer Uses information about how the data is stored to produce an efficient execution plan for evaluating the query. Execution plan A blueprint for evaluating a query, and is usually represented as a tree of relational operators. File and access methods Includes a variety of S/W for supporting the concept of a file, which is a collection of pages or a collection of records. Typically supports file of unordered pages as well as indexes. Source: Ramakrishnan R. & Gehrke J. (2000) Database Management Systems Structure of a DBMS (with some simplification) The Memory Hierarchy Buffer manager Bring pages in from disk to main memory. Disk space manager Deals with management of space on disk, where the data is stored. Transaction manager, lock manager, and recovery manager Supports concurrency and crash recovery by scheduling user requests and maintaining log of all changes to the database. Slower devices Less expensive Request for data Larger storage CPU CACHE MAIN MEMORY MAGNETIC DISK TAPE Data satisfying request Primary storage Secondary storage Tertiary storage 1 of 7
Characteristics of magnetic tapes: Magnetic Tapes The structure of a disk in simplified form. Inexpensive Store very large amounts of data. Good choice for archival storage used for maintaining data for a long period but do not expect to access it very often Support sequential access - stepping through all the data in order and cannot directly access a given location on tape. Block Unsuitable for storing operational data, but mostly used to back up operational data periodically. Source: Silberschatz A., Korth H., Sudarchan S. (2002) Database System Concepts Magnetic disks support direct access to a desired location and are widely used for database applications. Data is store on disk in units called disk blocks. Blocks are arranged in concentric rings called tracks, on one or more platters (single-sided or double sided). Each track is divided into arcs called sectors. The set of all tracks with the same diameter is called a cylinder. An array of disk heads, is moved as a unit. When one head is positioned over a block, the other heads are in identical position with respect to their platters. A disk controller manages the movement of the arm assembly and transferring of data to and from the disk surfaces. The time to access a disk block has several components: Access time = Seek time + Rotational delay + Transfer time Seek time The time taken to move the disk heads to the track on which a desired block is located. Rotational delay The waiting time for the desired block to rotate under the disk head. (Usually less than seek time) Transfer time The time for the disk to rotate over the block. Example: The seek time is 60 milliseconds (msec), rotation time is 12.5 msec., and transfer rate is 250 characters per msec. Find the access time if the length of the data, which is going to read, is 1000 characters. Access time = Seek time + Rotational delay + Transfer time = 60 + 12.5 + (1000/250) = 60 + 12.5 + 4 = 76.5 msec. Performance implications of disk structure: 1. Data must be in memory for the DBMS to operate on it. 2. The unit for data transfer between disk and main memory is a block; if a single item on a block is needed, the entire block is transferred. Reading or writing a disk clock is called an I/O (for input/output) operation. 3. The time to read or write a block varies, depending on the location of the data 2 of 7
RAID Note: Time taken for database operations is affected significantly by how data is stored on disks. Sequential access minimizes seek time and rotational delay and is much faster than random access. The Redundant Arrays of Independent Disks (RAID) is disk organization techniques that manage a large numbers of disks, providing a view of a single disk of high capacity and high speed by using multiple disks in parallel, and high reliability by storing data redundantly, so that data can be recovered even if a disk fails. e.g. mirroring Two identical copies of the data on two different disks are maintained. RAID Data stripping The data is segmented into equal-size partitions distributed over multiple disks. Redundancy Disks have a higher failure probability early and late in their lifetimes. Mean-time-to-failure (MTTF) is a failure probability during the actual usage time of the storage system. MTTF affects the reliability of a disk. If a disk fails, the redundant information is used to reconstruct the data on the failed disk. Redundancy can immensely increase the MTTF of a disk array. The role of disk space manager: manages space on disk supports the concept of a page as a unit of data provides commands to allocate or deallocate a page, and read or write page. The size of a page is chosen to be the size of a disk block s are stored as disk blocks. Reading or writing a page can be done in one disk I/O. Useful capability: Allocate a sequence of pages as a contiguous sequence of blocks to hold data. This capability is essential for exploiting the advantages of sequential accessing disk blocks. The disk space manager hides details of the underlying hardware and allows higher levels of the s/w to think of the data as a collection of pages. Managing space on disk: Keeping track of free blocks A database grows and shrinks as records are inserted and deleted over time. It is likely that blocks are initially allocated sequentially on disk, subsequent allocations and deallocations could in general create holes. 3 of 7
Keeping track of free blocks (continued) Two ways to keep track of block usage is to maintain: (1) A list of free blocks - Deallocating blocks are added to the free list for future use. (2) Bitmap - Indicating whether a block is in use or not - One bit for each disk block - Allowing fast identification and allocation of contiguous areas on disk Using OS file systems to manage disk space Operating system supports the abstraction of a file as a sequence of bytes. OS translates requests Read byte i of file f into Read block m of track t of cylinder c of disk d. The entire database could reside in one or more OS files for which a number of blocks are allocated (by the OS) and initialized. The disk space manager is then responsible for managing the space in these OS files. Using OS file systems to manage disk space (continued) Note: Using OS file is not popular for disk space management because different OS platforms has different disk space management and typical OS files cannot span disk devices. DBMS requires portability and may want to access a large single file. The role of buffer manager: Brings pages from disk to main memory as needed. Uses replacement policy to decide which page to replace. Manages the available main memory by partitioning it into a collection of pages or buffer pool. The main memory pages in the buffer pool are called frames. Disk page Free frame If a requested page is not in the pool and the pool is full, the buffer manager s replacement policy controls which existing page is replaced. requests from higher-level code DB MAIN MEMORY DISK pin_count and dirty variables: pin_count dirty pin_count = the no. of times that the page currently in a given frame has been requested but not released Incrementing pin_count is called pinning the page. Decrementing pin_count is called unpinning the page. dirty = the boolean variable indicating whether the page has been modified since it was brought into the buffer pool from disk. request & release 4 of 7
The buffer replacement policies: The replacement can affect the time taken for database operations. Different policies are suitable in different situations. Least recently used (LRU) A queue of pointers 1 st queue Clock replacement Similar behavior as LRU, but less overhead. Clock s face current variable = clock hand N 1 N= No. of buffer frame 2 3 4 5 pin_count = 0 end of queue A page that becomes a candidate for replacement Current frame is considered for replacement. If the frame is not chosen for replacement or pin_count >0, current is incremented and the next frame is considered. Other replacement policies include: first in first out (FIFO) Higher levels of the DBMS see data as a collection of records. We consider how a collection of records can be arranged on a page. most recently used (MRU) Random record 2 A record is identified by record id or rid (page id, slot number). Alternative approaches to managing slots on a page: Record slots are uniform in the same length and can be arranged within a page. Some slots are occupied or unoccupied by records. How we keep track of empty slots and how we locate all records on a page? 5 of 7
Two alternative hinges: First alternative: Store records in the first N slots (where N is the number of records on the page). record 2 Slot 3 record 3 Whenever a record is deleted,. record 2 Slot 3 record 3.. we move the last record on the page into the vacated slot. Slot 3 record 3 Second alternative: Use an array of bits to handle deletions, one bit per slot. Slot 3 Slot j header 1 0 1 j j 2 1 Number of slots To locate records, the page requires scanning the bit array to find slots whose bit is on. When a record is deleted, its bit is turned off. The page cannot be divided into a fixed collection of slots. DATA AREA rid=(i,n) rid=(i,1) PAGE i Problem is that when a new record is to be inserted, we have to find an empty slot of just the right length. rid=(i,2) length=24 Important rules: 1) To insert a record, we must allocate just the right amount of space for it. 2) To delete a record, we must move records to fill the hole created by the deletion to ensure the contiguousness of all free space. The most flexible organization for variable-length records is to maintain a directory of slots (maintain pointers). Pointer to start of free space FREE SPACE SLOT DIRECTORY No. of entries in slot directories N 2 1 20 16 24 N 6 of 7
How to organize fields within record. We must consider whether the fields of the record are of fixed or variable length. No. of fields and field types is stored in the system catalog. Alternative approaches to organization of records: Each field has a fixed length (that is the value in this field is of the same length in all records), and the number of fields is also fixed. The fields of such a record can be stored consecutively. The address of a particular field can be calculated using information about the lengths of preceding fields, which is available in the system catalog. Organization of records with fixed-length fields F1 F2 F3 F4 L1 L2 L3 L4 Base address (B) Address = B+L1+L2 Fi = field i Li = length of field i This organization stores fields consecutively, separated by delimiters. It requires a scan of the record in order to locate a desired field. At the beginning of a record, some space is reserved for use as an array of integer offsets the i th integer in this array is the starting address of the i th field value relative to the start of the record. Organization for variable-length fields F1 $ F2 $ F3 $ F4 Fields delimited by special symbol $ Fi = field i F1 F2 F3 F4 Array of field offsets 7 of 7