Principles of Data Management. Lecture #3 (Managing Files of Records)

Transcription

1 Principles of Management Lecture #3 (Managing Files of Records) Instructor: Mike Carey base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today s Topics v Today should fill in your Project 1 blanks! v Remember to read the course wiki page v And also follow the Piazza page home Hopefully everyone has been benefitting from the Q&A traffic there (and seeing the response time benefits of asking Q s there instead of via ) base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 2

2 REVIEW: Disk Space & I/O v Lowest layer of DBMS software manages the space on disk and disk-level I/O calls v Higher levels call upon this layer to: allocate/de-allocate a page read/write a page v A request for a sequence of pages must be satisfied by allocating the pages sequentially on disk! (Note: We are handing this job over to the OS filesystem for Project 1 in this class!) base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 3 REVIEW: Buffer Management Requests from Higher Levels disk page free frame BUFFER POOL Note: Project 1 s dfilemanager class would do the buffering inside if we were doing it! MAIN MEMORY DISK DB choice of frame dictated by replacement policy v must be in RAM for DBMS to operate on it! v Table of <frame#, pageid> pairs is maintained. base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 4

3 Files of Records v or block is OK when doing I/O, but higher levels of DBMS operate on records, and thus want files of records. v FILE: A collection of pages, each containing a collection of records. Must support: Insert (append)/delete/modify record read a particular record (specified using record id) scan all records (possibly with some conditions on the records to be retrieved) base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 5 Unordered ( Heap ) Files v Simplest file structure that contains records in no particular (logical) order. v As file grows and shrinks, disk pages are allocated and de-allocated. v To support record level operations, we must: keep track of the pages in a file keep track of free space on pages keep track of the records on a page keep track of fields within records v There are many alternatives for each. base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 6

4 Record Formats: Fixed Length F1 F2 F3 F4 L1 L2 L3 L4 Base address (B) of record Address of F3 = B+L1+L2 v Information about field types is the same for all records in file; it is stored in the system catalogs. (Note: Record field info in Project 1 passed in from above!) v Finding the i th field of a record does not require scanning the record. base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 7 Record Formats: Variable Length v Several alternative formats (# fields is fixed): F1 F2 F3 F4 $ $ $ $ Fields Delimited by Special Symbols F1 F2 F3 F4 L1 L2 L3 L4 Fields Preceded by Field Lengths Some thought questions for you: (1) What s true of the second format but not the first? (2) What annoying disadvantage does both formats share? (3) And, how do we know the field count in each case? base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 8

5 Record Formats: Variable Length (continued) v Variable-length fields with a directory: 4 F1 F2 F3 F4 Array of Field Offsets (a.k.a. Directory) This format: (1) Offers direct access to the i'th field. (2) Can support efficient storage of null values. (Q: How?) (3) Just requires a small directory overhead. (4) Can even help with ALTER TABLE ADD COLUMN! (Q: How?) base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 9 Heap File Implemented as a List Full s Header s with Free Space v The header page id and Heap file name must be stored someplace. (Project 1 note: The OS filesystem can help! J ) v Each page contains two extra pointers in this case. v Refinement: Use several lists for different degrees of free space (to mention just one of many possibilities). base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 10

6 Heap File Using a Directory Header 1 2 DIRECTORY N v entries can include the number of free bytes on each page. (Q: Why would a rough number be better?) v Directory is a collection of pages; linked list just one possible implementation. (Note: Can also do extents!) Directory much smaller than linked list of all HF pages base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 11 Formats: Fixed Length Records Slot 1 Slot 2 Slot N Free Space Slot 1 Slot Slot N PACKED N number of records Slot M M Record id = <page id, slot #>. In the first (packed) alternative, records will move around for free space management: Rids change may be unacceptable! base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 12 1 M UNPACKED, BITMAP number of slots

7 Formats: Variable Length Records Rid = (i,n) Rid = (i,2) i Rid = (i,1) (Free space) N N #slots SLOT DIRECTORY (offset, length) Pointer to start of free space Can move records within page w/o changing rids; so, also not unattractive for fixed-length records. base Management Systems 3ed, R. Ramakrishnan and J. Gehrke System Catalogs ( Up Above ) v For each relation: name, file name, file structure (e.g., Heap) name, type, and length (if fixed) for each attribute index name, target, and kind for each index also integrity constraints, defaults, nullability, etc. v For each index: structure (e.g., B+ tree) and search key fields v For each view: view name and definition (including query) v Plus statistics, authorization, buffer pool size, etc. Catalogs themselves stored as record-based files too! base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 14

8 Attr_Cat(attr_name, rel_name, type, position) attr_name rel_name type position attr_name Attr_Cat string 1 rel_name Attr_Cat string 2 type Attr_Cat string 3 position Attr_Cat integer 4 sid Students string 1 name Students string 2 login Students string 3 age Students integer 4 gpa Students real 5 fid Faculty string 1 fname Faculty string 2 sal Faculty real base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 15 3 Files of Records: Basic Summary v Disks provide cheap, non-volatile storage. Random access, but cost depends on location of page on disk; important to arrange data sequentially to minimize seek and rotation delays. v Buffer manager brings pages into RAM. stays in RAM (at least!) until unpinned by last among concurrent requestors. Written to disk when frame chosen for replacement (some time after dirtying requestor unpins the page). Choice of frame to replace based on replacement policy. Could be worth prefetching several pages at a time. base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 16

9 Summary (Contd.) v DBMS vs. OS File Support DBMS needs features not found in many OS s, such as forcing a page to disk, controlling the order of the page writes to disk, letting files span disks, controlling prefetching and page replacement policies based on (predictable) DB access patterns, etc. v Variable length record format with field offset directory offers support for direct access to the i'th field and also supports null values. v Slotted page format supports variable length records and allows records to move in a page. base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 17 Summary (Contd.) v File layer keeps track of pages in a file and supports abstraction of a collection of records. s with free space identified using linked list or directory structure (similar to how pages in file itself are tracked; may be integrated with that). v Indexes support efficient retrieval of records by mapping from values in fields to rids. v Catalog relations store information about relations, indexes and views. (Information common to all records in a given collection.) base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 18