Database Systems II. Record Organization

Size: px

Start display at page:

Download "Database Systems II. Record Organization"

Amber Harvey
5 years ago
Views:

1 Database Systems II Record Organization CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 75 Introduction We have introduced secondary storage devices, in particular disks. Disks use blocks as basic units of transfer and storage. In a DBS, we have to manage entities, typically represented as records with attributes (relational model). Attribute values (data items) can be complex: texts, images, videos. How to organize records on disk? CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 76

2 Introduction Data Items Records Blocks Files Memory CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 77 Data Items Short Integer, unsigned (16 Bits) e.g., 35 is Short Integer, signed e.g., -35 is CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 78

3 Data Items Floating point (32 Bits, single precision) 1 bit for sign, m for exponent, n bits for mantissa (binary) = 1.25 (decimal) = value = (-1) sign x 2 exponent bias x 1.fraction bias = 2 m-1-1 = 127 CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 79 Data Items Characters various coding schemes suggested most popular is ASCII 1 Character = 1 Byte = 8 Bits examples A: a: : LF: CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 80

4 CHAR VARCHAR Data Items String of characters NULL-terminated e.g., c a t m o u s e length indicator e.g., 3 c a t 4 m o u s e fixed length do not need terminating NULL or length indicator CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 81 Records Fixed format e.g., relational data model record is list of data items number and type of data items fixed vs. variable format e.g., XML number of data items variable Fixed length vs. variable length e.g., VARCHAR, blobs, repeating fields CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 82

5 Records Record header keeps general information about the record. Header contains (some of) the following: - pointer to schema, - record types, - record length (to skip record without consulting schema), - timestamp of last access, - pointers (offsets) to record attributes. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 83 Fixed-Length Records Fixed length records are the simplest format. Header contains only pointer to schema, record length and timestamp. Example (1) id, 2 byte integer (2) name, 10 char. Schema (3) dept, 2 byte code 55 s m i t h j o n e s 01 Records CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 84

6 Variable-Length Records Variable length records first store fixed length fields, followed by variable-length fields. Header contains also pointers (offsets) to variable-length fields (except first one). NULL values can be efficently represented by null pointers in the header. header birthdate name address fixed variable CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 85 Variable Format Records Variable format records are self-describing. Simplest representation as sequence of tagged fields. Record header contains number of fields. Tag contains - attribute name, - attribute type, - field length (if not apparent from attribute type). CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 86

7 # Fields Code identifying field as E# Integer type Value Code for EName String type Length of str. Value Variable Format Records Example 2 5 I 46 4 S 4 F O R D CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 87 Packing Records Into Blocks Introduction How to pack records into blocks? Need to address the following issues - separating records, - spanned vs. unspanned, - mixing record types, - ordering records, - addressing records, - BLOBs. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 88

8 Packing Records Into Blocks Separating records for fixed-length records, no need to separate for variable-length records use special marker or store record lengths (or offsets) - within each record - in block header CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 89 Packing Records Into Blocks Spanned vs. unspanned Unspanned records on a single block R1 R2 R3 R4 R5 block 1 block 2 Spanned records split into fragments that are distributed over multiple blocks R1 R2 R3 (a) R3 (b) R4 R5 R6 R7 (a) block 1 block 2 CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 90

9 Packing Records Into Blocks Spanned vs. unspanned Spanning necessary if records do not fit in a block, e.g. if they have very long fields (VARCHAR, blob). Spanning useful for better space utilization, even if records fit in a block. R1 R bytes wasted bytes wasted 2046 CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 91 Packing Records Into Blocks Spanned vs. unspanned Spanned records requires extra header information in records and record fragments: - is it fragment? (bit) - if fragment, is it first or last? (bit) - if applicable, pointers to previous/next fragment. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 92

10 Packing Records Into Blocks Mixing record types Can we mix records of different types (relations)in same block? Why should we? Records that are frequently accessed together should be in the same block (clustering). EMP e1 DEPT d1 DEPT d2 Compromise: no mixing on same block, but keep related records in same cylinder. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 93 Packing Records Into Blocks Ordering records Want to efficiently read records in order Ordering records in file and block by some key value (sequential file) Implementation options - next record physically contiguous R1 Next (R1) - link to next record R1 Next (R1) CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 94

11 Packing Records Into Blocks Overflow area records in sequence Ordering records header R1 R2 R3 R4 R5 R2.1 R1.3 R4.7 links more flexible under modifications, but require random I/O CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 95 Packing Records Into Blocks Addressing records In memory records have virtual memory address. How to address a record on disk? Trade-off between efficiency of dereferencing and flexibility in moving records over time Many options, from purely physical to fully indirect (logical) CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 96

12 Packing Records Into Blocks Addressing records Purely physical direct address on disk Record address (record ID) consists of cylinder # track # block # offset in block efficient access, but cannot move record within block or across blocks CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 97 Packing Records Into Blocks Addressing records Fully indirect (logical) address (record ID) can be any bit string, needs to be mapped to physical address map Rec ID Physical addr. Typically 8 Bytes Often 8-16 Bytes! inefficient access, but can move record anywhere CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 98

13 Packing Records Into Blocks Indirection in block Header Addressing records offsets Free space R3 R4 R1 R2 compromise between physical and fully indirect CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 99 Packing Records Into Blocks BLOBs Sometimes, very large data items, e.g. images, videos,... Store them as Binary Large Objects (BLOBs) Stored as spanned sequence of blocks - preferably contiguous on disk, - possibly even striped over several disks. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 100

14 Record Modifications Insertions If record order does not matter, just append record to file, i.e. find a block of that file with empty space. If order does matter, may be more complicated. First locate corresponding block. If block has enough space left, insert record there and rearrange records in block. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 101 Record Modifications Insertions If block does not have enough space left, use one of the two following strategies: - find space in nearby block predecessor or successor in sort order - create overflow block and link it into chain of blocks only if order implemented by links CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 102

15 Record Modifications Insertions Overflow blocks raise the following issues: - How much free space to leave in each block, track, cylinder? - How often to reorganize file and overflow area? Free space CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 103 Record Modifications Try to reclaim space. Deletions Either move around records within a block or maintain available-space list. Must avoid dangling pointers to deleted records. Typical approach: place tombstone in place of deleted record. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 104

16 Record Modifications Deletions physical record IDs This space never re-used This space can be re-used CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 105 Record Modifications Deletions logical record IDs map ID LOC 7788 Never reuse ID 7788 nor space in map... CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 106

17 Buffer Management Introduction The part of the DB relevant for the current transactions needs to reside in memory, i.e. in the DB buffer. BUFFER POOL disk block free frame MAIN MEMORY DISK DB choice of frame dictated by replacement policy CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 107 Buffer Management Block request If requested block is not in buffer pool, choose a frame for replacement. If this frame is dirty, write it to disk. Read requested page into chosen frame. Pin the page and return its address. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 108

18 Buffer Management Block replacement Requestor of page must unpin it, as soon as block is no longer needed. Requestor must also indicate whether block has been modified: dirty bit is used for this. Block in pool may be requested many times, and a pin count is used. Block is candidate for replacement if and only if pin count = 0. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 109 Buffer Management Replacement policy Frame is chosen for replacement by a replacement policy: - Least-recently-used (LRU), - Clock, efficient approximation of LRU - Most-recently-used (MRU) etc. Policy can have big impact on number of I/O s, depends on the access pattern. Will discuss more in the context of query optimization. CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 110

Data Storage and Query Answering. Data Storage and Disk Structure (4)

Data Storage and Query Answering Data Storage and Disk Structure (4) Introduction We have introduced secondary storage devices, in particular disks. Disks use blocks as basic units of transfer and storage.