Lecture 31 11/16/15. CMPSC431W: Database Management Systems. Instructor: Yu- San Lin

Transcription

1 CMPSC431W: Database Management Systems Lecture 31 11/16/15 Instructor: Yu- San Lin Course Website: hcp:// Slides based on McGraw- Hill & Dr. Wang- Chien Lee 1

2 CHAPTER 8: STORAGE AND INDEXING 2

3 Overview Overview of the storage engine How does a DBMS store and access persistent data? File organizawons and access methods Indexes QuanWtaWve analysis I/O cost Choice of indexes and performance 3

4 Basic AbstracWon of Data Data in a DBMS is a collecwon of in Each file consists of one or many The and organize data to support fast access of records Each file organizawon usually makes certain operawon efficient but other operawons expensive 4

5 Basic AbstracWon of Data (cont.) DB Files Pages Records 5

6 Basic AbstracWon of Data: Example A file of employee records sorted by salary (Q1) Find employees who makes $90K 100K a year (Q2) Find employees who is 25 years old 6

7 External Storage The data in DBMS is stored on external storage devices and fetched into main memory for processing : can retrieve random page at fixed cost Reading several consecuwve pages is much cheaper than reading them in random order Most important external storage devices Unit of a read or write is (typically 4KB or 8KB) 7

8 External Storage (cont.) : can only read pages in sequence Cheaper than disks; used for archival storage Cache Main memory Flash memory MagneWc disk OpWcal disk MagneWc tapes 8

9 Storage Engine File manager The file and access methods layer Makes calls to the buffer manager Provides the nowon of to the query engine Buffer manager Stages from external storage to main memory buffer pool Disk space manager Manages space (in terms of ) on disk 9

10 File OrganizaWons Methods of arranging a file of records on external storage is used to physically locate record A storage engine may support a number of alternawve file organizawons In addiwon to file organizawons, auxiliary data structures, i.e.,, allow us to find the rids of records with given values in index search key fields 10

11 File Manager: Basic OperaWons The file manager stores the records in a file (which may be in alternawve organizawons) of a collecwon of disk pages Basic operawons supported by a file manager : step through all records in a file Retrieve parwcular record based on its : one I/O Retrieve a set of records based on a : equity search, range search : insert, delete 11

12 AlternaWve File OrganizaWons Suitable when typical access if a file scan retrieving all records Best if records must be retrieved in some order, or only a range or records is needed Data structures to organize records 12

13 AlternaWve File OrganizaWons (cont.) Indexes (cont.) Can be combined with files to create Like sorted files, they speed up for a subset of records, based on values in certain (search key) fields are much faster than in sorted files There is a 13

14 Heap File OrganizaWon A record can be placed anywhere in a file where there is a space for the record There is no ordering of the record Typically, a single file for each relawon page 101 John 23 30K 199 Tom 23 28K 123 Lynn 33 50K 106 Ken 29 35K 102 Paul 53 70K 121 Jane 43 43K 14

15 Sorted File OrganizaWon Records are stored sequenwally in a sorted order based on the values of a search key Very expensive to update (have to maintain the sorted order) May use to manage inserwons and delewons John K 30K Paul K 70K Ken Ken K 35K Jane K 43K Lynn K 50K Tom K 28K 15

16 Indexes An index on a file speeds up selecwons on the search key fields for the index Any subset of the fields (or acributes) of a relawon can be the search key for an index on the relawon Search key is NOT the same as key (minimal set of fields that uniquely idenwfy a record in a relawon) 16

17 Indexes (cont.) An index contains a collecwon of A data entry with search key k (denoted as k*) contains enough informawon to locate data records with search key value k Supports efficient retrieval of all data entries k* with a given key value k and then use them to obtain data records Based on the above definiwon, can we use sorted files as an index? How? 17

18 AlternaWves for Data Entry k* in Index Three alternawves 1. An actual data record with key value k 2. <k, rid of a data record with search key k> 3. <k, list of rids of data records with search key k> 18

19 AlternaWves for Data Entry k* in Index (cont.) Choice of alternawves for data entries is to the indexing technique used to locate data entries with a given key value k Examples of indexing techniques: B+ tree, hash- based structures Typically, index contains auxiliary informawon that directs searches to the desired data entries 19

20 AlternaWve 1: Indexed Files The index structure is actually a file organizawon for data records (instead of a Heap file or sorted file) Typically, index on a given collecwon of data records uses AlternaWve 1 Otherwise, data records are duplicated, leading to redundant storage and potenwal inconsistency If data records are very large, containing data entries is high Implying the index size is also large 20

21 AlternaWve 1: Indexed Files (cont.) Data entry = data record Internal index node 21

22 AlternaWve 2 & 3 Data entries typically much smaller than data records BeCer than AlternaWve 1 with large data records, especially if search keys are small PorWon of index structure used to direct search, which depends on size of data entries, is much smaller than with AlternaWve 1 22

23 AlternaWve 2 Data entry = data record Internal index node Data record 23

24 AlternaWve 3 Data entry = data record Internal index node Data record 24

25 AlternaWve 2 & 3 (cont.) AlternaWve 3 more compact than AlternaWve 2, but leads to variable sized data entries even if search keys are of fixed length 25

26 Index ClassificaWon Primary v.s. Secondary: if search key contains primary key, then called primary index Be aware that these terms may be used with a different meaning Unique index: search key contains a candidate key 26

27 Index ClassificaWon (cont.) Clustered v.s. Unclustered: if order of data records is the same as, or close to, order of data entries, then called clustered index AlternaWve implies clustered. In pracwce, clustered also implies AlternaWve 1 (since sorted files are rare) A file can be clustered on search key Cost of retrieving data records through index varies greatly based on whether index is clustered or not 27

28 Don t Forget Homework #5 posted, due on 11/30 Project demo #2 the week of 11/30 28