Hash-Based Indexes. Chapter 11. Comp 521 Files and Databases Fall

Transcription

1 Has-Based Indexes Capter 11 Comp 521 Files and Databases Fall

2 Introduction Hasing maps a searc key directly to te pid of te containing page/page-overflow cain Doesn t require intermediate page fetces for internal steering nodes of tree-based indices Has-based indexes are best for equality selections. Tey do not support efficient range searces. Static and dynamic asing tecniques exist wit trade-offs similar to ISAM vs. B+ trees. Comp 521 Files and Databases Fall

3 Static Hasing # primary pages fixed, allocated sequentially, never de-allocated; overflow pages if needed. (k) mod M = bucket to wic data entry wit key k belongs. (M = # of buckets) (key) mod N key 0 2 N-1 Primary bucket pages Overflow pages Comp 521 Files and Databases Fall

4 Static Hasing (Contd.) Buckets contain data entries. Has function maps a searc key to a bin number (key) 0 M-1. Ideally uniformly. (key) = (a * key + b) mod M, usually works well. a and b are constants; lots known about ow to tune. Long overflow cains can develop and degrade performance. Dynamic tecniques (Extendible and Linear Hasing) address tis problem. Comp 521 Files and Databases Fall

5 Static Hasing Example Initially built over Ages attribute, wit 4 records/page and (Age) = Age mod * 24* 32* 1 41* 25* 29* 45* 2 26* 62* 18* 34* 3 31* Initial Index Note: records need not be ordered Average Occupancy? Comp 521 Files and Databases Fall

6 Static Hasing Example Adding 28, 33 Deleting 31, (leads to empty page) 0 20* 24* 32* 28* 1 41* 25* 29* 45* 2 26* 62* 18* 34* 3 31* 33* Comp 521 Files and Databases Fall

7 Hasing s Acilles Heel Maintaining Balance Data is often clustered Has function sould uniformly distribute keys over buckets. Demands a good as function (lots of researc in tis area) Bucket Spills Wat if M buckets are not enoug? Solution: new as function Families of as functions 0 (key), 1 (key), n (key) Transitions only redistribute overflowed buckets Comp 521 Files and Databases Fall

8 Extendible Hasing Situation: Bucket (primary page) becomes full. Cange asing function and reorganize. Reorganizes index by doubling # of buckets Reading and writing all pages is expensive! Key Idea: Use directory of pointers to buckets, double # of buckets by doubling te directory, splitting just te bucket tat overflowed! Directory muc smaller tan file, so doubling it is muc ceaper. Only spilt pages are split. No overflows! Trick lies in ow as function is adjusted! Comp 521 Files and Databases Fall

9 Example Directory is array of size 4. To find bucket for r, take last `global dept # bits of (r); we denote r by (r). LOCAL DEPTH GLOBAL DEPTH 1*! 5*! 21*! 13*! If (r) = 5 = binary 101, 01! it is in bucket pointed to 10! by ! 10*! Insert: If bucket is full, split it DIRECTORY! (allocate new page, re-distribute). If necessary, double te directory. 15*! 7*! 19*! (Decision is based on comparing te directory s global dept wit local dept of te bucket.) 00! 4*! 12*! 32*!16*! DATA! Bucket A! Bucket B! Bucket C! Bucket D! Comp 521 Files and Databases Fall

10 Insert (r)=20 (Causes Doubling) LOCAL DEPTH GLOBAL DEPTH 4*!12*! 32*!16*! Bucket A! LOCAL DEPTH GLOBAL DEPTH 3! 32*! 16*! Bucket A! 00! 01! 10! 11! DIRECTORY! 1*! 5*! 21*!13*! 10*! 15*!7*! 19*! 20*! Bucket B! Bucket C! Bucket D! Bucket A (`split image'! of Bucket A)! Comp 521 Files and Databases Fall ! 001! 010! 011! 100! 101! 110! 111! 3! DIRECTORY! 1*! 5*! 21*!13*! 10*! 15*!7*! 3! 19*! 4*! 12*! 20*! Bucket B! Bucket C! Bucket D! Bucket A (`split image'! of Bucket A)!

11 Points to Note 20 = binary Last 2 bits (00) tell us r belongs in A or A2. Last 3 bits needed to tell wic. Global dept of directory: Max # of bits needed to tell wic bucket an entry belongs to. Local dept of a bucket: # of bits used to determine if an entry belongs to tis bucket. Wen does bucket split cause directory doubling? Before insert, local dept of bucket = global dept. Insert causes local dept to become > global dept; directory is doubled by copying it over and fixing pointer to split image page. (Use of least significant bits enables efficient doubling via copying of directory!) Comp 521 Files and Databases Fall

12 Directory Doubling Wy use least significant bits in directory? Allows for doubling via copying! 6 = ! 3! 6 = ! 3! 0! 1! 1! 6* 00! 01! 10! 11! 6* 001! 010! 011! 100! 101! 110! 6* 0! 1! 1! 6* 00! 10! 01! 11! 6* 100! 010! 110! 001! 101! 011! 6* 111! 111! Least Significant vs. Most Significant Comp 521 Files and Databases Fall

13 Comments on Extendible Hasing If directory fits in memory, equality searc answered wit one disk access; else two. 100MB file, 100 bytes/rec, 4K pages contains 1,000,000 records (as data entries) and 25,000 directory elements; cances are ig tat directory will fit in memory. Directory grows in spurts, and, if te distribution of as values is skewed, directory can grow large. Multiple entries wit same as value cause problems! Delete: If removal of data entry makes bucket empty, it can be merged wit its split image. If eac directory element points to same bucket as its split image, can alve directory. Comp 521 Files and Databases Fall

14 Linear Hasing Tis is anoter dynamic asing sceme, an alternative to Extendible Hasing. LH avoids te need for a directory, yet andles te problem of long overflow cains. Idea: Use a family of as functions 0, 1, 2,... i (key) = (key) mod(2 i N); N = initial # buckets is some as function (range is not 0 to N-1) If N = 2 d0, for some d0, i consists of applying and looking at te last di bits, were di = d0 + i. i+1 doubles te range of i (similar to directory doubling) Comp 521 Files and Databases Fall

15 Linear Hasing (Contd.) Directory avoided in LH by allowing overflow pages, and always splitting te next bucket (in a round-robin fasion). Splitting proceeds in `rounds. Round ends wen all N R initial (for round R) buckets are split. Buckets 0 to Next-1 ave been split; Next to N R yet to be split. Current round number is Level. Searc: To find bucket for data entry r, find Level (r): If Level (r) in range Next to N R, r belongs ere. Else, r could belong to bucket Level (r) or bucket Level (r) + N R ; must apply Level+1 (r) to find out. Comp 521 Files and Databases Fall

16 Overview of LH File In te middle of a round. Bucket to be split Next Buckets tat existed at te beginning of tis round: tis is te range of Level Buckets split in tis round: If Level (searc key value) is in tis range, must use Level-1 (searc key value) to decide if entry is in `split image' bucket. `split image' buckets: created (troug splitting of oter buckets) in tis round Comp 521 Files and Databases Fall

17 Linear Hasing (Contd.) Insert: Find bucket by applying Level / Level+1 : If bucket to insert into is full: Add overflow page and insert data entry. Split Next bucket and any associated overflow pages and increment Next. Te bucket tat is split may not be te same as te one tat overflowed! Can coose alternate criterions to trigger split Next must be updated sequentially. Since buckets are split round-robin, long overflow cains don t develop! Doubling of directory in Extendible Hasing is similar; switcing of as functions is implicit in ow te # of bits examined is increased Comp 521 Files and Databases Fall

18 Example of Linear Hasing On split, Level+1 is used to redistribute entries. If bucket is full, Spill, Split Next, Move Next Level=0, N=4 Insert 43 Level=0 1 0 PRIMARY Next=0 1 0 PRIMARY OVERFLOW * 44* 36* 9* 25* 5* Data entry r wit (r)= * Next=1 9* 25* 5* * 18* 10* 30* 31* 35* 7* 11* Primary bucket page * 18* 10* 30* 31* 35* 7* 11* 43* (Tis info is for illustration only!) (Te actual contents of te linear ased file) * 36* Comp 521 Files and Databases Fall

19 Insert 37 ( ) References page Next, ceck 0 page, fits, no action Level=0 1 0 PRIMARY OVERFLOW * Next= * 25* 5* 37* * 18* 10* 30* * 35* 7* 11* 43* * 36* Comp 521 Files and Databases Fall

20 Insert 29 ( ) References page Next, ceck 0 page Spill, split, move Next Level=0 1 0 Level=0 PRIMARY OVERFLOW * PRIMARY OVERFLOW * Next=1 9* 25* 5* 37* * Next=2 25* 14* 18* 10* 30* * 18* 10* 30* * 35* 7* 11* 43* * 35* 7* 11* 44* 36* 43* * 36* 101 5* 37* 29* If we ad inserted 28 instead, ten page < Next, so we d need to consider 1 to determine te correct bucket. Comp 521 Files and Databases Fall

21 Insert 22 ( ) 1 References page Next, ceck 0 page spill, split, move Next 0 Level=0 PRIMARY OVERFLOW Level=0 32* PRIMARY OVERFLOW * * 25* * Next=2 25* 14* 18* 10* 30* * 10* Next=3 31* 35* 7* 11* 43* * 35* 7* 11* 43* * 36* * 36* 101 5* 37* 29* 101 5* 37* 29* * 30* 22* Comp 521 Files and Databases Fall

22 Add 51 ( ): End of a Round * 9* PRIMARY 25* 18* 10* Next=3 31* 35* 7* 11* OVERFLOW 43* PRIMARY Next=0 32* 9* 25* 18* 10* 35* 11* 51* 44* 36* OVERFLOW * 36* * 37* 29* 101 5* 37* 29* * 30* 22* * 30* 22* * 31* Comp 521 Files and Databases Fall

23 LH Described as a Variant of EH Te two scemes are actually quite similar: Begin wit an EH index were directory as N elements. Use overflow pages, split buckets round-robin. First split is at bucket 0. (Imagine directory being doubled at tis point.) But elements <1,N+1>, <2,N+2>,... are te same. So, need only create directory element N, wic differs from 0, now. Wen bucket 1 splits, create directory element N+1, etc. So, directory can double gradually. Also, primary bucket pages are created in order. If tey are allocated in sequence too (so tat finding i t is easy), we actually don t need a directory! Voila, LH. Comp 521 Files and Databases Fall

24 Summary Has-based indexes: best for equality searces, cannot support range searces. Static Hasing can lead to long overflow cains. Extendible Hasing avoids overflow pages by splitting a full bucket wen a new data entry is to be added to it. (Duplicates may require overflow pages.) Directory to keep track of buckets, doubles periodically. Can get large wit skewed data; additional I/O if tis does not fit in main memory. Comp 521 Files and Databases Fall

25 Summary (Contd.) Linear Hasing avoids a directory by splitting buckets round-robin, and using overflow pages. Overflow pages not likely to be long, nor around for long. Duplicates andled easily. Space utilization could be lower tan Extendible Hasing, since splits not concentrated on `dense data areas. Can tune criterion for triggering splits to trade-off sligtly longer cains for better space utilization. For as-based indexes, a skewed data distribution is one in wic te as values of data entries are not uniformly distributed! Comp 521 Files and Databases Fall