CS 5614: (Big) Data Management Systems. B. Aditya Prakash Lecture #4: Constraints, Storing and Indexes

Transcription

1 CS 5614: (Big) Data Management Systems B. Aditya Prakash Lecture #4: Constraints, Storing and Indexes

2 Constraints in Rela;onal Algebra and SQL

3 Maintaining Integrity of Data Data is dirty. How does an applica>on ensure that a database modifica>on does not corrupt the tables? Two approaches: Applica>on programs check that database modifica>ons are consistent. Use the features provided by SQL. 3

4 Maintaining Integrity of Data Data is dirty. How does an applica>on ensure that a database modifica>on does not corrupt the tables? Two approaches: Applica>on programs check that database modifica>ons are consistent. Use the features provided by SQL. 4

5 Integrity Checking in SQL PRIMARY KEY and UNIQUE constraints. FOREIGN KEY constraints. Constraints on a\ributes and tuples. Triggers (schema- level constraints). How do we express these constraints? How do we check these constraints? What do we do when a constraint is violated? 5

6 Keys in SQL A set of a\ributes S is a key for a rela>on R if every pair of tuples in R disagree on at least one a\ribute in S. Select one key to be the PRIMARY KEY; declare other keys using UNIQUE. 6

7 Primary Keys in SQL Modify the schema of Students to declare PID to be the key. CREATE TABLE Students( PID VARCHAR(8) PRIMARY KEY, Name CHAR(20), Address VARCHAR(255)); What about Courses, which has two a\ributes in its key? CREATE TABLE Courses(Number integer, DeptName: VARCHAR(8), CourseName VARCHAR(255), Classroom VARCHAR(30), Enrollment integer, PRIMARY KEY (Number, DeptName) ); 7

8 Effect of Declaring PRIMARY KEYs Two tuples in a rela>on cannot agree on all the a\ributes in the key. DBMS will reject any ac>on that inserts or updates a tuple in viola>on of this rule. A tuple cannot have a NULL value in a key a\ribute. 8

9 Other Keys in SQL If a rela>on has other keys, declare them using the UNIQUE keyword. Use UNIQUE in exactly the same places as PRIMARY KEY. There are two differences between PRIMARY KEY and UNIQUE: A table may have only one PRIMARY KEY but more than one set of a\ributes declared UNIQUE. A tuple may have NULL values in UNIQUE a\ributes. 9

10 Enforcing Key Constraints Upon which ac>ons should an RDBMS enforce a key constraint? Only tuple update and inser>on. RDMBS searches the tuples in the table to find if any tuple exists that agrees with the new tuple on all a\ributes in the primary key. To speed this process, an RDBMS automa>cally creates an efficient search index on the primary key. User can instruct the RDBMS to create an index on one or more a\ributes 10

11 Foreign Key Constraints Referen>al integrity constraint: in the rela>on Teach (that connects Courses and Professors), if Teach relates a course to a professor, then a tuple corresponding to the professor must exist in Professors. How do we express such constraints in Rela>onal Algebra? Consider the Teach(ProfessorPID, Number, DeptName) rela>on. every non- NULL value of ProfessorPID inteach must be a valid ProfessorPID in Professors. RA πprofessorpid(teach) π PID(Professors). 11

12 Foreign Key Constraints in SQL every non- NULL value of ProfessorPID inteach must be a valid ProfessorPID in Professors. In Teach, declare ProfessorPID to be a foreign key. CREATE TABLE Teach(ProfessorPID VARCHAR(8) REFERENCES Professor(PID), Name VARCHAR(30)...); CREATE TABLE Teach(ProfessorPID VARCHAR(8), Name VARCHAR(30)..., FOREIGN KEY ProfessorPID REFERENCES Professor(PID)); If the foreign key has mul>ple a\ributes, use the second type of declara>on. 12

13 Requirements for FOREIGN KEYs If a rela>on R declares that some of its a\ributes refer to foreign keys in another rela>on S, then these a\ributes must be declared UNIQUE or PRIMARY KEY in S. Values of the foreign key in R must appear in the referenced a\ributes of some tuple in S. 13

14 Enforcing Referen;al Integrity Three policies for maintaining referen>al integrity. Default policy: reject viola>ng modifica>ons. Cascade policy: mimic changes to the referenced a\ributes at the foreign key. Set- NULL policy: set appropriate a\ributes to NULL. 14

15 Specifying Referen;al Integrity Policies in SQL SQL allows the database designer to specify the policy for deletes and updates independently. Op>onally follow the declara>on of the foreign key with ON DELETE and/or ON UPDATE followed by the policy: SET NULL or CASCADE. Constraints can be circular, e.g., if there is a one- one mapping between two rela>ons. In this case, SQL allows us to defer the checking of constraints. 15

16 Constraining ATributes and Tuples SQL also allows us to specify constraints on a\ributes in a rela>on and on tuples in a rela>on. Disallow courses with a maximum enrollment greater than 100. A chairperson of a department must teach at most one course every semester. How do we express such constraints in SQL? How can we change our minds about constraints? A simple constraint: NOT NULL Declare an a\ribute to be NOT NULL ajer its type in a CREATE TABLE statement. Effect is to disallow tuples in which this a\ribute is NULL. 16

17 ATribute- Based CHECK Constraints Disallow courses with a maximum enrollment greater than 100. This constraint only affects the value of a single a\ribute in each tuple. CREATE TABLE Courses(... Enrollment INT CHECK (Enrollment <= 100)...); The condi>on can be any condi>on that can appear in a WHERE clause. CHECK statement may use a subquery to men>on other a\ributes of the same or other rela>ons. An a\ribute- based CHECK constraint is checked only when the value of that a\ribute changes. 17

18 Tuple- Based CHECK Constraints Tuple- based CHECK constraints are checked whenever a tuple is inserted into or updated in a rela;on. Designer may add these constraints ajer the list of a\ributes in a CREATE TABLE statement. A chairperson of a department teaches at most one course in any semester. CREATE TABLE Teach(... CHECK ProfessorPID NOT IN ((SELECT ProfessorPID FROM Teach) INTERSECT (SELECT ChairmanPID FROM Departments) ) ); 18

19 Modifying Constraints SQL allows constraints to be named. Use CONSTRAINT followed by the name of the constraint in front of PRIMARY KEY, UNIQUE, or CHECK. Can use constraint names in ALTER TABLE statements to delete constraints: say DROP CONSTRAINT followed by the name of the constraint. Can add constraints in an ALTER TABLE statement using ADD CONSTRAINT followed by an op>onal name followed by the (required) CHECK statement. 19

20 Asser;ons These are database- schema elements, like rela>ons Defined by: CREATE ASSERTION <name> CHECK (<condi>on>); Condi>on may refer to any rela>on or a\ribute in the database schema. 20

21 Asser;ons: Example Can t have more courses than students ( Pigeonhole Principle ) CREATE ASSERTION FewStudents CHECK ( (SELECT COUNT(*) FROM Students) <= (SELECT COUNT(*) FROM Courses) ); 21

22 Triggers: Mo;va;on Asser>ons are powerful, but the DBMS ojen can t tell when they need to be checked. A\ribute- and tuple- based checks are checked at known >mesà but not powerful. Triggers let the user decide when to check for any condi>on. 22

23 OK, what could have been done? 23

24 Announcements Check updated webpage More details on Grading scheme Homework dates Project deliverables Lecture Schedule 24

25 STORING DATA 25

26 DBMS Layers: Queries Query Optimization and Execution Relational Operators Files and Access Methods TODAY! Buffer Management Disk Space Management DB 26

27 Leverage OS for disk/file management? Layers of abstrac>on are good but: 27

28 Leverage OS for disk/file management? Layers of abstrac>on are good but: Unfortunately, OS ojen gets in the way of DBMS 28

29 Leverage OS for disk/file management? DBMS wants/needs to do things its own way Specialized prefetching Control over buffer replacement policy LRU not always best (some>mes worst!!) Control over thread/process scheduling Convoy problem Arises when OS scheduling conflicts with DBMS locking Control over flushing data to disk WAL protocol requires flushing log entries to disk 29

30 Disks and Files DBMS stores informa>on on disks. but: disks are (rela>vely) VERY slow! Major implica>ons for DBMS design! 30

31 Disks and Files Major implica>ons for DBMS design: READ: disk - > main memory (RAM). WRITE: reverse Both are high- cost opera>ons, rela>ve to in- memory opera>ons, so must be planned carefully! 31

32 Why Not Store It All in Main Memory? 32

33 Why Not Store It All in Main Memory? Costs too much. disk: ~$1/Gb; memory: ~$100/Gb High- end Databases today in the TB range. Approx 60% of the cost of a produc>on system is in the disks. Main memory is volable. Note: some specialized systems do store en>re database in main memory. 33

34 The Storage Hierarchy Smaller, Faster Bigger, Slower 34

35 Main memory (RAM) for currently used data. Disk for the main database (secondary storage). Tapes for archiving older versions of the data (tertiary storage). The Storage Hierarchy Registers L1 Cache. Main Memory Magnetic Disk Magnetic Tape Smaller, Faster Bigger, Slower 35

36 Jim Gray s Storage Latency Analogy: How Far Away is the Data? 10 9 Tape Andromeda The image cannot be displayed. Your computer The image may not have enough memory to cannot open the be image, or the image may have been displayed. corrupted. Your Restart your computer, and computer then open may the file again. If the red x still not appears, have you may have to delete the image and then insert it The again. image cannot be displayed. Your computer may not have enough 2,000 Years 10 6 Disk Pluto 2 Years 100 Memory Boston 1.5 hr On Board Cache On Chip Cache Registers This Building This Room My Head 10 min 1 min 36

37 Disks Secondary storage device of choice. Main advantage over tapes: random access vs. sequenbal. Data is stored and retrieved in units called disk blocks or pages. Unlike RAM, >me to retrieve a disk page varies depending upon loca>on on disk. rela>ve placement of pages on disk is important! 37

38 Anatomy of a Disk Disk head Spindle Tracks Sector Track Cylinder Platter Block size = multiple of sector size (which is fixed) Arm assembly Arm movement Platters Sector #38

39 Accessing a Disk Page Time to access (read/write) a disk block:... 39

40 Accessing a Disk Page Time to access (read/write) a disk block: seek Bme: moving arms to posi>on disk head on track rotabonal delay: wai>ng for block to rotate under head transfer Bme: actually moving data to/from disk surface 40

41 Accessing a Disk Page Rela>ve >mes? seek Bme: rotabonal delay: transfer Bme: 41

42 Accessing a Disk Page Rela>ve >mes? seek Bme: about 1 to 20msec rotabonal delay: 0 to 10msec transfer Bme: < 1msec per 4KB page Seek Rotate transfer Transfer 42

43 Seek ;me & rota;onal delay dominate Key to lower I/O cost: reduce seek/rota>on delays! Also note: For shared disks, much >me spent wai>ng in queue for access to arm/controller Seek Rotate transfer 43

44 Arranging Pages on Disk Next block concept: blocks on same track, followed by blocks on same cylinder, followed by blocks on adjacent cylinder Accesing next block is cheap A useful op>miza>on: pre- fetching See textbook page

45 Rules of thumb 1. Memory access much faster than disk I/O (~ 1000x) Sequen>al I/O faster than random I/O (~ 10x) 45

46 Disk Arrays: RAID Just FYI Logical Physical Benefits: Higher throughput (via data striping ) Longer MTTF (via redundancy) 46

47 Recall: DBMS Layers Queries Query Optimization and Execution Relational Operators Files and Access Methods TODAY! Buffer Management Disk Space Management DB 47

48 Buffer Management in a DBMS (copy of a) disk page free frame Page Requests from Higher Levels Just FYI buffer pool MAIN MEMORY DISK DB choice of frame dictated by replacement policy 48

49 Files FILE: A collec>on of pages, each containing a collec>on of records. Must support: insert/delete/modify record read a par>cular record (specified using record id) scan all records (possibly with some condi>ons on the records to be retrieved) 49

50 Alterna;ve File Organiza;ons Several alterna>ves (w/ trade- offs): Heap files: Suitable when typical access is a file scan retrieving all records. Sorted Files: Index File Organiza>ons: 50

51 Variable length records SLOTTED PAGE organiza>on - popular. occupied records... page header 51

52 Conclusions- - - Storing Memory hierarchy Disks: (>1000x slower) - thus pack info in blocks try to fetch nearby blocks (sequen>ally) Record organiza>on: Slo\ed page 52

53 TREE INDEXES 53

54 Declaring Indexes No standard! Typical syntax: CREATE INDEX StudentsInd ON Students(ID); CREATE INDEX CoursesInd ON Courses(Number, DeptName); 54

55 Types of Indexes Primary: index on a key Used to enforce constraints Secondary: index on non- key a\ribute Clustering: order of the rows in the data pages correspond to the order of the rows in the index Only one clustered index can exist in a given table Useful for range predicates Non- clustering: physical order not the same as index order 55

56 Using Indexes (1): Equality Searches Given a value v, the index takes us to only those tuples that have v in the a\ribute(s) of the index. E.g. (use CourseInd index) SELECT Enrollment FROM Courses WHERE Number = 4604 and DeptName = CS 56

57 Using Indexes (1): Equality Searches Given a value v, the index takes us to only those tuples that have v in the a\ribute(s) of the index. Can use Hashes, but see next 57

58 Using Indexes (2): Range Searches ``Find all students with gpa > 3.0 may be slow, even on sorted file Hashes not a good idea! What to do? Page 1 Page 2 Page 3 Page N Data File 58

59 Range Searches ``Find all students with gpa > 3.0 may be slow, even on sorted file Solu>on: Create an `index file. k1 k2 kn Index File Page 1 Page 2 Page 3 Page N Data File 59

60 Range Searches More details: if index file is small, do binary search there Otherwise?? k1 k2 kn Index File Page 1 Page 2 Page 3 Page N Data File 60

61 B- trees the most successful family of index schemes (B- trees, B+- trees, B*- trees) Can be used for primary/secondary, clustering/non- clustering index. balanced n- way search trees Original Paper: Rudolf Bayer and McCreight, E. M. Organiza>on and Maintenance of Large Ordered Indexes. Acta Informa>ca 1, ,

62 B- trees Eg., B- tree of order d=1: <6 6 >6 <9 9 >

63 B - tree proper;es: each node, in a B- tree of order d: Key order at most n=2d keys at least d keys (except root, which may have just 1 key) all leaves at the same level if number of pointers is k, then node has exactly p1 pn k- 1 keys (leaves are empty) v1 v2 v n- 1 63

64 Proper;es block aware nodes: each node is a disk page O(log (N)) for everything! (ins/del/search) typically, if d = , then 2-3 levels u>liza>on >= 50%, guaranteed; on average 69% 64

65 Queries Algo for exact match query? (eg., ssn=8?) <6 6 >6 <9 9 >

66 JAVA anima;on h\p://slady.net/java/bt/ 66

67 Queries Algo for exact match query? (eg., ssn=8?) <6 6 >6 <9 9 >

68 Queries Algo for exact match query? (eg., ssn=8?) <6 6 >6 <9 9 >

69 Queries Algo for exact match query? (eg., ssn=8?) <6 6 >6 <9 9 >

70 Queries Algo for exact match query? (eg., ssn=8?) <6 6 >6 <9 9 >9 H steps (= disk accesses)

71 Queries what about range queries? (eg., 5<salary<8) Proximity/ nearest neighbor searches? (eg., salary ~ 8 ) 71

72 Queries what about range queries? (eg., 5<salary<8) Proximity/ nearest neighbor searches? (eg., salary ~ 8 ) <6 6 9 >6 <9 >

73 Queries what about range queries? (eg., 5<salary<8) Proximity/ nearest neighbor searches? (eg., salary ~ 8 ) <6 6 9 >6 <9 >

74 Queries what about range queries? (eg., 5<salary<8) Proximity/ nearest neighbor searches? (eg., salary ~ 8 ) <6 6 9 >6 <9 >

75 Queries what about range queries? (eg., 5<salary<8) Proximity/ nearest neighbor searches? (eg., salary ~ 8 ) <6 6 9 >6 <9 >

76 Varia;ons How could we do even be\er than the B- trees above? 76

77 B+ trees - Mo;va;on B- tree print keys in sorted order: <6 6 >6 <9 9 >

78 B+ trees - Mo;va;on B- tree needs back- tracking how to avoid it? <6 6 >6 <9 9 >

79 B+ trees - Mo;va;on Stronger reason: for clustering index, data records are sca\ered: <6 6 >6 <9 9 >

80 Solu;on: B+ - trees facilitate sequen>al ops They string all leaf nodes together AND replicate keys from non- leaf nodes, to make sure every key appears at the leaf level (vital, for clustering index!) 80

81 B+ trees <6 6 >=6 <9 9 >=

82 B+ trees <6 6 9 Index Pages >=6 <9 >= Data Pages 82

83 B+ trees More details: next (and textbook) In short: on split at leaf level: COPY middle key upstairs at non- leaf level: push middle key upstairs (as in plain B- tree) 83

84 Example B+ Tree Search begins at root, and key comparisons direct it to a leaf Search for 5*, 15*, all data entries >= 24*... Root * 3* 5* 7* 14* 16* 19* 20* 22* 24* 27* 29* 33* 34* 38* 39* Based on the search for 15*, we know it is not in the tree! 84

85 Inser;ng a Data Entry into a B+ Tree Find correct leaf L. Put data entry onto L. If L has enough space, done! Else, must split L (into L and a new node L2) Redistribute entries evenly, copy up middle key. parent node may overflow but then: push up middle key. Splits grow tree; root split increases height. 85

86 Example B+ Tree Inser;ng 30* Root * 3* 5* 7* 14* 16* 19* 20* 22* 23* 24* 27* 29* 86

87 Example B+ Tree Inser;ng 30* Root * 3* 5* 7* 14* 16* 19* 20* 22* 23* 24* 27* 29* 30* 87

88 Example B+ Tree - Inser;ng 8* Root * 3* 5* 7* 14* 16* 19* 20* 22* 23* 24* 27* 29* 88

89 Example B+ Tree - Inser;ng 8* Root * 3* 5* 7* 14* 16* 19* 20* 22* 23* 24* 27* 29* No Space 89

90 Example B+ Tree - Inser;ng 8* Root So Split! * 3* 5* 7* 14* 16* 19* 20* 22* 23* 24* 27* 29* * 3* 5* 14* 16* 19* 20* 22* 23* 24* 27* 5* 7* 8* 29* 90

91 Example B+ Tree - Inser;ng 8* Root So Split! * 3* 5* 7* 14* 16* 19* 20* 22* 23* 24* 27* 29* And then push middle UP * 3* 5* 14* 16* 19* 20* 22* 23* 24* 27* 5* 7* 8* 29* 91

92 Example B+ Tree - Inser;ng 8* Root * 3* 5* 7* 14* 16* 19* 20* 22* 23* 24* 27* 29* Final State <5 >= * 3* 14* 16* 19* 20* 22* 23* 24* 27* 5* 7* 8* 29* 92

93 Example B+ Tree - Inser;ng 21* Root * 3* 5* 7* 8* 14* 16* 19* 20* 22* 23* 24* 27* 29* 2* 3* 5* 7* 8* 14* 16* 19* 20* 22* 23* 24* 27* 29* 93

94 Example B+ Tree - Inser;ng 21* Root * 3* 5* 7* 8* 14* 16* 19* 20* 22* 23* 24* 27* 29* Root is Full, so split recursively 2* 3* 5* 7* 8* 14* 16* 19* 20* 21* 22* 23* 24* 27* 29* 94

95 Example B+ Tree: Recursive split Root * 3* 5* 7* 8* 14* 16* 19* 20* 21* 22* 23* 24* 27* 29* Notice that root was also split, increasing height. 95

96 Example: Data vs. Index Page Split leaf: copy non- leaf: push Data Page Split 2* 3* 5* 7* 8* 5 2* 3* 5* 7* 8* why not non- leaves? Index Page Split #96

97 Same Inser;ng 21*: The Deferred Root Split * 3* 5* 7* 8* 14* 16* 19* 20* 22* 23* 24* 27* 29* Note this has free space. So 97

98 Inser;ng 21*: The Deferred Split Root * 3* 5* 7* 8* 14* 16* 19* 20* 22* 23* 24* 27* 29* Root LEND keys to sibling, through PARENT! 2* 3* 5* 7* 8* 14* 16* 19* 20* 21* 22* 23* 24* 27* 29* 98

99 Inser;ng 21*: The Deferred Split Root * 3* 5* 7* 8* 14* 16* 19* 20* 22* 23* 24* 27* 29* Root Shorter, more packed, faster tree * 3* 5* 7* 8* 14* 16* 19* 20* 21* 22* 23* 24* 27* 29* 99

100 Inser;on examples for you to try Root (not shown) 2* 3* 5* 7* 8* 11* 14* 16* 21* 22* 23* Insert the following data entries (in order): 28*, 6*, 25* 100

101 Answer After inserting 28*, 6* * 3* 5* 6* 7* 8* 11* 14* 16* 21* 22* 23* 28* After inserting 25* 101

102 Answer After inserting 25* * 3* 5* 6* 7* 8* 11* 14* 16* 21* 22* 23* 25* 28* 102

103 Dele;ng a Data Entry from a B+ Tree Start at root, find leaf L where entry belongs. Remove the entry. If L is at least half- full, done! If L underflows Try to re- distribute, borrowing from sibling (adjacent node with same parent as L). If re- distribu>on fails, merge L and sibling. update parent and possibly merge, recursively 103

104 Dele;on from B+Tree Root * 3* 5* 7* 8* 14* 16* 19* 20* 21* 22* 23* 24* 27* 29* 104

105 Example: Delete 19* & 20* Root Dele>ng 19* is easy: * 3* 5* 7* 8* 14* 16* 19* 20* 20* 22* 22* 24* 27* 29* 33* 34* 38* 39* 3 Root * 3* 5* 7* 8* 14* 16* 22* 24* 27* 29* 33* 34* 38* 39* Dele>ng 20* - > re- distribu>on (no>ce: 27 copied up) 105

106 ... And Then Dele;ng 24* 3 Root * 3* 5* 7* 8* 14* 16* 22* 24* 27* 29* 33* 34* 38* 39* 4 Root * 3* 5* 7* 8* 14* 16* 22* 27* 29* 33* 34* 38* 39* Must merge leaves: OPPOSITE of insert 106

107 ... And Then Dele;ng 24* 3 Root * 3* 5* 7* 8* 14* 16* 22* 24* 27* 29* 33* 34* 38* 39* 4 Root 17 but are we done?? * 3* 5* 7* 8* 14* 16* 22* 27* 29* 33* 34* 38* 39* Must merge leaves: OPPOSITE of insert 107

108 ... Merge Non- Leaf Nodes, Shrink Tree 4 Root * 3* 5* 7* 8* 14* 16* 22* 27* 29* 33* 34* 38* 39* 5 Root * 3* 5* 7* 8* 14* 16* 22* 27* 29* 33* 34* 38* 39* 108

109 Example of Non- leaf Re- distribu;on Tree is shown below during dele>on of 24*. Now, we can re- distribute keys Root * 3* 5* 7* 8* 14* 16* 17* 18* 20* 21* 22* 27* 29* 33* 34* 38* 39* 109

110 Aqer Re- distribu;on need only re- distribute 20 ; did 17, too why would we want to re- distribute more keys? Root * 3* 5* 7* 8* 14* 16* 17* 18* 20* 21* 22* 27* 29* 33* 34* 38* 39* 110

111 Main observa;ons for dele;on If a key value appears twice (leaf + nonleaf), the above algorithms delete it from the leaf, only why not non- leaf, too? 111

112 Main observa;ons for dele;on If a key value appears twice (leaf + nonleaf), the above algorithms delete it from the leaf, only why not non- leaf, too? lazy dele>ons - in fact, some vendors just mark entries as deleted (~ underflow), and reorganize/compact later 112

113 Recap: main ideas on overflow, split (and push, or copy ) or consider deferred split on underflow, borrow keys; or merge or let it underflow

114 B+ Trees in Prac;ce Typical order: 100. Typical fill- factor: 67%. average fanout = 2*100*0.67 = 134 Typical capaci>es: Height 4: 1334 = 312,900,721 entries Height 3: 1333 = 2,406,104 entries 114

115 B+ Trees in Prac;ce Can ojen keep top levels in buffer pool: Level 1 = 1 page = 8 KB Level 2 = 134 pages = 1 MB Level 3 = 17,956 pages = 140 MB 115

116 B+ trees with duplicates Everything so far: assumed unique key values How to extend B+- trees for duplicates? Alt. 2: <key, rid> Alt. 3: <key, {rid list}> 2 approaches, roughly equivalent 116

117 B+ trees with duplicates approach#1: repeat the key values, and extend B+ tree algo s appropriately - eg. many 14 s * 3* 5* 7* 13* 14* 14* 14* 14* 14* 22* 23* 24* 27* 29* 117

118 B+ trees with duplicates approach#1: subtle problem with dele>on: treat rid as part of the key, thus making it unique * 3* 5* 7* 13* 14* 14* 14* 14* 14* 22* 23* 24* 27* 29* 118

119 B+ trees with duplicates approach#2: store each key value: once but store the {rid list} as variable- length field (and use overflow pages, if needed) * 3* 5* 7* 13* 14* {rid list} 22* 23* 24* 27* 29* {rid list, cont d} 119

120 B+trees in Prac;ce prefix compression; bulk- loading; order 120

121 Prefix Key Compression Important to increase fan- out. (Why?) Key values in index entries only `direct traffic ; can ojen compress them. Papadopoulos Pernikovskaya 121

122 Prefix Key Compression Important to increase fan- out. (Why?) Key values in index entries only `direct traffic ; can ojen compress them. Pap Per <room for more separators/keys> 122

123 Bulk Loading of a B+ Tree In an empty tree, insert many keys Why not one- at- a- >me? Too slow! 123

124 Bulk Loading of a B+ Tree Ini>aliza>on: Sort all data entries scan list; whenever enough for a page, pack <repeat for upper level> Root Sorted pages of data entries; not yet in B+ tree 3* 4* 6* 9* 10* 11* 12* 13* 20* 22* 23* 31* 35* 36* 38* 41* 44* 124

125 Bulk Loading of a B+ Tree Root Data entry pages not yet in B+ tree 3* 4* 6* 9* 10* 11* 12* 13* 20* 22* 23* 31* 35* 36* 38* 41* 44* Root Data entry pages not yet in B+ tree Prakash * 4* 6* 9* VT 10* CS 11* * 13* 20* 22* 23* 31* 35* 36* 38* 41* 44* 125

126 A Note on `Order Order (d) concept replaced by physical space criterion in prac>ce (`at least half- full ). Why do we need it? Index pages can typically hold many more entries than leaf pages. Variable sized records and search keys mean different nodes will contain different numbers of entries. Even with fixed length fields, mul>ple records with the same search key value (duplicates) can lead to variable- sized data entries (if we use Alterna>ve (3)). 126

127 A Note on `Order Many real systems are even sloppier than this: they allow underflow, and only reclaim space when a page is completely empty. (what are the benefits of such slopiness?) 127

128 Conclusions B+tree is the prevailing indexing method Excellent, O(logN) worst- case performance for ins/del/search; (~3-4 disk accesses in prac>ce) guaranteed 50% space u>liza>on; avg 69% 128

129 Conclusions Can be used for any type of index: primary/ secondary, sparse (clustering), or dense (non- clustering) Several fine- extensions on the basic algorithm deferred split; prefix compression; (underflows) bulk- loading duplicate handling 129