Topics to Learn. Important concepts. Tree-based index. Hash-based index

Transcription

1 CS143: Index 1

2 Topics to Learn Important concepts Dense index vs. sparse index Primary index vs. secondary index (= clustering index vs. non-clustering index) Tree-based vs. hash-based index Tree-based index Indexed sequential file B+-tree Hash-based index Static hashing Extendible hashing 2

3 Basic Problem SELECT * FROM Student WHERE sid = 40 sid name GPA 20 Elaine Peter Susan 3.7 How can we answer the query? 3

4 Random-Order File How do we find sid=40? sid name GPA 20 Susan James Peter Elaine Christy 2.9 4

5 Sequential File Table sequenced by sid. Find sid=40? sid name GPA 20 Susan James Peter Elaine Christy 2.9 5

6 100,000 records Binary Search Q: How many blocks to read? Any better way? In a library, how do we find a book? 6

7 Basic Idea Build an index on the table An auxiliary structure to help us locate a record given a key

8 Dense, Primary Index Dense Index Sequential File Primary index (clustering index) Index on the search key Dense index (key, pointer) pair for every record Find the key from index and follow pointer Maybe through binary search Q: Why dense index? Isn t binary search on the file the same? 8

9 Why Dense Index? Example 10,000,000 records (900-bytes/rec) 4-byte search key, 4-byte pointer 4096-byte block. Unspanned tuples Q: How many blocks for table (how big)? Q: How many blocks for index (how big)? 9

10 Sparse, Primary Index Sparse Index Sequential File Sparse index (key, pointer) pair per every block (key, pointer) pair points to the first record in the block Q: How can we find 60? 10

11 Multi-level index Sparse 2nd level st level Q: Why multi-level index? Sequential File Q: Does dense, 2nd level index make sense? 11

12 Secondary (non-clustering) Index Sequence field Secondary (non-clustering) index When tuples in the table are not ordered by the index search key Index on a non-search-key for sequential file Unordered file Q: What index? Does sparse index make sense? 12

13 Sparse and secondary index?

14 Secondary index sparse High level First level is always dense Sparse from the second level

15 Important terms Dense index vs. sparse index Primary index vs. secondary index Clustering index vs. non-clustering index Multi-level index Indexed sequential file Sometimes called ISAM (indexed sequential access method) Search key ( primary key) 15

16 Insertion Insert Q: Do we need to update higher-level index? 16

17 Insertion Insert Q: Do we need to update higher-level index? 17

18 Insertion Insert Q: Do we need to update higher-level index? 18

19 Potential performance problem After many insertions Main index overflow pages (not sequential) 19

20 Traditional Index (ISAM) Advantage Simple Sequential blocks Disadvantage Not suitable for updates Becomes ugly (loses sequentiality and balance) over time 20

21 B+Tree Most popular index structure in RDBMS Advantage Suitable for dynamic updates Balanced Minimum space usage guarantee Disadvantage Non-sequential index blocks 21

22 B+Tree Example (n=3) 70 root Non leaf Leaf Susan James Peter 1.8 Balanced: All leaf nodes are at the same level 22

23 Sample Leaf Node (n=3) From a non-leaf node Last pointer: to the next leaf node points to tuple 20 Susan James Peter 1.8 n: max # of pointers in a node All pointers (except the last one) point to tuples At least half of the pointers are used. (more precisely, (n+1)/2 pointers) 23

24 Sample Non-leaf Node (n=3) To keys k<23 To keys 23 k<56 To keys 56 k Points to the nodes one-level below - No direct pointers to tuples At least half of the ptrs used (precisely, n/2 ) - except root, where at least 2 ptrs used 24

25 Search on B+tree Find 30, 60, 70? Find a greater key and follow the link on the left (Algorithm: Figure on textbook) 25

26 Nodes are never too empty Use at least Non-leaf: n/2 pointers Leaf: (n+1)/2 pointers n=4 Non-leaf full node min. node Leaf

27 Number of Ptrs/Keys for B+tree Max Max Ptrs keys Min ptrs Min keys Non-leaf (non-root) n n-1 n/2 n/2-1 Leaf (non-root) n n-1 (n+1)/2 (n-1)/2 Root n n

28 B+Tree Insertion (a) simple case (no overflow) (b) leaf overflow (c) non-leaf overflow (d) new root 28

29 (a) Simple case (no overflow) 29

30 Insertion (Simple Case) Insert

31 Insertion (Simple Case) Insert

32 (b) Leaf overflow 32

33 Insertion (Leaf Overflow) Insert No space to store 55 33

34 Insertion (Leaf Overflow) Insert Overflow! Split the leaf into two. Put the keys half and half 34

35 Insertion (Leaf Overflow) Insert

36 Insertion (Leaf Overflow) Insert Copy the first key of the new node to parent 36

37 Insertion (Leaf Overflow) Insert No overflow. Stop Q: After split, leaf nodes always half full? 37

38 (c) Non-leaf overflow 38

39 Insertion (Non-leaf Overflow) Insert Leaf overflow. Split and copy the first key of the new node 39

40 Insertion (Non-leaf Overflow) Insert

41 Insertion (Non-leaf Overflow) Insert

42 Insertion (Non-leaf Overflow) Insert Overflow!

43 Insertion (Non-leaf Overflow) Insert Split the node into two. Move up the key in the middle. 43

44 Insertion (Non-leaf Overflow) Insert Middle key

45 Insertion (Non-leaf Overflow) Insert No overflow. Stop Q: After split, non-leaf at least half full? 45

46 (d) New root 46

47 Insertion (New Root Node) Insert

48 Insertion (New Root Node) Insert Overflow!

49 Insertion (New Root Node) Insert Split and move up the mid-key. Create new root

50 Insertion (New Root Node) Insert 25 Q: At least 2 ptrs at root?

51 B+Tree Insertion Leaf node overflow The first key of the new node is copied to the parent Non-leaf node overflow The middle key is moved to the parent Detailed algorithm: Figure

52 B+Tree Deletion (a) Simple case (no underflow) (b) Leaf node, coalesce with neighbor (c) Leaf node, redistribute with neighbor (d) Non-leaf node, coalesce with neighbor (e) Non-leaf node, redistribute with neighbor In the examples, n = 4 Underflow for non-leaf when fewer than n/2 = 2 ptrs Underflow for leaf when fewer than (n+1)/2 = 3 ptrs Nodes are labeled as a, b, c, d, 52

53 (a) Simple case (no underflow) 53

54 (a) Simple case a b c d e Delete 25 54

55 (a) Simple case a b c d e Underflow? Delete 25 Underflow? Min 3 ptrs. Currently 3 ptrs 55

56 (b) Leaf node, coalesce with neighbor 56

57 Delete 50 (b) Coalesce with sibling (leaf) b c d a e 60 57

58 Delete 50 (b) Coalesce with sibling (leaf) b c d a Underflow? Underflow? Min 3 ptrs, currently 2. e 60 58

59 Delete 50 (b) Coalesce with sibling (leaf) b c d Try to merge with a sibling a underflow! Can be merged? e 60 59

60 (b) Coalesce with sibling (leaf) Merge b c d a Delete 50 Merge c and d. Move everything on the right to the left. 40 e 60 60

61 (b) Coalesce with sibling (leaf) a b c d e 60 Delete 50 Once everything is moved, delete d 61

62 Delete 50 (b) Coalesce with sibling (leaf) After leaf node merge, a b c d From its parent, delete the pointer and key to the deleted node e 60 62

63 (b) Coalesce with sibling (leaf) a b c Underflow? e Delete 50 Check underflow at a. Min 2 ptrs, currently 3 63

64 (c) Leaf node, redistribute with neighbor 64

65 (c) Redistribute (leaf) a b c d e Delete 50 65

66 (c) Redistribute (leaf) a b c d e Underflow? 60 Delete 50 Underflow? Min 3 ptrs, currently 2 Check if d can be merged with its sibling c or e If not, redistribute the keys in d with a sibling Say, with c Can be merged? 66

67 Delete 50 (c) Redistribute (leaf) a Redistribute b c d e Redistribute c and d, so that nodes c and d are roughly half full Move the key 30 and its tuple pointer to the d 67

68 (c) Redistribute (leaf) a b c d e Delete 50 Update the key in the parent 68

69 (c) Redistribute (leaf) a 30 Underflow? b c d e Delete 50 No underflow at a. Done. 69

70 (d) Non-leaf node, coalesce with neighbor 70

71 (d) Coalesce (non-leaf) a b 30 c 70 d e f g Delete 20 Underflow! Merge d with e. Move everything in the right to the left 71

72 (d) Coalesce (non-leaf) a b 30 c 70 d e f g Delete 20 From the parent node, delete pointer and key to the deleted node 72

73 (d) Coalesce (non-leaf) a b underflow! c 70 Can be merged? d f g Delete 20 Underflow at b? Min 2 ptrs, currently 1. Try to merge with its sibling. Nodes b and c: 3 ptrs in total. Max 4 ptrs. Merge b and c. 73

74 merge b (d) Coalesce (non-leaf) a c 70 d f g Delete 20 Merge b and c Pull down the mid-key 50 in the parent node Move everything in the right node to the left. Very important: when we merge non-leaf nodes, we always pull down the mid-key in the parent and place it in the merged node

75 (d) Coalesce (non-leaf) a b c 70 d f g Delete 20 Merge b and c Pull down the mid-key 50 in the parent node Move everything in the right node to the left. Very important: when we merge non-leaf nodes, we always pull down the mid-key in the parent and place it in the merged node

76 (d) Coalesce (non-leaf) a 90 b c d f g Delete 20 Delete pointer to the merged node

77 (d) Coalesce (non-leaf) a 90 b d f g Delete 20 Underflow at a? Min 2 ptrs. Currently 2. Done

78 (e) Non-leaf node, redistribute with neighbor 78

79 (e) Redistribute (non-leaf) a b 30 c d e f g Delete 20 Underflow! Merge d with e. 79

80 (e) Redistribute (non-leaf) a b 30 c d e f g Delete 20 After merge, remove the key and ptr to the deleted node from the parent 80

81 (e) Redistribute (non-leaf) underflow! b c Can be merged? d f g a Delete 20 Underflow at b? Min 2 ptrs, currently 1. Merge b with c? Max 4 ptrs, 5 ptrs in total. If cannot be merged, redistribute the keys with a sibling. Redistribute b and c 81

82 (e) Redistribute (non-leaf) redistribute a b c d f g Delete Redistribution at a non-leaf node is done in two steps. Step 1: Temporarily, make the left node b overflow by pulling down the mid-key and moving everything to the left

83 (e) Redistribute (non-leaf) redistribute a 99 b temporary overflow c d f g Delete 20 Step 2: Apply the overflow handling algorithm (the same algorithm used for B+tree insertion) to the overflowed node Detailed algorithm in the next slide 83

84 (e) Redistribute (non-leaf) redistribute a 99 b c d f g Delete 20 Step 2: overflow handling algorithm Pick the mid-key (say 90) in the node and move it to parent. Move everything to the right of 90 to the empty node c. 84

85 (e) Redistribute (non-leaf) a b c 97 d f g Delete 20 Underflow at a? Min 2 ptrs, currently 3. Done 85

86 Important Points Remember: For leaf node merging, we delete the mid-key from the parent For non-leaf node merging/redistribution, we pull down the mid-key from their parent. Exact algorithm: Figure In practice Coalescing is often not implemented Too hard and not worth it 86

87 Where does n come from? n determined by Size of a node Size of search key Size of an index pointer Q: 1024B node, 10B key, 8B ptr à n? 87

88 Question on B+tree SELECT * FROM Student WHERE sid > 60?

89 Summary on tree index Issues to consider Sparse vs. dense Primary (clustering) vs. secondary (non-clustering) Indexed sequential file (ISAM) Simple algorithm. Sequential blocks Not suitable for dynamic environment B+trees Balanced, minimum space guarantee Insertion, deletion algorithms 89

90 Index Creation in SQL CREATE INDEX <indexname> ON <table>(<attr>,<attr>, ) Example CREATE INDEX stidx ON Student(sid) Creates a B+tree on the attributes Speeds up lookup on sid 90

91 Primary (Clustering) Index MySQL: Primary key becomes the clustering index DB2: CREATE INDEX idx ON Student(sid) CLUSTER Tuples in the table are sequenced by sid Oracle: Index-Organized Table (IOT) CREATE TABLE T (... ) ORGANIZATION INDEX B+tree on primary key Tuples are stored at the leaf nodes of B+tree Periodic reorganization may still be necessary to improve range scan performance 91

92 Next topic Hash index Static hashing Extendible hashing 92

93 What is a Hash Table? Hash Table Hash function h(k): key à integer [0 n] e.g., h( Susan ) = 7 Array for keys: T[0 n] Given a key k, store it in T[h(k)] h(susan) = 4 h(james) = 3 h(neil) = Neil 2 3 James 4 Susan 5 93

94 Hashing for DBMS (Static Hashing) search key h(key) Disk blocks (buckets) (key, record)

95 Overflow and Chaining Insert h(a) = 1 h(b) = 2 h(c) = 1 h(d) = 0 h(e) = d a c b e Delete h(b) = 2 h(c) =

96 Major Problem of Static Hashing How to cope with growth? Data tends to grow in size Overflow blocks unavoidable hash buckets overflow blocks

97 Extendible Hashing (two ideas) (a) Use i of b bits output by hash function h(k) b use i grows over time 97

98 Extendible Hashing (two ideas) (b) Use directory that maintains pointers to hash buckets (indirection) h(c) directory.. hash bucket c e 98

99 Example h(k) is 4 bits; 2 keys/bucket Insert 0111 i = 0 1 i = i =

100 Example Insert 1010 i = 0 1 i = i = overflow! Increase i of the bucket. Split it. 100

101 Example Insert 1010 i = i = i = overflow! Redistribute keys based on first i bits i = 2 101

102 Example Insert 1010 i = Update ptr in dir to new bkt 1? If no space, double directory size (increase i)

103 Example Insert 1010 i = 2 00 i = Copy pointers

104 Example Insert 1010 i = 2 00 i =

105 Example Insert 0000 i = Split bucket and increase i Overflow! 105

106 Example 2 Insert 0000 i = Redistribute keys Overflow! 106

107 Insert 0000 i = 2 00 Example Update ptr in directory

108 Insert 0000 i = Example

109 Insert Overflow! i = Split bucket, increase i, redistribute keys

110 Insert i = Update ptr in dir If no space, double directory

111 Insert 0011 i = i =

112 Insert 0011 i = i =

113 Extendible Hashing: Deletion Two options a) No merging of buckets b) Merge buckets and shrink directory if possible 113

114 Delete 1010 i = a b c 114

115 Delete 1010 i = a b c Can we merge a and b? b and c? 115

116 i = Delete 1010 Decrease i and merge buckets 2 Update ptr in directory Q: Can we shrink directory? a b c 116

117 Delete 1010 i = 0 1 i = a b

118 Bucket Merge Condition Bucket merge condition Bucket i s are the same First (i-1) bits of the hash key are the same Directory shrink condition All bucket i s are smaller than the directory i 118

119 Questions on Extendible Hashing Can we provide minimum space guarantee? 119

120 Space Waste i =

121 Static hashing Hash index summary Overflow and chaining Extendible hashing Can handle growing files No periodic reorganizations Indirection Up to 2 disk accesses to access a key Directory doubles in size Not too bad if the data is not too large 121

122 Hashing vs. Tree Can an extendible-hash index support? SELECT * FROM R WHERE R.A > 5 Which one is better, B+tree or Extendible hashing? SELECT * FROM R WHERE R.A = 5 122