CS143: Index. Book Chapters: (4 th ) , (5 th ) , , 12.10

Transcription

1 CS143: Index Book Chapters: (4 th ) , (5 th ) , ,

2 Topics to Learn Important concepts Dense index vs. sparse index Primary index vs. secondary index (= clustering index vs. non-clustering index) Tree-based vs. hash-based index Tree-based index Indexed sequential file B+-tree Hash-based index Static hashing Extendible hashing 2

3 Basic Problem SELECT * FROM Student WHERE sid = 40 sid name Elaine Peter Susan GPA How can we answer the query? 3

4 Random-Order File How do we find sid=40? sid name Susan James Peter Elaine Christy GPA

5 Sequential File Table sequenced by sid. Find sid=40? sid name Susan James Peter Elaine Christy GPA

6 100,000 records Binary Search Q: How many blocks to read? Any better way? In a library, how do we find a book? 6

7 Basic Idea Build an index on the table An auxiliary structure to help us locate a record given a key

8 Dense, Primary Index Dense Index Sequential File Primary index (clustering index) Index on the search key Dense index (key, pointer) pair for every record Find the key from index and follow pointer Maybe through binary search Q: Why dense index? Isn t binary search on the file the same? 8

9 Why Dense Index? Example 10,000,000 records (900-bytes/rec) 4-byte search key, 4-byte pointer 4096-byte block. Unspanned tuples Q: How many blocks for table (how big)? Q: How many blocks for index (how big)? 9

10 Sparse, Primary Index Sparse Index Sequential File Sparse index (key, pointer) pair per every block (key, pointer) pair points to the first record in the block Q: How can we find 60? 10

11 Multi-level index Sparse 2nd level st level Q: Why multi-level index? Sequential File Q: Does dense, 2nd level index make sense? 11

12 Secondary (non-clustering) Index Sequence field Secondary (non-clustering) index When tuples in the table are not ordered by the index search key Index on a non-search-key for sequential file Unordered file Q: What index? Does sparse index make sense? 12

13 Sparse and secondary index?

14 Secondary index sparse High level First level is always dense Sparse from the second level

15 Important terms Dense index vs. sparse index Clustering index vs. non-clustering index Primary index vs. secondary index Multi-level index Indexed sequential file Sometimes called ISAM (indexed sequential access method) Search key ( primary key) 15

16 Insertion Insert Q: Do we need to update higher-level index? 16

17 Insertion Insert 15 (overflow) Q: Do we need to update higher-level index? 17

18 Insertion Insert 15 (redistribute) Q: Do we need to update higher-level index? 18

19 Potential performance problem After many insertions Main index overflow pages (not sequential) 19

20 Traditional Index (ISAM) Advantage Simple Sequential blocks Disadvantage Not suitable for updates Becomes ugly (loses sequentiality and balance) over time 20

21 B+Tree Most popular index structure in RDBMS Advantage Suitable for dynamic updates Balanced Minimum space usage guarantee Disadvantage Non-sequential index blocks 21

22 B+Tree Example (n=3) 70 root Non leaf Leaf Susan James Peter Balanced: All leaf nodes are at the same level

23 Sample Leaf Node (n=3) From a non-leaf node Last pointer: to the next leaf node points to tuple 20 Susan James Peter 1.8 n: max # of pointers in a node All pointers (except the last one) point to tuples At least half of the pointers are used. (more precisely, (n+1)/2 pointers) 23

24 Sample Non-leaf Node (n=3) To keys k<23 To keys 23 k<56 To keys 56 k Points to the nodes one-level below - No direct pointers to tuples At least half of the ptrs used (precisely, n/2 ) - except root, where at least 2 ptrs used 24

25 Search on B+tree Find 30, 60, 70? Find a greater key and follow the link on the left (Algorithm: Figure on textbook) 25

26 Nodes are never too empty Use at least Non-leaf: n/2 pointers Leaf: (n+1)/2 pointers n=4 Non-leaf full node min. node Leaf

27 Number of Ptrs/Keys for B+tree Max Max Ptrs keys Min ptrs Min keys Non-leaf (non-root) n n-1 n/2 n/2-1 Leaf (non-root) n n-1 (n+1)/2 (n-1)/2 Root n n

28 B+Tree Insertion (a) simple case (no overflow) (b) leaf overflow (c) non-leaf overflow (d) new root 28

29 (a) Simple case (no overflow) 29

30 Insertion (Simple Case) Insert

31 Insertion (Simple Case) Insert

32 (b) Leaf overflow 32

33 Insertion (Leaf Overflow) Insert No space to store 55 33

34 Insertion (Leaf Overflow) Insert Overflow! Split the leaf into two. Put the keys half and half 34

35 Insertion (Leaf Overflow) Insert

36 Insertion (Leaf Overflow) Insert Copy the first key of the new node to parent 36

37 Insertion (Leaf Overflow) Insert No overflow. Stop Q: After split, leaf nodes always half full? 37

38 (c) Non-leaf overflow 38

39 Insertion (Non-leaf Overflow) Insert Leaf overflow. Split and copy the first key of the new node 39

40 Insertion (Non-leaf Overflow) Insert

41 Insertion (Non-leaf Overflow) Insert Overflow!

42 Insertion (Non-leaf Overflow) Insert Split the node into two. Move up the key in the middle. 42

43 Insertion (Non-leaf Overflow) Insert Middle key

44 Insertion (Non-leaf Overflow) Insert No overflow. Stop Q: After split, non-leaf at least half full? 44

45 (c) New root 45

46 Insertion (New Root Node) Insert

47 Insertion (New Root Node) Insert Overflow!

48 Insertion (New Root Node) Insert Split and move up the mid-key. Create new root

49 Insertion (New Root Node) Insert 25 Q: At least 2 ptrs at root?

50 B+Tree Insertion Leaf node overflow The first key of the new node is copied to the parent Non-leaf node overflow The middle key is moved to the parent Detailed algorithm: Figure

51 B+Tree Deletion (a) Simple case (no underflow) (b) Leaf node, coalesce with neighbor (c) Leaf node, redistribute with neighbor (d) Non-leaf node, coalesce with neighbor (e) Non-leaf node, redistribute with neighbor In the examples, n = 4 Underflow for non-leaf when fewer than n/2 = 2 ptrs Underflow for leaf when fewer than (n+1)/2 = 3 ptrs Nodes are labeled as a, b, c, d, 51

52 (a) Simple case (no underflow) 52

53 (a) Simple case a b c d e Delete 25 53

54 (a) Simple case a b c d e Underflow? Delete 25 Underflow? Min 3 ptrs. Currently 3 ptrs 54

55 (b) Leaf node, coalesce with neighbor 55

56 (b) Coalesce with sibling (leaf) a b c d e Delete 50 56

57 Delete 50 (b) Coalesce with sibling (leaf) a b c d e Underflow? Underflow? Min 3 ptrs, currently 2. 57

58 Delete 50 (b) Coalesce with sibling (leaf) Try to merge with a sibling a b c d e underflow! Can be merged? 58

59 Delete 50 (b) Coalesce with sibling (leaf) a Merge b c d e Merge c and d. Move everything on the right to the left

60 (b) Coalesce with sibling (leaf) a b c d e 60 Delete 50 Once everything is moved, delete d 60

61 (b) Coalesce with sibling (leaf) a b c d e 60 Delete 50 After leaf node merge, From its parent, delete the pointer and key to the deleted node 61

62 Delete 50 (b) Coalesce with sibling (leaf) a b c Underflow? e Check underflow at a. Min 2 ptrs, currently

63 (c) Leaf node, redistribute with neighbor 63

64 (c) Redistribute (leaf) a b c d e Delete 50 64

65 (c) Redistribute (leaf) a b c d e Underflow? Delete 50 Underflow? Min 3 ptrs, currently 2 Check if d can be merged with its sibling c or e If not, redistribute the keys in d with a sibling Say, with c Can be merged? 65

66 Delete 50 (c) Redistribute (leaf) a Redistribute b c d e Redistribute c and d, so that nodes c and d are roughly half full 60 Move the key 30 and its tuple pointer to the d 66

67 (c) Redistribute (leaf) a b c d e Delete 50 Update the key in the parent 67

68 (c) Redistribute (leaf) a b c d e Delete 50 No underflow at a. Done. 68

69 (d) Non-leaf node, coalesce with neighbor 69

70 (d) Coalesce (non-leaf) a b c d e f g Delete 20 Underflow! Merge d with e. Move everything in the right to the left 70

71 (d) Coalesce (non-leaf) a b c d e f g Delete 20 From the parent node, delete pointer and key to the deleted node 71

72 (d) Coalesce (non-leaf) bunderflow! c Can be merged? d f g a Delete 20 Underflow at b? Min 2 ptrs, currently 1. Try to merge with its sibling. Nodes b and c: 3 ptrs in total. Max 4 ptrs. Merge b and c. 72

73 merge b (d) Coalesce (non-leaf) a c 70 d f g Delete 20 Merge b and c Pull down the mid-key 50 in the parent node Move everything in the right node to the left. Very important: when we merge non-leaf nodes, we always pull down the mid-key in the parent and place it in the merged node. 73

74 (d) Coalesce (non-leaf) a 90 b c d f g Delete 20 B+tree after merge 74

75 (d) Coalesce (non-leaf) a 90 b d f g Delete 20 Delete pointer to the merged node. 75

76 (d) Coalesce (non-leaf) a 90 b d f g Delete 20 Underflow at a? Min 2 ptrs. Currently 2. Done. 76

77 (e) Non-leaf node, redistribute with neighbor 77

78 (e) Redistribute (non-leaf) a b c d e f g Delete 20 Underflow! Merge d with e. 78

79 (e) Redistribute (non-leaf) a b c d e f g Delete 20 After merge, remove the key and ptr to the deleted node from the parent 79

80 (e) Redistribute (non-leaf) underflow! b c Can be merged? d f g a Delete 20 Underflow at b? Min 2 ptrs, currently 1. Merge b with c? Max 4 ptrs, 5 ptrs in total. If cannot be merged, redistribute the keys with a sibling. Redistribute b and c 80

81 (e) Redistribute (non-leaf) redistribute a b c d f g Delete Redistribution at a non-leaf node is done in two steps. Step 1: Temporarily, make the left node b overflow by pulling down the mid-key and moving everything to the left. 81

82 (e) Redistribute (non-leaf) redistribute a 99 b temporary overflow c d f g Delete 20 Step 2: Apply the overflow handling algorithm (the same algorithm used for B+tree insertion) to the overflowed node Detailed algorithm in the next slide 82

83 (e) Redistribute (non-leaf) redistribute a 99 b c d f g Delete 20 Step 2: overflow handling algorithm Pick the mid-key (say 90) in the node and move it to parent. Move everything to the right of 90 to the empty node c. 83

84 (e) Redistribute (non-leaf) a b c d f g Delete 20 Underflow at a? Min 2 ptrs, currently 3. Done 84

85 Important Points Remember: For leaf node merging, we delete the mid-key from the parent For non-leaf node merging/redistribution, we pull down the mid-key from their parent. Exact algorithm: Figure In practice Coalescing is often not implemented Too hard and not worth it 85

86 Where does n come from? n determined by Size of a node Size of search key Size of an index pointer Q: 1024B node, 10B key, 8B ptr n? 86

87 Question on B+tree SELECT * FROM Student WHERE sid > 60?

88 Summary on tree index Indexed sequential file (ISAM) Sparse vs. dense Primary (clustering) vs. secondary (nonclustering) Not suitable for dynamic environment B+trees Balanced, minimum space guarantee Insertion, deletion algorithms 88

89 Index Creation in SQL CREATE INDEX <index_name> ON <table>(<attr>,<attr>, ) Example CREATE INDEX st_id ON Student(sid) Creates a B+tree on the attributes Speeds up lookup on sid Clustering index (in DB2) CREATE INDEX cls_idx ON Student(sid) CLUSTER Tuples are sequenced by sid 89

90 Next topic Hash index Static hashing Extendible hashing 90

91 What is a Hash Table? Hash Table Hash function h(k): key integer [0 n] e.g., h( Susan ) = 7 Array for keys: T[0 n] Given a key k, store it in T[h(k)] h(susan) = 4 h(james) = 3 h(neil) = Neil James Susan 91

92 Hashing for DBMS (Static Hashing) 0 Disk blocks (buckets) search key h(key) 1 2 (key, record)

93 Overflow and Chaining Insert h(a) = 1 h(b) = 2 h(c) = 1 h(d) = 0 h(e) = 1 Delete h(b) = 2 h(c) =

94 Major Problem of Static Hashing How to cope with growth? Data tends to grow in size Overflow blocks unavoidable hash buckets overflow blocks

95 Extendible Hashing (two ideas) (a) Use i of b bits output by hash function h(k) b use i grows over time 95

96 Extendible Hashing (two ideas) (b) Use directory that maintains pointers to hash buckets (indirection) h(c) directory.. hash bucket c e 96

97 Example h(k) is 4 bits; 2 keys/bucket Insert 0111 i = 0 1 i = i =

98 Example Insert 1010 i = 0 1 i = i = overflow! Increase i of the bucket. Split it. 98

99 Example Insert 1010 i = i = i = overflow! Redistribute keys based on first i bits i = 2 99

100 Example Insert 1010 i = Update ptr in dir to new bkt 1? If no space, double directory size (increase i)

101 Example Insert 1010 i = 2 00 i = Copy pointers

102 Example Insert 1010 i = 2 00 i =

103 Example Insert 0000 i = Split bucket and increase i Overflow! 103

104 Example 2 Insert 0000 i = Redistribute keys Overflow! 104

105 Insert 0000 i = 2 00 Example Update ptr in directory

106 Insert 0000 i = Example

107 Insert Overflow! i = Split bucket, increase i, redistribute keys

108 Insert i = Update ptr in dir If no space, double directory

109 Insert 0011 i = i =

110 Insert 0011 i = i =

111 Extendible Hashing: Deletion Two options a) No merging of buckets b) Merge buckets and shrink directory if possible 111

112 Delete 1010 i = a b c 112

113 Delete 1010 i = a b c Can we merge a and b? b and c? 113

114 i = Delete 1010 Decrease i and merge buckets 2 Update ptr in directory a b c Q: Can we shrink directory? 114

115 Delete 1010 i = 0 1 i = a b

116 Bucket Merge Condition Bucket merge condition Bucket i s are the same First (i-1) bits of the hash key are the same Directory shrink condition All bucket i s are smaller than the directory i 116

117 Questions on Extendible Hashing Can we provide minimum space guarantee? 117

118 Space Waste i =

119 Static hashing Hash index summary Overflow and chaining Extendible hashing Can handle growing files No periodic reorganizations Indirection Up to 2 disk accesses to access a key Directory doubles in size Not too bad if the data is not too large 119

120 Hashing vs. Tree Can an extendible-hash index support? SELECT FROM R WHERE R.A > 5 Which one is better, B+tree or Extendible hashing? SELECT FROM R WHERE R.A = 5 120