Algorithms. Algorithms 3.3 BALANCED SEARCH TREES. 2 3 search trees red black BSTs B-trees (see book or videos) ROBERT SEDGEWICK KEVIN WAYNE

Size: px

Start display at page:

Download "Algorithms. Algorithms 3.3 BALANCED SEARCH TREES. 2 3 search trees red black BSTs B-trees (see book or videos) ROBERT SEDGEWICK KEVIN WAYNE"

Anne Burke
6 years ago
Views:

1 lgorithms OBT DGWICK KVIN WYN 3.3 BLNCD C T lgorithms F O U T D I T I O N 2 3 search trees red black BTs B-trees (see book or videos) OBT DGWICK KVIN WYN Last updated on 10/17/17 9:09 M

2 ymbol table review implementation guarantee search insert delete search hit average case insert delete ordered ops? key interface sequential search (unordered list) binary search (ordered array) n n n n n n equals() log n n n log n n n compareto() BT n n n log n log n n compareto() goal log n log n log n log n log n log n compareto() Challenge. Guarantee performance. optimized for teaching and coding; introduced to the world in this course! This lecture. 2 3 trees and left-leaning red black BTs. 3

3 3.3 BLNCD C T lgorithms 2 3 search trees red black BTs B-trees OBT DGWICK KVIN WYN

4 2 3 tree llow 1 or 2 keys per node. 2-node: one key, two children. 3-node: two keys, three children. ymmetric order. Inorder traversal yields keys in ascending order. erfect balance. very path from root to null link has same length. M how to maintain? 3-node 2-node smaller than J larger than J C L X between and J null link 5

5 2 3 tree demo earch. Compare search key against key(s) in node. Find interval containing search key. Follow associated link (recursively). search for M J C L X 6

6 2 3 tree: insertion Insertion into a 2-node at bottom. dd new key to 2-node to create a 3-node. insert G L L C X C G X 7

7 2 3 tree: insertion Insertion into a 3-node at bottom. dd new key to 3-node to create temporary 4-node. Move middle key in 4-node into parent. epeat up the tree, as necessary. If you reach the root and it s a 4-node, split it into three 2-nodes. insert Z L L X C X C Z 8

8 2 3 tree construction demo insert 9

9 2 3 tree construction demo 2 3 tree L C X 10

10 2 3 tree: global properties Invariants. Maintains symmetric order and perfect balance. f. ach transformation maintains symmetric order and perfect balance. root a b c a b c parent is a 3-node left d e b d e a b c a c parent is a 2-node left a b c d a b d c middle a e b c d a c e b d right a a c right a b a b d b c d b d c d e c e 11

11 2 3 tree: performance plitting a 4-node is a local transformation: constant number of operations. a e b c d less than a between a and b between b and c between c and d between d and e greater than e a c e b d less than a between a and b between b and c between c and d between d and e greater than e 12

12 Balanced search trees: quiz 1 What is the maximum height of a 2 3 tree with n keys?. ~ log3 n B. ~ log2 n C. ~ 2 log2 n D. ~ n 13

13 2 3 tree: performance erfect balance. very path from root to null link has same length. Tree height. Worst case: lg n. Best case: log [all 2-nodes] 3 n.631 lg n. [all 3-nodes] Between 12 and 20 for a million nodes. Between 18 and 30 for a billion nodes. Bottom line. Guaranteed logarithmic performance for search and insert. 14

14 T implementations: summary implementation guarantee search insert delete search hit average case insert delete ordered ops? key interface sequential search (unordered list) n n n n n n equals() binary search (ordered array) log n n n log n n n compareto() BT n n n log n log n n compareto() 2 3 tree log n log n log n log n log n log n compareto() but hidden constant c is large (depends upon implementation) 15

15 2 3 tree: implementation? Direct implementation is complicated, because: Maintaining multiple node types is cumbersome. Need multiple compares to move down tree. Need to move back up the tree to split 4-nodes. Large number of cases for splitting. fantasy code public void put(key key, Value val) { Node x = root; Beautiful while (x.getthecorrectchild(key) algorithms are not always!= null) the most useful. } { x = x.getthecorrectchildkey(); Donald Knuth if (x.is4node()) x.split(); } if (x.is2node()) x.make3node(key, val); else if (x.is3node()) x.make4node(key, val); Bottom line. Could do it, but there s a better way. 16

16 3.3 BLNCD C T lgorithms 2 3 search trees red black BTs B-trees OBT DGWICK KVIN WYN

17 ow to implement 2 3 trees with binary trees? Challenge. ow to represent a 3 node? pproach 1. egular BT. No way to tell a 3-node from a 2-node. Cannot map from BT back to 2 3 tree. pproach 2. egular BT with red glue nodes. Wastes space for extra node. Code probably messy. pproach 3. egular BT with red glue links. Widely used in practice. rbitrary restriction: red links lean left. 18

18 Left-leaning red black BTs (Guibas-edgewick 1979 and edgewick 2007) 1. epresent 2 3 tree as a BT. 2. Use internal left-leaning links as glue for 3 nodes. 3-node a b b larger key is root a less than a between a and b greater than b less than a between a and b greater than b ncoding a 3-node with two 2-nodes red links glue nodes within a 3-node black links connect 2-nodes and 3-nodes J M ck tree J L M X C L X C 2 3 tree corresponding red black BT 19

19 Left-leaning red black BTs: 1 1 correspondence with 2 3 trees Key property. 1 1 correspondence between 2 3 and LLB. red black tree J M C L X horizontal red links C J L M X 2-3 tree M J C L X 20

20 n equivalent definition BT such that: No node has two red links connected to it. very path from root to null link has the same number of black links. ed links lean left. perfect black balance ck tree M J C L X 21

21 Balanced search trees: quiz 2 Which is a red black BT?. 5 B C. 5 D

22 earch implementation for red black BTs Observation. earch is the same as for elementary BT (ignore color). public Value get(key key) { Node x = root; while (x!= null) { int cmp = key.compareto(x.key); if (cmp < 0) x = x.left; else if (cmp > 0) x = x.right; else if (cmp == 0) return x.val; } return null; } but runs faster (because of better balance) ck tree emark. Many other ops (floor, iteration, rank, selection) are also identical. C J L M X 23

23 ed black BT representation ach node is pointed to by precisely one link (from its parent) can encode color of links in nodes. private static final boolean D = true; private static final boolean BLCK = false; private class Node { Key key; Value val; Node left, right; h.left.color is D C D h G J h.right.color is BLCK } boolean color; // color of parent link private boolean ised(node x) { if (x == null) return false; return x.color == D; } null links are black 24

24 Insertion into a LLB tree: overview Basic strategy. Maintain 1 1 correspondence with 2 3 trees. During internal operations, maintain: ymmetric order. erfect black balance. [ but not necessarily color invariants ] right-leaning red link two red children (a temporary 4-node) To restore color invariant: apply elementary ops (rotations and color flips). left-left red (a temporary 4-node) left-right red (a temporary 4-node) 25

25 lementary red black BT operations Left rotation. Orient a (temporarily) right-leaning red link to lean left. rotate left (before) less than h between and x greater than private Node rotateleft(node h) { assert ised(h.right); Node x = h.right; h.right = x.left; x.left = h; x.color = h.color; h.color = D; return x; } Invariants. Maintains symmetric order and perfect black balance. 26

26 lementary red black BT operations Left rotation. Orient a (temporarily) right-leaning red link to lean left. rotate left (after) less than h between and x greater than private Node rotateleft(node h) { assert ised(h.right); Node x = h.right; h.right = x.left; x.left = h; x.color = h.color; h.color = D; return x; } Invariants. Maintains symmetric order and perfect black balance. 27

27 lementary red black BT operations ight rotation. Orient a left-leaning red link to (temporarily) lean right. rotate right (before) x less than between and h greater than private Node rotateight(node h) { assert ised(h.left); Node x = h.left; h.left = x.right; x.right = h; x.color = h.color; h.color = D; return x; } Invariants. Maintains symmetric order and perfect black balance. 28

28 lementary red black BT operations ight rotation. Orient a left-leaning red link to (temporarily) lean right. rotate right (after) less than x between and h greater than private Node rotateight(node h) { assert ised(h.left); Node x = h.left; h.left = x.right; x.right = h; x.color = h.color; h.color = D; return x; } Invariants. Maintains symmetric order and perfect black balance. 29

29 lementary red black BT operations Color flip. ecolor to split a (temporary) 4-node. flip colors less (before) between h between greater private void flipcolors(node h) { assert!ised(h); assert ised(h.left); assert ised(h.right); h.color = D; h.left.color = BLCK; h.right.color = BLCK; } than and and than Invariants. Maintains symmetric order and perfect black balance. 30

30 lementary red black BT operations Color flip. ecolor to split a (temporary) 4-node. flip colors (after) less between h between greater private void flipcolors(node h) { assert!ised(h); assert ised(h.left); assert ised(h.right); h.color = D; h.left.color = BLCK; h.right.color = BLCK; } than and and than Invariants. Maintains symmetric order and perfect black balance. 31

31 Insertion into a LLB tree Warmup 1. Insert into a tree with exactly 1 node. left a b b root search ends at this null link root red link to new node containing a converts 2-node to 3-node right a a a search ends at this null link attached new node with red link b b root root rotated left to make a legal 3-node 32

32 Insertion into a LLB tree Case 1. Insert into a 2-node at the bottom. Do standard BT insert; color new link red. If new red link is a right link, rotate left. to maintain symmetric order and perfect black balance to restore color invariants insert C add new node here right link red so rotate left C C C Insert into a 2-node 33

33 Insertion into a LLB tree Warmup 2. Insert into a tree with exactly 2 nodes. larger a a a b b b c search ends at this null link attached new node with red link colors flipped to black c smaller a a a b b search ends at this null link b b c c attached new node with red link c rotated right b colors flipped to black c ed between a a a a a b b b b c search ends at this null link c c attached new node with red link rotated left c rotated right colors flipped to black c 34

34 Insertion into a LLB tree Case 2. Insert into a 3-node at the bottom. Do standard BT insert; color new link red. otate to balance the 4-node (if needed). Flip colors to pass red link up one level. otate to make lean left (if needed). to maintain symmetric order and perfect black balance to restore color invariants inserting C add new node here two lefts C both children red so flip colors node here C two lefts in a row so rotate right right link red so rotate left C C Insert into a 3-node right link red 35

35 Insertion into a LLB tree: passing red links up the tree Case 2. Insert into a 3-node at the bottom. inserting Do standard BT insert; color new link red. otate to balance the 4-node (if needed). Flip colors to pass red link up one Clevel. otate to make lean left (if needed). epeat case 1 or case 2 up the tree (if needed). C M add new node here right link red so rotate left C M C M inserting C right link red both so rotate children left red so flip colors C C M M M M add new node here two lefts in a row so rotate right both children red so flip colors both children red so flip colors C M add new node here both children red so flip colors right link red C M both children so rotate left red so to maintain symmetric order flip colors and perfect black balance C Mright link red so rotate left two lefts in a rowto so rotate right C restore M color invariants C C C both children red so flip colors C M M two lefts in a row so rotate right M C assing a red link up C the tree M assing a red link up the tree both children red so flip colors M M M 36

36 ed black BT construction demo insert C X M L M C L X 37

37 Insertion into a LLB tree: Java implementation Can distill down to three cases! ight child red; left child black: rotate left. Left child red; left left grandchild red: rotate right. Both children red: flip colors. h h left rotate h right rotate flip colors private Node put(node h, Key key, Value val) { if (h == null) return new Node(key, val, D); int cmp = key.compareto(h.key); if (cmp < 0) h.left = put(h.left, key, val); else if (cmp > 0) h.right = put(h.right, key, val); else if (cmp == 0) h.val = val; if (ised(h.right) &&!ised(h.left)) h = rotateleft(h); if (ised(h.left) && ised(h.left.left)) h = rotateight(h); if (ised(h.left) && ised(h.right)) flipcolors(h); assing a red link up a red-black tree insert at bottom (and color it red) lean left balance 4-node split 4-node } return h; only a few extra lines of code provides near-perfect balance 38

38 Insertion into a LLB tree: visualization 255 insertions in ascending order 39

39 Insertion into a LLB tree: visualization 255 insertions in descending order 40

40 Insertion into a LLB tree: visualization 255 random insertions 41

41 Balanced search trees: quiz 4 What is the maximum height of a LLB tree with n keys?. ~ log3 n B. ~ log2 n C. ~ 2 log2 n D. ~ n 42

42 Balance in LLB trees roposition. eight of tree is 2 lg n in the worst case. f. Black height = height of corresponding 2 3 tree lg n. Never two red links in-a-row. mpirical observation. eight of tree is ~ 1.0 lg n in typical applications. 43

43 T implementations: summary implementation guarantee search insert delete search hit average case insert delete ordered ops? key interface sequential search (unordered list) n n n n n n equals() binary search (ordered array) log n n n log n n n compareto() BT n n n log n log n n compareto() 2 3 tree log n log n log n log n log n log n compareto() red black BT log n log n log n log n log n log n compareto() hidden constant c is small (at most 2 lg n compares) 44

44 TDD T Threaded set. Implement the following I: public class Threadedet Threadedet() void add(tring s) create an empty threaded set add the string to the set (if it is not already in the set) boolean contains(tring s) is the string s in the set? tring previouskey(tring s) the string added to the set immediately before s (null if s is the first string added or s not in set) erformance requirement. log n time per operation (worst case). 45

$War story: why red black? Xerox C innovations. [1970s] lto. GUI. thernet. malltalk. Laser printing. Bitmapped display. WYIWYG text editor.... Xerox lto DIClIOlV1TIC FUl\l\V()K Fon BLNCD T Leo J.$ $BTUCT I() this paper we present a uniform framework for the implementation and study of halanced tree algorithms. \Ve show how to imhcd in this the way down towards a leaf.$

45 War story: why red black? Xerox C innovations. [1970s] lto. GUI. thernet. malltalk. Laser printing. Bitmapped display. WYIWYG text editor.... Xerox lto DIClIOlV1TIC FUl\l\V()K Fon BLNCD T Leo J. Guibas.Xerox alo lto esearch Center, alo lto, California, and Carnegie-fellon University and obert edgewick* rogram in Computer cience Brown University rovidence,. I. BTUCT I() this paper we present a uniform framework for the implementation and study of halanced tree algorithms. \Ve show how to imhcd in this the way down towards a leaf. s we will see, this has a number of significant advantages ovcr the older methods. We shall cxamine a numhcr of variations on a common theme and exhibit full implementations which are notable for their brcvity. One imp1cn1entation is exatnined carefully, and some properties about its 47

xtended telephone service outage. should allow for 2 40 keys Main cause = height bound exceeded!

46 War story: red black BTs Telephone company contracted with database provider to build real-time database to store customer information. Database implementation. ed black BT. xceeding height limit of 80 triggered error-recovery process. xtended telephone service outage. should allow for 2 40 keys Main cause = height bound exceeded! Telephone company sues database provider. Legal testimony: If implemented properly, the height of a red black BT with n keys is at most 2 lg n. expert witness 48

Balanced trees in the wild ed black trees are widely used as

duler, linux/rbtree.h. macs: conservative stack scanning.

B-trees (and cousins) are widely used for file systems and

47 Balanced trees in the wild ed black trees are widely used as system symbol tables. Java: java.util.treemap, java.util.treeet. C++ TL: map, multimap, multiset. Linux kernel: completely fair scheduler, linux/rbtree.h. macs: conservative stack scanning. Other balanced BTs. VL trees, splay trees, randomized BTs,. B-trees (and cousins) are widely used for file systems and databases. Windows: NTF. Mac: F, F+. Linux: eiserf, XF, xt3f, JF, BTF. Databases: OCL, DB2, ING, QL, ostgreql. 49

48 ed black BTs in the wild Common sense. ixth sense. Together they're the FBI's newest team. 50

Algorithms. Algorithms 3.3 BALANCED SEARCH TREES. 2 3 search trees red black BSTs ROBERT SEDGEWICK KEVIN WAYNE.

Algorithms. Algorithms 3.3 BALANCED SEARCH TREES. 2 3 search trees red black BSTs ROBERT SEDGEWICK KEVIN WAYNE. lgorithms OBT DGWICK KVIN WYN 3.3 BLNCD C T 2 3 search trees red black BTs lgorithms F O U T D I T I O N OBT DGWICK KVIN WYN http://algs4.cs.princeton.edu Last updated on 10/11/16 9:22 M BT: ordered symbol