Algorithms. Algorithms. Algorithms 3.1 SYMBOL TABLES. API elementary implementations ordered operations

Size: px

Start display at page:

Download "Algorithms. Algorithms. Algorithms 3.1 SYMBOL TABLES. API elementary implementations ordered operations"

Erik Shepherd
5 years ago
Views:

1 lgorithms OBT DGWIK KVI WY 3.1 YBOL TBL 3.1 YBOL TBL lgorithms F O U T D I T I O PI elementary implementations lgorithms PI elementary implementations OBT DGWIK KVI WY OBT DGWIK KVI WY ymbol tables ymbol table applications Key-value pair abstraction. Insert a value with specified key. Given a key, search for the corresponding value. x. D lookup. Insert UL with specified IP address. Given UL, find corresponding IP address. application purpose of search key value dictionary find definition word definition book index find relevant pages term list of page numbers file share find song to download name of song computer ID financial account process transactions account number transaction details web search find relevant web pages keyword list of page names UL IP address compiler find properties of variables variable name type and value routing table route Internet packets destination best route D find IP address given UL UL IP address reverse D find UL given IP address IP address UL genomics find markers D string known positions file system find file on disk filename location on disk key value 3 4

2 Basic symbol table PI ssociative array abstraction. ssociate one value with each key. public class T<Key, Value> T() void put(key key, Value val) Value get(key key) create a symbol table put key-value pair into the table (remove key from table if value is null) value paired with key (null if key is absent) void delete(key key) remove key (and its value) from table boolean contains(key key) is there a value paired with key? boolean ismpty() is the table empty? int size() number of key-value pairs in the table Iterable<Key> keys() all the keys in the table PI for a generic basic symbol table a[key] = val; a[key] onventions Values are not null. ethod get() returns null if key not present. ethod put() overwrites old value with new value. Intended consequences. asy to implement contains(). public boolean contains(key key) return get(key)!= null; an implement lazy version of delete(). public void delete(key key) put(key, null); 5 6 Keys and values quality test Value type. ny generic type. ll Java classes inherit a method equals(). Key type: several natural assumptions. ssume keys are omparable, use compareto(). ssume keys are any generic type, use equals() to test equality. ssume keys are any generic type, use equals() to test equality; use hashode() to scramble key. specify omparable in PI. Java requirements. For any references x, y and z: eflexive: x.equals(x) is true. ymmetric: x.equals(y) iff y.equals(x). Transitive: if x.equals(y) and y.equals(z), then x.equals(z). on-null: x.equals(null) is false. equivalence relation built-in to Java (stay tuned) Best practices. Use immutable types for symbol table keys. Immutable in Java: Integer, Double, tring, java.io.file, utable in Java: tringbuilder, java.net.ul, arrays,... Default implementation. (x == y) do x and y refer to the same object? ustomized implementations. Integer, Double, tring, java.io.file, User-defined implementations. ome care needed. 7 8

3 Implementing equals for user-defined types Implementing equals for user-defined types eems easy. eems easy, but requires some care. typically unsafe to use equals() with inheritance (would violate symmetry) public class Date implements omparable<date> private final int month; private final int day; private final int year;... public boolean equals(date that) public final class Date implements omparable<date> private final int month; private final int day; private final int year;... public boolean equals(object y) if (y == this) return true; if (y == null) return false; if (y.getlass()!= this.getlass()) return false; must be Object. Why? xperts still debate. optimize for true object equality check for null objects must be in the same class (religion: getlass() vs. instanceof) if (this.day!= that.day ) return false; if (this.month!= that.month) return false; if (this.year!= that.year ) return false; return true; check that all significant fields are the same 9 Date that = (Date) y; if (this.day!= that.day ) return false; if (this.month!= that.month) return false; if (this.year!= that.year ) return false; return true; cast is guaranteed to succeed check that all significant fields are the same 10 quals design T test client for traces "tandard" recipe for user-defined types. Optimization for reference equality. heck against null. heck that two objects are of the same type and cast. ompare each significant field: if field is a primitive type, use == if field is an object, use equals() if field is an array, apply to each entry Best practices. o need to use calculated fields that depend on other fields. ompare fields mostly likely to differ first. ake compareto() consistent with equals(). apply rule recursively alternatively, use rrays.equals(a, b) or rrays.deepquals(a, b), but not a.equals(b) x.equals(y) if and only if (x.compareto(y) == 0) Build T by associating value i with i th string from standard input. public static void main(tring[] args) T<tring, Integer> st = new T<tring, Integer>(); for (int i = 0;!tdIn.ismpty(); i++) tring key = tdin.readtring(); st.put(key, i); for (tring s : st.keys()) tdout.println(s + " " + st.get(s)); keys values P L output L 11 9 P

4 T test client for analysis Frequency counter implementation Frequency counter. ead a sequence of strings from standard input and print out one that occurs with highest frequency. % more tinytale.txt it was the best of times it was the worst of times it was the age of wisdom it was the age of foolishness it was the epoch of belief it was the epoch of incredulity it was the season of light it was the season of darkness it was the spring of hope it was the winter of despair % java Frequencyounter 1 < tinytale.txt it 10 % java Frequencyounter 8 < tale.txt business 122 % java Frequencyounter 10 < leipzig1.txt government tiny example (60 words, 20 distinct) real example (135,635 words, 10,769 distinct) real example (21,191,455 words, 534,580 distinct) public class Frequencyounter public static void main(tring[] args) int minlen = Integer.parseInt(args[0]); T<tring, Integer> st = new T<tring, Integer>(); while (!tdin.ismpty()) tring word = tdin.readtring(); ignore short strings if (word.length() < minlen) continue; if (!st.contains(word)) st.put(word, 1); else st.put(word, st.get(word) + 1); tring max = ""; st.put(max, 0); for (tring word : st.keys()) if (st.get(word) > st.get(max)) max = word; tdout.println(max + " " + st.get(max)); create T read string and update frequency print a string with max freq equential search in a linked list Data structure. aintain an (unordered) linked list of key-value pairs. lgorithms OBT DGWIK KVI WY 3.1 YBOL TBL PI elementary implementations earch. can through all keys until find a match. Insert. can through all keys until find a match; if no match add to front. key value P 10 L first P 10 L 11 L 11 red nodes are new black nodes are accessed in search circled entries are changed values gray nodes are untouched P P Trace of linked-list T implementation for standard indexing client 16

5 lementary T implementations: summary Binary search in an ordered array Data structure. aintain an ordered array of key-value pairs. T implementation sequential search (unordered list) worst-case cost (after inserts) average case (after random inserts) search insert search hit insert ordered iteration? hallenge. fficient implementations of both search and insert. key interface / 2 no equals() ank helper function. ow many keys < k? successful search for P lo hi m L P unsuccessful search for Q keys[] L P L P L P lo hi m L P L P L P L P loop exits with lo > hi: return 7 entries in black are a[lo..hi] entry in red is a[m] loop exits with keys[m] = P: return 6 Trace of binary search for rank in an ordered array Binary search: Java implementation Binary search: trace of standard indexing client public Value get(key key) if (ismpty()) return null; int i = rank(key); if (i < && keys[i].compareto(key) == 0) return vals[i]; else return null; private int rank(key key) number of keys < key int lo = 0, hi = -1; while (lo <= hi) int mid = lo + (hi - lo) / 2; int cmp = key.compareto(keys[mid]); if (cmp < 0) hi = mid - 1; else if (cmp > 0) lo = mid + 1; else if (cmp == 0) return mid; return lo; Problem. To insert, need to shift all greater keys over. key value keys[] vals[] entries in red were inserted entries in gray did not move entries in black moved to the right circled entries are changed values P 10 P L 11 L P L P L P

6 lementary T implementations: summary T implementation worst-case cost (after inserts) average case (after random inserts) search insert search hit insert ordered iteration? key interface 3.1 YBOL TBL sequential search (unordered list) binary search (ordered array) / 2 no equals() log log / 2 yes compareto() lgorithms PI elementary implementations OBT DGWIK KVI WY hallenge. fficient implementations of both search and insert. 21 xamples of ordered symbol table PI Ordered symbol table PI min() get(09:00:13) floor(09:05:00) select(7) keys(09:15:00, 09:25:00) ceiling(09:30:00) max() size(09:15:00, 09:25:00) is 5 rank(09:10:25) is 7 keys values 09:00:00 hicago 09:00:03 Phoenix 09:00:13 ouston 09:00:59 hicago 09:01:10 ouston 09:03:13 hicago 09:10:11 eattle 09:10:25 eattle 09:14:25 Phoenix 09:19:32 hicago 09:19:46 hicago 09:21:05 hicago 09:22:43 eattle 09:22:54 eattle 09:25:52 hicago 09:35:21 hicago 09:36:14 eattle 09:37:44 Phoenix public class T<Key extends omparable<key>, Value> void Value T() put(key key, Value val) get(key key) create an ordered symbol table put key-value pair into the table (remove key from table if value is null) value paired with key (null if key is absent) void delete(key key) remove key (and its value) from table boolean contains(key key) is there a value paired with key? boolean ismpty() is the table empty? int size() number of key-value pairs Key min() smallest key Key max() largest key Key floor(key key) largest key less than or equal to key Key ceiling(key key) smallest key greater than or equal to key int rank(key key) number of keys less than key Key select(int k) key of rank k void deletein() delete smallest key void deleteax() delete largest key int size(key lo, Key hi) number of keys in [lo..hi] Iterable<Key> keys(key lo, Key hi) keys in [lo..hi], in sorted order Iterable<Key> keys() all keys in the table, in sorted order 23 PI for a generic ordered symbol table 24

Binary search: ordered symbol table operations summary lgorithms OBT DGWIK KVI WY sequential search binary search search insert / delete min / max floor / ceiling rank lg 1 lg lg lgorithms F O U T D

7 Binary search: ordered symbol table operations summary lgorithms OBT DGWIK KVI WY sequential search binary search search insert / delete min / max floor / ceiling rank lg 1 lg lg lgorithms F O U T D I T I O 3.2 BIY T BTs deletion select 1 OBT DGWIK KVI WY ordered iteration lg order of growth of the running time for ordered symbol table operations 25 Binary search trees lgorithms OBT DGWIK KVI WY 3.2 BIY T BTs deletion Definition. BT is a binary tree in symmetric order. binary tree is either: mpty. Two disjoint binary trees (left and right). ymmetric order. ach node has a key, and every node s key is: Larger than all keys in its left subtree. maller than all keys in its right subtree. a subtree left link of a left link parent of and keys smaller than null links 9 root right child of root key keys larger than value associated with 3

8 BT representation in Java BT implementation (skeleton) Java definition. BT is a reference to a root ode. ode is comprised of four fields: Key and a Value. reference to the left and right subtree. public class BT<Key extends omparable<key>, Value> private ode root; private class ode /* see previous slide */ root of BT smaller keys larger keys public void put(key key, Value val) /* see next slides */ private class ode private Key key; private Value val; private ode left, right; public ode(key key, Value val) this.key = key; this.val = val; BT ode key left val right BT with smaller keys BT with larger keys Binary search tree public Value get(key key) /* see next slides */ public void delete(key key) /* see next slides */ public Iterable<Key> iterator() /* see next slides */ Key and Value are generic types; Key is omparable 4 5 Binary search tree demo Binary search tree demo earch. If less, go left; if greater, go right; if equal, search hit. Insert. If less, go left; if greater, go right; if null, insert. successful search for insert G G 6 7

9 BT search: Java implementation BT insert Get. eturn value corresponding to given key, or null if no such key. public Value get(key key) ode x = root; while (x!= null) int cmp = key.compareto(x.key); if (cmp < 0) x = x.left; else if (cmp > 0) x = x.right; else if (cmp == 0) return x.val; return null; Put. ssociate value with key. earch for key, then two cases: Key in tree reset value. Key not in tree add new node. inserting L search for L ends at this null link create new node reset links on the way up L L P P P ost. umber of compares is equal to 1 + depth of node. Insertion into a BT 8 9 BT insert: Java implementation Put. ssociate value with key. public void put(key key, Value val) root = put(root, key, val); concise, but tricky, recursive code; read carefully! Tree shape any BTs correspond to same set of keys. umber of compares for search/insert is equal to 1 + depth of node. private ode put(ode x, Key key, Value val) if (x == null) return new ode(key, val); int cmp = key.compareto(x.key); if (cmp < 0) x.left = put(x.left, key, val); else if (cmp > 0) x.right = put(x.right, key, val); else if (cmp == 0) x.val = val; return x; best case typical case worst case ost. umber of compares is equal to 1 + depth of node. emark. Tree shape depends on order of insertion

BT insertion: random order visualization orrespondence between BTs and quicksort partitioning x. Insert keys in random order. P T D O U I Y L emark.

If distinct keys are inserted into a BT in random order, the expected number of compares for a search/insert is ~ 2 ln. Pf. 1-1 correspondence with quicksort partitioning. Proposition.

10 BT insertion: random order visualization orrespondence between BTs and quicksort partitioning x. Insert keys in random order. P T D O U I Y L emark. orrespondence is 1-1 if array has no duplicate keys BTs: mathematical analysis T implementations: summary Proposition. If distinct keys are inserted into a BT in random order, the expected number of compares for a search/insert is ~ 2 ln. Pf. 1-1 correspondence with quicksort partitioning. Proposition. [eed, 2003] If distinct keys are inserted in random order, expected height of tree is ~ ln. ow Tall is a Tree? Bruce eed, Paris, France reed@moka.ccr.jussieu.fr implementation sequential search (unordered list) binary search (ordered array) guarantee average case ordered operations ops? on keys search insert search hit insert /2 no equals() lg lg /2 yes compareto() BTT Let ~ be the height of a random binary search tree on n nodes. We show that there exists constants a = and/3 = such that (~) = c~logn -/31oglogn + O(1), We also show that Var(~) = O(1). BT 1.39 lg 1.39 lg next compareto() But Worst-case height is. (exponentially small chance when keys are inserted in random order) 14 15

11 inimum and maximum inimum. mallest key in table. aximum. Largest key in table. 3.2 BIY T lgorithms BTs deletion min () max OBT DGWIK KVI WY Q. ow to find the min / max? 17 Floor and ceiling omputing the floor Floor. Largest key a given key. eiling. mallest key a given key. floor(g) () floor(d) ceiling(q) ase 1. [k equals the key at root] The floor of k is k. ase 2. [k is less than the key at root] The floor of k is in the left subtree. ase 3. [k is greater than the key at root] The floor of k is in the right subtree (if there is any key k in right subtree); otherwise it is the key in the root. finding floor(g) G is greater than so floor(g) could be on the right floor(g)in left subtree is null G is less than so floor(g) must be on the left Q. ow to find the floor / ceiling? 18 result 19

12 omputing the floor ubtree counts public Key floor(key key) ode x = floor(root, key); if (x == null) return null; return x.key; private ode floor(ode x, Key key) if (x == null) return null; int cmp = key.compareto(x.key); if (cmp == 0) return x; if (cmp < 0) return floor(x.left, key); ode t = floor(x.right, key); if (t!= null) return t; else return x; finding floor(g) G is greater than so floor(g) could be on the right floor(g)in left subtree is null G is less than so floor(g) must be on the left In each node, we store the number of nodes in the subtree rooted at that node; to implement size(), return the count at the root. node count result 20 emark. This facilitates efficient implementation of rank() and select(). 21 BT implementation: subtree counts ank private class ode private Key key; private Value val; private ode left; private ode right; private int count; public int size() return size(root); private int size(ode x) if (x == null) return 0; return x.count; ok to call when x is null ank. ow many keys < k? asy recursive algorithm (3 cases!) node count number of nodes in subtree private ode put(ode x, Key key, Value val) if (x == null) return new ode(key, val); int cmp = key.compareto(x.key); if (cmp < 0) x.left = put(x.left, key, val); else if (cmp > 0) x.right = put(x.right, key, val); else if (cmp == 0) x.val = val; x.count = 1 + size(x.left) + size(x.right); return x; public int rank(key key) return rank(key, root); private int rank(key key, ode x) if (x == null) return 0; int cmp = key.compareto(x.key); if (cmp < 0) return rank(key, x.left); else if (cmp > 0) return 1 + size(x.left) + rank(key, x.right); else if (cmp == 0) return size(x.left); 22 23

return; inorder(x.left, q); q.enqueue(x.key); inorder(x.

13 Inorder traversal BT: ordered symbol table operations summary Traverse left subtree. nqueue key. Traverse right subtree. search sequential search binary search BT lg h public Iterable<Key> keys() Queue<Key> q = new Queue<Key>(); inorder(root, q); return q; private void inorder(ode x, Queue<Key> q) if (x == null) return; inorder(x.left, q); q.enqueue(x.key); inorder(x.right, q); BT key val left right BT with smaller keys BT with larger keys smaller keys, in order key larger keys, in order all keys, in order insert min / max floor / ceiling rank select ordered iteration h 1 h lg h lg h 1 h log h = height of BT (proportional to log if keys inserted in random order) Property. Inorder traversal of a BT yields keys in ascending order. order of growth of running time of ordered symbol table operations T implementations: summary implementation guarantee average case search insert delete search hit insert delete ordered iteration? operations on keys lgorithms 3.2 BIY T BTs deletion sequential search (linked list) binary search (ordered array) /2 /2 no equals() lg lg /2 /2 yes compareto() BT 1.39 lg 1.39 lg??? yes compareto() OBT DGWIK KVI WY ext. Deletion in BTs. 27

14 BT deletion: lazy approach Deleting the minimum To remove a node with a given key: et its value to null. Leave key in tree to guide search (but don't consider it equal in search). To delete the minimum key: Go left until finding a node with a null left link. eplace that node by its right link. Update subtree counts. go left until reaching null left link I delete I tombstone public void deletein() root = deletein(root); return that node s right link available for garbage collection ost. ~ 2 ln ' per insert, search, and delete (if keys in random order), where ' is the number of key-value pairs ever inserted in the BT. Unsatisfactory solution. Tombstone (memory) overload. private ode deletein(ode x) if (x.left == null) return x.right; x.left = deletein(x.left); x.count = 1 + size(x.left) + size(x.right); return x; update links and node counts after recursive calls ibbard deletion ibbard deletion To delete a node with key k: search for node t containing key k. To delete a node with key k: search for node t containing key k. ase 0. [0 children] Delete t by setting parent link to null. ase 1. [1 child] Delete t by replacing parent link. deleting node to delete replace with null link available for garbage collection update counts after recursive calls deleting update counts after recursive calls 5 node to delete replace with child link available for garbage collection

15 ibbard deletion ibbard deletion: Java implementation To delete a node with key k: search for node t containing key k. ase 2. [2 children] Find successor x of t. Delete the minimum in t's right subtree. Put x in t's spot. node to delete t go right, then go left until reaching null left link x search for key successor min(t.right) reaching null left link t.left x 5 x has no left child but don't garbage collect x still a BT deletein(t.right) 7 update links and node counts after recursive calls public void delete(key key) root = delete(root, key); private ode delete(ode x, Key key) if (x == null) return null; int cmp = key.compareto(x.key); if (cmp < 0) x.left = delete(x.left, key); else if (cmp > 0) x.right = delete(x.right, key); else if (x.right == null) return x.left; if (x.left == null) return x.right; ode t = x; x = min(t.right); x.right = deletein(t.right); x.left = t.left; x.count = size(x.left) + size(x.right) + 1; return x; search for key no right child no left child replace with successor update subtree counts ibbard deletion: analysis T implementations: summary Unsatisfactory solution. ot symmetric. implementation guarantee average case search insert delete search hit insert delete ordered iteration? operations on keys sequential search (linked list) /2 /2 no equals() binary search (ordered array) lg lg /2 /2 yes compareto() BT 1.39 lg 1.39 lg yes compareto() other operations also become if deletions allowed urprising consequence. Trees not random (!) sqrt () per op. Longstanding open problem. imple and efficient delete for BTs. ext lecture. Guarantee logarithmic performance for all operations

ELEMENTARY SEARCH ALGORITHMS

ELEMENTARY SEARCH ALGORITHMS BB - GOIT TODY DT. OF OUT GIIG KUT D ymbol Tables I lementary implementations Ordered operations TY GOIT ar., cknowledgement:.the$course$slides$are$adapted$from$the$slides$prepared$by$.$edgewick$ and$k.$wayne$of$rinceton$university.