Algorithms. Algorithms. Algorithms. API elementary implementations. ordered operations. API elementary implementations. ordered operations

Size: px

Start display at page:

Download "Algorithms. Algorithms. Algorithms. API elementary implementations. ordered operations. API elementary implementations. ordered operations"

Buck Todd
5 years ago
Views:

1 lgorithms OBT DGWIK K VI W Y Data structures mart data structures and dumb code works a lot better than the other way around. ric. aymond 3. YBOL T BL PI elementary implementations lgorithms F O U T ordered operations D I T I O OBT DGWIK K VI W Y 2 ymbol tables Key-value pair abstraction. Insert a value with specified key. Given a key, search for the corresponding value. 3. YBOL T BL PI elementary implementations lgorithms OBT DGWIK K VI W Y ordered operations x. D lookup. Insert domain name with specified IP address. Given domain name, find corresponding IP address. domain name IP address key value 4

2 ymbol table applications ymbol tables: context application purpose of search key value dictionary find definition word definition book index find relevant pages term list of page numbers file share find song to download name of song computer ID financial account process transactions account number transaction details web search find relevant web pages keyword list of page names lso known as: maps, dictionaries, associative arrays. Generalizes arrays. Keys need not be between 0 and. Language support. xternal libraries:, VisualBasic, tandard L, bash,... Built-in libraries: Java, #, ++, cala,... Built-in to language: wk, Perl, PP, Tcl, Javacript, Python, uby, Lua. compiler find properties of variables variable name type and value routing table route Internet packets destination best route every array is an associative array every object is an associative array table is the only primitive data structure D find IP address domain name IP address reverse D find domain name IP address domain name genomics find markers D string known positions file system find file on disk filename location on disk hasiceyntaxforssociativerrays["python"] = True hasiceyntaxforssociativerrays["java"] = False legal Python code 5 6 Basic symbol table PI ssociative array abstraction. ssociate one value with each key. public class T<Key, Value> onventions Values are not null. ethod get() returns null if key not present. java.util.ap allows null values ethod put() overwrites old value with new value. T() create an empty symbol table void put(key key, Value val) put key-value pair into the table Value get(key key) value paired with key boolean contains(key key) is there a value paired with key? Iterable<Key> keys() all the keys in the table void delete(key key) remove key (and its value) from table boolean ismpty() is the table empty? int size() number of key-value pairs in the table a[key] = val; a[key] Intended consequences. asy to implement contains(). public boolean contains(key key) return get(key)!= null; an implement lazy version of delete(). public void delete(key key) put(key, null); 7 8

3 Keys and values quality test Value type. ny generic type. ll Java classes inherit a method equals(). Key type: several natural assumptions. ssume keys are omparable, use compareto(). ssume keys are any generic type, use equals() to test equality. ssume keys are any generic type, use equals() to test equality; use hashode() to scramble key. specify omparable in PI. Java requirements. For any references x, y and z: eflexive: x.equals(x) is true. ymmetric: x.equals(y) iff y.equals(x). Transitive: if x.equals(y) and y.equals(z), then x.equals(z). on-null: x.equals(null) is false. equivalence relation built-in to Java (stay tuned) do x and y refer to the same object? Best practices. Use immutable types for symbol table keys. Immutable in Java: Integer, Double, tring, java.io.file, utable in Java: tringbuilder, java.net.ul, arrays,... Default implementation. (x == y) ustomized implementations. Integer, Double, tring, java.io.file, User-defined implementations. ome care needed. 9 0 Implementing equals for user-defined types Implementing equals for user-defined types eems easy. eems easy, but requires some care. typically unsafe to use equals() with inheritance (would violate symmetry) public class Date implements omparable<date> private final int month; private final int day; private final int year;... public boolean equals(date that) public final class Date implements omparable<date> private final int month; private final int day; private final int year;... public boolean equals(object y) if (y == this) return true; if (y == null) return false; if (y.getlass()!= this.getlass()) return false; must be Object. Why? xperts still debate. optimize for true object equality check for null objects must be in the same class (religion: getlass() vs. instanceof) if (this.day!= that.day ) return false; if (this.month!= that.month) return false; if (this.year!= that.year ) return false; return true; check that all significant fields are the same Date that = (Date) y; if (this.day!= that.day ) return false; if (this.month!= that.month) return false; if (this.year!= that.year ) return false; return true; cast is guaranteed to succeed check that all significant fields are the same 2

quals design T test client for analysis "tandard" recipe for user-defined types. Optimization for reference equality. heck against null. heck that two objects are of the same type; cast.

4 quals design T test client for analysis "tandard" recipe for user-defined types. Optimization for reference equality. heck against null. heck that two objects are of the same type; cast. ompare each significant field: if field is a primitive type, use == if field is an object, use equals() if field is an array, apply to each entry Best practices. o need to use calculated fields that depend on other fields. ompare fields mostly likely to differ first. ake compareto() consistent with equals(). apply rule recursively can use rrays.deepquals(a, b) but not a.equals(b) x.equals(y) if and only if (x.compareto(y) == 0) but use Double.compare() with double (to deal with -0.0 and a) e.g., cached anhattan distance Frequency counter. ead a sequence of strings from standard input and print out one that occurs with highest frequency. % more tinytale.txt it was the best of times it was the worst of times it was the age of wisdom it was the age of foolishness it was the epoch of belief it was the epoch of incredulity it was the season of light it was the season of darkness it was the spring of hope it was the winter of despair % java Frequencyounter 3 < tinytale.txt the 0 % java Frequencyounter 8 < tale.txt business 22 % java Frequencyounter 0 < leipzig.txt government tiny example (60 words, 20 distinct) real example (35,635 words, 0,769 distinct) real example (2,9,455 words, 534,580 distinct) 3 4 Frequency counter implementation public class Frequencyounter public static void main(tring[] args) int minlen = Integer.parseInt(args[0]); T<tring, Integer> st = new T<tring, Integer>(); while (!tdin.ismpty()) tring word = tdin.readtring(); ignore short strings if (word.length() < minlen) continue; if (!st.contains(word)) st.put(word, ); else st.put(word, st.get(word) + ); create T read string and update frequency lgorithms 3. YBOL TBL PI elementary implementations ordered operations tring max = ""; print a string with max frequency st.put(max, 0); for (tring word : st.keys()) if (st.get(word) > st.get(max)) max = word; tdout.println(max + " " + st.get(max)); OBT DGWIK KVI WY 5

5 equential search in a linked list lementary T implementations: summary key value Data structure. aintain an (unordered) linked list of key-value pairs. 0 earch. can through all keys until find a match. 2 Insert. 3 can 3 through 2 all keys until 0 find a match; if no match add to front. key value first get("") P 0 5 P 0 6 L 5 L put("", L 9) first red nodes are new 0 0 black nodes are accessed in search red nodes are 3 new 2 0 circled entries are black nodes 0 changed values 0 are accessed in search gray nodes are untouched circled 6 entries are changed values P P gray nodes are untouched 0 Trace 7 of linked-list 5 T implementation 4 3 for standard 8 indexing 6 client 0 implementation sequential search (unordered list) guarantee average case search insert search hit insert operations on keys equals() P 0 L P 0 L P hallenge. fficient implementations of both search and insert. 2 L P Trace of linked-list T implementation for standard indexing client Binary search in an ordered array Binary search in an ordered array Data structure. aintain parallel arrays for keys and values, sorted by keys. Data structure. aintain parallel arrays for keys and values, sorted by keys. earch. Use binary search to find key. earch. Use binary search to find key. Proposition. t most ~ lg compares to search a sorted array of length. keys[] vals[] keys[] successful search for P successful 7 8 search 9 for P keys[] lo hi m L P lo hi m get("p") successful 0 search 9 for 4 P L 5 6 P entries in black L P entries in black are a[lo..hi] are a[lo..hi] lo 5 hi 9 m7 L P L P L P entries in black are a[lo..hi] L P entry in red is a[m] L P entry in red is a[m] L P L P unsuccessful search for Q loop exits unsuccessful search with keys[m] = P: return 6 entry for in Q loop exits with keys[m] = P: return 6 red is a[m] lo 6 hi 6 m 6 L P lo hi m unsuccessful 0 search 9 4 for Q L P loop exits 0 with 9 4 keys[m] = P: return 6 L P lo 5 hi 9 m7 L P L P L P return 5 6 vals[6] 5 L P L P L P L P loop exits with lo > hi: return 7 loop exits with lo > hi: return L P loop Trace exits of with binary lo > search hi: return for rank 7 in an ordered array Trace of binary search for rank in an ordered array 9 public Value get(key key) int lo = 0, hi = -; while (lo <= hi) int mid = lo + (hi - lo) / 2; int cmp = key.compareto(keys[mid]); if (cmp < 0) hi = mid - ; else if (cmp > 0) lo = mid + ; else if (cmp == 0) return vals[mid]; return null; no matching key 20

6 lementary symbol tables: quiz FID T FIT Implementing binary search was. asier than I thought. B. bout what I expected.. arder than I thought. D. uch harder than I thought. Problem. Given an array with all 0s in the beginning and all s at the end, find the index in the array where the s begin. input I don't know. Variant. You are given the length of the array. Variant 2. You are not given the length of the array Binary search: insert lementary T implementations: summary Data structure. aintain an ordered array of key-value pairs. Insert. Use binary search to find place to insert; shift all larger keys over. Proposition. Takes linear time in the worst case. implementation guarantee average case search insert search hit insert operations on keys put("p", 0) keys[] vals[] P sequential search (unordered list) binary search (ordered array) equals() log log compareto() can do with log compares, but requires array accesses hallenge. fficient implementations of both search and insert

7 xamples of ordered symbol table PI keys values lgorithms OBT DGWIK KVI WY 3. YBOL TBL PI elementary implementations ordered operations min() get(09:00:3) floor(09:05:00) select(7) keys(09:5:00, 09:25:00) ceiling(09:30:00) max() 09:00:00 hicago 09:00:03 Phoenix 09:00:3 ouston 09:00:59 hicago 09:0:0 ouston 09:03:3 hicago 09:0: eattle 09:0:25 eattle 09:4:25 Phoenix 09:9:32 hicago 09:9:46 hicago 09:2:05 hicago 09:22:43 eattle 09:22:54 eattle 09:25:52 hicago 09:35:2 hicago 09:36:4 eattle 09:37:44 Phoenix size(09:5:00, 09:25:00) is 5 rank(09:0:25) is 7 26 Ordered symbol table PI K I OTD Y public class T<Key extends omparable<key>, Value> Key min() smallest key Problem. Given a sorted array of distinct keys, find the number of keys strictly less than a given query key. Key max() largest key Key floor(key key) largest key less than or equal to key Key ceiling(key key) smallest key greater than or equal to key int rank(key key) number of keys less than key Key select(int k) key of rank k 27 28

8 Binary search: ordered symbol table operations summary lgorithms OBT DGWIK KVI WY sequential search binary search search insert log 3.2 BIY T min / max floor / ceiling rank select log log lgorithms F O U T D I T I O BTs ordered operations iteration deletion (see book) OBT DGWIK KVI WY order of growth of the running time for ordered symbol table operations 30 Binary search trees lgorithms OBT DGWIK KVI WY BIY T BTs ordered operations iteration deletion Definition. BT is a binary tree in symmetric order. binary tree is either: mpty. Two disjoint binary trees (left and right). ymmetric order. ach node has a key, and every node s key is: Larger than all keys in its left subtree. maller than all keys in its right subtree. a subtree left link of a left link parent of and keys smaller than null links 9 root right child of root key keys larger than value associated with 3

9 Binary search tree demo Binary search tree demo earch. If less, go left; if greater, go right; if equal, search hit. Insert. If less, go left; if greater, go right; if null, insert. successful search for insert G G 4 5 BT representation in Java BT implementation (skeleton) Java definition. BT is a reference to a root ode. ode is composed of four fields: Key and a Value. reference to the left and right subtree. public class BT<Key extends omparable<key>, Value> private ode root; private class ode /* see previous slide */ root of BT smaller keys larger keys public void put(key key, Value val) /* see next slides */ private class ode private Key key; private Value val; private ode left, right; public ode(key key, Value val) this.key = key; this.val = val; BT ode key left val right BT with smaller keys BT with larger keys Binary search tree public Value get(key key) /* see next slides */ public Iterable<Key> iterator() /* see slides in next section */ public void delete(key key) /* see textbook */ Key and Value are generic types; Key is omparable 6 7

BT search: Java implementation BT insert Get. eturn value corresponding to given key, or null if no such key. public Value get(key key) ode x = root; while (x!= null) int cmp = key.compareto(x.

10 BT search: Java implementation BT insert Get. eturn value corresponding to given key, or null if no such key. public Value get(key key) ode x = root; while (x!= null) int cmp = key.compareto(x.key); if (cmp < 0) x = x.left; else if (cmp > 0) x = x.right; else if (cmp == 0) return x.val; return null; ost. umber of compares = + depth of node. Put. ssociate value with key. earch for key, then two cases: Key in tree reset value. Key not in tree add new node. inserting L search for L ends at this null link create new node reset links on the way up L L P P P Insertion into a BT 8 9 BT insert: Java implementation Put. ssociate value with key. Tree shape any BTs correspond to same set of keys. umber of compares for search/insert = + depth of node. public void put(key key, Value val) root = put(root, key, val); private ode put(ode x, Key key, Value val) if (x == null) return new ode(key, val); int cmp = key.compareto(x.key); if (cmp < 0) x.left = put(x.left, key, val); else if (cmp > 0) x.right = put(x.right, key, val); else if (cmp == 0) x.val = val; return x; best case typical case worst case Warning: concise but tricky code; read carefully! ost. umber of compares = + depth of node. Bottom line. Tree shape depends on order of insertion. 0

11 BT insertion: random order visualization Binary search trees: quiz x. Insert keys in random order. Given distinct keys, what is the name of this sorting algorithm?. huffle the keys. 2. Insert the keys into a BT, one at a time. 3. Do an inorder traversal of the BT.. Insertion sort. B. ergesort.. Quicksort. D. one of the above.. I don't know. 2 3 orrespondence between BTs and quicksort partitioning BTs: mathematical analysis P Proposition. If distinct keys are inserted into a BT in random order, the expected number of compares for a search/insert is ~ 2 ln. Pf. correspondence with quicksort partitioning. T D O U Proposition. [eed, 2003] If distinct keys are inserted into a BT I Y in random order, the expected height is ~ 4.3 ln. L expected depth of function-call stack in quicksort ow Tall is a Tree? Bruce eed, Paris, France reed@moka.ccr.jussieu.fr BTT Let ~ be the height of a random binary search tree on n nodes. We show that there exists constants a = and/3 = such that (~) = c~logn -/3oglogn + O(), We also show that Var(~) = O(). emark. orrespondence is if array has no duplicate keys. But Worst-case height is. [ exponentially small chance when keys are inserted in random order ] 4 5

12 T implementations: summary implementation sequential search (unordered list) binary search (ordered array) guarantee average case search insert search hit insert operations on keys equals() log log compareto() BT log log compareto() lgorithms 3.2 BIY T BTs iteration ordered operations deletion OBT DGWIK KVI WY Why not shuffle to ensure a (probabilistic) guarantee of log? 6 Binary search trees: quiz 2 Inorder traversal In what order does the traverse(root) code print out the keys in the BT? private void traverse(ode x) if (x == null) return; traverse(x.left); tdout.println(x.key); traverse(x.right);. B.. D.. I don't know. inorder() inorder() inorder() print inorder() print done done print inorder() inorder() print inorder() print done done print done done print inorder() print done done output: 8 9

13 Inorder traversal Traverse left subtree. nqueue key. Traverse right subtree. public Iterable<Key> keys() Queue<Key> q = new Queue<Key>(); inorder(root, q); return q; private void inorder(ode x, Queue<Key> q) if (x == null) return; inorder(x.left, q); q.enqueue(x.key); inorder(x.right, q); BT key val left right BT with smaller keys BT with larger keys smaller keys, in order key larger keys, in order all keys, in order LVL-OD TVL Level-order traversal of a binary tree. Process root. Process children of root, from left to right. Process grandchildren of root, from left to right. T Property. Inorder traversal of a BT yields keys in ascending order. level order traversal: T 20 2 LVL-OD TVL LVL-OD TVL Q. Given binary tree, how to compute level-order traversal? Q2. Given level-order traversal of a BT, how to (uniquely) reconstruct BT? x. T T queue.enqueue(root); while (!queue.ismpty()) ode x = queue.dequeue(); if (x == null) continue; tdout.println(x.item); queue.enqueue(x.left); queue.dequeue(x.right); T level order traversal: T 22 23

14 inimum and maximum inimum. mallest key in BT. aximum. Largest key in BT. 3.2 BIY T lgorithms BTs iteration ordered operations deletion min () m max OBT DGWIK KVI WY Q. ow to find the min / max? 25 Floor and ceiling omputing the floor Floor. Largest key in BT query key. eiling. mallest key in BT query key. Floor. Largest key in BT k? ase. [ key in node x = k ] The floor of k is k. finding floor() floor(g) () floor(d) Q. ow to find the floor / ceiling? m ceiling(q) 26 ase 2. [ key in node x > k ] The floor of k is in the left subtree of x. ase 3. [ key in node x < k ] The floor of k can't be in left subtree of x: it is either in the right subtree of x or it is the key in node x. finding floor(g) is less than G so floor(g) could be on the right is greater than G so floor(g) must be on the left 27 omputing the floor function

15 omputing the floor ank and select public Key floor(key key) return floor(root, key); private Key floor(ode x, Key key) if (x == null) return null; int cmp = key.compareto(x.key); if (cmp == 0) return x; if (cmp < 0) return floor(x.left, key); Key t = floor(x.right, key); if (t!= null) return t; else return x.key; finding floor(g) G is greater than so floor(g) could be on the right floor(g)in left subtree is null G is less than so floor(g) must be on the left Q. ow to implement rank() and select() efficiently for BTs?. In each node, store the number of nodes in its subtree. 2 subtree count result BT implementation: subtree counts omputing the rank private class ode private Key key; private Value val; private ode left; private ode right; private int count; public int size() return size(root); private int size(ode x) if (x == null) return 0; return x.count; ok to call when x is null ank. ow many keys in BT < k? ase. [ k < key in node ] o key in right subtree < k; some keys in left subtree < k. ase 2. [ k > key in node ] node count number of nodes in subtree ll keys in left subtree < k; private ode put(ode x, Key key, Value val) initialize subtree count to if (x == null) return new ode(key, val, ); int cmp = key.compareto(x.key); if (cmp < 0) x.left = put(x.left, key, val); else if (cmp > 0) x.right = put(x.right, key, val); else if (cmp == 0) x.val = val; x.count = + size(x.left) + size(x.right); return x; the key in the node is < k; some keys in right subtree may be < k. ase 3. [ k = key in node ] ll keys in left subtree < k; no key in right subtree < k. 30 3

16 ank BT: ordered symbol table operations summary ank. ow many keys in BT < k? asy recursive algorithm (3 cases!) node count search insert min / max sequential search binary search BT log h h h h = height of BT (proportional to log if keys inserted in random order) public int rank(key key) return rank(key, root); floor / ceiling log h private int rank(key key, ode x) if (x == null) return 0; int cmp = key.compareto(x.key); if (cmp < 0) return rank(key, x.left); else if (cmp > 0) return + size(x.left) + rank(key, x.right); else if (cmp == 0) return size(x.left); rank select ordered iteration log log h h order of growth of running time of ordered symbol table operations T implementations: summary implementation guarantee average case search insert search hit insert ordered ops? key interface sequential search (unordered list) equals() binary search (ordered array) log log compareto() BT log log compareto() red-black BT log log log log compareto() ext lecture. Guarantee logarithmic performance for all operations. 34

Algorithms. Algorithms. Algorithms 3.1 SYMBOL TABLES. API elementary implementations ordered operations

Algorithms. Algorithms. Algorithms 3.1 SYMBOL TABLES. API elementary implementations ordered operations OBT DGWIK KVIN WYN lgorithms OBT DGWIK KVIN WYN Data structures 3.1 YBOL TBL lgorithms F O U T D I T I O N PI elementary implementations mart data structures and dumb code works a lot better than the other