Selection, Bubble, Insertion, Merge, Heap, Quick Bucket, Radix

Transcription

1 Spring 2010

2 Review Topics Big O Notation Heaps Sorting Selection, Bubble, Insertion, Merge, Heap, Quick Bucket, Radix Hashtables Tree Balancing: AVL trees and DSW algorithm Graphs: Basic terminology and topological sort No GUIs on exam

3 Big O Notation f(n) is O(g(n)) if (c, n 0 ) such that n n 0, f(n) c g(n) there exists; for all (c, n 0 ) is called the witness pair f(n) is O(g(n)) sort of means that f(n) g(n) Meaning of n 0 60 mph car vs. a 40 mph with a head start Function could start behind but grow faster The faster car/function takes over at n 0 Ignore what happens at small values of n

4 Big O Notation Meaning of c We can measure a car s speed accurately Hard to measure an algorithm s speed What about two different computers? n running time, 3 GHz CPU vs. 2n running time, 1 GHz CPU Is the algorithm faster or the computer faster? Treat n, 2n, 0.5n, etc. the exact same Ignore the leading coefficient O(n) algorithm will run faster than an O(n 2 ) algorithm Even if a supercomputer runs the O(n 2 ) algorithm

5 Heaps Heaps are binary trees Every node is smaller than its two children (minheap) Smallest element is always at the top Heaps grow top to bottom Each level is filled left to right Heaps can be represented with arrays Children at indices 2*i + 1, 2*i + 2 Parent at index (i 1)/2 Adding/removing in a heap grows/shrinks array by 1

6 Heaps Adding to a heap Add the element to the first empty spot in the heap Swap up as long as the parent is larger Removing from a heap Remove the root, replace with the last node in the heap Swap down as long as either child is smaller Always pick the smaller of two children Heaps are used to implement priority queues add in O(log n) time, removemin in O(log n) time

7 Sorting Many ways to sort Same high level goal, same result What s the difference? Running time Best case, average case, worst case Extra space used

8 Sorting Swap operation: swap(x, i, j) temp = x[i] x[i] = x[j] x[j] = temp Many sorts are a fancy set of swap instructions Modifies the array in place, very space efficient Not space efficient to copy a large array

9 Selection Sort for (i = 0 n 2) Scan array from index i to the end Swap smallest element into index i E.g.: 1, 3, 5, 8, 9, 6, 7 1, 3, 5, 6, 9, 8, 7 After k iterations, first k elements absolutely sorted 1, 3, 5, 6 are sorted, rest of list is larger than 6 O(n 2 ) time, O(1) extra space O(n 2 ) time for any list, including sorted lists

10 Bubble Sort for (i = n 2 0) for (j = 0 i) Compare elements at indices j, j+1, swap if necessary E.g.: 2, 1, 8, 7, 3, 9 1, 2, 8, 7, 3, 9 1, 2, 8, 7, 3, 9 1, 2, 7, 8, 3, 9 1, 2, 7, 3, 8, 9 After k iterations, last k elements are absolutely sorted Largest element bubbles to the end in each iteration O(n 2 ) time, O(1) extra space O(n 2 ) time for any list, including sorted lists

11 Insertion Sort for (i = 1 n 1) Take element at index i, swap it back one spot until you hit the beginning of the list previous element is smaller than this one E.g.: 4, 7, 8, 6, 2, 9 4, 7, 6, 8, 2, 9 4, 6, 7, 8, 2, 9 After k iterations, first k elements are relatively sorted 4, 6, 7, 8 sorted, but 2 is the smallest in the list O(n 2 ) time, O(1) extra space O(n) time for sorted lists or nearly sorted lists

12 Merge Sort Copy left half, right half into two smaller arrays Recursively run merge sort each half Base case: 1 element array Merge two sorted halves back into original array E.g.: (1, 3, 6, 9), (2, 5, 7, 8) (1, 2, 3, 5, 6, 7, 8, 9) Running Time: O(n log n) Merge takes O(n) time Split the list in half about log(n) times Also uses O(n) extra space!

13 Heap Sort Use a max heap (represented as an array) Can move items up/down the heap with swaps for (i = 1 n 1) Add element at index i to the heap First part builds the heaps for (i = n 1 1) Remove largest element, put it in spot i O(n log n) time, O(1) extra space

14 Quick Sort Randomly pick a pivot Partition into two (unequal) halves Left partition smaller than pivot, right partition larger Recursively run quick sort on both partitions Expected Running Time: O(n log n) Partition takes O(n) time Best case: pivot is the median O(n log n) Worse case: pivot is the smallest/largest O(n 2 ) O(1) extra space: partition can be done in place

15 Bucket Sort Aforementioned sorts are comparison based sorts Work on any type E extends Comparable<E> Best any comparison sort can do is O(n log n) time Bucket sort limited to integers from 0 to m 1 Create m buckets, each starting with a value of 0 Increment bucket i if i is found in the array Reconstruct the array from the buckets O(n + m) time, O(m) extra space

16 Radix Sort Bucket sort by 1 s digit, 10 s digit, 100 s digit, etc Buckets now store a list of numbers with same digit Preserve the order of numbers with the same digit E.g.: 42, 13, 77, 21, 41, 12 21, 41, 42, 12, 13, 77 12, 13, 21, 41, 42, 77 (radix is 10) Numbers have radix r and k digits O(k(n + r)) running time, O(r) extra space Bucket sorting numbers from 0 to m 1: radix m, 1 digit

17 Hashtables Looking for a data structure with O(1) time to add, find, and remove an element Java s HashMap<K, V>, HashSet<E> First try n integers between 0 and m 1 The answer looks similar to bucket sort Limitations: Integers must have a limited range What about any comparable data type (e.g.: String)? Java generics: E extends Comparable<E>

18 Hashtables Hashtable: store n objects in an array of size m Hash function maps each object to one of m buckets Elements no longer sorted as in bucket sort Two different objects could go in the same bucket Chaining: store a linked list in each bucket (Also linear probing, quadratic probing) Good hash function has O(1) collisions Bad hash function will ruin the hashtable!

19 Hashtables Table Size If too large, we waste space If too small, everything collides with each other Resize table when n/m exceeds load factor λ (0 < λ 1) Double the table size, re add everything Worst case: add n items, double table on item n Doubling takes n + n/2 + n/4 + n/8 + < 2n time Some adds expensive, but the average is still O(1) time Table doubling also useful for growing an ArrayList

20 Tree Balancing

21 Graphs A graph has vertices (also called nodes) A graph has edges between two vertices n number of vertices; m number of edges Directed vs. undirected graph Directed edges can only be traversed one way Undirected edges can be traversed both way Weighted vs. unweighted graph Edges could have weights/costs assigned to them

22 Graphs Degree: number of edges touching a vertex Directed graphs have indegree, outdegree Indegree: number of edges entering a vertex Outdegree: number of edges leaving a vertex Cycles: path from a vertex back to itself Each edge in the cycle traversed only once Acyclic undirected graph is a tree Acyclic directed graph is a DAG

23 Graphs Adjacency matrix A(i, j) = 1 if there is an edge from vertex i to vertex j Could be something other than 1 in weighted graphs A(i, j) = A(j, i) in undirected graphs A(i, j) = 0 if there is no edge O(n 2 ) space Adjacency list Each vertex stores a list of adjacent vertices Also store weight in a weighted graph O(n + m) space good for sparse graphs

24 Topological Sort Topological sort is for directed graphs Topological sort algorithm: Delete a vertex with an indegree of 0 Delete its outgoing edges, too Repeat until no vertices have an indegree of 0 A topological sort cannot delete cycles Every node in a cycle has an indegree of 1 Need to delete another node in the cycle first A graph is DAG iff a topological sort deletes it iff if and only if

25 Topological Sort B A B C E D A E C D