Outline Purpose How to analyze algorithms Examples. Algorithm Analysis. Seth Long. January 15, 2010

Transcription

1 January 15, 2010

2 Intuitive Definitions Common Runtimes Final Notes

3 Compare space and time requirements for algorithms Understand how an algorithm scales with larger datasets

4 Intuitive Definitions Outline Intuitive Definitions Common Runtimes Final Notes Common Symbols: O (Big O) indicates an upper bound Ω indicates an upper bound Θ indicates an upper and lower bound That is, a comparable growth rate o (little o) indicates strictly an upper bound ω indicates strictly a lower bound O is most common, followed by Ω. When using these, leave out constants O(n 3 ), not O(2n 3 + n ) The smaller terms affect runtime, but not scaling

5 Math Definitions Outline Intuitive Definitions Common Runtimes Final Notes Let T (n) indicate the actual running time of the algorithm on dataset of size n Let f (n) be a function of the dataset size n T (n) = O(f (n)) iff there exist c and n 0 such that T (n) cf (n) when n n 0 T (n) = Ω(f (n)) iff there exist c and n0 such that T (n) cf (n) when n n 0 T (n) = Θ(f (n)) iff T (n) = O(f (n)) and T (n) = Ω(f (n)) T (n) = o(f (n)) iff c n 0 T (n) < cf (n) when n n 0 T (n) = ω(f (n)) iff c n 0 T (n) > cf (n) when n n 0 This does mean that if T (n) = O(n), T (n) = O(n 2 ) as well = is a little strange here (all orders equal?)

6 Graphs of Common Functions Intuitive Definitions Common Runtimes Final Notes

7 Final Notes Outline Intuitive Definitions Common Runtimes Final Notes Average case is hard to calculate (worst case normally used) Quicksort is O(n 2 ) in the worse case, but generally beats Mergesort (O(nlog(n))) The average case for Quicksort is O(nlog(n)) as well Best case may be relevant if it is common (using bubble sort on a previously-sorted list to which a small number of items have been added, for example) Complexity theory is based on this: Class P is all problems which can be solved in polynomial time Class NP (Non-deterministic Polynomial) is problems for which the solution can be verified (but not always found) in polynomial time P is a subset of NP NP-Complete problems are in NP but not P NP-Hard problems are at least as hard as the hardest NP-Complete problems

8 Bubble: O(n 2 ) for runtime O(1) for space (no extra space) O(n 2 ) copy operations Best case: Already sorted list (O(n)) Worse case: Sorted backwards Selection: O(n 2 ) for runtime O(1) for space O(n) copy operations! No best or worse case - all the same Some algorithms (for example, Mergesort) separate the list into multiple pieces, which requires extra memory

9 Binary Search Trees O(log(n)) for runtime Requires no extra space Compare to linear search Problems in which the solution is repeatedly halved are generally O(log(n))

10 This is (sadly) not a binary search tree Method #1: Depth First O(n) where n is number of items in the tree O(1) for space Method #2: Breadth First Solves problems with depth first O(n) for time O(w) where w is the width of the tree Note: The tree itself may not be stored! Consider 16-puzzle. What if the tree is infinitely deep? What if there are multiple solutions? Method #3: Iterative Depth First O(n) for time O(1) for space

11 Factorials: fact(x): if x = 1 return 1 return x * fact(x-1) This is O(n) for time, O(n) for space, unoptimized Consider the following algorithm for the Fibonacci sequence: fib(x): if x = 0 return 0 if x = 1 return 1 return fib(x-1) + fib(x-2) Running time? O( 3 2 n ) (basically, fib(n)) Space? Same thing

12 Towers of Hanoi Here s a harder one: Hanoi(n, A, C, B): if n!= 0: Hanoi(n - 1, A, B, C) print Move the plate from, A, to, C Hanoi(n - 1, B, C, A) This prints out the series of moves to solve the problem. In general, look for a c n sort of running time with multiple recursion. Like the opposite of a Binary Search Tree.

13 Sums It is possible to analyze the runtime to solve an equation without first converting to an algorithm. Example: The sums are loops Here is another: n i=0 j=0 i α ij β ij n log 2 i X [j] i=0 j=0