Algorithm Analysis. Part I. Tyler Moore. Lecture 3. CSE 3353, SMU, Dallas, TX

Transcription

1 Algorithm Analysis Part I Tyler Moore CSE 5, SMU, Dallas, TX Lecture how many times do you have to turn the crank? Some slides created by or adapted from Dr. Kevin Wayne. For more information see Some slides adapted from Dr. Steven Skiena. For more information see / 6 Brute force. For many nontrivial problems, there is a natural brute-force search algorithm that checks every possible solution. Typically takes time or worse for inputs of size. Unacceptable in practice. Desirable scaling property. When the input size doubles, the algorithm should only slow down by some constant factor. Def. An algorithm is poly-time if the above scaling property holds. choose C = d 4 5 / 6 4 / 6

2 We say that an algorithm is efficient if has a polynomial running time. Justification. It really works in practice! In practice, the poly-time algorithms that people develop have low constants and low exponents. Breaking through the exponential barrier of brute force typically exposes some crucial structure of the problem. Worst case. Running time guarantee for any input of size. Generally captures efficiency in practice. Draconian view, but hard to find effective alternative. Exceptions. Some exponential-time algorithms are used widely in practice because the worst-case instances seem to be rare. Exceptions. Some poly-time algorithms do have high constants and/or exponents, and/or are useless in practice. Q. Which would you prefer vs.? / 6 6 / 6 Worst case. Running time guarantee for any input of size. Ex. Heapsort requires at most compares to sort elements. Probabilistic. Expected running time of a randomized algorithm. Ex. The expected number of compares to quicksort elements is. Amortized. Worst-case running time for any sequence of operations. Ex. Starting from an empty stack, any sequence of push and pop operations takes operations using a resizing array. Average-case. Expected running time for a random input of size. Ex. The expected number of character compares performed by -way radix quicksort on uniformly random strings is. Also. Smoothed analysis, competitive analysis, / 6 8 / 6

3 Upper bounds. is if there exist constants and such that for all. Ex.. is. is also. choose c = 50, n0 = is neither nor. Typical usage. Insertion makes compares to sort elements. Equals sign. is a set of functions, but computer scientists often write instead of. Ex. Consider and. We have Thus,. Domain. The domain of is typically the natural numbers. Sometimes we restrict to a subset of the natural numbers. Other times we extend to the reals. Alternate definition. is if lim sup n T (n) f(n) <. Nonnegative functions. When using big-oh notation, we assume that the functions involved are (asymptotically) nonnegative. Bottom line. OK to abuse notation; not OK to misuse it. 9 / 6 0 / 6 Lower bounds. is Ω if there exist constants and such that for all. Ex.. is both Ω and Ω. is neither Ω nor Ω. choose c =, n0 = Tight bounds. is Θ if there exist constants,, and such that for all. Ex.. is Θ. is neither Θ nor Θ. choose c =, c = 50, n0 = Typical usage. Any compare-based sorting algorithm requires Ω compares in the worst case. Typical usage. Mergesort makes Θ compares to sort elements. Meaningless statement. Any compare-based sorting algorithm requires at least compares in the worst case. / 6 4 / 6

4 Big Oh Examples Big Omega Examples Definition Definition T (n) is O(f (n)) if there exist constants c > 0 and n0 0 such that T (n) c f (n) for all n n0. T (n) is Ω(f (n)) if there exist constants c > 0 and n0 0 such that T (n) c f (n) for all n n0. n 00n + 6 = O(n )? n Yes, because for c = and no, n > n 00n n + 6 = O(n )? Yes, because for c = 0.0 and no 0, 0.0n > n 00n + 6 n 00n + 6 = O(n)? n 00n + 6 = Ω(n )? n Yes, because for c = and no 00, n < n 00n n + 6 = Ω(n )? No, because for c = and no >, n 00n + 6 < n n 00n + 6 = Ω(n)? Yes, because for c = and n0 00, n < n 00n + 6 No, because c n < n when n > c / 6 Big Theta Examples 4 / 6 Exercises Definition T (n) is Θ(f (n)) if there exist constants c > 0, c > 0 and n0 0 such that c f (n) T (n) c f (n) for all n n0. n 00n + 6 = Θ(n )? Yes, because O and Ω apply n + 4n = O(n ) n + 0n = Ω(n ) n 00n + 6 = Θ(n )? Pick a suitable c and n0 to show that n 4n = Ω(n ) No, because only O applies n 00n + 6 = Θ(n)? No, because only Ω applies 5 / 6 6 / 6

5 Big Oh Addition/Subtraction Big Oh Multiplication Suppose f (n) = O(n ) and g(n) = O(n ). What do we know about g (n) = f (n) + g(n)? Adding bounding constants shows g (n) = O(n ) What do we know about g (n) = f (n) g(n)? Since bounding constants may not cancel, g (n) = O(n ) What about lower bounds? Does g (n) = Ω(n ) We know nothing about lower bounds on g and g because we don t know about the lower bounds on f and g. Multiplication by a constant does not change the asymptotics O(c f (n)) O(f (n)) Ω(c f (n)) Ω(f (n)) Θ(c f (n)) Θ(f (n)) But when both functions in a product are increasing, both are important O(f (n)) O(g(n)) O(f (n) g(n)) Ω(f (n)) Ω(g(n)) Ω(f (n) g(n)) Θ(f (n)) Θ(g(n)) Θ(f (n) g(n)) 7 / 6 8 / 6 Logarithms Upper bounds. is if there exist constants,, and such that for all and. Ex.. is both and. is neither nor. Typical usage. Breadth-first search takes time to find the shortest path from to in a digraph. It is important to understand deep in your bones what logarithms are and where they come from. A logarithm is simply an inverse exponential function. Saying b x = y is equivalent to saying that x = log b y. Logarithms reflect how many times we can double something until we get to n, or halve something until we get to. 7 0 / 6

6 Binary Search and Logarithms Logarithms and Binary Trees In binary search we throw away half the possible number of keys after each comparison. Thus twenty comparisons suffice to find any name in the million-name Manhattan phone book! Question: how many times can we halve n before getting to? How tall a binary tree do we need until we have n leaves? The number of potential leaves doubles with each level. How many times can we double until we get to n? / 6 / 6 Logarithms and Bits Logarithms and Multiplication How many bits do you need to represent the numbers from 0 to i? Recall that log a (xy) = log a (x) + log a (y) This is how people used to multiply before calculators, and remains useful for analysis. What if x = a? / 6 4 / 6

7 The Base is not Asymptotically Important Federal Sentencing Guidelines F.. Fraud and Deceit; Forgery; Offenses Involving Altered or Counterfeit Instruments other than Counterfeit Bearer Obligations of the United States. (a) Base offense Level: 6 (b) Specific offense Characteristics () If the loss exceeded $,000, Recall the definition, c log c x = x and that So for a = and c = 00: Since Big Oh log 00 log b a = log c a log c b log n = log 00 n log 00 = 6.64 is a constant, we can ignore it when calculating 5 / 6 increase the offense level as follows: Loss(Apply the Greatest) Increase in Level (A) $,000 or less no increase (B) More than $,000 add (C) More than $5,000 add (D) More than $0,000 add (E) More than $0,000 add 4 (F) More than $40,000 add 5 (G) More than $70,000 add 6 (H) More than $0,000 add 7 (I) More than $00,000 add 8 (J) More than $50,000 add 9 (K) More than $500,000 add 0 (L) More than $800,000 add (M) More than $,500,000 add (N) More than $,500,000 add (O) More than $5,000,000 add 4 (P) More than $0,000,000 add 5 (Q) More than $0,000,000 add 6 (R) More than $40,000,000 add 7 (Q) More than $80,000,000 add 8 The increase in punishment level grows logarithmically in the amount of money stolen. Thus it pays to commit one big crime rather than many small crimes totaling the same amount. In other words, Make the Crime Worth the Time 6 / 6