Big-O-ology. How do you tell how fast a program is? = How do you tell how fast a program is? (2nd Try) Answer? Run some test cases.

Transcription

1 How do you tell how fast a program is? Answer? Run some test cases Jim Royer January 16, 2019 CIS 675 Problem You can ly run a few test cases There will be many inputs you w t test and BAD things may be happening that stuff FIX? Test all possible inputs! With a 100 element int array as input with 32 bit ints, there are = 3200 bits in the input Each bit can be 0 or 1 So there possible inputs where HA! CIS 675 1/ 19 CIS 675 2/ = So you can t test that many inputs! CIS 675 3/ 19 How do you tell how fast a program is? (2nd Try) ANOTHER ANSWER: Do some run time analysis A simple, sample method public static int findmin(int a[]) { // RETURNS: the minimal value in the array a int n = alength; int minval = a[0]; for (int j=1; j < n; j++) if (a[j]<minval) { minval = a[j]; return minval; Q: How much time does this take? CIS 675 4/ 19

2 How much time does findmin take? Dealing with Problem 1: Different size arrays produce different run times Problems with the questi: 1 Different size arrays will produce different run times You would expect findmin to take lger a length array than a length 3 array 2 On a particular input, you will see different run times under: different machines different Java compilers different Java run-time systems Obvious fix: Make the answer depend the length of the array: T(n) = the run time for a length n array Finding T(n) exactly is usually hard But an estimate is almost always good enough WARNING: Oversimplified CIS 675 5/ 19 CIS 675 6/ 19 Dealing with Problem 2: Different setups produce different run times Assumpti 1: Proportiality Given two setups, there are cstants c low and c high such that c low the run-time for setup 1 a size n input the run-time for setup 2 a size n input c high Suppose we have two computers, an old WONDERBOX 4 and a new WONDERBOX CLOUD 9 We expect programs the newer machine to be between 25 and 38 times faster than the old e Dealing with Problem 2: Different setups produce different run times Assumpti 2: Straight-line code takes cstant time Straight-line code = code with no loops or recursis The cstant depends the setup (of machine, compiler, ) (Code: straight-line and otherwise) if (a[j] < minval) { minval = a[j]; n = alength; minval = a[0]; for (j=1; j < n; j++) if (a[j] < minval) { minval = a[j]; WARNING: Also oversimplified CIS 675 7/ 19 CIS 675 8/ 19

3 Back to: How much time does findmin take? public static int findmin(int a[]) { int n = alength, minval = a[0]; for (int j=1; j < n; j++) if (a[j] < minval) { minval = a[j]; return minval; ( ) is the straight line code that is buried deepest in for-loops For a given value of n, ( ) is executed n 1 times So, for a given setup, there are cstants c low and c high c low (n 1) the run time of findmin c high (n 1) Use testing to get estimates for c low and c high ( ) public static void selectisort(int a[]) { // find minj {i,,n-1 a[minj] = min {a[j] : i j n-1 if (a[j]<minval) { minj = j; minval = a[j]; // Now: a[0] a[1] a[i] min {a[j] : i < j n-1 // end of for-loop // end of selectisort CIS 675 9/ 19 CIS / 19 public static void selectisort(int a[]) { // find minj {i,,n-1 a[minj] = min {a[j] : i j n-1 if (a[j]<minval) { minj = j; minval = a[j]; // Now: a[0] a[1] a[i] min {a[j] : i < j n-1 // end of for-loop // end of selectisort public static void selectisort(int a[]) { // find minj {i,,n-1 a[minj] = min {a[j] : i j n-1 if (a[j]<minval) { minj = j; minval = a[j]; // Now: a[0] a[1] a[i] min {a[j] : i < j n-1 // end of for-loop // end of selectisort Step 1: Identify the pieces of straightline code that are buried deepest if (a[j]<minval) { minj = j; minval = a[j]; In selectisort this is: since it occurs within both loops Whereas both minj = i; minval = a[i]; a[minj] = a[i]; a[i] = minval; occur within ly the outermost loop Step 2: Count how many times these innermost pieces of the program are executed for a given value of n We handle the two loops in selectisort e at a time, starting with the innermost THE INNERMOST LOOP: This is So: if (a[j]<minval) {minj = j; minval = a[j]; the first iterati has j==i+1, the last iterati has j==n-1, and j increases by 1 with each iterati there are (n-1) - (i+1) +1 == n-i-1 many iteratis for particular values of i and n the innermost code is executed n-i-1 times every time the innermost for loop is executed

4 public static void selectisort(int a[]) { // find minj {i,,n-1 a[minj] = min {a[j] : i j n-1 if (a[j]<minval) { minj = j; minval = a[j]; // Now: a[0] a[1] a[i] min {a[j] : i < j n-1 // end of for-loop // end of selectisort public static void selectisort(int a[]) { // find minj {i,,n-1 a[minj] = min {a[j] : i j n-1 if (a[j]<minval) { minj = j; minval = a[j]; // Now: a[0] a[1] a[i] min {a[j] : i < j n-1 // end of for-loop // end of selectisort THE OUTERMOST LOOP: This is for (i=0; i<n-1; i++) { minj = i; minval = a[i]; the innermost loop a[minj] = a[i]; a[i] = minval; The first iterati thus has i = 0, the last iterati has i = n 2, and i increases by 1 with each iterati So, the number of times the innermost code is executed is: iterati # value of i # of executis = (n i 1) 1 0 n n n-3 n-3 n-4 3 n-2 n-3 2 n-1 n-2 1 Therefore, the total # of executis is (n 1) By a standard bit of math this last sum = (n 1) n 2 = 1 2 n2 1 2 n Step 3: Take the count from step 2 and cclude the order of the runtime any particular machine So, there are cstants c l, c u > 0 such that, for any array of length n, c l ( 1 2 n2 1 n) µ secs 2 the runtime of selectisort c u ( 1 2 n2 1 n) µ secs 2 For a slightly different choice of positive cstants c l and c u, we have c l n2 for sufficiently large n µ secs the runtime of selectisort c u n 2 µ secs A Third public static void insertisort(int a[]) { int i, j, key, n = alength; for (i=1; i < n; i++) { // Assume: a[0] a[i-1] Insert the value of a[i] // within a[0 (i-1)] making a[0] a[i] key = a[i]; j = i-1; while (j>=0 && a[j]>key) { a[j+1] = a[j]; j = j -1; a[j+1] = key; // Now a[0] a[1] a[i] CIS / 19 A Third InsertiSort is much faster some length-n inputs than others Eg, A Third 1 If initially a[0] < a[1] < < a[n 1], InsertiSort runs in Θ(n) time 2 If initially: a[0] > a[1] > > a[n 1], InsertiSort runs in Θ(n 2 ) time public static void insertisort(int a[]) { int i, j, key, n = alength; for (i=1; i < n; i++) { // Assume: a[0] a[i-1] Insert the value of a[i] // within a[0 (i-1)] making a[0] a[i] key = a[i]; j = i-1; while (j>=0 && a[j]>key) { a[j+1] = a[j]; j = j -1; a[j+1] = key; // Now a[0] a[1] a[i] The best case run time a size-n input is the smallest run time of the algorithm such inputs The worst case run time a size-n input is the biggest run time of the algorithm such inputs For example, findmin has Θ(n) best and worse case run times selectisort has Θ(n 2 ) best and worse case run times insertisort has Θ(n) best case and Θ(n 2 ) worse case

5 Big-Θ Cventi N = { 0, 1, 2, N + = { 1, 2, 3, f, g: N N + The collecti of functis that have linear growth rate is the set: for some c l, c u > 0, Θ(n) = f : for all sufficiently large n c l f (n)/n c u The collecti of functis that have quadratic growth rate is the set: for some c l, c u > 0, Θ(n 2 ) = f : for all sufficiently large n c l f (n)/n 2 c u CIS / 19 Big-Θ, The general case Suppose g:n N + is given The collecti of functis that have growth rate proportial g is: Θ(g(n)) = f : for some c l, c u > 0, for all sufficiently large n c l f (n) g(n) c u c 2 = c u and c 1 = c l Image from: CIS / 19 Some s Suppose f is given by: f is in Θ(n 2 ) f (n) = 100n n + 97 Some s f (n)/n for all n > n + 97 n 2 0 Some s Suppose f is given by: f (n) = 100n n + 97 f is in Θ(n 2 ) Suppose f is given by: f (n) = 8000n + 97 f is not in Θ(n 2 ) Suppose f is given by: f (n) = 8000n + 97 f is not in Θ(n 2 ) CIS / 19

6 The Limit Rule for Big-Θ The Limit Rule (Versi 1) f (n) Suppose that lim = c, where 0 c + n g(n) (a) If 0 < c < +, then f is in Θ(g(n)) (b) If c = 0 or c =, then f is not in Θ(g(n)) Suppose f and f are given by: f (n) = 100n n + 97 f (n) = 8000n + 97 f is in Θ(n 2 ) and f is not in Θ(n 2 ) Big-O and Big-Ω Recall: Θ(g(n)) = f : for some c l, c u > 0, for all sufficiently large n c l f (n)/g(n) c u To talk about upper bounds and lower bounds growth rates, we break Θ in two parts for some c u > 0, O(g(n)) = f : for all sufficiently large n f (n)/g(n) c u Ω(g(n)) = f : for some c l > 0, for all sufficiently large n c l f (n)/g(n) CIS / 19 CIS / 19 Big-O and Big-Ω, ctinued The Limit Rule, Again Intuitively: Θ(g(n)) is the collecti of functis that have growth rates proportial to g(n) O(g(n)) is the collecti of functis f (n) such that f (n)/g(n) is no worse than some cstant for large n Ω(g(n)) is the collecti of functis f (n) such that f (n)/g(n) is no smaller than some positive cstant for large n The Limit Rule (Versi 2) Suppose that where c is such that 0 c + f (n) lim n g(n) = c (a) If 0 < c < +, then f is in Θ(g(n)), O(g(n)), and Ω(g(n)) (b) If c = 0, then f is in O(g(n)), but not in either Θ(g(n)) or Ω(g(n)) (c) If c =, then f is in Ω(g(n)), but not in Θ(g(n)) or O(g(n)) typo fix CIS / 19 CIS / 19

7 Applying the Limit Rule (a) n (8000n + 97) is in O(n 2 ) (b) n (100n n + 97) is in Ω(n) Limitatis of the Limit Rule (a) n n 2+sin n is in Ω(n) and O(n 3 ) but is not in any Θ(n k ) class (b) n (2 + sin n)n 2 is in Θ(n 2 ) but CIS / 19