Computational Complexity: Measuring the Efficiency of Algorithms

Size: px

Start display at page:

Download "Computational Complexity: Measuring the Efficiency of Algorithms"

Maryann Ball
6 years ago
Views:

1 Computational Complexity: Measuring the Efficiency of Algorithms Rosen Ch. 3.2: Growth of Functions Rosen Ch. 3.3: Complexity of Algorithms Walls Ch. 10.1: Efficiency of Algorithms Measuring the efficiency of algorithms We have two algorithms: alg1 and alg2 that solve the same problem. Our application needs a fast running time. How do we choose between the algorithms? Measuring the efficiency of algorithms Implement the two algorithms in Java and compare their running times Issues with this approach: How are the algorithms coded? We want to compare the algorithms, not the implementations. What computer should we use? Results may be sensitive to this choice. What data should we use? Measuring the efficiency of algorithms Want: techniques that analyze algorithms independently of specific details such as implementation, hardware, or data. Solution: analyze the number of operations the algorithm will perform for an input of given size Example: copying an array with n elements requires. operations. Growth rates Algorithm A requires n 2 / 2 operations to solve a problem of size n Algorithm B requires 5n+10 operations to solve a problem of size n For large enough problem size algorithm B is more efficient We focus on the growth rate: Algorithm A requires time proportional to n 2 Algorithm B requires time proportional to n Order of magnitude analysis Big O notation: A function f(x) is O(g(x)) if there exist two positive constants, c and k, such that f(x) c*g(x) x > k k c*g(x) f(x) n 1

2 Order of magnitude analysis Big O notation: A function f(x) is O(g(x)) if there exist two positive constants, c and k, such that f(x) c*g(x) x > k Order of magnitude analysis Big O notation: A function f(x) is O(g(x)) if there exist two positive constants, c and k, such that f(x) c*g(x) x > k Common Shapes: Constant O(1) Focus is on the shape of the function Ignore the multiplicative constant Focus is on large x k allows us to ignore behavior for small x c and k are witnesses to the relationship that f(x) is O(g(x)) If there is one pair of witnesses (c,k) then there are infinitely many. examples? Common Shapes: Linear Linear Other Shapes: Sublinear O(n) f(n) = a*n + b Example: copying an array for (int i = 0; i < a.size; i++){ a[i] = b[i]; 2

3 Common Shapes: logarithm Logarithms (cont.) Quadratic log b n is the number x such that b x = n 2 3 = 8 log 2 8 = = 16 log 2 16 = 4 log b n: (# of digits to represent n in base b) 1 We usually work with base 2 Properties of logarithms log(x y) = log x + log y log(x a ) = a log x log a n = log b n / log b a logarithm is a very slow-growing function examples of logarithmic complexity? O(n 2 ): for (int i=0; i < n; i++){ for (int j=0; j < n; j++) { Other Shapes: Superlinear Simplifying Big-O Combinations of Functions Polynomial (x a ), exponential (a x ) Theorem: Let f (x) = a n x n + a n 1 x n a 1 x + a 0 where a n,a n 1...,a 1,a 0 are real numbers. Then f (x) is O(x n ) Example: x 2 + 5x is O(x 2 ) Additive Theorem: Suppose that f 1 (x) is O(g 1 (x)) and f 2 (x) is O(g 2 (x)). Then ( f 1 + f 2 )(x) is O(max( g 1 (x), g 2 (x) ). Multiplicative Theorem: Suppose that f 1 (x) is O(g 1 (x)) and f 2 (x) is O(g 2 (x)). Then ( f 1 f 2 )(x) is O(g 1 (x)g 2 (x)). 3

4 Practical Analysis - Combinations Sequential Big-O bound: Steepest growth dominates Example: copying of array, followed by binary search n + log(n) O(?) Embedded code Big-O bound multiplicative Example: a for loop with n iterations and a body taking O(log n) O(?) Other Notations Big-Omega (Ω): lower bound Big-Theta (Θ): tight upper bound Big-Omega: A function f(x) is Ω(g(x)) if there exist two positive constants, c and k, such that f(x) c*g(x) x > k Big-Theta: A function f(x) is Θ(g(x)) if f(x) is O(g(x)) and f(x) is Ω(g(x)). We then say that f(x) is of order g(x). Best, Worst and Average Time Complexity Worst case just how bad can it get: the maximal number of steps Average case amount of time expected usually In this course we will hand wave when it comes to average case Example: searching for an item in an unsorted array Examples. Practical Analysis - Loops 1 public void insertelementat(object obj, int index) { 2 for (i = elementcount; i > index; i--) { 3 elementdata[i] = elementdata[i-1];... How many times will line 3 repeat? On what does the number depend? Practical Analysis Dependent loops... for (i = 0; i < n; i++) { for (j = 0; j < i; j++) { i = 0: inner-loop iters =0 i = 1: inner-loop iters =1... i = n-1: inner-loop iters =n-1 Total = (n-1) f(n) = n*(n-1)/2 O(n 2 ) Practical Analysis - Recursion Number of operations depends on : number of calls work done in each call Examples: factorial: how many recursive calls? binary search? We will devote more time to analyzing recursive algorithms later in the course. 4

5 Examples Copying an array O(n), where n is size of the array Linear search in a list for a particular value Worst case O(n), Average O(n), where n is the list size Enumerate all lowercase char strings of length n O(26 n ) Insert element in an array Worst case O(n), Average case O(n), where n is the number of elements in the array Example: Selection Sort Basic idea Keep already sorted array at high-end Find the biggest element in unsorted part Swap it into the highest position in unsorted part Iterate Invariant: each pass guarantees that one more element is in the correct position and all elements in lower part of array are smaller Selection Sort - Code public static void selectionsort(comparable[] thearray, int n) { // Sorts the items in an array into ascending order. // Precondition: thearray is an array of n items. // Postcondition: thearray is sorted into ascending order. // last = index of the last item in the subarray of items yet to be sorted // largest = index of the largest item found for (int last = n-1; last >= 1; last--) { // Invariant: thearray[last+1..n-1] is sorted and > thearray[0..last] // select largest item in thearray[0..last] int largest = indexoflargest(thearray, last+1); // swap largest item thearray[largest] with thearray[last] Comparable temp = thearray[largest]; thearray[largest] = thearray[last]; thearray[last] = temp; indexoflargest private static int indexoflargest(comparable[] thearray, int size) { // Finds the largest item in an array. // Precondition: thearray is an array of size items; size >= 1. // Postcondition: Returns the index of the largest item in the array. int indexsofar = 0; // index of largest item found so far // Invariant: thearray[indexsofar]>=thearray[0..currindex-1] for (int currindex = 1; currindex < size; ++currindex) { if (thearray[currindex].compareto(thearray[indexsofar])>0) { indexsofar = currindex; // end if // end for return indexsofar; Selection Sort Algorithm Complexity How many comparisons are made? How many swaps are made? How much space? Selection Sort Algorithm Complexity How many comparisons are made? (n-1)+(n-2)+...+1, or O(n 2 ) How many swaps are made? Note swap is run even if already in position n, or O(n) How much space? In-place algorithm 5

6 Final Comments Order-of-magnitude analysis focuses on large problems If the problem size is always small, you can probably ignore an algorithm s efficiency Weigh the trade-offs between an algorithm s time requirements and its memory requirements, expense of programming/maintenance 6

9/3/12. Outline. Part 5. Computational Complexity (1) Software cost factors. Software cost factors (cont d) Algorithm and Computational Complexity

9/3/12. Outline. Part 5. Computational Complexity (1) Software cost factors. Software cost factors (cont d) Algorithm and Computational Complexity Outline Part 5. Computational Compleity (1) Measuring efficiencies of algorithms Growth of functions Asymptotic bounds for the growth functions CS 200 Algorithms and Data Structures 1 2 Algorithm and Computational