CS 231 Data Structures and Algorithms Fall Algorithm Analysis Lecture 16 October 10, Prof. Zadia Codabux

Transcription

1 CS 231 Data Structures and Algorithms Fall 2018 Algorithm Analysis Lecture 16 October 10, 2018 Prof. Zadia Codabux 1

2 Agenda Algorithm Analysis 2

3 Administrative No quiz this week 3

4 Algorithm Analysis 4

5 Algorithm Analysis An algorithm is a sequence of instructions or a set of rules that are followed to accomplish a task. e.g. making coffee requires a series of step To accomplish a task, there are more than one ways (algorithms). E.g., you can use K-cup coffee maker or French press pot. Some solutions are more efficient, taking less resources than others. How can we tell which solutions are more efficient? How fast they run / how long they take to execute (time complexity). How much memory or space they consume (space complexity). We will focus on time complexity. Time complexity analysis is a simplified mathematical way of analyzing how long an algorithm with a given number of input(s) will take to complete it s task. It is usually defined using Big-O notation. 5

6 Algorithm Analysis Recall the grocery checkout example If you were to choose a data structure to store the prices of items. At checkout, when the cashier scans the barcode of an item, the price is retrieved. You have a choice between different data structures. How do you decide which is the best for the job? What is defined as better or best? 6

7 Phonebook Example Scenario 1: Look up business phone number given page number and business name. Scenario 2: Given a phone number, find the person or business with that number. Scenario 3: Let n indicate the number of people in the phone book. Let's say everyone in the phone book gets a copy of the book. Let's say there is a numeric error in each person's number in each copy of the book. 7

8 Big-Oh Notation We use Big-O notation to describe the order of operations that an algorithm will take to run as the size of problem (n) increases. For example, iterating through a linked list will take time proportional to n. Suppose it took T(n) = 3.25n milliseconds. To use Big-O notation, we ignore the constants that don t have that big impact on the overall complexity of the problem. (Note that this does not mean the constants are unimportant for actual runtime or efficiency!) 8

9 Big-Oh Notation Formally, Wikipedia puts it like this So our function T(n) = 3.25n + 42 is O(n). 9

10 Big-Oh Notation Likewise, if we have another algorithm whose runtime follows a function more like this: T(n) = 5n n , we examine each term and use the term that grows the fastest as n grows. That is 5n 2. Then, we ignore that constant and say that this second algorithm is O(n 2 ). In short, we look at the term in our expression that has the potential greatest impact on the value that our expression will return. (This changes as the constant gets extremely large and n gets small, but let s not worry about about that for now.) There are several common time complexities with simple examples. 10

11 Constant Time: O(1) The number of steps the algorithm takes to accomplish the task is not proportional to the input size n. int[] ary = {1,2,3,4,5,6,7,8,9}; int get (int index) { return ary[index]; } Other examples: adding two numbers, assigning a value to a variable, comparing two numbers 11

12 Logarithmic Time: O(log n) The amount of work the algorithm needs to do decreases at each step (binary search in a sorted array) int[] ary = {1,2,3,4,5,6,7,8,9}; boolean search (int num, int start, int end, int[] ary) { intmidpos = (end-start)/2 + 1; if (end < start) return false; if (ary[midpos] == num) return true; else if (ary[midpos] > num) { return search(num, start, midpos-1, ary); } else { return search(num, midpos+1, end, ary); }} search(3, 0, ary.length-1, ary); 12

13 Linear Time: O(n) The number of steps the algorithm needs to accomplish the task is proportional to the input size n. Here, n=a.length int[] a = {1,2,3,4,5,6,7,8,9}; for (int i = 0; i < a.length; i++) { System.out.println(a[i]); } 1 initialization of I (constant) n comparisons of i against a.length n increments of i n array indexing operations (to compute a[i]) n invocations of System.out.println T(n)= 4n+1. All operations are not created equal. But overall, T(n)= n. 13

14 Linear Time: O(n) Imagine a case with doubly-nested loops where only the outer loop is dependent on the problem size n, and the inner loop always executes a constant number of times, say 3 times: for ( int i = 0; i < n; i++ ) { for ( int j = 0; j < 3; j++ ) { // these statements are executed O(n) times } } Time Complexity: n 14

15 Quadratic Time: O(n 2 ) // print out the 2D array in a grid for (int r = 0; r < n; r++) { for (int c = 0; c < n; c++) System.out.print(matrix[r][c]); } Time complexity of the nested for loop 1 initialization of r (constant) n initialization of c n comparisons of r against matrix.length n x n comparisons of c against matrix[0].length n increments of r n x n increments of c n x n array indexing operations (to compute matrix[r][c]) n x n invocations of System.out.println T(n) = 4 x n x n + 3 x n + 1 = 4n 2 + 3n + 1 Overall, T(n) = O(n 2 ) 15

16 Cubic Time: O(n 3 ) for ( int i = 0; i < n; i++ ) { for ( int j = 0; j < n; j++ ) { for ( int k = 0; k < n; k++ ) { // these statements are executed O(n 3 ) times } } } 16

17 Exponential Time: O(c n ) Given an input of size n, the number of steps the algorithm takes to accomplish a task is a constant to the n power. Example: trying to find every combination of letters for a password of length n. 17

18 Algorithm Analysis Source: Wikipedia 18

19 Best, Worst and Average Case Worst-case - An upper bound on the running time for any input of given size Average-case - Assume all inputs of a given size are equally likely Best-case - The lower bound on the running time Complexity of algorithms is usually evaluated in the worst case (most unfavorable scenario). This means in the average case they can work faster, but in the worst case they work with the evaluated complexity and not slower. 19

20 Best, Worst and Average Case Example: Sequential search in a list of size n To find the searched key in the worst case, we have to check all the elements in the array. Worst-case: n comparisons O(N) In the best case we will have luck and we will find the element at first position. Best-case: 1 comparison O(1) In the average case we can expect to check half the elements in the array until we find the one we are looking for. Average-case: n/2 comparisons O(N/2) = O(N) ignore the constants 20

21 Time Complexity of Data Structures Array/ ArrayList Get (using index) : O(1) Search (a value) : O(n) Add/Remove (at end) : O(1) Add/Remove (at end) when full: O(n) Singly Linked List AddFirst/ RemoveFirst : O(1) Traversing and Add/Remove (at index) : O(n) Get (using index) : O(n) 21

22 Time Complexity of Data Structures Array-based/ Node-based Stack Push : O(1) Pop : O(1) Peek : O(1) 22