Algorithmic Analysis. Go go Big O(h)!

Transcription

1 Algorithmic Analysis Go go Big O(h)! 1

2 Corresponding Book Sections Pearson: Chapter 6, Sections 1-3 Data Structures:

3 What is an Algorithm? Informally, any well defined computational procedure that takes some value or set of values as input and produces some value, or set of values, as output - Cormen et al. 3

4 How Do You Compare Algorithms? What makes one algorithm more efficient than another? What makes one algorithm faster than another? How do we prove that one algorithm is superior to another without having to run input cases to test it? 4

5 What is Algorithmic Analysis? Primarily focused on the runtime of an algorithm We actually do not care about the actual runtime in terms of time, that is more performance analysis Care about the runtime of an algorithm in relation to the size of its input, usually denoted by n. 5

6 BubbleSort vs. FailSort public static void bubblesort(int[] a) { for (int i = 0; i < a.length; i += 1) { for (int j = 0; j < a.length - 1; j += 1) { if (a[j] > a[j + 1]) { // swap a[j] and a[j+] int temp = a[j]; a[j] = a[j + 1]; a[j + 1] = temp; public static void failsort(int[] a) { for (int i = 0; i < a.length; i += 1) { for (int j = 0; j < a.length - 1; j += 1) { if (a[j] > a[j + 1]) { // copy a[j] int temp = a[j]; // remove a[j] for (int k = j; k < a.length - 1; k += 1) a[k] = a[k + 1]; // reinsert as a[j+1] for (int k = a.length - 1; k > j; k -= 1) a[k] = a[k - 1]; a[j + 1] = temp; 6

7 Asymptotic Notation What is an asymptote? We want a function that describes the boundary of the maximum runtime (ceiling) We generally want the tightest bound possible, something bounded under an n 2 asymptote will also be bounded by a n 3 one. Which one gives us a narrower area? There are other functions which describe how an algorithm, but they are beyond the scope of this course 7

8 Big Oh Notation Expressed as f(n) O(g(n)) ( sometimes written f(n) = O(g(n)) ) n is the size of the input f(n) is the actual runtime of the algorithm in relation to n g(n) is some function that has the property below: f(n) c*g(n) for all n > k where c > 0 and k 1 O(g(n)) is a set of all functions bounded by c*g(n) 8

9 Big Oh Graph In this graph, the algorithm s actual runtime is f(n), but it is described as O(g(n)), as f(n) c*g(n) for all n>k Constant s don t matter in f(n) because we can just adjust c in the above equation to compensate As n approaches infinity, the values of f(n) will be defined by the largest n term in f(n), which will generally be what g(n) is defined by Graph from: xw2k.nist.gov/dads//html/bigonotation.html 9

10 Properties of Big Oh Notation When measuring the the Big Oh of an algorithm s runtime, f(n), the only part that matters is the largest n value in f(n), with constants being ignored due to how g(n) is defined. So if an algorithm has a runtime of 3n 3 + 5n n + 13, the algorithm runs in O(n 3 ) 10

11 O(1) - Constant Time O(1) means something runs in constant time, so it is not dependent on the size of the input Examples: swap of two elements, array element lookup, calling an accessor or mutator method 11

12 O(log n) - Logarithmic Time In computer science notation, log n log2n = lg n O(lg n) time means an algorithm s runtime is proportional to the logarithm of the input size, which is very fast This comes up frequently with algorithms that use divide and conquer methods, such as binary search 12

13 O(n) - Linear Time O(n) time refers to something that has a runtime that has a linear relationship to the input size Examples: Looping through an array of size n or a string of length n 13

14 O(n 2 ) - Quadratic Time Typically referred to as running in n squared time Usually involves a loop and a nested loop that both traverse the input Examples: Bubble sort, insertion sort, gnome sort 14

15 O(n 3 ) - Cubic Time Typically referred to as n cubed time, sometime referred to as cubic time Example: Matrix multiply of two nxn matrices (sorta) 15

16 O(b n ) - Exponential Time b is some constant term, usually just use 2 Examples: Brute forcing a Rubik s cube, recursive Fibonacci algorithm 16

17 O(n!) - Factorial Time Also equivalent to nn runtime Examples: Bozo sort!, Brute forcing the Traveling Salesman Problem or other permutation problems. 17

18 O(n lg n) - Linearthmic Time Commonly referred to as n log n time The fastest algorithmic runtime of any comparison sort, hence why it is a common term Examples: Quicksort, Heapsort, Mergesort 18

19 Graphical Comparison of Common Big-Os O(1) < O(lg n) < O(n) < O(n lg n) < O(n 2 ) <O(n 3 ) < O(2 n ) < O(n!) O(1) O(lg n) O(n) O(n lg n) O(n 2 ) O(n 3 ) O(2 n ) O(n!) Relative Execution Time Problem Size 19

20 BubbleSort vs. FailSort Algorithmic Analysis where n = length of array a public static void bubblesort(int[] a) { for (int i = 0; i < a.length; i += 1) { for (int j = 0; j < a.length - 1; j += 1) { if (a[j] > a[j + 1]) { // swap a[j] and a[j+] int temp = a[j]; a[j] = a[j + 1]; a[j + 1] = temp; Two nested for loops which both go from 0 to n, so bubblesort runs in O(n 2 ) public static void failsort(int[] a) { for (int i = 0; i < a.length; i += 1) { for (int j = 0; j < a.length - 1; j += 1) { if (a[j] > a[j + 1]) { // copy a[j] int temp = a[j]; // remove a[j] for (int k = j; k < a.length - 1; k += 1) a[k] = a[k + 1]; // reinsert as a[j+1] for (int k = a.length - 1; k > j; k -= 1) a[k] = a[k - 1]; a[j + 1] = temp; Two deep nested for loops which go from 0 to n containing additional for loops which can run up to n times each, so failsort runs in O(n 3 ) 20

21 Big Oh of Loops What is the Big Oh notation of each of the nested loops with respect to n? for(int i = 0; i < n; i++) { for(int j = i; j < n; j++) { k = k + i + j; for(int i = 0; i < n; i++) { for(int j = i; j < 5; j++) { k = k + i + j; for(int i = 0; i < n; i=i+n/5) { for(int j = i; j < n; j++) { k = k + i + j; 21

22 Pop Quiz! Write the Big Oh notation for the runtime of each of the nested loops with respect to n. //First Problem for(int i = 0; i < n; i++) { for(int j = 0; j < i; j++) { for(int k = i; k < n; k++) { total = total + i + j + k; //Second Problem for(int i = 0; i < n; i=i+n/4) { for(int j = 0; j < i; j++) { for(int k = i; k < n; k=k+n/3) { total = total + i + j + k; 22

23 First Problem Solution //First Problem for(int i = 0; i < n; i++) { for(int j = 0; j < i; j++) { for(int k = i; k < n; k++) { n times = O(n) total = total + i + j + k; n/2 times average run = O(n) n/2 times average run = O(n) Nested loops are multiplied in runtime, so this runs in O(n 3 ) 23

24 Second Problem Solution //Second Problem for(int i = 0; i < n; i=i+n/4) { for(int j = 0; j < i; j++) { for(int k = i; k < n; k=k+n/3) { total = total + i + j + k; 4 times = O(1) 3n/8 times average run = O(n) max 3 times = O(1) Nested loops are multiplied in runtime, so this runs in O(n) 24

25 What about O(k/n)? int k = 1000; for(int i = 0; i < (k/n); i++) { total = total + i; When n>k, (k/n)==0, thus the loop never executes. So, the big-oh is O(1), due to the i=0, which always executes. 25