CSc 120. Introduc/on to Computer Programming II. 08: Efficiency and Complexity. Adapted from slides by Dr. Saumya Debray

Transcription

1 CSc 120 Introduc/on to Computer Programming II Adapted from slides by Dr. Saumya Debray 08: Efficiency and Complexity

2 EFFICIENCY MATTERS 2

3 reasoning about performance 3

4 Reasoning about efficiency Consider two different programs that sum the integers from 1 to n def sumv1(n): num = 0 for i in range(1,n+1): num += i return num def sumv2(n): num = (n*(n+1))/2 return num 4

5 Reasoning about efficiency How would we compare them to see which is "beger"? def sumv1(n): num = 0 for i in range(1,n+1): num += i return num def sumv2(n): num = (n*(n+1))/2 return num 5

6 Reasoning about efficiency We could compare the difference in running Jmes: Download sumv1(n) o summer18/notes/sumv1.py o run this for these values of n: 10,000, 100,000, 1,000,000 Download sumv2(n) o hgp://www2.cs.arizona.edu/classes/cs120/summer18/notes/ sumv2.py o this for these values of n: 10,000, 100,000, 1,000,000 6

7 Reasoning about efficiency ObservaJons on sumv1(n) vs sumv2(n): For sumv1, as we increase n, the running Jme increases o increases in proporjon to n For sumv2, as we increase n, the running Jme stays the same We nojced this by running the programs But this depends on many external factors 7

8 Reasoning about efficiency The Jme taken for a program to run can depend on: o processor properjes that have nothing to do with the program (e.g., CPU speed, amount of memory) o what other programs are running (i.e., system load) o which inputs we use (some inputs may be worse than others) We would like to compare different algorithms: without requiring that we implement them both first focusing on running Jme (not memory usage) abstracjng away processor- specific details considering all possible inputs 8

9 Reasoning about efficiency Algorithms vs. programs Algorithm: o a step- by- step list of instrucjons for solving a problem Program: o an algorithm that been implemented in a given language We would like to compare different algorithms abstractly 9

10 Comparing algorithms Search for a word my_word in a dicjonary (a book) A dic>onary is sorted Algo 1 (search from the beginning): start at the first word in the dicjonary if the word is not mword, then go to the next word conjnue in sequence unjl my_word is found Algo 2: start at the middle of the dicjonary if my_word is greater than the word in the middle, start with the middle word and conjnue from there to the end if my_word is less than the word in the middle, start with the middle word and conjnue from there to the beginning 10

11 Comparing algorithms Which is beger, Algo 1 (search from the beginning) or Algo 2? Algo 2 in most cases (seemingly) What is the reason? When is Algo 1 beger? Algo 1 is beger if the word is close to the beginning How close to the beginning? When considering which is beger, what measure are we using? The number of comparisons 11

12 Comparing algorithms Call comparison a primi>ve operajon an abstract unit of computajon We want to characterize an algorithm in terms of how many primijve operajons are performed best case and worst case We want to express this in terms of the size of the data (or size of its input) 12

13 Primi/ve opera/ons Abstract units of computajon convenient for reasoning about algorithms approximates typical hardware- level operajons Includes: assigning a value to a variable looking up the value of a variable doing a single arithmejc operajon comparing two numbers accessing a single element of a Python list by index calling a funcjon returning from a funcjon 13

14 Primi/ve ops and running /me A primijve operajon typically corresponds to a small constant number of machine instrucjons No. of primijve operajons executed no. of machine instrucjons executed actual running Jme 14

15 Example Code Primi4ve opera4ons def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 len(list_) : 1 range( ) : 1 in : 1 for : 2 list_[i] : 1 str_ : 1 == : 1 if : 1 each iterajon: 9 primijve ops 15

16 Primi/ve ops and running /me We consider how a funcjon's running Jme depends on the size of its input which input do we consider? 16

17 Best case vs. worst case inputs # lookup(str_, list_): returns the index where str_ occurs in list_ def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 Best- case scenario?: Worst- case scenario?: 17

18 Best case vs. worst case inputs # lookup(str_, list_): returns the index where str_ occurs in list_ def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 Best- case scenario: str_ == list_[0] # first element loop does not have to iterate over list_ at all running Jme does not depend on length of list_ does not reflect typical behavior of the algorithm 18

19 Best case vs. worst case inputs # lookup(str_, list_): returns the index where str_ occurs in list_ def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 Worst- case scenario: str_ == list_[- 1] # last element loop iterates through list_ running Jme is proporjonal to the length of list_ captures the behavior of the algorithm beger 19

20 Best case vs. worst case inputs # lookup(str_, list_): returns the index where str_ occurs in list_ def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 In reality, we get something in between but "average- case" is difficult to characterize precisely 20

21 What about average case? Running Jme worst- case Jme average case? best- case Jme 1 0 A B C D E F G H Inputs 21

22 Worst- case complexity Considers worst- case inputs Describes the running Jme of an algorithm as a funcjon of the size of its input ("Jme complexity") Focuses on the rate at which the running Jme grows as the input gets large Typically gives a beger characterizajon of an algorithm's performance This approach can also be applied to the amount of memory used by an algorithm ("space complexity") 22

23 Example Code Primi4ve opera4ons def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 len(list_) : 1 range( ) : 1 in : 1 for : 2 list_[i] : 1 str_ : 1 == : 1 if : 1 each iterajon: 9 primijve ops 23

24 Example Code Primi4ve opera4ons def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 Total primi>ve ops executed: 1 iterajon: 9 ops n iterajons: 9n ops + return at the end: 1 op len(list_) : 1 range( ) : 1 in : 1 for : 2 list_[i] : 1 str_ : 1 == : 1 if : 1 each iterajon: 9 primijve ops total worst- case running Jme for a list of length n = 9n

25 EXERCISE # What is the total worst- case running >me of the following code fragment expressed in terms of n? for i in range(n): k =

26 EXERCISE # What is the total worst- case running >me of the following code fragment expressed in terms of n? a = 5 b = 10 for i in range(n): x = i * b for j in range(n): z += b 26

27 asympto/c complexity 27

28 Asympto/c complexity In the worst- case, lookup(str_, list_) executes 9n + 1 primijve operajons given a list of length n To translate this to running Jme: suppose each primijve operajon takes k Jme units then worst- case running Jme is (9n + 1)k But k depends on specifics of the computer, e.g.: Processor speed k running 4me slow n + 20 medium 10 90n + 10 fast 3 27n

29 Asympto/c complexity depends on processor- specific characterisjcs worst case running Jme = An + B depends on how the algorithm processes data 29

30 Asympto/c complexity For algorithm analysis, we focus on how the running Jme grows as a funcjon of the input size n usually, we do not look at the exact worst case running Jme it's enough to know proporjonalijes E.g., for the lookup() funcjon: we say only that its running Jme is "propor>onal to the input length n" 30

31 Example Code Primi4ve opera4ons def list_posijons(list1, list2): posijons = [] for value in list1: idx = lookup(value, list2) posijons.append(idx) return posijons 31

32 Example Code Primi4ve opera4ons def list_posijons(list1, list2): posijons = [] for value in list1: idx = lookup(value, list2) posijons.append(idx) return posijons 1 in : 1 for : 2 9n iterates n Jmes Worst case behavior: primijve operajons = n(9n + 5) + 2 = 9n 2 + 5n + 2 running Jme = k(9n 2 + 5n + 2) 32

33 Example Code def list_posijons(list1, list2): posijons = [] for value in list1: idx = lookup(value, list2) posijons.append(idx) return posijons Primi4ve opera4ons Worst case: 9n 2 + 5n + 2 As n grows, the 9n 2 term grows faster than 5n+2 for large n, the n 2 term dominates running Jme depends primarily on n 2 33

34 Example As n grows larger, the n 2 term dominates the contribujon of the other terms becomes insignificant 9n 2 + 5n + 2 9n 2 9n 2 + 5n + 2 9n 2 9n 2 + 5n + 2 9n 2 34

35 Example 2: 2x x x x x x x 2 2x 2 2x x x 2 35

36 Example 3: x x x x x x x x x x x x x 3 x 3 x 3 x x x x 3 36

37 Growth rates As input size grows, the fastest- growing term dominates the others the contribujon of the smaller terms becomes negligible it suffices to consider only the highest degree (i.e., fastest growing) term For algorithm analysis purposes, the constant factors are not useful they usually reflect implementajon- specific features to compare different algorithms, we focus only on proporjonality ignore constant coefficients 37

38 Comparing algorithms Growth rate n Growth rate n 2 def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 def list_posijons(list1, list2): posijons = [] for value in list1: idx = lookup(value, list2) posijons.append(idx) return posijons 38

39 Summary so far Want to characterize algorithm efficiency such that: does not depend on processor specifics accounts for all possible inputs count primijve operajons consider worst- case running Jme We specify the running Jme as a funcjon of the size of the input consider proporjonality, ignore constant coefficients consider only the dominant term o e.g., 9n 2 + 5n + 2 n 2 39

40 big- O nota/on 40

41 Big- O nota/on Big- O is formalizes this intuijve idea: consider only the dominant term o e.g., 9n 2 + 5n + 2 n 2 allows us to say, "the algorithm runs in Jme proporjonal to n 2" 41

42 Big- O nota/on IntuiJon: When we say we mean "f(n) is O(g(n))" "f is growing at most as fast as g" "big- O notajon" 42

43 Big- O nota/on Captures the idea of the growth rate of funcjons, focusing on proporjonality and ignoring constants Defini4on: Let f(n) and g(n) be funcjons mapping posijve integers to posijve real numbers. Then, f(n) is O( g(n) ) if there is a real constant c and an integer constant n 0 1 such that f(n) c g(n) for all n > n 0 43

44 Big- O nota/on f(n) is O( g(n) ) if there is a real constant c and an integer constant n 0 1 such that f(n) c g(n) for all n > n 0 Once the input gets big enough, c g(n) is (always) larger than f(n) 44

45 Big- O nota/on: proper/es If g(n) is growing faster than f(n): f(n) is O(g(n)) g(n) is not O(f(n)) If f(n) = a 0 + a 1 n a k n k, then: f(n) = O(n k ) i.e., coefficients and lower- order terms can be ignored g(n) f(n) 45

46 Some common growth- rate curves O(n 3 ) O(n 2 ) O(n log(n)) O(n) O(log n) 46

47 using big- O nota/on 47

48 Compu/ng big- O complexi/es Given the code: Given the code line 1 line 2... line k... O(f 1 (n))... O(f 2 (n))... O(f k (n)) loop... O(f1(n)) iterajons line1... O(f2(n)) The overall complexity is O(max(f 1 (n), f s (n),..., f k (n))) The overall complexity is O( f 1 (n) x f 2 (n) ) 48

49 Using big- O nota/on Code Big- O complexity O(1) str_ == list_[i] O(1) O(1) O(1) 49

50 Using big- O nota/on Code Big- O complexity O(1) if str_ == list_[i]: return i O(1) O(1) O(1) 50

51 Using big- O nota/on Code Big- O complexity for i in range(len(list_)): if str_ == list_[i]: return i O(n) O(n) (worst- case) (n = length of the list) O(1) 51

52 Using big- O nota/on Code Big- O complexity def lookup(str_, list_): for i in range(len(list_)): if str_ == list_[i]: return i return - 1 O(n) O(1) O(n) 52

53 Using big- O nota/on Code Big- O complexity def list_posijons(list1, list2): posijons = [] for value in list1: idx = lookup(value, list2) posijons.append(idx) O(n) (worst- case) (n = length of list1) return posijons O(n 2 ) O(n) (worst- case) (n = length of list2) 53

54 Using big- O nota/on Code Big- O complexity O(1) def list_posijons(list1, list2): posijons = [] for value in list1: O(n 2 ) idx = lookup(value, list2) posijons.append(idx) return posijons O(n 2 ) 54

55 ICA- 6 WARM- UP Do the first 2 problems. Is analyzing worst- case running Jme important? How many Web pages does Google search? hgps:// crawling- indexing/ 55

56 Compu/ng big- O complexi/es Given the code: Given the code line 1 line 2... line k... O(f 1 (n))... O(f 2 (n))... O(f k (n)) loop... O(f1(n)) iterajons line1... O(f2(n)) The overall complexity is O(max(f 1 (n), f s (n),..., f k (n))) The overall complexity is O( f 1 (n) x f 2 (n) ) 56

57 EXERCISE # my_rfind(mylist, elt) : find the distance from the # end of mylist where elt occurs, - 1 if it does not def my_rfind(mylist, elt): pos = len(mylist) 1 while pos >= 0: if mylist[pos] == elt: return pos pos - = 1 return - 1 Worst- case big- O complexity =??? 57

58 EXERCISE # for each element of a list: find the biggest value # between that element and the end of the list def find_biggest_arer(arglist): pos_list = [] for idx0 in range(len(arglist)): biggest = arglist[idx0] for idx1 in range(idx0+1, len(arglist)): biggest = max(arglist[idx1], biggest) pos_list.append(biggest) return pos_list Worst- case big- O complexity =??? 58

59 Input size vs. run /me: max() RunJme (ms) Input size 59

60 EXERCISE # for each element of a list: find the biggest value # between that element and the end of the list def find_biggest_arer(arglist): pos_list = [] for idx0 in range(len(arglist)): biggest = max(arglist[idx0:]) # library code pos_list.append(biggest) return pos_list Worst- case big- O complexity =??? 60