Algorithm Analysis. Review of Terminology Some Consequences of Growth Rates. The book s big example: Detecting anagrams. There be math here!

Transcription

1 Algorithm Analysis Review of Terminology Some Consequences of Growth Rates There be math here! The book s big example: Detecting anagrams 4 different algorithms!

2 Big O Definition: T(N) = O(f(N)) if there exist positive constants c, n 0 such that: T(N) c f(n) for all N n 0 Idea: We are concerned with how the function grows when N is large. We are not picky about constant factors: coarse distinctions among functions Lingo: "T(N) grows no faster than f(n)." 2

3 Big O c, n 0 > 0 such that f(n) c g(n) when N n 0 f(n) grows no faster than g(n) for large N 3

4 Preferred big-o usage pick tightest bound. If f(n) = 5N, then: f(n) = O(N 5 ) f(n) = O(N 3 ) f(n) = O(N log N) f(n) = O(N) preferred ignore constant factors and low order terms T(N) = O(N), not T(N) = O(5N) T(N) = O(N 3 ), not T(N) = O(N 3 + N 2 + N log N) 4

5 Big-O of selected functions 5

6 An O(1) algorithm is constant time. The running time of such an algorithm is essentially independent of the input. Such algorithms are rare, since they cannot even read all of their input. An O(log b n) [for some b] algorithm is logarithmic time. We do not care what b is. An O(n) algorithm is linear time. Such algorithms are not rare. This is as fast as an algorithm can be and still read all of its input. An O(n log b n) [for some b] algorithm is log-linear time. This is about as slow as an algorithm can be and still be truly useful (scalable). An O(n 2 ) algorithm is quadratic time. These are usually too slow. An O(b n ) [for some b] algorithm is exponential time. These algorithms are much too slow to be useful. 6

7 Hierarchy of Big-O Functions, ranked in increasing order of growth: 1 log log n log n n n log n n 2 n 2 log n n n n! n n 7

8 Various growth rates T ( n) ( 1):Constant time T ( n) (log n): logarithmic time T ( n) ( n): linear time T ( n) ( nlog n):famous for sorting T ( n) ( n T ( n) ( n T ( n) (2 2 k n ): quadratic ): polynomial time ), T ( n) ( n n time ), T ( n) ( n!): exponential Practical for small values of n (e.g., n =10 or n = 20) time; 8

9 Program loop runtimes Suppose we have a list, mylist, with n elemenst: for x in mylist: // O(n) statement(s) The loop runtime grows linearly when compared to its maximum value n. 9

10 Program loop runtimes counter = 0 while counter < n: statement(s) counter += 1 What s the big-o? 10

11 Program loop runtimes counter = 0 while counter < n: statement(s) counter += 2 What s the big-o? 11

12 Program loop runtimes counter = 0 while counter < n*n: statement(s) counter += 1 What s the big-o? 12

13 Program loop runtimes Suppose we have a list, mylist, with n elemenst: for x in mylist: for y in mylist: statement(s) Nesting loops multiplies their runtimes. The outer loop is O(n). The inner loop is O(n). Multiply. The total big-o is O(n 2 ). 13

14 Program loop runtimes counter = 0 while counter < n: counter2 = 0 while counter2 < n: counter3 = 0 while counter3 < n: statements counter3 += 1 counter2 += 1 counter += 1 What s the big-o? 14

15 How does this translate to practical problems? If linear relationship, If input doubles, time doubles. If input triples, time triples. If quadratic relationship, If input doubles, time quadruples (2 2 ) If input triples, time increases ninefold (3 2 ) If cubic relationship, If input doubles, time increases eightfold If input triples, time increases twenty-sevenfold How about n log n?

16 Let s do some examples Suppose it takes 100 ms to process 1000 items in a list. How long will it take to process 10,000 items? It depends Is the algorithm Linear? Quadratic? Cubic? NlogN? 16

17 The linear solution Suppose it takes 100 ms to process 1000 items in a list. How long will it take to process 10,000 items? 100/1000 = t/10000 t = 1000 It will take 1,000 ms or 10 times longer. 17

18 The quadratic solution Suppose it takes 100 ms to process 1000 items in a list. How long will it take to process 10,000 items? 100/(1000) 2 = t/(10000) 2 100/ = t/ t = It will take 10,000 ms or 100 times longer. 18

19 The cubic solution Suppose it takes 100 ms to process 1000 items in a list. How long will it take to process 10,000 items? 100/(1000) 3 = t/(10000) 3 100/ = t/ t = 100,000 It will take 100,000 ms or 1000 times longer. 19

20 The log-linear solution Suppose it takes 100 ms to process 1000 items in a list. How long will it take to process 10,000 items? 100/1000log(1000) = t/10000log(10000) ~100/(1000*10) = t/(10000*13) ~1,300 It will take 1,300 ms. 20

21 The same problem, but with a different unknown variable Suppose it takes 100 ms to process 1000 items in a list. How many can we process if with have 400 ms? It depends Is the algorithm Linear? Quadratic? Cubic? NlogN? 21

22 The linear solution Suppose it takes 100 ms to process 1000 items in a list. How many can we process if with have 400 ms? 100/1000 = 400/n n = 4000 We can process 4000 items (or 4 times more) 22

23 The quadratic solution Suppose it takes 100 ms to process 1000 items in a list. How many can we process if with have 400 ms? 100/(1000) 2 = 400/(n) 2 100/ = 400/n 2 n 2 = 4,000,000 We can process 2,000 elements twice as many (or SQRT(4) as many). 23

24 The cubic solution Suppose it takes 100 ms to process 1000 items in a list. How many can we process if with have 400 ms? 100/(1000) 3 = 400/(n) 3 100/ = 400/n 3 n 3 = 4,000,000,000 We can process 1,587 items. 24

25 The log-linear solution Suppose it takes 100 ms to process 1000 items in a list. How many can we process if with have 400 ms? 100/1000log(1000) = 400/nlog(n) ~100/(1000*10) = 400/nlog(n) ~40,000 = nlogn (Use Wolfram Alpha) ~3,408 items 25

26 The Anagram Detection Example (2.2.2) What is an anagram? heart and earth python and typhon Our problem: Assume we have two words of equal length and all lower case How can we tell if they are anagrams? 26

27 The Book Gives Us 4 Solutions Checking Off: Listing 2.6 Sort and Compare Listing 2.7 Brute Force No Listing (darn it!) Count and Compare Listing

28 Checking Off For every character in the first string, make sure there is a corresponding character somewhere in the 2 nd string. When a character has been checked off, we want to mark it with the special value None. But strings are immutable, so the authors first convert the 2 nd string to a list 28

29 Here s the code: Pay attention to Big-O def anagramsolution1(s1,s2): alist = list(s2) pos1 = 0 stillok = True while pos1 < len(s1) and stillok: pos2 = 0 found = False while pos2 < len(alist) and not found: if s1[pos1] == alist[pos2]: found = True else: pos2 = pos2 + 1 if found: alist[pos2] = None else: stillok = False pos1 = pos1 + 1 return stillok 29

30 Big-O of Checking Off For every element of the first string (of length n), We will visit as many as n characters in the 2 nd string Each character in the 2 nd string must be marked In the worst case, finding the first character might take n compares. Finding the second character would take n 1. Finding the third would take n 2. Etc n n(n+1)/2 = (n^2 + n)/2 = O(n 2 ) 30

31 Look back at the code and see if it looks quadratic Notice the loop inside of the loop. What s the worst case for the big-o of each loop? Multiply nested loops. 31

32 Solution 2: Sort and Compare If the characters are sorted in each string, the end result must be equal strings! So make a list of each string, Sort them Compare the resulting lists. 32

33 Solution 2: Code def anagramsolution2(s1,s2): alist1 = list(s1) alist2 = list(s2) alist1.sort() alist2.sort() pos = 0 matches = True while pos < len(s1) and matches: if alist1[pos]==alist2[pos]: pos = pos + 1 else: matches = False return matches 33

34 Big O of Solution 2 Sorting takes nlogn time So the big O of this code looks like O(nlogn + nlogn + n) = O(2nlogn + n) O(nlogn) 34

35 Solution 3 (Brute Force) Take string1. Generate all of its anagrams. See if any of them are string2. Why didn t the book implement this? No good reason. I ll give a quick and dirty version on the next page. 35

36 Solution which hides all the dirty work from itertools import permutations def anagramsolution3(s1,s2): perms = [''.join(p) for p in permutations(s1)] for x in perms: if x == s2: return True return False 36

37 So why didn t the book show you the code? How many permutations are there of a string of length n? How many choices for the first character? The next? The next?. n*(n-1)*(n-2)* *3*2*1 N! Yikes. 37

38 Solution 4: Count and Compare Key insight: Each string must contain the same number of a s, b s, c s, and so on. So for each string, count up the occurrence of each of the 26 characters. Then at the end, see if those counts of the 26 characters are the same. 38

39 Solution 4 Code: Pay attention to big-o def anagramsolution4(s1,s2): c1 = [0]*26 c2 = [0]*26 for i in range(len(s1)): pos = ord(s1[i])-ord('a') c1[pos] = c1[pos] + 1 for i in range(len(s2)): pos = ord(s2[i])-ord('a') c2[pos] = c2[pos] + 1 j = 0 stillok = True while j<26 and stillok: if c1[j]==c2[j]: j = j + 1 else: stillok = False return stillok 39

40 Let s try out anagramsolution1 In listing_2_6, add some lines to set two varables, maybe a and b, to two strings like bat and tab. Run the anagram program on those two strings and print the result. 40

41 Using the Debugger in PyCharm Debugging is an important skill in programming. Often we debug by inserting print statements in our programs which works really well for limited debugging. But, for really in-depth debugging we can use debugging tools within good IDEs. 41

42 In PyCharm I ll show you how to set breakpoints There are also some YouTube videos if you can t remember what we do in lab Once breakpoints are set, use Run Debug. The code will stop at designated breakpoints. You can look at variable values You can step line by line, stepping into or over methods as you desire. 42

43 Let s also step through the other two programs First Sort and Compare. Then Count and Compare. 43

44 Problem 3 The book gives code for 3 solutions for determining whether two strings are anagrams of each other (Listing 2.6, Listing 2.7, and Listing 2.8). In this problem, you will experimentally determine the relative growth rates of each method by running them on strings of length 100 up to 1,000 (step 50) Here s a random method for generating a random string of length size: import string import random def randomstring(size): return ''.join(random.choice(string.ascii_lowercase) for _ in range(size)) 44

45 More on Problem 3 Each method will take the longest if it works on an anagram. Here s a way to rearrange a string randomly: def makeanagram(s): temp = s; t = '' while temp!= '': i = random.randrange(0, len(temp)) t += temp[i] temp = temp[:i] + temp[(i+1):] return t 45

46 More on Problem 3 o In a for-loop, vary n from 100 to 1000, step 50. Create a random string of length n. Call it s. Make an anagram of s by calling makeanagram. Call it t. Now time each of the three methods by passing in the parameters s and t. o Time how long each method to determine whether the string is an anagram. Make your output look something like this so that it is easy to graph: N Sol1 Sol2 Sol

47 More on Problem 3 o You should notice that Sol1 (Solution 1, Listing 2.6) takes much, much longer than the other two methods. o Now let s just look at the last two methods. We can run these for much larger sizes. Let s do another experiment without doing Listing 2.6. Just looking at the last two methods, time them for n = up to 100,000, step 5,000. Graph them. 47