CSC236 Week 5. Larry Zhang

Transcription

1 CSC236 Week 5 Larry Zhang 1

2 Logistics Test 1 after lecture Location : IB110 (Last names A-S), IB 150 (Last names T-Z) Length of test: 50 minutes If you do really well... 2

3 Recap We learned two types of functions according to how they are defined closed-form function Recursively defined function We want to find the closed form of a recursively defined function, why? We learned a method to find the closed form Repeated substitution 3

4 That s a good amount of math, now let s analyse the runtime of recursive algorithms 4

5 ConcepTest What s the worst-case runtime T(n) of the following algorithm? def factorial(n): 1 if n == 1: 2 return 1 3 else: 4 return n * factorial(n-1) A. Θ(1) B. Θ(log n) C. Θ(n) D. Θ(n²) E. None of above 5

6 Prove it formally: Strategy 1. Look at the base case, get its runtime 2. Look at the recursive calls, get the runtime in terms of the runtime of the recursive calls. def factorial(n): 1 if n == 1: 2 return 1 3 else: 4 return n * factorial(n-1) 6

7 Base case T(1)? def factorial(n): 1 if n == 1: 2 return 1 3 else: 4 return n * factorial(n-1) The amount of work is constant T(1) = c where c is some constant 7

8 The recursive calls T(n)? def factorial(n): 1 if n == 1: 2 return 1 3 else: 4 return n * factorial(n-1) The amount of work include time spent on the factorial(n-1): T(n-1) some constant amount of work T(n) = T(n-1) + d, where d is some constant 8

9 So altogether... def factorial(n): 1 if n == 1: 2 return 1 3 else: 4 return n * factorial(n-1) This is a recursively defined function, we want to find the closed form of this function, and we know how to do it. 9

10 Apply repeated substitution Guess: 10

11 Solve n - k = 1, get k = n - 1 Plug in k = n - 1 Potential closed form which can be proven to be correct. Do the proof yourself! 11

12 So far, we can say def factorial(n): 1 if n == 1: 2 return 1 3 else: 4 return n * factorial(n-1) This recursive algorithm has worst-case runtime which is in Θ(n) (we know how to formally prove this, too) 12

13 Now, finally, you can say with confidence... def factorial(n): 1 if n == 1: 2 return 1 3 else: 4 return n * factorial(n-1) This is a Θ(n) algorithm. 13

14 ConcepTest 14

15 ConcepTest What s the asymptotic runtime of fib(n), knowing A. Θ(1) B. Θ(log n) C. Θ(n) D. Θ(n²) E. None of above It is exponential 15

16 Analysing runtime of recursive algorithm Approach #1 This is typically a lot of work. We may be able to save some time since we don t really care about the exact closed form when we only need a Theta bound. 16

17 Analysing runtime of recursive algorithm Approach #2 17

18 ConcepTest 18

19 ConcepTest 19

20 Analysing runtime of recursive algorithm Approach #3 There is a theorem that can help you get the asymptotic bound on the runtime super fast... 20

21 Master Theorem 21

22 Master Theorem 22

23 How to use master theorem First make sure you can actually use the master theorem for some recurrences you cannot use it, like T(n) = T(n-1) + 1 the recurrence must be of the above form if cannot use master theorem there are more powerful theorem s available, but they are not allowed in this course use the good old repeated substitution If master theorem apply, start by calculating logb a Then compare logb a to k, and decide which case it belongs to 23

24 Exercises 24

25 Consider the following functions. For each, decide which case of the master theorem applies (if any), and give the asymptotic worst-case runtime 25

26 EX 1 26

27 EX 2 27

28 EX 3 28

29 EX 4 29

30 EX 5 30

31 EX 6 Cannot use the master theorem directly, but can still do some bounding 31

32 Home Exercise Prove the master theorem by doing repeated substitution on This exercise will make you REALLY understand the theorem, and many other things. Proof 32

33 Next Example 33

34 First, clarify a notation The dot dot notation : [a..b] means all numbers from a to b, inclusive. e.g., [0..5] include 0, 1, 2, 3, 4, 5 [0..0] is just 0 [0..-1] is an empty list This is different from the Python colon (:) operator. 34

35 def xxx_xxxxx(a, x): Pre: A is a non-empty sorted list in non-decreasing order Post: Returns True if and only if x is in A 1 if len(a) == 1: 2 return A[0] == x 3 else: 4 m = len(a) // 2 # integer division, rounds down 5 if x <= A[m-1]: 6 return xxx_xxxxx(a[0..m-1], x) 7 else: 8 return xxx_xxxxx(a[m..len(a)-1], x) 35

36 def bin_search(a, x): Pre: A is a non-empty sorted list in non-decreasing order Post: Returns True if and only if x is in A 1 if len(a) == 1: 2 return A[0] == x 3 else: 4 m = len(a) // 2 # integer division, rounds down 5 if x <= A[m-1]: 6 return bin_search(a[0..m-1], x) 7 else: 8 return bin_search(a[m..len(a)-1], x) Binary Search 36

37 ConcepTest Assuming list slicing takes constant time, the asymptotic runtime of this binary_search is... def bin_search(a, x): Pre: A is a non-empty sorted list in non-decreasing order Post: Returns True if and only if x is in A 1 if len(a) == 1: 2 return A[0] == x 3 else: 4 m = len(a) // 2 # integer division, rounds down 5 if x <= A[m-1]: 6 return bin_search(a[0..m-1], x) 7 else: 8 return bin_search(a[m..len(a)-1], x) 37

38 ConcepTest In reality, list slicing takes linear time, so the real asymptotic runtime of this binary_search is... def bin_search(a, x): Pre: A is a non-empty sorted list in non-decreasing order Post: Returns True if and only if x is in A 1 if len(a) == 1: 2 return A[0] == x 3 else: 4 m = len(a) // 2 # integer division, rounds down 5 if x <= A[m-1]: 6 return bin_search(a[0..m-1], x) 7 else: 8 return bin_search(a[m..len(a)-1], x) 38

39 Better implemented binary search def bin_search(a, x, first, last): Pre: A is a non-empty sorted list in non-decreasing order Post: Returns True if and only if x is in A 1 if first == last: Idea: Manipulating indices on one list rather than making copies of the list. 2 return A[first] == x 3 else: 4 m = (first + last + 1) // 2 # integer division, rounds down 5 if x <= A[m-1]: 6 return bin_search(a, x, first, m-1) 7 else: 8 return bin_search(a, x, m, last) After this optimization, the runtime of the algorithm is really O(log n). 39

40 Takeaway This mathematical analysis of the runtime is a very powerful tool for algorithm design. You know exactly what impact each modification has on the overall runtime of the algorithm. Like how a master chef knows the impact of each ingredient. You can predict, before you code it, what is worth optimizing, and what is not. You can be super confident about it because everything is proven! Rather than just reusing existing algorithms created by other people, you can invent your own delicious dishes (efficient algorithms). 40