Algorithms. Notations/pseudo-codes vs programs Algorithms for simple problems Analysis of algorithms

Transcription

1 Algorithms Notations/pseudo-codes vs programs Algorithms for simple problems Analysis of algorithms Is it correct? Loop invariants Is it good? Efficiency Is there a better algorithm? Lower bounds * DISC MATH, NCHU 1

2 What is an algorithm? Complete pseudo-codes are necessary Given a problem, can we always design an algorithm to solve it? Design and Analysis of Algorithms Different design paradigms Different analysis techniques Intractability results * DISC MATH, NCHU 2

3 Swapping Problem Swapping two numbers in memory tmp = x; x = y; y= tmp; Can we do it without using tmp? x = x+y; y = x-y; x = x-y; Why does this work? Does it always work? * DISC MATH, NCHU 3

4 Making changes Problem Want to make change for ANY amount using the fewest number of coins Simple greedy algorithm: keep using the largest denomination possible Works for coins/bills: 1,5,10,25,100,500,1000. Does it always work? Fails for the following coins: 1,5,7,10 e.g: 14 = , 14 = Read proof from the text * DISC MATH, NCHU 4

5 Finding MAX Problem Q1. How do you find the max of n numbers (stored in array A?) Formal specs: INPUT: A[1..n] - an array of integers OUTPUT: an element m of A such that A[j] m, 1 j length(a) Find-max (A) 1. max A[1] How many comparisons? 2. for j 2 to length(a) 3. do if (max < A[j]) 4. max A[j] 5. return max Q2. Can you think of another algorithm? Take a minute. How many comparisons does it take? * DISC MATH, NCHU 5

6 Reasoning (formally) about algorithms 1. I/O specs: Needed for correctness proofs and performance analysis. e.g. for sorting: INPUT: A[1..n] - an array of integers OUTPUT: a permutation B of A such that B[1] B[2]. B[n] 2. CORRECTNESS: The algorithm satisfies the output specs for EVERY valid input. 3. ANALYSIS: Compute the running time, the space requirements, number of cache misses, disk accesses, network accesses,. * DISC MATH, NCHU 6

7 Correctness proofs of algorithms (1/3) INPUT: A[1..n] - an array of integers OUTPUT: an element m of A such that m A[j], 1 j length(a) Find-max (A) 1. max A[1] 2. for j 2 to length(a) 3. do if (max < A[j]) 4. max A[j] 5. return max Prove that for any valid Input, the output of Find-max satisfies the output condition. Proof 1 [by contradiction]: Suppose the algorithm is incorrect. Then for some input A, (a) max is not an element of A or (b) ( j max < A[j]). Max is initialized to and assigned to elements of A (a) is impossible. For (b): after the j th iteration of the forloop (lines 2 4), max A[j]. From lines 3,4, max only increases. Therefore, upon termination, max A[j], which contradicts (b). * DISC MATH, NCHU 7

8 Correctness proofs of algorithms (2/3) INPUT: A[1..n] - an array of integers OUTPUT: an element m of A such that m A[j], 1 j length(a) Find-max (A) 1. max A[1] 2. for j 2 to length(a) 3. do if (max < A[j]) 4. max A[j] 5. return max Prove that for any valid Input, the output of Find-max satisfies the output condition. Proof 2 [use loop invariants]: (identify invariant) I(j): At the beginning of iteration j of for loop, max contains the maximum of A[1..j-1]. (Proof) True for j=2. For j > 2, assume that (j-1) holds. So at the beginning of iteration j-1, max = max.of A[1..j-2]. * DISC MATH, NCHU 8

9 Correctness proofs of algorithms (3/3) Case (a) A[j] is the maximum of A[1..j]. In lines 3,4, max is set to A[j]. Case (b) A[j] is not the maximum of A[1..j], so the maximum of A[1..j] is in A[1..j-1]. By our assumption max already has this value and by lines 3-4 max is unchanged in this iteration. * DISC MATH, NCHU 9

10 Loop invariant proofs STRATEGY: We proved that the invariant holds at the beginning of iteration j for each j used by Find-max. Upon termination, j = length(a)+1. (WHY?) The invariant holds, and so max contains the maximum of A[1..n] -- STRUCTURED PROOF TECHNIQUE -- VERY SIMILAR TO INDUCTION Advantages: Rather than reason about the whole algorithm, reason about SINGLE iterations of SINGLE loops. * DISC MATH, NCHU 10

11 Extension of finding MAX Q1. How do you find the max and min of n numbers (stored in array A?) Q2. Can you think of a FASTER algorithm? * DISC MATH, NCHU 11

12 Detecting palindromes Problem reads the same forwards and backwards e.g. pop, noon [Approach 1] Using an auxiliary array Assume the array is named as a[1],a[2],,a[n] compare a[i] to a[n+1-i] for equality [Approach 2] Without using an auxiliary array Initially, answer = true For i := 1 to [n/2] if a i a n+1-i then answer := false * DISC MATH, NCHU 12

13 A Harder Problem INPUT: A[1..n] - an array of integers, k, Think 1 k length(a) for a minute OUTPUT: an array B[1..n], such that the element B[k] is the k th largest element in A. Brute Force: For i= n to 1 do Find the maximum (by executing finding MAX) remove it from A and put it in B[i] (that is, repeat n times of finding MAX ) Q: How good is this algorithm? Q: Is there a better algorithm? * DISC MATH, NCHU 13

14 Sorting and Searching Very basic operations Used very,very often in real applications LOTS of new ideas * DISC MATH, NCHU 14

15 Searching an array Given an array A[1..m] does there exist a number (key) n? Unsorted array: linear search Sorted array: Can you do better? YES Binary search: Use the sorted property to eliminate large parts of the array. * DISC MATH, NCHU 15

16 Linear Searching Given an integer array A[1..n] does there exist an integer x? Procedure linear-search (x:integer, a[1], a[2],, a[n]: integers) i=1 while (i n and x a[i]) {i := i+1} if i n then location := i else location := 0 * DISC MATH, NCHU 16

17 Pseudocode for Binary Search * DISC MATH, NCHU 17

18 Data structures By preprocessing (sorting) the data into a data structure (sorted array), we are able to speed up search queries. Very common idea in Computer Science Many other data structures are commonly used: linked lists, trees, graph, hash tables,. * DISC MATH, NCHU 18

19 Sorting using Find-Max Simple sorting algorithm using Find-max 1. j=n 2. while (j>1){ 3. maxindex = index of max A[1..j] 4. swap A[maxindex], A[j] 5. j=j-1 6. } Is this the fastest possible sort? * DISC MATH, NCHU 19

20 Sorting: Insertion sort We maintain a subset of elements sorted within a list. The remaining elements are off to the side somewhere. Initially, think of the first element in the array as a sorted list of length one. One at a time, we take one of the elements that is off to the side and we insert it into the sorted list where it belongs. This gives a sorted list that is one element longer than it was before. When the last element has been inserted, the array is completely sorted. English descriptions: - Easy, intuitive. - Often imprecise, may leave out critical details. * DISC MATH, NCHU 20

21 Insertion sort (of an array A) for j=2 to length(a) do key=a[j] i=j-1 while i>0 and A[i]>key do A[i+1]=A[i] i-- A[i+1]=key Can you understand The algorithm? I would not know this is insertion sort Moral: document code What is a good loop invariant? It is easy to write a loop invariant if you understand what the algorithm does. Key technique: Using assertions. * DISC MATH, NCHU 21

22 Using Assertions (1/3) An assertion is a statement about the current state of the data structure that is either true or false. Useful for thinking about algorithms developing describing proving correctness An assertion need not consist of formal/math mumbo jumbo Use an informal description An assertion is not a task for the algorithm to perform. It is only a comment that is added for the benefit of the reader. * DISC MATH, NCHU 22

23 Assertions (2/3) Example of Assertions Preconditions: Any assumptions that must be true about the input instance. Postconditions: The statement of what must be true when the algorithm/program returns. Correctness: <PreCond> & <code> <PostCond> If the input meets the preconditions, then the output must meet the postconditions. If the input does not meet the preconditions, then nothing is required. * DISC MATH, NCHU 23

24 Assertions (3/3) Example of Assertions <precond> codea loop <loop-invariant> exit when <exit Cond> codeb endloop codec <postcond> * DISC MATH, NCHU 24

25 Correctness of Insertion sort (1/3) for j=2 to length(a) do key=a[j] Insert A[j] into the sorted sequence A[1..j-1] i=j-1 while i>0 and A[i]>key do A[i+1]=A[i] i-- A[i+1]:=key Invariant: at the start of each for loop, A[1 j-1] consists of elements originally in A[1 j-1] but in sorted order Initialization: j = 2, the invariant trivially holds because A[1] is a sorted array * DISC MATH, NCHU 25

26 Correctness of Insertion sort (2/3) for j=2 to length(a) do key=a[j] i=j-1 while i>0 and A[i]>key do A[i+1]=A[i] i-- A[i+1]:=key Invariant: at the start of each for loop, A[1 j-1] consists of elements originally in A[1 j-1] but in sorted order Maintenance: the inner while loop moves elements A[j-1], A[j-2],, A[k] one position right without changing their order. Then the former A[j] element is inserted into k th position so that A[k-1] A[k] A[k+1]. A[1 j-1] sorted + A[j] A[1 j] sorted * DISC MATH, NCHU 26

27 Correctness of Insertion sort (3/3) for j=2 to length(a) do key=a[j] Insert A[j] into the sorted sequence A[1..j-1] i=j-1 while i>0 and A[i]>key do A[i+1]=A[i] i-- A[i+1]:=key Invariant: at the start of each for loop, A[1 j-1] consists of elements originally in A[1 j-1] but in sorted order Termination: the loop terminates, when j=n+1. Then the invariant states: A[1 n] consists of elements originally in A[1 n] but in sorted order * DISC MATH, NCHU 27

28 Many, many other sorts Good algorithms: merge sort, quick sort Terrible algorithms: bubble sort * DISC MATH, NCHU 28

29 MEMO Read section 3.1. Get familiar with bubble sort, insertion sort, Binary search, and linear serach. What is so called pseudocode? How can you prove the correctness of an algorithm? HW #9,11,13,17,21 of 3.1 * DISC MATH, NCHU 29