CS502 Design & Analysis of Algorithms

Transcription

1 1. The word Algorithm comes from the name of the Muslim author Abu Ja far Mohammad ibn Musaal-Khowarizmi or Al-Khwarizmi Khwarizm Kheva Uzbekistan 2. In order to say anything meaningful about our algorithms, it will be important for us to settle on a. C++ program Java program Pseudo program Mathematical model of computation pg A RAM is an idealized machine with random-access memory. 256MB 512MB an infinitely large pg GB 4. Which formula is used for calculating worst case running time? Tworst n) = max T ( ) ( I = n I 1 Tworst( n) = max I = 1 T ( I) Tworst ( n) = max I = 1 T ( n) Tworst( n) = max I = n T ( n)

2 2 CS502 Design & Analysis of Algorithms 5. Divide-and-conquer involves breaking the problem into a small number of Sieve pivot Sub problems pg 34 Selection 6. In the analysis of Selection algorithm, we eliminate a constant fraction of the array with each phase; we get the convergent series in the analysis, linear arithmetic geometric pg 37 harmonic 7. One of the clever aspects of heaps is that they can be stored in arrays without using any. Pointers pg 40 constants variables functions 8. Quick sort can work well in virtual memory environment. It is true It is false Sorting is not performed in virtual memory environment Either true or false 9. What is common between Bubble sort, Insertion sort, Selection sort, Quick sort, and Heap sort? All are in-place algorithms pg 54 All are stable algorithms None of these All are unstable algorithms 10. Counting sort algorithm sorts in Θ(n + k) Θ(n - k) Θ(n) Θ(k) 11. If there are Θ (n 2 ) entries in edit distance matrix then the total running time is Θ (1) Θ (n 2 ) pg 84 Θ (n) Θ (n log n) 12. A product of matrices is if it is either single matrix or the product of two matrix products, surrounded by parentheses. Fully parenthesized Partially parenthesized Not parenthesized None of the options 13. Time complexity of chain matrix multiplication is Θ (n 3 ) and space complexity is Θ (n 2 )

3 3 CS502 Design & Analysis of Algorithms Θ (n 3 ) Θ (n log n) Θ (log n) 14. Suppose we have three items as shown in the following table, and suppose the capacity of the knapsack is 50 i.e. W = 50. Item Value Weight The optimal solution is to pick Items 1 and 2 Items 1 and 3 Items 2 and 3 None of these 15. Dynamic programming algorithms Store the results of end points Store the problems statment Do not store the subproblem results Do store the results of intermediate subproblems pg Algorithm s essential elements are Step wise solution Stepwise solution and finite time Step wise solution finite inputs Stepwise approach in which time and memory does not matter. 17. Suppose we have an algorithm that carries out N 2 operations for an input of size N. Let us say that a computer takes 1 microsecond (1/ second) to carry out one operation. How long does the algorithm run for an input of size 3000? 90 seconds 9 seconds 0.9 seconds 0.09 seconds 18. What is the solution to the recurrence T(n) = T(n/2)+n, T(1) = 1? O(logn) O(n) 37 O(nlogn) O(n 2 ) 19. Which of the following is true? The worst case time complexity of the quick sort is NlogN The best case time complexity of the merge sort is logn The worst case time complexity of the selection sort is N^2 The worst case time complexity of the merge sort is logn 20. In Quick Sort, partition algorithm partitions the array into sub arrays. 2 3 pg 46 1

4 4 Q # 01: In order to say anything meaningful about our algorithms, it will be important for us to settle on a. C++ program Java program Pseudo program Mathematical model of computation Q # 03: Divide-and-conquer involves breaking the problem into a small number of Pivot Sub problems Selection Sieve Q # 04: For the heap sort, access to nodes involves simple operations. Arithmetic 41 binary algebraic logarithmic Q # 05: If a sorting algorithm solely based on comparisons of keys in the array then it is impossible to sort more efficiently than Ω (n lg n) Θ (n lg n) Ο (n lg n) None of these Q # 06: The comparison based algorithm defines a(n). decision tree 54 array linked list stack 4 Q # 07: The main shortcoming of counting sort is that it is useful for Small Integers 71 Small characters Floats None of these Q # 08: Memoization is a process To avoid unnecessary repetitions by writing down the results of recursive calls and looking them again if needed later 74 To avoid repeated data by writing down the results of input array and looking them again if needed later. None of these All options are correct Q # 09: Matrix multiplication is An associative but not commutative operation 85

5 An associative and commutative operation Not associative but commutative operation Neither associative nor commutative operation Q # 10: We can find the product A x B of matrices A and B, only if they are compatible which means, No of Columns of A must be equal to No of Rows of B No of Columns of A must be equal to No of Columns of B No of Rows of A must be equal to No of Rows of B Order of A must be equal to order of B Q # 11: A product of matrices is if it is either single matrix or the product of two matrix products, surrounded by parentheses. Fully parenthesized Partially parenthesized Not parenthesized None of the options Q # 12: Merge sort requires Extra storage 54 No need of extra storage Sometimes requires extra storage Extra time than other sorting algorithms Q # 13: Heap sort is In place and stable Out place and stable Out place but not stable In place but not stable 54 Q # 14: Worst case running time of Quick Sort algorithm for an array with n elements is? 2 n n n 2 n 8 n 5 Q # 15: Which of the following is calculated with Big O notation? Medium bounds Lower bounds Upper bounds 25 Both upper and lower bound Q # 16: Which of the following is calculated with Big Theta notation? Lower bounds 25 Upper bounds Both upper and lower bound Medium bounds Q # 17: Identify the maximal points in given set, according to 2-D maxima (the points that are NOT dominated by other points).

6 {( 2,5 ),( 4, 4 ),( 4,11 ),( 5,1 ),( 7,7 ),( 7,13 ),( 9,10 ),( 11,5 ),( 12,12 ),( 13,3 ),( 14,10 ),( 15,7) } { } ( 7,13 ),( 12,12 ),( 14,10 ),( 15,7) { } ( 7,7 ),( 7,13 ),( 9,10 ),( 11,5 ),( 14,10) { } ( 2,5 ),( 4, 4 ),( 4,11 ),( 5,1 ),( 14,10) ( 4, 4 ),( 4,11 ),( 7,13 ),( 9,10)( 14,10) { } n i= 1 n Q # 18: is equal to, n(n+1) n(n) n(n+1)/3 n+1 20 Q # 19: What is the running time of the above sorting algorithm in worst case? 2 Θ( n ) 3 Θ( ) n 39 Θ( log ) n n Θ( log n) 6 Q # 20: The reason for introducing Sieve Technique algorithm is that it illustrates a very important special case of Divide-and-conquer 34 Decrease and conquer Greedy nature 2-dimension Maxima MIDTERM EXAMINATION

7 Q # 01: Random access machine or RAM is a/an Machine build by Al-Khwarizmi Mechanical machine Electronics machine Mathematical model 10 What is common between Bubble sort, Insertion sort, Selection sort, Quick sort, and Heap sort? All are in-place algorithms (pg54) All are stable algorithms None of these All are unstable algorithms 7 In order to say anything meaningful about our algorithms, it will be important for us to settle on a. C++ program Java program Pseudo program Mathematical model of computation (pg10) Q # 04: In which order we can sort? increasing order or decreasing order 39 both at the same time increasing order only decreasing order only Q # 05: For the heap sort we store the tree nodes in level-order traversal 40 in-order traversal pre-order traversal post-order traversal Q # 06: Quick sort procedure was invented by Hoare in 1960 Sedgewick Mellroy Coreman Q # 07: For sorting algorithms which one is the efficient algorithm having running time? O(n 2 ) O(n log n) not sure O(n 2 log n) O(n 3 ) Q # 08: When a recursive algorithm revisits the same problem over and over again, we say that the optimization problem has sub-problems. Overlapping Over costing Optimized None of these

8 Q # 09: A problem exhibits if an optimal solution to the problem contains within it optimal solution to sub-problems. Optimal structure 92 Efficient structure Inefficient structure Unknown behavior Q # 10: A p q matrix A can be multiplied with a q r matrix B. The result will be a p r matrix C. There are (p. r) total entries in C and each takes to compute. (q) 84 (1) (n2) (n3) Q # 11: Dynamic programming is Other name of rescursion Other name of divide and conquer Recursion without repetition 75 Recurrence development technique Q # 12: Dynamic programming algorithms Store the results of end points Store the problems statement Do not store the subproblem results Do store the results of intermediate subproblems 75 Q # 13: The worst case running time of the algorithm given below is, 8 6 Θ( ) n 2n 6 Θ n 2 Θ( n ) Θ( 2nlg 6) 14 page Q # 14: Due to left-complete nature of binary tree, heaps can be stored in Arrays 40 Structures

9 Link list Stack 9 QNo.1 What is heap and what is heap order? (Mark2) Answer:- The heap is the section of computer memory where all the variables created or initialized at runtime are stored. The heap order property: in a (min) heap, for every node X, the key in the parent is smaller than or equal to the key in X. QNo.2 Quick sort such that sort the array in to non-increasing order? (Mark2) Answer:- Quick sorting, an array A[1..n] of n numbers We are to reorder these elements into increasing (or decreasing) order. More generally, A is an array of objects and we sort them based on one of the attributes - the key value. The key value need not be a number. It can be any object from a totally ordered domain. Totally ordered domain means that for any two elements of the domain, x and y, either x < y, x = y or x > y. QNo.3 Draw the cost table for chain multiplication problem with initial states(mark3) Answer:- Cost table for chain multiplication problem in initial stage is the diagonal entries (all m [i, i]) filled with zeros and rest of the table entries are empty which are to be filled in next steps of the table calculation. (A1)(A2A3A4...An) or (A1A2)(A3A4...An) or (A1A2A3)(A4...An) or (A1A2A3A4...An 1)(An) QNo.4 we can avoid unnecessary repetitions for recursive calls? (Mark3) Answer:- We can avoid these unnecessary repetitions by writing down the results of recursive calls and looking them up again if we need them later. This process is called memorization. Worst case for edit distance algorithm? What is the simple change that can change the worst case time? Answer:- Analysis of DP edit distance There are (n2 ) entries in the matrix. Each entry E(i,j) takes (1) time to compute. The total running is 2 (n ) Recursion clearly leads to the same repetitive call pattern that we saw in Fibonnaci sequence. To avoid this, we will use the DP approach. We will build the solutionb bottom-up. We will use the base case E(0,j) to fill first row and the base case E(I,0) to fill first column. We will fill the remaining E matrix row by row. Q:Describe an efficient algorithm to find the median of a set of 106 integers; it is known that there are fewer than 100 distinct integers in the set. Solution:- Step1:Start Step2:Find the 100 distinct numbers among 10^6 integers. Step3:Sort the 100 distinct numbers Step4:Count the distinct numbers Step5: if count is odd,middle number is the median Step6:if count is even,add the middle two numbers then divide by 2,the result is the median Step5:End number. What is the formula for generating Catalan numbers?

10 10 CS502 Design & Analysis of Algorithms Solution Equation (22) is a recurrence relation. C_(n+1) = C_n * [2(2n+1)] / (n+2) we have the values of n in one column and the values of C_n in another, then to put this formula in Excel, on the (n+1)-th row just replace C_n and n with the appropriate cells from the previous row. What are Catalan numbers? Give the formula. Catalan numbers form a sequence of natural numbers that occur in various counting problems, often involving recursively defined objects Formula is C(n) = 2n Cn / (n+1) Q-Write a pseudo code Fibonacci With memorization? -- (3) Sol: MEMOFIB(n) 1 if (n < 2) 2 then return n 3 if (F[n] is undefined) 4 then F*n+ MEMOFIB(n 1) + MEMOFIB(n 2) 5 return F[n] Q Write Down the steps of Dynamic programming (5) Dynamic programming is essentially recursion without repetition. Developing a dynamic programming algorithm generally involves two separate steps: Formulate problem recursively. Write down a formula for the whole problem as a simple combination of answers to smaller sub problems. Build solution to recurrence from bottom up. Write an algorithm that starts with base cases and works its way up to the final solution. Dynamic programming algorithms need to store the results of intermediate sub problems. This is often but not always done with some kind of table. We will now cover a number of examples of problems in which the solution is based on dynamic programming strategy. Q: How we build heap Sol:We build a max heap out of the given array of numbers A[1..n]. We repeatedly extract the maximum item from the heap. Once the max item is removed, we are left with a hole at the root. Q:Write Pseudo code for KNAPSACK algorithm? 5 marks Solution: KNAPSACK (n, W) 1 for w=0,w 2 do V[0,w] 0 3 for i=0,n 4 do V[i,0] 0 5 for w=0,w 6 do if(wi w i v +V[i-1,w- i w ]>V[i-1,w]) 7 then V[I,w] i v +V[i-1,w] 8 else V[i,w] V[i-1,w] The time complexity is clearly O(n,W), it must be cautioned that as n and W get large, both time and space complexity become significant. Q:Spelling correction in edit distance? 3 marks Sol:A better way to display this editing process is to place the words with the other: S D I M D M, M A - T H S, A - R T - S THE FIRST WORD HAS AGAP FOR EVERY INSERTION (1)AND THE SECOND WORD HAS A GAP FOR EVERY DELETION (D). MATHES (M) DO NOT COUNT. THE EDIT TRANSCRIPT IS DEFINED AS A STRING OVER THE ALPHABETM,S,I,d THAT DESCRIBES A TRANSFORMATION OF ONE STRING INTO OTHER. FOR EXAMPLE S D I M D M =4 Q: Differentiate b/w Bubble sort, insertion sort and selection sort? 3 marks SOLUTION:Bubble sort: scan the array. Whenever two consecutive items are found that are out of order, swap them. Repeat until all consecutive items are in order. Insertion sort: assume that A*1 i-1] have already been sorted. Insert A[i] into its properposition in this sub array. Create this position by shifting all larger elements to the right. Selection sort: Assume that A[1..i-1] contain the i-1 smallest elements in sorted order. Find the smallest in A*i.n+ swap it with A*i+. Write down the steps of dynamic programming strategy. (2 marks)

11 11 CS502 Design & Analysis of Algorithms Solution: Developing a dynamic programming algorithm generally involves two separate steps: 1_formulate problem recursively. Write down a formula for the whole problem as a simple combination of answers to smaller sub problems. 2_ Build solution to recurrence from bottom up: Q: Solve the recursion problem. (5marks.) Solution: Recursion clearly leads to the same repetitive call pattern that we saw in Fibonnaci sequence. To avoid this, we will use the DP approach. We will build the solution bottom-up. We will use the base case E(0,j) to fill first row and the base case E(I,0) to fill first column. We will fill the remaining E matrix row by row.if we trace through the recursive calls to MemoFib, we find that array F[] gets filled from bottom up i.e., first F*2+, then F*3+, and so on, upto F[n]. we can replace recursion with a simple for-loop that just fills up the array F[] in that order. We are given an array of n elements of x1, x2,x3,,,,,xn, Q: Suggest best sorting algorithm of order On. (5 marks). Solution:The main shortcoming of counting sort is that it is useful for small integers, i.e., 1 k where k is small. If this were a million more, the size of the rank array would also be a million. Radix sort provides a nice work around this limitation by sorting numbers one digit at a time. Q: What are the two steps generally involved while developing dynamic programming algorithm. Solution: Developing a dynamic programming algorithm generally involves two separate steps: Q: How we build heap? (2) Solution: We build a max heap out of the given array of numbers A[1..n]. We repeatedly extract the maximum item from the heap. Once the max item is removed, we are left with a hole at the root. What are the applications of edit distance technique? Name any three (3) Solution: 1. Spelling Correction 2. Plagiarism Detection 3. Computational Molecular Biology 4. Speech recognition. What is the worst case running time for the bucket sort? What simple change is required in the algorithm to preserve its linear expected running time and makes it worst case time Θ(n log n) (5) Solution: The worst case for bucket sort occurs when the all inputs falls into single bucket, for example. Since we use insertion sort for sorting buckets and insertion sort as a worst case of O(n2). the worst case runtime for bucket sort is O(n2). By using an algorithm with worst case runtime of O(nlgn) instead of insertion sort for sorting buckets, we can ensure that worst case is O(nlgn) without affecting the average case behavior. Given an unordered list of n x0, x1, x2,, xn and elements is common, if there are atleast n/5 copies of it.we want to identify all the common numbers in our list. Give O(n log n) to solve the problem. (5) Solution:We define R n to be the set of ordered n-tuples of real numbers, Rn := {(x1, x2... xn) : x1,..., xn R}.The elements of Rn are called vectors. Given a vector x = (x1,..., xn), the numbers x1,..., xn are called the components of x. You are already quite familiar with Rnfor small values of n. Q: Find the out cost A1=5 x 4 A2= 4 x 6 A3= 6x2 (2 marks) Solution:-((A1A2)A3) = (5 4 6) + (5 6 2)= 180 (A1(A2A3)) = (4 6 2) + (5 4 2) = 88

12 Q: How to construct an optimal solution for o/1 knapsack problem? Sol: Construction of optimal solution: Construct an optimal solution from computed information. Let us try this: If items are labelled 1, 2,..., n, then a subproblem would be to find an optimal solution for S k = items labelled 1, 2,..., k This is a valid subproblem definition. The question is: can we describe the final solution S n in terms of subproblems S k? Unfortunately, we cannot do that. Here is why. Consider the optimal solution if we can choose items 1 through 4 only. Solution Items chosen are 1, 2, 3, 4 Total weight: = 14 Total value: = 20 Now consider the optimal solution when items 1 through 5 are available Q: Effect of calling max heap Solution:-The smallest key is in the root in a min heap; in the max heap, the largest is in the root. Q: Suggest the criteria for measuring algorithms. Also discuss the issues need to be discussed in the algorithm design. Solutions:- In order to design good algorithms, we must first agree the criteria for measuring algorithms. The emphasis in this course will be on the design of efficient algorithm, and hence we will measure algorithms in terms of the amount of computational resources that the algorithm requires. These resources include mostly running time and memory. Depending on the application, there may be other elements that are taken into account, such as the number disk accesses in a database program or the communication bandwidth in a networking application Q: Write the STABLE OR IN-PLACE of QUICK, HEAP, COUNTING, MERGE Sorts. [5 MARKS] Answer: In-place Stable In-place Stable Merge sort Bubble sort Bubble sort Quick sort Insertion sort Insertion sort Heap sort Selection sort Counting sort 12 Q: Define MEMOIZATION? [2 MARKS] Answer: We can avoid unnecessary repetition by write down the recursive calls and look them when we need. Define worst case and average case of Quick sort? Answer: Worst case: We maximize over all possible values of q, means the selection of pivot q which gives the maximum (worst) time for sorting. The worst case running time is O (n2). Average case: Average case analysis of quick sort, the average is computed over all possible random choices of the pivot index q. The average case running time for quick sort is. (n log n). Q-We can avoid unnecessary repetitions for recursive calls?( I thnk how can we...) [PAGE 74] Answer: By using Fibonacci Sequence We can avoid this unnecessary repetition by writing down the results of recursive calls and looking them up again if we need them later. This process is called memoization. Spelling correction in edit distance? 3 marks

13 13 CS502 Design & Analysis of Algorithms Answer: Spelling Correction: If a text contains a word that is not in the dictionary, a 'close' word, i.e. one with a small edit distance, may be suggested as a correction. Most word processing applications, such as Microsoft Word, have spelling checking and correction facility. When Word, for example, finds an incorrectly spelled word, it makes suggestions of possible replacements. Q-What is Bubble sort? Answer: Bubble sort: Scan the array. Whenever two consecutive items are found that are out of order, swap them. Repeat until all consecutive items are in order. Q-What is the worst case running time for the Quick sort? What simple change is required in the algorithm to preserve its linear expected running time and makes it worst case time.(n log n). Answer Worst case running time is O (n). The simple change which can change the running time of the edit distance Algorithm is the number of entries n2. Q: In edit distance what is the simple change that can change the worst case time? Ans: The simple change which can change the running time of edit distance algorithm is the number of entries n2,or you can say the term n. In other words you can say the length of the strings determine the running time of the computation. Q:Define the worst case of edit distance algorithm? Answer: The worst case running time of edit distance DP algorithm is O (n2), which is the number of entries in the table n2 as each entry need constant time to compute. Q:Average-case Analysis of Quicksort Answer: Average case analysis of Quick sort: The running time of Quick sort depends on the selection of pivot q which is done randomly. For average case analysis of quick sort, average is computed over all possible random choices of the pivot index q. The average case running time for quick sort is. (N log n). Q: Worst case Analysis of Quick sort? Answer: Worst case analysis of Quick sort: For worst case we maximize over all possible values of q, means the selection of pivot q which gives the maximum (worst) time for sorting. The chances are that the pivot values q=1 (start) or n (end) or n/2 (middle) happens to give maximum time values. The worst case running time is O (n2). Q: What is radix sort and explain it with examples. Answer: Radix sort: In linear time sorting, counting sort is used for the numbers in the range 1 to k where k is small and it is based on determining the rank of each number in final sorted array. But it is useful only for small integers i.e., 1...k where k is small. But if k were very large, then the size of the rank array formed would also be very large which is not efficient. So solution for such cases is the Radix sort which works by sorting one digit at a time. Example: [1] 8[4]1 [1] [3] 3[7]3 [3] [5] 1[8]5 [8]41 Q: Essential constraint for the counting sort. Answer: Essential constraint for the counting sort is that numbers to be sorted must be small integers i.e., 1...k where k is small. Q: Better appropriate to cope with chain matrix multiplication.

14 Answer: Better appropriate to cope with chain matrix multiplication is to do its Dynamic Programming formulation. It is breaking up the problem into sub problems, solving and string and then combining solutions to those sub problems to solve the global problem. Q: Heapify procedure and Build Heap explain in simple words with example. Answer: If an element in the Heap is not at its proper place means it is violating the Heap Order, the Heapify procedure is used to fix it and place it at its proper position. In Heapify, we recursively swap the element with its larger one child and stop at a stage when this element is larger than both of its children or it becomes the leaf node. Build Heap procedure is used for building Heap from any list and it is done by applying the Heapify procedure on each element starting from bottom and going upward to the root. Starting is done at second last level as the leaf nodes have no children so already in heap order. Q: Difference between worst case \Average cases Analysis of Quicksort explain in simple words with example. Answer: The running time of Quick sort depends on the selection of pivot q which is done randomly. For worst case we maximize over all possible values of q, means the selection of pivot q which gives the maximum (worst) time for Sorting. The chances are that the pivot values q=1 (start) or n (end) or n/2 (middle) happen to give maximum time values. The worst case running time is O (n2). For average case analysis of quick sort, the average is computed over all possible random choices of the pivot index q. The average case running time for quick sort is. (n log n). Q: Edit Distance in Speech Recognition. Answer: Algorithm is similar to those for the edit-distance problem is used in some speech recognition systems. Find a close match between a new utterance and one in library of classified utterance. Q: What is sorting? Describe slow running sorting algorithms. 5 marks Answer: Sorting is the process of arranging items in sequence. Slow running sorting algorithms: [PAGE 39] There are a number of well-known slow O(n2) sorting algorithms. 1. Bubble sort: Scan the array. Whenever two consective item are found that are out of order, swap them. Repeat until all consecutive items are in order. 2. Insertion sort: Assume that A[1..i-1] have already been sorted. Insert A[i] into its proper position in this sub array. Create this position by shifting all larger elements to the right. 3. Selection sort: Assume that A[1.i-1] contain the i-1 smallest element in A[i.n] swap it with A[i]. Q: What is Catalan Numbers and write their formula. 3 marks Answer: The Catalan numbers are famous functions in combinatrics. Formula: 14 Slow Bubble sort: Insertion sort Selection sort: O(n2) Fast Quicksort Merge sort Heap sort O(n log n) Fastest Count Radix Bucket Ω(n log n) Lower Bounds for Sorting: The best we have seen so far is O(n log n) algorithms for sorting. Is it possible to do better than O(n log n)? If a sorting algorithm is solely based on

15 comparison of keys in the array then it is impossible to sort more efficiently than (n log n) time. All algorithms we have seen so far are comparison-based sorting algorithms. Consider sorting three numbers a1, a2, and a3. There are 3! = 6 possible combinations: (a1, a2, a3), (a1, a3, a2), (a3, a2, a1) (a3, a1, a2), (a2, a1, a3), (a2, a3, a1) One of these permutations leads to the numbers in sorted order. The comparison based algorithm defines a decision tree. Here is the tree for the three numbers. Q: Following is the cost table for chain matrix multiplication problem with initial state. Mark the cells with question mark? which are updated after first iteration Ans: m[1, 2] = m[1, 1] +m[2, 2] + p0 p1 p2 = = 120 m[2, 3] = m[2, 2] +m[3, 3] + p1 p2 p3 = = 48 m[3, 4] = m[3, 3] +m[4, 4] + p2 p3 p4 = = 84 m[4, 5] = m[4, 4] +m[5, 5] + p3 p4 p5 = = 42 Q: Why do we analyze the average case performance of a randomized algorithm and not its worst case performance? Average case performance :The analysis of the algorithm s performance is the same for all inputs.in this case the average is computed over all possible random choices that the algorithm might make for the choice of the pivot index in the second step of the QuickSort procedure. worst case performance is a recursive program, it is natural to use a recurrence to describe its running time. But unlike MergeSort, where we had control over the sizes of the recursive calls, here we do not. It depends on how the pivot is chosen. Suppose that we are sorting an array of size n, A[1 : n], and further suppose that the pivot that we select is of rank q, for some q in the range 1 to n. It takes _(n) time to do the partitioning and other overhead, and we make two recursive calls. The first is to the subarray A[1 : q 1] which has q 1 elements, and the other is to the subarray A[q + 1 : n] which has n q elements. So if we ignore the _(n) (as usual) we get the recurrence: T(n) = T(q 1) + T(n q) + n Q: Suggest and describe modifications of the implementation of QuickSort that will improve its performance. Ans:The running time of quicksort depends heavily on the selection of the pivot. If the rank of the pivot is very large or very small then the partition (BST) will be unbalanced. Since the pivot is chosen randomly in our algorithm, the expected running time is O(n log n). The worst case time, however, is O(n2). Luckily, this happens rarely. 15

16 16 Algorithm: Informal Definition: An algorithm is any well-defined computational procedure that takes some values, or set of values, as input and produces some value, or set of values, as output. An algorithm is thus a sequence of computational steps that transform the input into output. Algorithms, Programming: A good understanding of algorithms is essential for a good understanding of the most basic element of computer science: programming. Unlike a program, an algorithm is a mathematical entity, which is independent of a specific programming language, machine, or compiler. A RAM is an idealized machine with an infinitely large random-access memory. Instructions are executed one-by-one (there is no parallelism). 2-dimension maxima: Let us do an example that illustrates how we analyze algorithms. Suppose you want to buy a car. You want the pick the fastest car. But fast cars are expensive; you want the cheapest. You cannot decide which is more important: speed or price. Definitely do not want a car if there is another that is both faster and cheaper. We say that the fast, cheap car dominates the slow, expensive car relative to your selection criteria. So, given a collection of cars, we want to list those cars that are not dominated by any other. Here is how we might model this as a formal problem. Let a point p in 2-dimensional space be given by its integer coordinates, p = (p.x, p.y). A point p is said to be dominated by point q if p.x _ q.x and p.y _ q.y. Given a set of n points, P = {p1, p2,..., pn} in 2-space a point is said to be maximal if it is not dominated by any other point in P. Running Time Analysis: The main purpose of our mathematical analysis will be measuring the execution time. We will also be concerned about the space (memory) required by the algorithm. The running time of an implementation of the algorithm would depend upon the speed of the computer, programming language, optimization by the compiler etc. Although important, we will ignore these technological issues in our analysis. To measure the running time of the brute-force 2-d maxima algorithm, we could count the number of steps of the pseudo code that are executed or, count the number of times an element of P is accessed or, the number of comparisons that are performed. Worst-case time is the maximum running time over all (legal) inputs of size n. Let I denote an input instance, let I denote its length, and let T(I) denote the running time of the algorithm on input I. Then Tworst(n) = max I =n T(I)

17 17 CS502 Design & Analysis of Algorithms Average-case time is the average running time over all inputs of size n. Let p(i) denote the probability of seeing this input. The average-case time is the weighted sum of running times with weights being the probabilities: Tavg(n) =X I =n p(i)t(i) We will almost always work with worst-case time. Average-case time is more difficult to compute; it is difficult to specify probability distribution on inputs. Worst-case time will specify an upper limit on the running time. Worst-case running time is _(n2). This is called the symptotic growth rate of the function. Sorting takes _(n log n); we will show this later when we discuss sorting. The for loop executes ntimes. The inner loop (seemingly) could be iterated (n 1) times. It seems we still have an n(n סּ 1) or ( n2 )סּ algorithm. Asymptotic Notation: You may be asking that we continue to use the notation _() but have never defined it. Let s remedy this now. Given any function g(n), we define _(g(n)) to be a set of functions that asymptotically equivalent to g(n). Formally: _(g(n)) = {f(n) there exist positive constants c1, c2 and n0 such that 0 _ c1g(n) _ f(n) _ c2g(n) for all n _ n0} Lower bound: f(n) grows asymptotically at least as fast as n2. For this, need to show that there exist positive constants c1 and n0, such that f(n) _ c1n2 for all n _ n0. Consider the reasoning f(n) = 8n2 + 2n 3 _ 8n2 3 = 7n2 + (n2 3) _ 7n2 Thus c1 = 7. We implicitly assumed that 2n _ 0 and n2 3 _ 0 These are not true for all n but if n _ p3, then both are true. So select n0 _ p3. We then have f(n) _ c1n2 for all n _ n0. Upper bound: f(n) grows asymptotically no faster than n2. For this, we need to show that there exist positive constants c2 and n0, such that f(n) _ c2n2 for all n _ n0. Consider the reasoning f(n) = 8n2 + 2n 3 _ 8n2 + 2n _ 8n2 + 2n2 = 10n2 The O-notation is used to state only the asymptotic upper bounds. The Ω-notation allows us to state only the asymptotic lower bounds. Divide: the problem into a small number of pieces Conquer: solve each piece by applying divide and conquer to it recursively Combine: the pieces together into a global solution. (Divide:) split A down the middle into two subsequences, each of size roughly n/2 (Conquer:) sort each subsequence by calling merge sort recursively on each. (Combine:) merge the two sorted subsequences into a single sorted list. Sieve Technique The reason for introducing this algorithm is that it illustrates a very important special case of divide-and-conquer, which I call the sieve technique. We think of divide-and-conquer as reaking the problem into a small number of smaller subproblems, which are then solved recursively. The sieve technique is a special case, where the number of subproblems is just 1. Selection Algorithm:It is easy to see that the rank of the The rank of the pivot x is q p + 1 in A[p..r]. Let rank x = q p + 1. If k = rank x then the pivot is kth smallest. If k < rank x then search A[p..q 1] recursively. If k > rank x then search A[q + 1..r] recursively. Find element of rank (k q) because we eliminated q smaller elements in A. A heap is a left-complete binary tree that conforms to the heap order. The heap order property: in a (min) heap, for every node X, the key in the parent is smaller than or equal to the key in X. In other words, the parent node has key smaller than or equal to both of its

18 children nodes. Similarly, in a max heap, the parent has a key larger than or equal both of its children Thus the smallest key is in the root in a min heap; in the max heap, the largest is in the root. An in-place sorting algorithm is one that uses no additional array for storage. A sorting algorithm is stable if duplicate elements remain in the same relative position after sorting. Theorem 1: Any comparison-based sorting algorithm has worst-case running time (n log n). Edit Distance The words computer and commuter are very similar, and a change of just one letter, p- m, will change the first word into the second. The word sport can be changed into sort by the deletion of the p, or equivalently, sort can be changed into sport by the insertion of p. The edit distance of two strings, s1 and s2, is defined as the minimum number of point mutations required to change s1 into s2, where a point mutation is one of: change a letter, insert a letter or delete a letter For example, the edit distance between FOOD and MONEY is at most four: FOOD! MOOD! MONfD! MONED! MONEY Constructing the Optimal Solution The algorithm for computing V[i, j] does not keep record of which subset of items gives the optimal solution. To compute the actual subset, we can add an auxiliary boolean array keep[i, j] which is 1 if we decide to take the ith item and 0 otherwise. We will use all the values keep[i, j] to determine the optimal subset T of items to put in the knapsack as follows: If keep[n,w] is 1, then n 2 T. We can now repeat this argument for keep[n 1,W wn]. If kee[n,w] is 0, the n 62 T and we repeat the argument for keep[n 1,W]. 18