Department of Computer Science University of Babylon LECTURE NOTES OF ALGORITHMS: DESIGN TECHNIQUES AND ANALYSIS By Faculty of Science for Women( SCIW), University of Babylon, Iraq Samaher@uobabylon.edu.iq
Outlines Divide and Conquer Algorithm and design paradigm to various problems O(n log n) Algorithm for Counting Inversions I O(n log n) Algorithm for Counting Inversions II Strassen's Sub cubic Matrix Multiplication Algorithm O(n log n) Algorithm for Closest Pair I O(n log n) Algorithm for Closest Pair II QUZI Homwork
Divide and Conquer Algorithm and Design Paradigm to Various Problems Both merge sort and quicksort employ a common algorithmic paradigm based on recursion. This paradigm, divide-and-conquer, breaks a problem into subproblems that are similar to the original problem, recursively solves the subproblems, and finally combines the solutions to the subproblems to solve the original problem. Because divide-and-conquer solves subproblems recursively, each subproblem must be smaller than the original problem, and there must be a base case for subproblems. You should think of a divide-and-conquer algorithm as having three parts: Divide the problem into a number of subproblems that are smaller instances of the same problem. Conquer the subproblems by solving them recursively. If they are small enough, solve the subproblems as base cases. Combine the solutions to the subproblems into the solution for the original problem. You can easily remember the steps of a divide-and-conquer algorithm as divide, conquer, combine. Here's how to view one step, assuming that each divide step creates two subproblems (though some divide-and-conquer algorithms create more than two):
Divide and Conquer Algorithm and Design Paradigm to Various Problems If we expand out two more recursive steps, it looks like this: Because divide-and-conquer creates at least two subproblems, a divide-and-conquer algorithm makes multiple recursive calls.
Divide and Conquer Algorithm and Design Paradigm to Various Problems When the divide and conquer become useful or not useful? 1. Divide and conquer is useful if and only if the main problem can be divide into sub problems form the same type (i.e., natural) of original problem. At this case time complexity become g(n) t(n) t(n1) t(n 2 )... t(n k ) f (n),, if n small otherwise Where, f(n) the time require to divided the original problem into many sub problems g(n) the time require to solve solution of small problems t(n) time require divide and conquer for the input have size n 2. Divide and conquer is not useful into two cases a. Problem have the size n divide into two or more sub problems each one have approximation the size n (such as fib. Problem) because this lead to exponational complexity b. Problem have the size n divide into sub problems each one have approximation the size (n/c) where c is constant. because this lead to the complexity of algorithm nθ(log n)
Divide and Conquer Algorithm and Design Paradigm to Various Problems Merge + Sort =
Divide and Conquer Algorithm and Design Paradigm to Various Problems Examples and Motivation What is the largest-possible number of inversions that a 6-element array can have? a. 15 b. 21 c. 36 d. 64 Sol: =(6(6-1))/2= (6*5)/2=30/2=15
Divide and Conquer Algorithm and Design Paradigm to Various Problems
Divide and Conquer Algorithm and Design Paradigm to Various Problems Q: Suppose the input array A has no split inversions. What is the relationship between the sorted subarrays B and C? a. B has the smallest element of A, C the second-smallest, B the third-smallest, and so on. b. All elements of B are less than all elements of C. c. All elements of B are greater than all elements of C. d. There is not enough information to answer this question. Pseudocode for Merge: D = output [length = n] B = 1 st sorted array [n/2] C = 2 nd sorted array [n/2] i = 1 j = 1 for k = 1 to n if B(i) < C(j) D(k) = B(i) i++ else [C(j) < B(i)] D(k) = C(j) j++ End (ignores end cases)
Merge_and_CountSplitInv
Strassen s Subcubic Matrix Multiplication Algorithm Appling Divide and Conquer
What is the asymptotic running time of the straightforward iterative algorithm for matrix multiplication? θ(n log n) θ(n 2 ) θ(n 3 ) Correct answer θ(n 4 )
The Closest Pair Problem
The Closest Pair Problem
ClosestPair(P x, P y ) Q: Suppose we can correctly implement the ClosestSplitPair subrouine in O(n) time. What will be the overall running time of the Closest Pair algorithm? (Choose the smallest upper bound that applies.) O(n) O(n log n) O(n (log n) 2 ) O(n 2 ) Correct answer
Summary and Analysis of the 2-D Algorithm mindist = infinity for i = 1 to length(p) - 1 for j = i + 1 to length(p) let p = P[i], q = P[j] if dist(p, q) < mindist: mindist = dist(p, q) closestpair = (p, q) return closestpair ClosestPair of a set of points: 1. Divide the set into two equal sized parts by the line l, and recursively compute the minimal distance in each part. 2. Let d be the minimal of the two minimal distances. 3. Eliminate points that lie farther than d apart from l 4. Sort the remaining points according to their y-coordinates 5. Scan the remaining points in the y order and compute the distances of each point to its five neighbors. 6. If any of these distances is less than d then update d. Steps 2-6 define the merging process which must be repeated logn times because this is a divide and conquer algortithm: Step 2 takes O(1) time Step 3 takes O(n) time Step 4 is a sort that takes O(nlogn) time Step 5 takes O(n) time (as we saw in the previous section) Step 6 takes O(1) time Hence the merging of the sub-solutions is dominated by the sorting at step 4, and hence takes O(nlogn) time. This must be repeated once for each level of recursion in the divide-and-conquer algorithm,
Quiz Q1: 3-way-Merge Sort : Suppose that instead of dividing in half at each step of Merge Sort, you divide into thirds, sort each third, and finally combine all of them using a three-way merge subroutine. What is the overall asymptotic running time of this algorithm? (Hint: Note that the merge step can still be implemented in O(n) time.) a. n b. n2log(n) c. n(log(n))2 d. nlog(n) Q2: You are given functions f and g such that f(n)=o(g(n)). Is f(n) log2(f(n)c)=o(g(n) log2(g(n)))? (Here c is some positive constant.) You should assume that f and g are non decreasing and always bigger than 1. a. Sometimes yes, sometimes no, depending on the functions f and g b. False c. Sometimes yes, sometimes no, depending on the constant c d. True
Quiz Q3: Assume again two (positive) non decreasing functions f and g such that f(n)=o(g(n)). Is 2f(n)=O(2g(n))? (Multiple answers may be correct, you should check all of those that apply.) a. Yes if f(n) g(n) for all sufficiently large n b. Sometimes c. Never d. Always Q4: The Fibonacci of number (11 ) is a. 12 b. 13 c. 20 d. 19 Q5: The main advantages of Asymptotic Analysis are a. it's Sweet spot for discussing the high level performance(reasoning) of algorithms b. it's sharp enough to be useful. In particular, to make predictive comparisons between different high level algorithmic approaches to solving a common problem. c. it is a mathematical concept d. coarse enough to suppress all of the details that you want to ignore. Details that depend on the choice of architecture, the choice of programming language, the choice of compiler}.
Quiz Q6: k-way-merge Sort. Suppose you are given k sorted arrays, each with n elements, and you want to combine them into a single array of kn elements. Consider the following approach. Using the merge subroutine taught in lecture, you merge the first 2 arrays, then merge the 3rd given array with this merged version of the first two arrays, then merge the 4th given array with the merged version of the first three arrays, and so on until you merge in the final (kth) input array. What is the running time taken by this successive merging algorithm, as a function of k and n? (Optional: can you think of a faster way to do the k-way merge procedure?) Q7 : Give Suitable word for each the following : a. algorithm +data structure = b. information + interactive with environment = c. pseudo = d. Person can give active solution for any problem in any domain = e. Any algorithm can be analysis based on two domains =
Quiz Q8: Arrange the following functions in increasing order of growth rate (with g(n) following f(n) in your list if and only if f(n)=o(g(n))). Write your 5-letter answer, i.e., the sequence in lower case letters in the space provided. For example, if you feel that the answer is a->b->c->d->e (from smallest to largest), then type abcde in the space provided without any spaces in between the string. You can assume that all logarithms are base 2 (though it actually doesn't matter).
Homwork You are given as input an unsorted array of n distinct numbers, where n is a power of 2. Give an algorithm that identifies the second-largest number in the array, and that uses at most n+log2n 2 comparisons. You are a given a unimodal array of n distinct elements, meaning that its entries are in increasing order up until its maximum element, after which its elements are in decreasing order. Give an algorithm to compute the maximum element that runs in O(log n) time. You are given a sorted (from smallest to largest) array A of n distinct integers which can be positive, negative, or zero. You want to decide whether or not there is an index i such that A[i] = i. Design the fastest algorithm that you can for solving this problem.