LESSON 16: ALGORITHM STRATEGIES

Transcription

1 LESSON 16: ALGORITHM STRATEGIES Objective Divide and Conquer Backtracking Now you are familiar with greedy algorithm Divide and Conquer Definition: An algorithmic technique. To solve a problem on an instance of size n, a solution is found either directly because solving that instance is easy (typically, because the instance is small) or the instance is divided into two or more smaller instances. Each of these smaller instances is recursively solved, and the solutions are combined to produce a solution for the original instance. Note: The name divide and conquer is because the problem is conquered by dividing it into several smaller problems. This technique yields elegant, simple and quite often very efficient algorithms. Well-known examples include heapify, merge sort, quicksort, Strassen s fast matrix multiplication, the Fast Fourier Transform (FFT), and binary search. (Why is binary search included? The dividing part just picks which segment in which to search, and combining the solutions is trivial: take the answer from the segment in which you searched.) Similar principles are at the heart of several important data structures such as binary search tree, multiway search trees, tries, skip lists, multidimensional search trees (k-d trees, quadtrees), etc. Implementation How many odd numbers in an array? In addition to correctness, programming style is also important, so please write modular, well-documented code. 1. Implement the recursive, divide-and-conquer function ADD-ODDS (A,p,r) that returns the sum of the odd numbers found in the positive integer 2. The main part of your program will read in data from a file whose name is given as the first argument to your executable. The first line of the input file contains the number of lines (1-99) in the rest of the file. Each subsequent line begins with the number of elements (1-99) in the array followed by the elements (1-99). Your main program should process each line one at a time by reading in the array, calling ADD-ODDS, and then printing the result. For example, /*Program1*/ /*name:anan Tongprasith*/ /*course: cse2320 sec 501*/ /*compile: cc addodd.c*/ #include<stdio.h> /* The main program reads data from input file and put the data into an*/ /*array name element. Then we call function addodds to recursively add*/ /*odd numbers from the array. */ main(int argc, char *argv[]) FILE *fp; char newline[100],newc; int numofline,i,element[100],a=0; fp=fopen(argv[1], r ); /*Open input file*/ if(fgets(newline,100,fp)!= NULL) /*Read first line*/ numofline=atoi(newline); /*Num of line=first line*/ for (i=numofline;i>0;i-=1) /*Do it line by line*/ fgets(newline,100,fp); /*Read a raw data line*/ a=testme(element,newline); /*Put data into an array*/ printf( %d\n,addodds(1,a,element)); /*Addodds the array*/ else fclose(fp); return 0; /* Function testme reads a data line and put the data into an array.*/ int testme(int *ele,char *new) int i=-1,j=0,k=0,linelength=strlen(new);char *piece; 54 3B.582

2 do /*check for j=0;printf( \0 ); do /*Loop for data separation*/ i++; /*buffer=data*/ piece[j++]=new[i]; while(new[i]!= && new[i]!= \0'); space between data and end of line*/ ele[k]=atoi(piece); /*Put data into an array*/ *piece= \0'; k=k+1; while(i<linelength); k=ele[0];ele=&ele[1]; /*Remove first item (# of data)*/ /*reset buffer*/ return k; / *Return # of data*/ /* Function addodds receives starting point, end point, and data array.*/ /* It sums up all odd numbers in the array between starting point and */ /* end points and return the result.*/ int addodds(int min,int max,int *ele) int x,y; if (min==max) /*Base case*/ if ((ele[min]%2)==1) /*Checking for odds*/ *Return odds*/ return 0; else /*Recursive case*/ else return ele[min]; / /*Return 0 s if even*/ x=addodds(min,(min+max)/2,ele); /*Recursive call 1st half*/ y=addodds(1+(min+max)/2,max,ele); /* 2nd half*/ return x+y; /*Return the sum*/ Three Divide and Conquer Sorting Algorithms Today we ll finish heapsort, and describe both mergesort and quicksort. Why do we need multiple sorting algorithms? Different methods work better in different applications. Heapsort uses close to the right number of comparisons but needs to move data around quite a bit. It can be done in a way that uses very little extra memory. It s probably good when memory is tight, and you are sorting many small items that come stored in an array. Merge sort is good for data that s too big to have in memory at once, because its pattern of storage access is very regular. It also uses even fewer comparisons than heapsort, and is especially suited for data stored as linked lists. Quicksort also uses few comparisons (somewhat more than the other two). Like heapsort it can sort in place by moving data in an array. Heapification Recall the idea of heapsort: heapsort(list L) make heap H from L make empty list X while H nonempty remove smallest from H and add it to X return X Remember that a heap is just a balanced binary tree in which the value at any node is smaller than the values at its children. We went over most of this last time. The total number of comparisons is n + however many are needed to make H. The only missing step: how to make a heap? To start with, we can set up a binary tree of the right size and shape, and put the objects into the tree in any old order. This is all easy and doesn t require any comparisons. Now we have to switch objects around to get them back in order. The divide and conquer idea: find natural subproblems, solve them recursively, and combine them to get an overall solution. Here the obvious subproblems are the subtrees. If we solve them recursively, we get something that is close to being a heap, except that perhaps the root doesn t satisfy the heap property. To make the whole thing a heap, we merely have to percolate that value down to a lower level in the tree. heapify(tree T) if (T is nonempty) heapify(left subtree) heapify(right subtree) let x = value at tree root while node containing x doesn t satisfy heap propert switch values of node and its smallest child 3B

3 The while loop performs two comparisons per iteration, and takes at most iterations, so the time for this satisfies a recurrence T(n) <= 2 T(n/2) + 2 How to Solve It? Divide and Conquer Recurrences In general, divide and conquer is based on the following idea. The whole problem we want to solve may too big to understand or solve at once. We break it up into smaller pieces, solve the pieces separately, and combine the separate pieces together. We analyze this in some generality: suppose we have a pieces, each of size n/b and merging takes time f(n). (In the heapification example a=b=2 and f(n)=o() but it will not always be true that a=b - sometimes the pieces will overlap.) The easiest way to understand what s going on here is to draw a tree with nodes corresponding to subproblems (labeled with the size of the subproblem) n / \ n/b n/b n/b / \ / \ / \ For simplicity, let s assume n is a power of b, and that the recursion stops when n is 1. Notice that the size of a node depends only on its level: size(i) = n/(b^i). What is time taken by a node at level i? time(i) = f(n/b^i) How many levels can we have before we get down to n=1? For bottom level, n/b^i=1, so n=b^i and i=()/(log b). How many items at level i? a^i. So putting these together we have ()/(log b) T(n) = sum a^i f(n/b^i) This looks messy, but it s not too bad. There are only a few terms (logarithmically many) and often the sum is dominated by the terms at one end (f(n)) or the other (n^(log a/log b)). In fact, you will generally only be a logarithmic factor away from the truth if you approximate the solution by the sum of these two, O(f(n) + n^(log a/log b)). Let s use this to analyze heapification. By plugging in parameters a=b=2, f(n)=, we get T(n) = 2 sum 2^i log(n/2^i) Rewriting the same terms in the opposite order, this turns out to equal T(n) = 2 sum n/2^i log(2^i) = 2n sum i/2^i infty <= 2n sum i/2^i = 4n So heapification takes at most 4n comparisons and heapsort takes at most n + 4n. (There s an n n lower bound so we re only within O(n) of the absolute best possible.) This was an example of a sorting algorithm where one part used divide and conquer. What about doing the whole algorithm that way? Merge sort According to Knuth, merge sort was one of the earliest sorting algorithms, invented by John von Neumann in Let s look at the combine step first. Suppose you have some data that s close to sorted - it forms two sorted lists. You want to merge the two sorted lists quickly rather than having to resort to a general purpose sorting algorithm. This is easy enough: merge(l1,l2) list X = empty while (neither L1 nor L2 empty) compare first items of L1 & L2 remove smaller of the two from its list add to end of X catenate remaining list to end of X return X Time analysis: in the worst case both lists empty at about same time, so everything has to be compared. Each comparison adds one item to X so the worst case is X -1 = L1 + L2-1 comparisons. One can do a little better sometimes e.g. if L1 is smaller than most of L2. Once we know how to combine two sorted lists, we can construct a divide and conquer sorting algorithm that simply divides the list in two, sorts the two recursively, and merges the results: merge sort(l) 56 3B.582

4 if (length(l) < 2) return L else split L into lists L1 and L2, each of n/2 elements L1 = merge sort(l1) L2 = merge sort(l2) return merge(l1,l2) This is simpler than heapsort (so easier to program) and works pretty well. How many comparisons does it use? We can use the analysis of the merge step to write down a recurrence: C(n) <= n-1 + 2C(n/2) As you saw in homework 1.31, for n = power of 2, the solution to this is n - n + 1. For other n, it s similar but more complicated. To prove this (at least the power of 2 version), you can use the formula above to produce C(N) <= sum 2^i (n/2^i - 1) = sum n - 2^i = n( + 1) - (2n - 1) = n - n + 1 So the number of comparisons is even less than heapsort. Quicksort Quicksort, invented by Tony Hoare, follows a very similar divide and conquer idea: partition into two lists and put them back together again It does more work on the divide side, less on the combine side. Merge sort worked no matter how you split the lists (one obvious way is to take first n/2 and last n/2 elements, another is to take every other element). But if you could perform the splits so that everything in one list was smaller than everything in the other, this information could be used to make merging much easier: you could merge just by concatenating the lists. How to split so one list smaller than the other? e.g. for alphabetical order, you could split into A-M, N-Z so could use some split depending on what data looks like, but we want a comparison sorting algorithm that works for any data. Quicksort uses a simple idea: pick one object x from the list, and split the rest into those before x and those after x. quicksort(l) if (length(l) < 2) return L else pick some x in L L1 = y in L : y < x L2 = y in L : y > x L3 = y in L : y = x quicksort(l1) quicksort(l2) return concatenation of L1, L3, and L2 (We don t need to sort L3 because everything in it is equal). Quicksort analysis The partition step of quicksort takes n-1 comparisons. So we can write a recurrence for the total number of comparisons done by quicksort: C(n) = n-1 + C(a) + C(b) where a and b are the sizes of L1 and L2, generally satisfying a+b=n-1. In the worst case, we might pick x to be the minimum element in L. Then a=0, b=n-1, and the recurrence simplifies to C(n)=n-1 + C(n-1) = O(n^2). So this seems like a very bad algorithm. Why do we call it quicksort? How can we make it less bad? Randomization! Suppose we pick x=a[k] where k is chosen randomly. Then any value of a is equally likely from 0 to n-1. To do average case analysis, we write out the sum over possible random choices of the probability of that choice times the time for that choice. Here the choices are the values of k, the probabilities are all 1/n, and the times can be described by formulas involving the time for the recursive calls to the algorithm. So average case analysis of a randomized algorithm gives a randomized recurrence: n-1 C(n) = sum (1/n)[n C(a) + C(n-a-1)] a=0 To simplify the recurrence, note that if C(a) occurs one place in the sum, the same number will occur as C(n-a-1) in another term - we rearrange the sum to group the two together. We can also take the (n-1) parts out of the sum since the sum of 1/n copies of 1/n times n-1 is just n-1. n-1 C(n) = n sum (2/n) C(a) a=0 The book gives two proofs that this is O(n ). Of these, induction is easier. One useful idea here: we want to prove f(n) is O(g(n)). The O() hides too much information, instead we need to prove f(n) <= a g(n) but we don t know what value a should take. We work it out with a left as a variable then use the analysis to see what values of a work. We have C(1) = 0 = a (1 log 1) for all a. Suppose C(i) <= a i log i for some a, all i<n. Then C(n) = n-1 + sum(2/n) C(a) <= n-1 + sum(2/n)ai log i 3B

5 = n-1 + 2a/n sum(i=2 to n-1) (i log i) <= n-1 + 2a/n integral(i=2 to n)(i log i) = n-1 + 2a/n (n^2 / 2 - n^2/4-2 ln 2 + 1) = n-1 + a n - an/2 - O(1) and this will work if n-1 < an/2, and in particular if a=2. So we can conclude that C(n) <= 2 n. Note that this is worse than either merge sort or heap sort, and requires random number generator to avoid being really bad. But it s pretty commonly used, and can be tuned in various ways to work better. (For instance, let x be the median of three randomly chosen values rather than just one value). Backtracking Definition An algorithmic technique to find solutions by trying one of several choices. If the choice proves incorrect, computation backtracks or restarts at the point of choice and tries another choice. It is often convenient to maintain choice points and alternate choices using recursion. Note: Conceptually, a backtracking algorithm does a depth-first search of a tree of possible (partial) solutions. Each choice is a node in the tree. The backtracking method is based on the systematically inquisition of the possible solutions where through the procedure,set of possible solutions are rejected before even examined so their number is getting a lot smaller. An important requirement which must be fulfilled is that there must be the proper hierarchy in the systematically produce of solutions so that sets of solutions that do not fulfill a certain requirement are rejected before the solutions are produced. For this reason the examination and produce of the solutions follows a model of non-cycle graph for which in this case we will consider as a tree. The root of the tree represents the set of all the solutions. Nodes in lower levels represent even smaller sets of solutions,based on their properties. Obviously,leaves will be isolated solutions.it is easily understood that the tree (or any other graph) is produced during the examination of the solutions so that no rejected solutions are produced. When a node is rejected, the whole sub-tree is rejected, and we backtrack to the ancestor of the node so that more children are produced and examined. Because this method is expected to produce subsets of solutions which are difficult to process, the method itself is not very popular. The Queens Problem We consider a grid of squares, dimensioned nxn, partly equivalent to a chessboard containing n 2 places. A queen placed in any of the n 2 squares controls all the squares that are on its row, its column and the 45 0 diagonals. The problem asked, is how to put n queens on the chessboard, so that any other queen does not control the square of every queen. Obviously for n=2 there is no problem to the solution, while for n=4 a valid solution is given by the drawing below. A possible position on the grid is set by the pair of pointers (i,j) where 1<i,j<n, and i stands for the number of column and j stands for the number of row. Up to this point, for the same i there are n valid values for j. For a candidate solution though, only one queen can be on each column, that is only one value j=v(i).therefor the solutions are represented with the n values of the matrix V=[V(1),...V(n)].All the solutions for which V(i)=V(j) are rejected because 2 queens can not be on the same row. Now the solutions are the permutes of n pointers, which is n!,still a forbiddingly big number. Out of all these solution the correct one is the one which satisfies the last requirement:2 queens will not belong in the same diagonal, which is: V(j)-V(i)<>±(i-j) for i<>j. (5.8-1) A backtracking algorithm or this problem constructs the permutes [V(1),...V(n)] of the 1,...,n pointers, and examines them as to the property (5.8-1).For example there are (n-2)! permutes in the shape of [3,4...].These will not be produced and examined if the systematically construction of them has already ordered them in a sub-tree with root [3,4] which will be rejected by the condition, and will also reject all the (n-2)! permutes. On the contrary,the same way of producingexamining will go even further to the examination of more permutes in the shape of p=1,4,2,... since, so far the condition is satisfied. The next node to be inserted that is j:=v(4) must also satisfies these:j-1<>3,j-4<>2,j-4<>-2,j-2<>1,j-2<>- 1.All the j pointers satisfying these requirements produce the following permutes:[1,4,2,j,...] which connect to the tree as children of Mean while large sets of permutes such as [1,4,2,6,...] have already been rejected. A typical declaration of this algorithm: The root of all solutions, has as children n nodes [1],...,[n],where [j] represents all the permutes starting with j(and whose number is (n-1)! for every j).inductive if a node includes the k nodes j 1,...j k we attempt to increase it with another node j 1,...,j k,j k+1 so that the condition (5.8-1) is fulfilled. For n=4 this method produces the indirect graph of the following picture, and does not produce the 4!=24 leaves of all the candidate solutions. 58 3B.582

6 Point to Ponder The name divide and conquer is because the problem is conquered by dividing it into several smaller problems. Quicksort, invented by Tony Hoare, follows a very similar divide and conquer idea: A backtracking algorithm does a depth-first search of a tree of possible (partial) solutions. Each choice is a node in the tree. Question Explain the Divide & Conquer Algorithm? Explain the Backtracking Algorithm? References Fundamentals of Algorithmic by Notes Gilles Brassard Paul Bratley 3B