Introduction to Algorithms December 9, 2005 Massachusetts Institute of Technology Professors Erik D. Demaine and Charles E. Leiserson Handout 33

Transcription

1 Introduction to Algorithms December 9, 005 Massachusetts Institute of Technology 6.046J/18.410J Professors Erik D. Demaine and Charles E. Leiserson Handout 33 Quiz Solutions 14 1 # of students Quiz Score Problem 1. Ups and downs Moonlighting from his normal job at the National University of Technology, Professor Silvermeadow performs magic in nightclubs. The professor is developing the following card trick. A deck of n cards, labeled 1,,..., n, is arranged face up on a table. An audience member calls out a range [i, j], and the professor flips over every card k such that i k j. This action is repeated many times, and during the sequence of actions, audience members also query the professor about whether particular cards are face up or face down. The trick is that there are no actual cards: the professor performs these manipulations in his head, and n is huge. Unbeknownst to the audience, the professor uses a computational device to perform the manipulations, but the current implementation is too slow to work in real time. Help the professor by designing an efficient data structure that supports the following operations on n cards: FLIP(i, j): Flip over every card in the interval [i, j]. IS-FACE-UP (i): Return TRUE if card i is face up and FALSE if card i is face down.

2 Handout 33: Quiz Solutions Solution: Let F be the number of FLIP operations requested by the audience. Notice that performing a single flip FLIP(i, j) is equivalent to performing two flips, FLIP(i, ) and FLIP(j + 1, ). One fast solution is to use a dynamic order-statistic tree, where an element with key i represents a flip of the interval [i, ). For both x = i and x = j + 1, FLIP inserts x into the tree T if x T, and deletes x from T if x T. IS-FACE-UP is implemented by returning true if the number of elements in the tree less than i (call it z) is even, and false if z is odd. One way to compute z is to insert i into T, set z to be one less than the rank of i, and then delete i from T. This data structure requires O(min {F, n}) space and supports FLIP and IS-FACE-UP operations each in O(lg(min {F, n})) time. A completely correct solution of this type received full credit. The following list describes other types of solutions that students submitted and the approximate number of points awarded to each. 1. Another solution is to create a static 1-d range tree T containing the elements from 1 to n as its keys. Each node x in the tree stores a bit that corresponds to a flip of the interval of all elements in the subtree rooted at x. FLIP(i, j) performs a range query on [i, j], and flips the stored bit for the O(lg n) disjoint subtrees that are found by the query. IS-FACE-UP(i) walks down the range tree to the node for i and returns true if the sum of the bits of nodes along the path from the root to i is even. This data structure uses O(n) space and supports both operations in O(lg n) time. This type of solution received about 0 points.. Other students assumed that F n, and stored each of the flip intervals [i, j] in an interval tree. The IS-FACE-UP(i) procedure simply queries the interval tree and returns true if the number of intervals that overlap i is even. If the tree stores overlapping flip intervals, then FLIP and IS-FACE-UP run in O(lg F ) time and O(F lg F ) time, respectively. If we do not allow overlapping intervals in the tree, then FLIP runs in O(F lg F ) time but IS-FACE-UP runs in O(lg F ) time. This type of solution received about 13 points. 3. A simple slow solution is to use a bit-vector of length n to record which bits are flipped. This solution supports FLIP and IS-FACE-UP in O(j i) and O(1) time, respectively. Similarly, another simple solution that stores all F flip-intervals in a linked list supports FLIP and IS-FACE-UP in O(1) and O(F ) time, respectively. Either of these solutions received about 8 points. 4. Any solution that ran slower than the simple slow solution (for example, O(lg n + j i) for FLIP) received about 5 points.

3 Handout 33: Quiz Solutions 3 Problem. The Data Center The world-famous architect Gary O Frank has been commissioned to design a new building, called the Data Center. Gary wants his top architectural protégé to design a scale model of the Data Center using precision-cut sticks, but he wants to preclude the model from inadvertently containing any right angles. Gary fabricates a set of n sticks, labeled 1,,..., n, where stick i has length x i. Before giving the sticks to the protégé, he shows them to you and asks you whether it is possible to create a right triangle using any three of the sticks. Give an efficient algorithm for determining whether there exist three sticks a, b, and c such that the triangle formed from them having sides of lengths x a, x b, and x c is a right triangle (that is, x + x b = x ). Solution: Let X[1..n] be the array with stick sizes. We are asked to determine whether there exist distinct indices i, j, and k such that X[i] + X[j] = X[k]. As an initial step we sort X[1..n] in ascending order. This can be done in worst-case O(n lg n) time by for example heapsort. As soon as the array is sorted we first observe that it is sufficient to check X[i] + X[j] = X[k] for indices i, j, and k with i < j < k. In order to obtain a worst-case O(n ) algorithm, we increase k from 1 to n in an outerloop and we search in linear time in an innerloop for indices i and j, i < j < k, with X[i] + X[j] = X[k]. 1 Sort X[1..n] in ascending order k 1 3 while k n 4 do i, j 1, k 1 5 while i < j 6 if X[i] + X[j] = X[k] then return true 7 if X[i] + X[j] < X[k] 8 then i i else j j 1 10 return false a c We will prove the invariant: < j [1 i < j k 1 and X[i ] + X[j ] = X[k] ] implies [i i j]. Initially, i = 1 and j = k 1 and the invariant holds. For the inductive step, assume that the invariant holds when we enter the innerloop. Notice that X[1..n] is sorted in increasing order. If X[i] + X[j] < X[k], then, for i < j j, X[i] + X[j ] X[i] + X[j] < X[k] and the invariant holds for i i + 1. If X[i] + X[j] > X[k], then, for i i < j, X[i ] + X[j] X[i] + X[j] > X[k]

4 Handout 33: Quiz Solutions 4 and the invariant holds for j j 1. If the innerloop finishes, then i = j and the invariant shows that there are no indices i and j, 1 i < j k 1, such that X[i ] + X[j ] = X[k]. This proves the correctness of the algorithm. The innerloop has worst-case O(n) running time, hence, together with the outerloop the algorithm runs in worst-case O(n ) time.

5 Handout 33: Quiz Solutions 5 Problem 3. Nonnegativizing a matrix by pivoting A matrix M [1.. n, 1.. n] contains entries drawn from { }. Each row contains at most 10 finite values, some of which may be negative. The goal of the problem is to transform M so that every entry is nonnegative by using only pivot operations: PIVOT(M, i, x) 1 for j 1 to n do M [i, j] M [i, j] + x 3 M [j, i] M [j, i] x Give an efficient algorithm to determine whether there exists a sequence of pivot operations with various values for i and x such that, at the end of the sequence, M [i, j] 0 for all i, j = 1,,..., n. Solution: Suppose that we have a sequence PIVOT(M, i 1, x 1 ), PIVOT(M, i, x ),..., PIVOT(M, i k, x k ) of pivots. Since each pivot operation simply adds to columns and subtracts from rows, by the associativity and commutativity of addition, the order of the pivot operations is irrelevant. Moreover, since PIVOT(M, i 1, x 1 ) followed by PIVOT(M, i, x ) is equivalent to PIVOT(M, i 1, x 1 + x ), there is no need to pivot more than once on any index. Thus, any sequence of pivots is equivalent to a sequence of exactly n pivots PIVOT(M, 1, x 1 ), PIVOT(M,, x ),..., PIVOT(M, n, x n ), where some of the x i may be 0. The only pivots that affect a matrix entry M [i, j] are PIVOT(M, i, x i ) and PIVOT(M, j, x j ). Thus, after the entire sequence of pivots has been performed, the resulting matrix M has entries M [i, j] = M [i, j] + x i x j. We want every entry M [i, j] to be nonnegative, which is to say that M [i, j] + xi x j 0, or x j x i M [i, j]. Thus, we only need to solve a set of difference constraints. We can ignore the difference constraints where M [i, j] =, which leaves at most 10n difference contraints in n variables. It takes us O(n ) time to find the at-most 10n finite entries, and we can use the Bellman-Ford algorithm [CLRS, Section 4.4] to solve them in O(n (10n)) = O(n ) time, for a total of O(n ) time.

6 Handout 33: Quiz Solutions 6 Problem 4. Augmenting the Queueinator TM By applying his research in warm fission, Professor Uriah s company is now manufacturing and selling the Queueinator TM, a priority-queue hardware device which can be connected to an ordinary computer and which effectively supports the priority-queue operations INSERT and EXTRACT-MIN in O(1) time per operation. The professor s company has a customer, however, who actually needs a double-ended priority queue that supports not only the operations INSERT and EXTRACT-MIN, but also EXTRACT-MAX. Redesigning the Queueinator TM hardware to support the extra operation will take the professor s company a year of development. Help the professor by designing an efficient double-ended priority queue using software and one or more Queueinator TM devices. Solution: Keep two Queueinators, MIN and MAX, where MAX has the keys negated, of roughly equal size and keep an element mid which is the smallest element in MAX. While inserting an element with key k, compare it to mid and insert it in MIN if k < mid and into MAX otherwise. For EXTRACT-MIN, extract from MIN and for EXTRACT-MAX, extract from MAX. If one empties, extract all the elements from the other into a sorted array. Split the array into half and insert the smaller half into MIN and the larger half into MAX. This solution provides O(1) amortized cost for all operations. Proof of Correctness: The invariant is that all the elements in MIN are smaller than mid and all the elements in MAX are larger than or equal to mid. The other invariant is that MIN and MAX together contain all the elements that have been inserted and not yet been extracted. The invariants are maintained during all operations. Therefore, extractions return the correct results. Running time analysis: The potential function is Φ(i) = MIN MAX, where MIN and MAX are the number of elements in the MIN and MAX queueinators respectively. Both MIN and MAX are empty initially, and the potential is 0. The potential obviously never becomes negative. Let us look at the amortized costs. 1.INSERT Inserts into MIN or MAX. The real cost is 1 and the potential either increases by (if you inserted into the bigger queue) or decreases by 1 (if you insert into the smaller queue). Thus the cost is always O(1)..EXTRACT-MIN If the MIN is non-empty, then the real cost is 1 and the potential either increases or decreases by. If MIN is empty, then you have to rebalance the queues. The potential before the rebalance is equal to the number of elements in MAX, and the potential after the rebalance is either 0 (or if the number of elements is odd, but that doesn t change much). The real cost of rebalancing is the number of elements in MAX, since all the elements are extracted and then inserted again. Thus the change in potential pays for the rebalancing, and the amortized cost is O(1). 3.EXTRACT-MAX It is analogous to EXTRACT-MIN. Therefore the amortized cost of the operations is 0(1). There were other good solutions, some of them are given below

7 Handout 33: Quiz Solutions 7 1.Use 4 queueinators where of them to keep track of the deleted elements. Those solutions lost a couple of points due to the use of the extra hardware..use extra fields in the items to keep track of deletions. These also lost a couple of points due to extra assumptions that you can change the elements. 3.Use hash tables to keep track of deleted elements. This only provides expected O(1) amortized time, and assumes that the keys are integers in a range. Therefore, they lost about 5 points. 4.Use direct access tables to keep track of deleted elements, lost about 6-7 points due to the extra space. The analysis was very important for the solutions and the solutions that had incomplete or incorrect running time analysis lost points for that. In addition, a convincing correctness argument was required for full credit. People who gave the correct algorithm but did not claim or prove amortized bounds lost many points.

8 Handout 33: Quiz Solutions 8 Problem 5. Spam distribution Professor Hormel is designing a spam distribution network. The network is represented by a rooted tree T = (V, E) with root r V and nonnegative edge-weight function w : E. Each vertex v V represents a server with one million addresses, and each edge e E represents a communication channel that costs w(e) dollars to purchase. A server v V receives spam precisely if the entire path from the root r to v is purchased. The professor wants to send spam from the root r to k V servers (including the root) by spending as little money as possible. Help the professor by designing an algorithm that finds a minimum-weight connected subtree of T with k vertices including the root. (For partial credit, solve the problem when each vertex v V has at most children in T.) Solution: Executive Overview. The solution to this problem is based on dynamic programming. We define k V subproblems, one for each combination of a vertex v V and a number k k. Each subproblem is of the form Find the minimum-weight connected subtree of rooted at v with k vertices. Each subproblem can be solved in O(k) time, resulting in a running time of O(k V ). Notation. Throughout this solution, we use the following notation: MWCT(v,i): a minimum-weight connected subtree of i nodes rooted at v child(v,i): child i of node v w(v, i): the weight of the edge from v to child i W (S): the sum of the weights of all the edges in tree S deg(v): the number of children of node v Optimal Substructure. This problem exhibits both optimal substructure and overlapping subproblems. For intuition, consider a binary tree T. Let v be a node in the tree, and consider a MWCT of T rooted at v with l nodes. If T 1 is the left subtree of v and has l 1 nodes, then T 1 is a MWCT rooted at v s left child with l 1 nodes. Similarly, if T is the right subtree of v and has l nodes, then T is a MWCT rooted at v s right child with l nodes. The proof follows by a cut-and-paste argument. Assume that T 1 is not a MWCT rooted at v s left child. Then there is another tree, S, rooted at v s left child with l 1 nodes and with less weight than T 1. We can then reduce the weight of the tree rooted at v by substituting S for tree T 1, contradicting our assumption that the tree rooted at v is a minimum-weight connected subtree with k nodes. We can now observe how to use the overlapping subproblems. Once we have calculated the MWCT(v, l) for all v V and all l k, we can determine the MWCT(v, k + 1). In particular, we choose subtrees rooted at v s children that contain k nodes in total. That is, if there are l nodes in the left subtree of v, then there must be k l nodes in the right subtree. We choose l to

9 Handout 33: Quiz Solutions 9 split the tree between v s left and right children to minimize the total weight. We now proceed to generalize this to arbitrary degree trees, and explain the algorithm in more detail. Algorithm Description. Let T [v] be the subtree of T rooted at v. Let T [v, c] be the subtree of T rooted at v consisting of v and the trees rooted at children 1.. c. More formally, T [v, c] = v {T [w] : w = childv, i, 1 i c}. We create two V k V arrays: C[1.. V, 1.. k, 1.. V ] and B[1.. V, 1.. k], 1.. V. C[v, k, c] holds the cost of the minimum-weight connected subtree of T [v, c] with k nodes. The array B is used to reconstruct the tree after the dynamic program has terminated. B[v, k, c] holds the number of children in the subtree of the MWCT of T [v, c] with k nodes rooted at child c. Notice that the resulting arrays are three-dimensional tables of size k V. Fortunately, we will only have to fill in k V of the entries, leaving the rest empty/uninitialized, as we will see when the algorithm is analyzed. (For simplicity, we assume that arbitrary-sized memory blocks can be allocated in O(1) time. If memory allocation is more costly, we can reduce the memory usage to O(k V ).) The main procedure to calculate the MWCT of V with k nodes is as follows: MWCT(V, k) 1 for all v V do for c = 1 to deg(v) 3 do C[v, 1, c] 0 4 B[v, 1, c] 0 5 for l = to k 6 do for all v V 7 do w child(v, 1) 8 C[v, l, 1] w(v, w) + C[w, l 1, deg(w)] 9 B[v, l, 1] l 1 10 for c = to deg(v) 11 do i FIND-NUM-CHILDREN(v, l, c) 1 B[v, l, c] i 13 if i = 0 then C[v, l, c] C[v, l, c 1] 14 if i = l then C[v, l, c] w(v, w) + C[w, l 1, deg(w)] 15 if 0 < i < l then C[v, l, c] C[v, l i] + w(v, w) + C[w, i, deg(w)] We begin in lines 1 4 by initializing C[v, 1, c] = 0 for all v V and all c V. These are the smallest subproblems considered: the one node minimum-weight connected subtree rooted at v consists simply of v itself, and hence has cost zero. We proceed with three nested loops to fill in the arrays. In the outer loop (line 5), we iterate l from to k. In the second loop (line 6), we iterate v over all nodes in V. In the inner loop (line 10), we iterate c from to deg(v).

10 Handout 33: Quiz Solutions 10 Within the loop, we are examining the subtree T [v, c] and the node w = child(v, c). Let S be the MWCT of T [v, c] containing l nodes. We want to calculate C[v, l, c], the cost of S. First consider the case where c = 1. In this case, all the nodes in S are in the subtree rooted at w. Therefore, S consists of node v and the MWCT of w with l 1 nodes. Now consider the case where c. Notice that some of the nodes in S may be in subtrees rooted at the first c 1 children of v, and some may be in the subtree rooted at w. The function FIND-NUM-CHILDREN(v, l, c) calculates the number of nodes in S that are in the subtree rooted at w; the remaining l i 1 are in the subtrees rooted at the first c 1 children of v. It is then easy to determine the cost of S. If no nodes in S are in the subtree rooted at w, then S is just the MWCT of T [v, c 1] with l nodes (line 11). If all the nodes in S are in the subtree rooted at w, then S consists of node v along with the MWCT of w with l 1 nodes (line 1). Finally, if i is between 0 and l, then S consists of the MWCT of T [v, c 1] with l i nodes, plus the MWCT of w with i nodes (line 13). It remains to discuss the FIND-NUM-CHILDREN procedure: FIND-NUM-CHILDREN(v, l, c) 1 w child(v, c) min-weight C[v, l, c 1] 3 num 0 4 for i = 1 to l 5 do wt C[v, l i, c 1] + w(v, c) + C[w, i, deg(w)] 6 if wt < min-weight 7 then min-weight wt 8 num i 9 if min-weight > w(v, w) + C[w, l 1, deg(w)] 10 then num l 11 return num The FIND-NUM-CHILDREN procedure compares all possible ways of dividing l nodes between T [v, c 1] and the subtree rooted at w = child(v, c). First, it considers the case where all l children are in T [v, c 1] and zero children are in the tree rooted at w (line 3). Next, it iterates through the loop l times, placing l i nodes in T [v, c 1] and i nodes in the subtree rooted at w (lines 4 8). Finally, it considers the case where l 1 children are in the subtree rooted at w (lines 9 10). It then returns the choice which minimizes the weight. The final procedure in the algorithms prints out the tree, performing a depth-first traversal of the resulting tree:

11 Handout 33: Quiz Solutions 11 PRINT-TREE(v, k) 1 PRINT(v) l k 3 for c = deg(v) downto 1 4 do if l > 0 and B[v, l, c] = 0 5 then w child(v, c) 6 PRINT-TREE(v, B[v, l, c]) 7 l l B[v, l, c] The PRINT-TREE procedure starts with the last child of v, and examines B[v, l, c] to determine the number of nodes in the subtree root at child c. It then recurses into that subtree, and updates the number of nodes remaining in the MWCT. Running Time. First, examine lines 1 4 of MWCT: the two loops iterate over every edge of every node, resulting on O( E ) = O( V ) operations. Next, the for-loop on line 5 repeats O(k) time. Now we examine lines The innermost loop executes once for every edge of every node, i.e., O( E ) = O( V ) times. Each iteration of the loop calls FIND-NUM-CHILDREN(v, l, c) which takes O(l) = O(k) time. Therefore each iteration of lines 6 15 costs O(k V ). Therefore the entire cost of MWCT is O(k V ). Finally, printing the output costs O( V ). Correctness. The correctness follows by induction on k using a cut-and-paste argument. The loop invariant maintained by the innermost loop is that for all c c, C[v, l, c ] is equal to the cost of the MWCT of T [v, c ]. The loop invariant of the outer loop is that for all v, for all l l, for all c deg(v), C[v, l, c ] is equal to the cost of the MWCT of T [v, c ] with l nodes. These two invariants need to be proved together by induction. (We omit mention of the B array here, though its correctness follows by the same argument as follows.) Consider immediately after the innermost loop has terminated for some v, l, and c. We argue that C[v, l, c] is the cost of the MWCT of T [v, c]. Assume not. Then there is some other subtree S of T [v, c] with l nodes and cost < C[v, l, c]. Consider the subtree of S rooted at child c of v with i nodes. (S may be the empty tree.) We know that the cost of this subtree must be at least x = C[child(v, c), i, deg(child(v, c))], since by induction x is the MWCT of child c of v with i nodes. Similarly, consider the subtree of S consisting of all the nodes in T [v, c 1]. Again, the cost of this subtree must be at least y = C[v, l i, c 1], since by induction y is the MWCT of T [v, c 1] with l i nodes. But FIND-MIN-CHILDREN ensures that the C[v, l, c] is no more than x+ y + w(v, c), contradicting our assumption. Finally, the inductive step of the outer loop invariant follows from the inner loop invariant, concluding the proof.

12 Handout 33: Quiz Solutions 1 Problem 6. A tomato a day Professor Kerry loves tomatoes! The professor eats one tomato every day, because she is obsessed with the health benefits of the potent antioxidant lycopene and because she just happens to like them very much, thank you. The price of tomatoes rises and falls during the year, and when the price of tomatoes is low, the professor would naturally like to buy as many tomatoes as she can. Because tomatoes have a shelf-life of only d days, however, she must eat a tomato bought on day i on some day j in the range i j < i + d, or else the tomato will spoil and be wasted. Thus, although the professor can buy as many tomatoes as she wants on any given day, because she consumes only one tomato per day, she must be circumspect about purchasing too many, even if the price is low. The professor s obsession has led her to worry about whether she is spending too much money on tomatoes. She has obtained historical pricing data for n days, and she knows how much she actually spent on those days. The historical data consists of an array C[1.. n], where C[i] is the price of a tomato on day i. She would like to analyze the historical data to determine what is the minimum amount she could possibly have spent in order to satisfy her tomato-a-day habit, and then she will compare that value to what she actually spent. Give an efficient algorithm to determine the optimal offline (0/0 hindsight) purchasing strategy on the historical data. Given d, n, and C[1.. n], your algorithm should output B[1.. n], where B[i] is the number of tomatoes to buy on day i. Solution: You can solve this problem in Θ(n) time, which is the fastest possible asymptotic solution because it requires Ω(n) time to scan the cost array. For every day i, you must buy the tomato for that day in the window W (i) [i d + 1, i] [1, n]. Note that the greedy choice works: in the optimal solution, you should buy the tomato for day i on the minimum cost day T i in W (i). Once you calculate T i for all i, you can easily compute B in Θ(n) time. You can compute all T i in O(n) time by computing the sliding window minimum T i for each i in amortized O(1) time. Use a double-ended queue (deque) of days Q = q 1, q,..., q m such that day q 1 is the minimum cost day in W (i) and each entry q k>1 is the minimum cost day of the days in W (i) following q k 1. Then q 1 = T i and this invariant can be maintained in amortized O(1) time using the following algorithm:

13 Handout 33: Quiz Solutions 13 BUYTOMATOES(d, C[1.. n]) 1 Q B[1.. n] 0 3 for i 1 to n 4 do if not EMPTY(Q) and HEAD(Q) i d 5 then POP-FRONT(Q) Remove old day 6 while not EMPTY and C[i] C[TAIL(Q)] 7 do POP-BACK(Q) Remove non-minimum-cost days 8 PUSH-BACK(Q, i) Add current day 9 B[HEAD(Q)] B[HEAD(Q)] return B Each day i is added once to Q and removed at most once from Q, so the total number of deque operations is O(n). Each deque operation can be done in O(1) time. The loop in the algorithm executes for n iterations, so the overall running time is Θ(n). The deque requires O(min(n, d)) space since there can be at most one element for each of the past d days. The return array B requires Θ(n) space, so the overall space used is Θ(n). There are other ways to compute T i in amortized O(1) time that also received full credit. Slower solutions that received partial credit include solutions that compute T i in O(log d) or O(d) time, solutions that compute B directly by looking ahead up to d days at each step, dynamic programming solutions, and solutions that sort the tomato prices and maximize the number of tomatoes purchased at lower prices.