Randomized algorithms: Two examples and Yao s Minimax Principle

Transcription

1 Randomized algorithms: Two examles and Yao s Minimax Princile Maximum Satisfiability Consider the roblem Maximum Satisfiability (MAX-SAT). Bring your knowledge u-to-date on the Satisfiability roblem. Maximum Satisfiability: Given a set of clauses, set each of the variables to True or False such as to maximize the number of clauses that become True, i.e. of which at least one of the literals is true. There is a trivial reduction from Satisfiability to Maximum Satisfiability, roving NP-hardness of MAX-SAT. In this course I assume everyone to have basic knowledge on comlexity theory (P, NP, and NP-Comleteness (NPhardness)). Those who need can make themselves u-to-date by reading e.g. the first few chaters of Garey and Johnson s Comuters and Intractability: a guide to the theory of NP-Comleteness or any other basic theoretical comuter science book on algorithms. Consider the following extremely simle randomised algorithm (RA) for MAX- SAT: Set each of the variables indeendently to True with robability 1/2. Theorem 0.1. RA gives a solution with an exected number of True clauses greater than or eual to 1/2 times the otimal number of True clauses. Proof. Suose we have a Boolean exression (instance of MAX-SAT) with m clauses. For i = 1,..., m let Z i = 1 denote the event that clause C i is True, and Z i = 0 otherwise. If C i contains k literals then the robability that none of them is True is 2 k, by indeendence of the random True-False assignment. Thus, P r{z i = 1} = 1 2 k 1/2. Hence, E[Z i ] 1/2. This holds for all i. The exected number of True clauses is E[ m i=1 Z i]. Using the useful linearity roerty of exectations gives m m E[ Z i ] = E[Z i ] 1 2 m. i=1 i=1 Since m is clearly an uer bound on the otimal number of True clauses, this roves the theorem. The examle shows the ower of randomisation. It is a simle algorithm that erforms reasonably well. It also reveals the interretation of a randomised algorithm as a robability distribution over the class of deterministic algorithms: in this case, denoting by n the number of variables, it gives robability 2 n to each of the 2 n deterministic algorithms that blindly set variables to True or 1

2 False and robability 0 to any other algorithm. Notice that for these blind algorithms it is easy to construct a Boolean exression on n variables of which none of the clauses will be True. Randomisation avoids the bad erformance of the deterministic algorithms that receive a ositive robability of being chosen. The MAX-SAT examle was concerned with aroximation. What if we would be interested in the exact answer: what is the robability that RA roduces an otimal solution. With resect to this uestion RA erforms very badly: if there are c otimal solutions then RA will roduce any of them with robability c2 n. We will find out that we cannot exect an efficient randomised algorithm to erform essentially better for MAX-SAT. Minimum Cut in a grah A cut in a grah is a set of edges whose removal slits the grah into two or more comonents. We are interested in the minimum cardinality cut. We know the roblem is well solved through solving a maximum flow roblem on it. But let us study a much simler randomised algorithm. RA: Select from the grah G = (V, E) uniformly at random an edge and contract it. Contracting an edge means identifying its two end vertices. If the two vertices incident to an edge have a common neighbour then contraction of it creates arallel edges. We kee the arallel (multile) edges. We reeat the rocedure iteratively to the newly created grah. At each ste the number of vertices of the grah is reduced by 1. We continue until only two vertices are left. Notice the a cut in the grah after one contraction is also a cut in the grah before the contraction. Hence, the multile edges between the two vertices remaining at the end of the algorithm corresond to a cut in the original grah. We study the robability that this simle randomised algorithm finds a minimum cut. Let C be an otimal cut and let it have size k. Notice that C will not be found if any of its edges is contracted (selected) by the algorithm. So we should study the robability that none of the k edges of C are contracted by RA. First consider the first iteration of RA. First observation: each edge of E is selected with robability 1/ E. Second observation: None of the vertices has degree < k, otherwise such a vertex would induce a cut of size smaller than k and therefore C could not be otimal. Therefore, E nk/2. Hence, Therefore, P r{c does not survive 1st ste} P r{c survives 1st ste} 1 2 n. k nk/2 = 2 n. 2

3 Now consider the second iteration: P r{c survives 2nd ste} = P r{c survives 2nd ste C survived 1st ste}p r{c survives 1st ste}+ P r{c survives 2nd ste C has not survived 1st ste}p r{c does not survive 1st ste} = (1) P r{c survives 2nd ste C survived 1st ste}p r{c survives 1st ste} Again we will bound the robability that C does not survive the 2nd ste given that it has survived the 1st ste. We have now n 1 vertices and, by the same argument as before, each of these has degree at least k, whence the number of edges is at least (n 1)k/2. Thus, P r{c does not survive 2nd ste C survived 1st ste and therefore, k (n 1)k/2 = 2 n 1 P r{c survives 2nd ste C survived 1st ste} 1 2 n 1. Inserting into (1) yields P r{c survives 2nd ste} (1 2 n 1 )(1 2 n ). Similarly, by induction we can rove that P r{c survives h th ste} Π h i=1(1 2 n (i 1) ) and in articular, after n 2 stes a grah on 2 vertices survives: P r{c survives} = Π n 2 i=1 (1 2 n i + 1 ) = n i 1 Πn 2 i=1 n i + 1 = 2 n(n 1) > 2 n 2. Thus, the algorithm gives an error in declaring that the cut found is a min cut with robability at most 1 2 n 2. Reeating the algorithm n 2 /2 times brings the robability of not finding a min cut down to (1 2 n 2 )n2 /2 = 1 e. This can even be decreased further to any arbitrarily small robability ɛ by a number of reetitions of the algorithm which is olynomial both in n and in 1/ɛ. The algorithm is an examle of a Monte Carlo algorithm. These are algorithms with a so-called one-sided robability of failure. If the algorithm were to answer the uestion if in a grah a cut of size at most k exists, then if the algorithm oututs a yes-answer, then it is certainly correct, since it will have found a cut 3

4 of size at most k. But if it oututs a no-answer then it could be that it failed to find a min cut. Thus a no-answer may not be correct. Related to such Monte Carlo algorithms is the definition of a randomised comlexity class, the class RP. For this class we introduce the elementary comuter oeration of a fair coin-fli. Euivalently we may assume that it takes one oeration to draw an element from a set of size olynomially bounded in the size of the inut. RP is the class of decision roblems Π for which a randomised algorithm A exists that runs in olynomial time (deterministically), such that for any instance π of Π the following holds: a) If π has a yes-answer P r{a(π) gives yes answer} 1/2; b) If π has a no-answer P r{a(π) gives no answer} = 1. In this definition A(π) is the outcome of alying A to π. Clearly P RP NP. The last one follows from the fact that for a yes-answer instance of a roblem in NP, a single certificate exists on which an algorithm verifies in olynomial time the yes-answer. Since for roblem in RP there exist for each yes-answer a robability at least 1/2 of obtaining a yes-answer by a olynomial time algorithm, then at least one running (realisation) of the randomised algorithm must lead to a yes-answer. In the definition the robability 1/2 is chosen arbitrarily. As we mentioned with the min cut algorithm reeating the algorithm several times can bring the robability of failure down to any arbitrarily small constant as long as the robability of failure of a single run is bounded by 1/oly( π ), the recirocal of a number that is bounded by a olynomial function of the inut size. Many oen uestions remain with resect to this class in the sirit of the NP=P? uestion. One of them is RP NP co-np? For other related uestions and some more related comlexity classes based on randomised algorithms I refer to the last art of Chater 1 of the book. Our conclusion for the Min Cut roblem based on the analysis of the simle randomised algorithm is that the roblem is in RP. This is no news, since we already knew it belonged to P. Yao s Minimax Princile We will exlore a game theory aroach to the erformance analysis of randomised algorithms which will lead to a articularly useful tool for establishing lower bounds on the erformance of randomised algorithms. 4

5 To start, think of the world-wide known roblem Paer-Scissors-Stone. Call the two layers R (for the row-layer) and C (for the column-layer). We define the ay-off matrix M of the game by letting M ij denote the amount that C ays to R if R lays strategy i and C lays strategy j. Thus in Paer-Scissors-Stone the ay-off matrix becomes: Paer Scissors Stone Paer Scissors Stone Clearly R likes to maximise what C needs to ay to her: he seeks V R = max min M ij, i j whereas C seeks to minimize what she needs to ay to R: V C = min j max M ij. i In Paer-Scissors-Stone V R = 1 and V C = 1. This game is said to have no solution. The main aroach to get around this roblem is introducing randomisation. In game theory deterministic strategies are called ure strategies and randomised strategies are called mixed strategies. A mixed strategy for R is a vector of robabilities i with which R lays strategy i, i = 1,..., m, corresonding to the rows of M. Similarly, = ( 1,..., n ) defines a randomised strategy for C, corresonding to the columns of M. The ay-off becomes now a random variable with exected value T M. Again R tries to maximise its exected ay-off and C tries to minimise its exected ayments: W R = max min T M; W C = min max T M. Von Neumann showed that for any two-erson zero-sum game W R = W C. (In a zero-sum game the money remains among the layers.) Nowadays we can show this easily by strong LP-duality. Let us consider max min T M more closely. Once layer R chooses her strategy, then layer C will choose such as to minimise T M, which is written out n j=1 (T M j ) j, where we use M j to denote the j-th column of M. But given, T M j is a fixed number, say a j. Clearly, minimising j a j j subject to j 0 and j j = 1 is solved by selecting min j a j and setting the corresonding coordinate of to 1 and all the others to 0. Thus, given the best 5

6 strategy for C is a deterministic strategy which we reresent by the unit-vector e j having 1 at coordinate j and 0 s elsewhere: max Similarly we can argue that min min max T M = max min T Me j. j T M = min Van Neumann now imlies Loomis Theorem: max min j T Me j = min max e T i M. i max e T i M. i As a result min j T Me j max i e T i M for every air of strategies,. Yao discovered the usefulness of these game theoretical results for studying the erformance of randomised algorithms. Think of solving a roblem as a twoerson zero-sum game in which C is the layer who lays (chooses) an algorithm and R is the layer who lays an instance of the roblem. The ay-off matrix can be any erformance measure we are interested in, such as the ratio between the solution value obtained by the algorithm layed by C and the otimal value on an instance layed by R. Or it could be the running time of an algorithm layed by C on an instance layed by R. Define A as the class of all ossible (deterministic) algorithms that C could choose from (for instance all olynomial time algorithms) and I as the class of all ossible instances of the roblem that R could choose from. Let M(I, A) be the desired erformance measure of algorithm A when alied to instance I. A randomised strategy I for R is nothing more than selecting a random roblem instance. Therefore E[M(I, A)] is the exected erformance measure of algorithm A on random instance I. Analysis of this exectation is known in the literature under the name robabilistic analysis of algorithms. On the other hand a random strategy A for C is nothing more than a randomised algorithm. Therefore E[M(I, A )] is the exected erformance measure of randomised algorithm A on instance I. And max I I E[M(I, A )] is the worst-case exected erformance measure of randomised algorithm A. Remembering the corollary of Loomis Theorem we have min E[M(I, A)] max E[M(I, A )]. A A I I In words: the exected erformance of the best deterministic algorithm on any random instance I is a lower bound on the worst-case exected erformance of any randomised algorithm. This is what is called Yao s Minimax Princile. 6

7 Thus to get a lower bound on the worst-case erformance of any randomised algorithm it suffices to secify a random instance of the roblem and analyse what any deterministic algorithm can achieve on this instance. Of course, better bounds are obtained by choosing more clever random instances. In fact, we know that there must be a random instance on which the best deterministic algorithm will give exactly the best attainable worst-case erformance of any randomised algorithm, but it may not always be easy to detect that random instance. Material During the first lecture I treated from [MR] Section 1.1 and arts of 1.5. Section and