1 Introduction to Game Theory

Transcription

1 15-451/651: Design & Analysis of Algorithms October 12, 2015 Lecture #12: Game Theory last changed: October 9, 2015 In today s lecture, we ll talk about game theory and some of its connections to comuter science. The toics we ll cover are: 2-layer Zero-sum games, and the concet of (minimax) otimal strategies. The connection of 2-layer zero-sum games to randomized algorithms. And time ermitting, general sum-games, and the idea of a Nash euilibrium. 1 Introduction to Game Theory Game theory is the study of how eole behave in social and economic interactions, and how they make decisions in these settings. It is an area originally develoed by economists, but given its general scoe, it has alications to many other discilines, including comuter science. A clarification at the very beginning: a game in game theory is not just what we traditionally think of as a game (chess, checkers, oker, tennis, or football), but is much more inclusive a game is any interaction between arties, each with their own interests. And game theory studies how these arties make decisions during such interactions. 1 Since we very often build large systems in comuter science, which are used by multile users, whose actions affect the erformance of all the others, it is natural that game theory would lay an imortant role in many CS roblems. For examle, game theory can be used to model routing in large networks, or the behavior of eole on social networks, or auctions on Ebay, and then to make ualitative/uantitative redictions about how eole would behave in these settings. In fact, the two areas (game theory and comuter science) have become increasingly closer to each other over the ast two decades the interaction being a two-way street with game-theorists roving results of algorithmic interest, and comuter scientists roving results of interest to game theory itself. 2 Some Definitions and Examles In a game, we have A collection of articiants, often called layers. Each layer has a set of choices, called actions, from which layers choose about how to lay (i.e., behave) in this game. Their combined behavior leads to ayoffs (think of this as the hainess or satisfaction level ) for each of the layers. Let us look at some examles: all of these basic examles have only two layers which will be easy to icture and reason about. 2.1 The Shooter-Goalie Game This game abstracts what haens in a game of soccer, when some team has a enalty shot. There are two layers in this game. One called the shooter, the other is called the goalie. Hence this is a 1 Robert Aumann, Nobel rize winner, has suggested the term interactive decision theory instead of game theory. 1

2 2-layer game. The shooter has two choices: either to shoot to her left, or shoot to her right. The goalie has two choices as well: either to dive to the shooter s left, or to the shooter s right. Hence, in this case, both the layers have two actions, denoted by the set {L, R}. 2 Now for the ayoffs. If both the shooter and the goalie choose the same strategy (say both choose L, or both choose R) then the goalie makes a save. Note this is an abstraction of the game: for now we assume that the goalie always makes the save when diving in the correct direction. This brings great satisfaction for the goalie, not so much for the shooter. On the other hand, if they choose different strategies, then the shooter scores the goal. (Again, we are modeling a erfect shooter.) This brings much hainess for the shooter, the goalie is disaointed. Being mathematically-minded, suose we say that the former two choices lead to a ayoff of +1 for the goalie, and 1 for the shooter. And the latter two choices lead to a ayoff of 1 for the goalie, and +1 for the shooter. We can write it in a matrix (called the ayoff matrix) thus: ayoff goalie matrix M L R shooter L ( 1, 1) (1, 1) R (1, 1) ( 1, 1) The rows of the game matrix are labeled with actions for one of the layers (in this case the shooter), and the columns with the actions for the other layer (in this case the goalie). The entries are airs of numbers, indicating who wins how much: e.g., the L, L entry contains ( 1, 1), the first entry is the ayoff to the row layer (shooter), the second entry the ayoff to the column layer (goalie). In general, the ayoff is (r, c) where r is the ayoff to the row layer, and c the ayoff to the column layer. In this case, note that for each entry (r, c) in this matrix, the sum r + c = 0. Such a game is called a zero-sum game. The zero-sum-ness catures the fact that the game is urely cometitive. Note that being zero-sum does not mean that the game is fair in any sense a game where the ayoff matrix has (1, 1) in all entries is also zero-sum, but is clearly unfair to the column layer. One more comment: for 2-layer games, textbooks often define the row-ayoff matrix R which consists of the ayoffs to the row layer, and the column-ayoff matrix C consisting of the ayoffs to the column layer. The tules in the ayoff matrix M contain the same information, i.e., M ij = (R ij, C ij ). The game being zero-sum now means that R = C, or R + C = 0. In the examle above, the matrix R is ayoff goalie matrix M L R shooter L 1 1 R Note carefully: we have defined things so that left and right are with resect to the shooter. From now on, when we say the goalie dives left, it should be clear that the goalie is diving to the shooter s left. 2

3 2.2 Pure and Mixed Strategies Now given a game with ayoff matrix M, the two layers have to choose strategies, i.e., decide how to lay. One strategy would be for the row layer to decide on a row to lay, and the column layer to decide on a column to lay. Say the strategy for the row layer was to lay row/ I and the column layer s strategy was to lay column J, then the ayoffs would be given by the tule (R IJ, C IJ ) in location I, J: ayoff R IJ to the row layer, and C IJ to the column layer In this case both layers are laying deterministically. (E.g., the goalie decides to always go left, etc.) A strategy that decides to lay a single action is called a ure strategy. But very often ure strategies are not what we lay. We are trying to comete with the worst adversary, and we may like to hedge our bets. Hence we may use a randomized algorithm: e.g., the layers dive/shoot left or right with some robability, or when laying the classic game of Rock-Paer-Scissors the layer choose one of the otions with some robability. This means the row layer decides on a non-negative real i for each row, such that i i = 1 (this gives a robability distribution over the rows). Similarly, the column layer decides on a i > 0 for each column, such that i i = 1. The robability distributions, are called the mixed strategies for the row and column layer, resectively. And then we look at the exected ayoff V R (, ) := ij Pr[row layer lays i, column layer lays j] = ij i j R ij for the row layer (where we used that the row and column layer have indeendent randomness), and V C (, ) := i j C ij ij for the column layer. This being a two-layer zero-sum game, we know that V R (, ) = V C (, ), so we will just mention the ayoff to one of the layers (say the row layer). For instance, if = (0.5, 0.5) and = (0.5, 0.5) in the shooter-goalie game, then V R = 0, whereas = (0.75, 0.25) and = (0.6, 0.4) gives V R = = Minimax-Otimal Strategies What does the row layer want to do? She wants to find a vector that maximizes the exected ayoff to her, over all choices of the oonent s strategy. The mixed strategy that maximizes the minimum ayoff. I.e., the row layer wants to find Make sure you arse this correctly: lb := max min V R(, ) mixed strategy that maximizes the minimum exected ayoff {}}{ lb := max min V R (, ) }{{} ayoff when oonent lays her otimal strategy against our choice Loosely, this much ayoff the row layer can guarantee to herself no matter what the column layer does. The uantity lb is a lower bound on the row-layer s ayoff. 3

4 What about the column layer? She wants to find some that maximizes her own exected ayoff, over all choices of the oonent s strategy. She wants to otimize max min V C(, ) But this is a zero-sum game, so this is the same as max min ( V R(, )) And ushing the negative sign through, we get the column layer is trying to otimize her own worst-case ayoff, which is min max V R(, ) So the ayoff in this case to the row layer is ub := min max V R(, ) The column layer can guarantee that the row layer does not get more than this much ayoff, no matter what the row-layer does. This is an uer bound on the row layer s ayoff. We have two uantities: lb and ub. How do they comare? To figure this out, we first make a simle but imortant observation: suose we want to find the row layer s minimax-otimal strategy. Then we can assume that the column layer lays a ure strategy (a single column). Why? Once the row layer fixes a mixed strategy, the column layer then has no reason to randomize: her ayoffs will be some average of the ayoffs from laying the individual columns, so she can just ick the best column for her. In other words, we get that the uantity ub can be euivalently defined as lb = max Similarly, the column layer wants to otimize ub = min max i The Balanced Game Examle j min j i R ij. j R ij = max i min i j C ij. For the shooter-goalie game, we claim that the minimax-otimal strategies for both layers is (0.5, 0.5). How to calculate this? Row Player: For the row layer (shooter), suose = ( 1, 2) is the mixed strategy. Note that 1 0, 2 0 and = 1. So it is easier to write the strategy as = (, 1 ) with [0, 1]. OK. If the column layer (goalie) lays L, then this strategy gets the shooter a ayoff of ( 1) + (1 ) (1) = 1 2. If the column layer (goalie) lays R, then this strategy gets the shooter a ayoff of (1) + (1 ) ( 1) = 2 1. So we want to choose some value [0, 1] to maximize lb = min(1 2, 2 1) In this case, this maximum is achieved at = 1/2. (One way to see it is by drawing these two lines.) And the minimax-otimal exected ayoff to the shooter is 0. Column Player: The calculation for the column layer (goalie) is very similar in this case. The minimax-otimal strategy for the goalie is also (0.5, 0.5) and the guarantees that the shooter cannot make more than 0 ayoff. 4 j

5 An observation: the shooter can guarantee a ayoff of lb = 0, and the goalie can guarantee that the shooter s ayoff is never more than ub = 0. Since lb = ub, in this case the value of the game is said to be lb = ub = The Asymmetric Goalie Examle Let s change the game slightly. Suose the goalie is weaker on the left. What haens if the ayoff matrix is now: L R shooter L ( 1 2, 1 2 ) (1, 1) R (1, 1) ( 1, 1) Row Player: For the row layer (shooter), suose = (, 1 ) is the mixed strategy, with [0, 1]. If the column layer (goalie) lays L, then this strategy gets the shooter a ayoff of ( 1/2) + (1 ) (1) = 1 (3/2). If the column layer (goalie) lays R, then this strategy gets the shooter a ayoff of (1) + (1 ) ( 1) = 2 1. So we want to choose some value [0, 1] to maximize lb = min(1 (3/2), 2 1) In this case, this maximum is achieved at = 4/7. And the minimax-otimal exected ayoff to the shooter is 1/7. Note that the goalie being weaker means the shooter s ayoff increases. Column Player: What about the calculation for the column layer (goalie)? If her strategy is = (, 1 ) with [0, 1], then if the shooter lays L then the shooter s ayoff is ( 1/2) + (1 ) (1) = 1 (3/2). If she lays R, then it is 2 1. So the goalie will try to minimize ub = max(1 (3/2), 2 1) which will again give (4/7, 3/7) and guarantees that the exected loss is never more than 1/7. Again, the shooter guarantees a ayoff of lb = 1/7, and the goalie can guarantee that the shooter s ayoff is never more than ub = 1/7. In this case the value of the game is said to be lb = ub = 1/7. Exercise 1: What if both layers have somewhat different weaknesses? What if the ayoffs are: (-1/2, 1/2) (3/4, -3/4) (1, -1) (-3/2, 3/2) Show that minimax-otimal strategies are = (2/3, 1/3), = (3/5, 2/5) and value of game is 0. Exercise 2: For the game with ayoffs: (-1/2, 1/2) (3/4, -3/4) (1, -1) (-2/3, 2/3) Show that minimax-otimal strategies are = ( 4, 3 17 ), = (, 18 ) and value of game is Exercise 3: For the game with ayoffs: (-1/2, 1/2) (-1, 1) (1, -1) (2/3, -2/3) Show that minimax-otimal strategies are = (0, 1), = (0, 1) and value of game is

6 3 Von Neumann s Minimax Theorem In all the above examles of 2-layer zero-sum games, we saw that the row layer has a strategy that guarantees some ayoff lb for her, no matter what strategy the column layer lays. And the column layer has a strategy that guarantees that the row layer cannot get ayoff more than ub, no matter what strategy the row layer lays. The remarkable fact in the examles was that lb = ub in all these cases! Was this just a coincidence? No: a celebrated result of von Neumann 3 shows that we always have lb = ub in (finite) 2-layer zero-sum games. Theorem 1 (Minimax Theorem (von Neumann, 1928)) Given a finite 2-layer zero-sum game with ayoff matrices R = C, lb = max min V R(, ) = min This common value is called the value of the game. max V R(, ) = ub. The theorem imlies that in a zero-sum game, both the row and column layers can even ublish their minimax-otimal mixed strategies (i.e., tell the strategy to the other layer), and it does not hurt their exected erformance as long as they lay otimally. 4 Von Neumann s Minimax Theorem is an imortant result in game theory, but it has beautiful imlications to comuter science as well as we see in the next section. 4 Lower Bounds for Randomized Algorithms In order to rove lower bounds, we thought of us coming u with algorithms, and the adversary coming with some inuts on which our algorithm would erform oorly take a long time, or make many comarisons, etc. We can encode this as a zero-sum game with row-layer ayoff matrix R. (Kee sorting as an examle alication in your mind.) The columns are various algorithms for the roblem (for sorting n elements). The rows are all the ossible inuts (all n! of them). The entry R ij is the cost of the algorithm j on the inut i (say the number of comarisons). This may be a huge matrix, but we ll never write it down. It s just a concetual guide. But what does the matrix tell us? A lot, as it turns out: A deterministic algorithm with good worst-case guarantee is a column that does well against all rows: all entries in this column are small. A randomized algorithm with good exected guarantee is a robability distribution over columns, such that the exected cost for each row i is small. This is a mixed strategy for the column layer. It gives an uer bound. 3 John von Neumann, mathematician, hysicist, and olymath. 4 It s like telling your rock-aer-scissors oonent that you will lay each action with eual robability, it does not buy them anything to know your strategy. This is not true in general non-zero-sum games; there if you tell your oonent the mixed-strategy you re laying, she may be able to do better. Note carefully that you are not telling them the actual random choice you will make, just the distribution from which you will choose. 6

7 Ideally we want to find the minimax-otimal distribution achieving the value of this game. This would be the best randomized algorithm. What is a lower bound for randomized algorithm? It is a mixed-strategy (a robability distribution over the inuts) such that for every algorithm j, the exected cost of j (under distribution ) is high. So to rove a lower bound for randomized algorithms, if suffices to show that lb is high for this game. I.e., give a strategy for the row layer (a distribution over inuts) such that every column (deterministic algorithm) incurs a high cost on it. 4.1 A Lower Bound for Sorting Algorithms Recall from Lecture #2 we showed that any deterministic comarison-based sorting algorithm must erform log 2 n! = n lg n O(n) comarisons in the worst case. The next theorem extends this result to randomized algorithms. Theorem 2 Let A be any randomized comarison-based sorting algorithm (that always oututs the correct answer). Then there exist inuts on which A erforms Ω(lgn!) comarisons in exectation. Proof: Suose we construct a matrix R as above, where the columns are ossible (deterministic) sorting algorithms for n elements, the rows are the n! ossible inuts, and entry R ij is the number of comarisons algorithm j makes on inut i. We claim that the value of this game is Ω(lg n!): this imlies that the best distribution over columns (i.e., the best randomized algorithm) must suffer at least this much cost on some column (i.e., inut). To show the value of the game is large, we show a robability distribution over the columns (i.e., inuts) such that the exected cost of every row (i.e., deterministic algorithm) is Ω(lg n!). This robability distribution is the uniform distribution: each of the n! inuts is eually likely. Now consider any deterministic algorithm: as in Lecture #2, this is a decision tree with at least n! leaves. No two inuts go to the same leaf. In this tree, how many leaves can have deth at most (lg n!) 10? At most the number of nodes at deth at most (lg n!) 10 in a binary tree. Which in turn is So (lg n!) n! 1024 n! fraction of the leaves in this tree have deth more than (lg n!) 10. In other words, if we ick a random inut, it will lead to a leaf at deth more than (lg n!) 10 with robability Which gives the exected deth of a random inut to be 0.99((lg n!) 10) = Ω(lg n!). 5 General-Sum Two-Player Games In general-sum games, we don t deal with urely cometitive situations, but cases where there are win-win and lose-lose situations as well. For instance, the coordination game of chicken, a.k.a. what side of the street to drive on? It has the ayoff matrix: Note that we are now using the convention that a layer choosing L is driving on their left. Note that if both layers choose the same side, then both win. And if they choose oosite sides, both crash and lose. (Both layers can choose to drive on the left like Britain, India, etc. or both on the right, like the rest of the world, but they must coordinate. Both these are stable solutions and give a ayoff of 1 to both arties.) 7

8 ayoff Bob matrix M L R Alice L (1, 1) ( 1, 1) R ( 1, 1) (1, 1) Consider another coordination game that we call which movie? Two friends are deciding what to do in the evening. One wants to see Citizen Kane, and the other Dumb and Dumber. They d rather go to a movie together than searately (so the strategy rofiles C, D and D, C have ayoffs zero to both), but C, C has ayoffs (8, 2) and D, D has ayoffs (2, 8). ayoff Bob matrix M C D Alice C (8, 2) (0, 0) D (0, 0) (2, 8) Finally, yet another game is Prisoner s Dilemma (or to ollute or not? ) with the ayoff matrix: ayoff Bob matrix M collude defect Alice collude (2, 2) ( 1, 3) defect (3, 1) (0, 0) 5.1 Nash Euilibria In this case, a good notion is to look for a Nash Euilibrium 5 which is a stable set of (mixed) strategies for the layers. Stable here means that given strategies (, ), neither layer has any incentive to unilaterally switch to a different strategy. I.e., for any other mixed strategy for the row layer row layer s new ayoff = ij i j R ij ij i j R ij = row layer s old ayoff and for any other ossible mixed strategy for the column layer column layer s new ayoff = ij i jc ij ij i j C ij = column layer s old ayoff. Here are some examles of Nash euilibria: In the chicken game, both { = (1, 0), = (1, 0)} and { = (0, 1), = (0, 1)} are Nash euilibria, as is { = ( 1 2, 1 2 ), = ( 1 2, 1 2 )}. In the movie game, the only Nash euilibria are { = (1, 0), = (1, 0)} and { = (0, 1), = (0, 1)}. In risoner s dilemma, the only Nash euilibrium is to defect (or ollute). So we need extra incentives for overall good behavior. 5 Named after John Nash: mathematician and Nobel rize winner. 8

9 It is easy to come u with games where there are no stable ure strategies. The main result in this area was roved by Nash in 1950 (which led to his name being attached to this concet) Theorem 3 (Existence of Stable Strategies) Every finite layer game (each with a finite number of strategies) has at least one Nash euilibrium. This theorem imlies the Minimax Theorem (Theorem 1) as a corollary: indeed, take any twolayer zero-sum game and consider any Nash euilibrium (, ), with value V = ij i j R ij = ij i j C ij. Since this is stable, neither layer can do better by deviating, even knowing the other layer s strategy. So they must be laying minimax-otimal. 9