Computing Optimal Strategies in Matrix Games

Transcription

1 Computing Optimal Strategies in Matrix Games Jonathan Dando BSc (Hons) Computer Science 2006

2 COMPUTING OPTIMAL STRATEGIES IN MATRIX GAMES Submitted by: Jonathan Dando COPYRIGHT Attention is drawn to the fact that the copyright of this thesis rests with the author. The Intellectual Property Rights of the products produced as part of the project belong to the University of Bath (see This copy of the thesis has been supplied on condition that anyone who consults it is understood to recognise that its copyright rests with its author and that no quotation from the thesis and that no information derived from it may be published without prior written consent of the author. DECLARATION This dissertation is submitted to the University of Bath in accordance with the requirements of the degree of Batchelor of Science in the Department of Computer Science. No portion of the work in this dissertation has been submitted in support of an application for any other degree or qualification of this or any other university or institution of learning. Except where specifically acknowledged, it is the work of the author. Signed... This thesis may be made available for consultation with the University Library and may be photocopied or lent to other libraries for the purpose of consultation. Signed...

3 Abstract Game theory was first developed in the eighteenth century but was only considered seriously in the early twentieth century by mathematicians such as von Neumann and Nash. This dissertation introduces and briefly discusses the history of game theory and its application to today s world. It then goes on to discuss many important aspects of game theory such as payoff matrices, utility, pure and mixed strategies. Algorithms for solving two player zero sum matrix games are discussed. A system for solving these games is designed and implemented.

4 Acknowledgments I would like to thank my supervisor Professor Nicolai Vorobjov for his constant support throughout the project. I would also like to thank my father Stuart for his meticulous proof reading and encouragement. I would also like to express further thanks to my mother Susan for her never ending encouragement and patience without which this project would not have been possible. i

5 Contents 1 Introduction 1 2 Literature Survey Representation of Games Extensive Form Normal Form Utility Strategies Pure Strategies Mixed Strategies Calculating Optimal Mixed Strategies Linear Programming Existing Solutions Requirements Introduction ii

6 3.2 Project Requirements System Requirements Functional Requirements Non-Functional Requirements Domain Requirements Design Introduction Structure of the System Saddle Points Domination Games n Games Special Cases m 2 Games m n Games Implementation Introduction Programming Language Generating the Payoff Matrix Removing Dominated Strategies iii

7 5.5 The Optimal Strategy Games n Games m 2 Games Transposing the Payoff Matrix Swapping the Optimal Strategies m n Games Finding the Pivot Performing the Pivot The Driver Function Memory Allocation Error Messages Testing Introduction Validation Defect Testing Black-box Testing Equivalence Partitioning Conclusion Project Evaluation iv

8 7.2 Possible Extensions Personal Evaluation References 52 A Testing 54 A.1 Validation of Requirements Specification A.2 Black-box Testing A.2.1 The Double Linked List A.2.2 Invalid User Input A.2.3 Solving Games A.2.4 Some Random Games B Source Code 83 B.1 DoubleLinkedList.h B.2 DoubleLinkedList.c B.3 strategies.h B.4 strategies.c B.5 extrafunctions.h B.6 extrafunctions.c B.7 ComputeOptimalStrategy.c B.8 DoubleLinkedListTest.c v

9 List of Figures 2.1 Extensive Form Normal Form A Game in Equilibrium Two Player Zero Sum Game Zero Sum Game in Matrix Form A Game Without Saddle Points A 2 n game A Graphical Representation of a Linear Programming Problem A potential pivot p with corresponding values of r and c A General Transformation Examples of games with and without saddle points A 2 n Game Representation A Game Where The Optimal Solution Consists of More Than Two Strategies A Game With an Interval Strategy vi

10 4.5 Other Cases An m 2 Game A Multidimensional Array in Memory Black-box Testing vii

11 Chapter 1 Introduction Dictionary.com [online] [2005] defines a game as competitive activity or sport in which players contend with each other according to a set of rules. Games have been played for a very long time in one form or another. Parlour games are a specific kind of game that are usually played indoors and were very popular in Victorian times. These games start from a given point and the players play the game using a set of strategies. Along with the players strategies there could be an element of luck or chance such as throwing a dice or choosing a number of cards from a random deck. There are many examples of this kind of game with varying complexity. Games such as stone, paper, scissors are played purely by chance. There is no skill whatsoever involved. Other games such as bridge involve a lot more skill but there is still a degree of chance. In chess, chance does not play a part at all. Chess and bridge demonstrate another aspect of games that play an important role. The amount of information that is available to each player is very different in both games. In chess, every move that has been made so far is available to the player. In contrast, the players in bridge have a very imperfect knowledge. This means that each player has to choose a strategy without knowing the exact position of the game. These games terminate in some sort of payoff to each player. This payoff maybe monetary or simply pure satisfaction. In order to maximise this payoff it is necessary to find an optimal strategy. The derivation of these strategies have been studied under a branch of mathematics that came to be known as game theory. While it can be used to solve the simple problems 1

12 described above these problems are often too trivial. Game theory was first used as a way to predict economic behaviour, but it has been used in many fields ranging from biology to philosophy. Game theory was first formalised by John von Neumann before and during the Cold War due to its application in military strategy. Other theorists such as Nash have contributed to further study. In more recent times game theory has attracted attention from computer scientists due to its application in artificial intelligence and cybernetics. The purpose of this dissertation is to study the history of game theory and then design and implement an application to find optimal strategies for certain games. The dissertation is split into different sections that represent the progression of the application. Chapter 2 is a literature survey. This chapter is dedicated to research of the topic and an investigation of existing solutions to the problem. Chapter 3 formalises the requirements, which leads onto chapter 4 which explains how the application will be developed. Chapter 5 describes the final implementation. Chapter 6 shows all the testing that took place to make sure the application functions correctly. Finally chapter 7 is a conclusion which ties the whole dissertation together. The appendix contains the source code and all the test data for the project. A full code listing and instructions on how to use the application are included on the attached CD. 2

13 Chapter 2 Literature Survey The purpose of this section is to provide an initial investigation into the problem including current methods for solving the problem and any existing systems that implement these solutions. It is generally accepted that the ideas of game theory were introduced in the eighteenth century, however it wasn t until John von Neumann s book, Theory of Games and Economic Behavoir, was published in 1944 that it really took off and other theorists continued the development. There needs to be a way of representing games or situations in order to develop a solution to the problem. It is common for all the participants to have a set of strategies to choose. The payoff to each of the players is determined according to the strategy that each of them chooses 2.1 Representation of Games There are two common ways of representing this information that will now be considered Extensive Form In this form the game is represented by a tree where each node indicates the current state of the game. There is a unique node called the initial node that represents the 3

14 start of the game. Any node that only has one edge connected to it is called a terminal node and these represent the end of the game. Every edge represents a possible action that a player can take. At each of the terminal nodes the payoff for every player is indicated. Figure 2.1: Extensive Form Figure 2.1 is an example of a representation of an extensive game. The game starts at the top with player 1 having a choice of either L or R. Player 2 then gets a choice of either A, B or C, D. The payoffs for all of these strategies are shown at the bottom. This way of representing games is often used if the players are aware of everyone else s strategies. If they have no information about the other players strategies then the normal form is used instead Normal Form The normal form (also known as the strategic form or matrix form) is a matrix where the players strategies are shown on the edges and the payoffs to each player make up the matrix itself. This representation is good when both players play their strategies simultaneously with no prior knowledge of the other player s strategy H T H (-1, 1) (1, -1) T (1, -1) (-1,1) Figure 2.2: Normal Form The example in figure 2.2 is one taken from Owen [1982]. It represents a game known as matching pennies. The idea is for both players to choose either heads (H) or tails (T). If both players choose the same side then player II gets one cent. If the players 4

15 choose differently then player I gets one cent. The values on the rows indicate player I s strategies whereas the values on the columns indicate the strategies that player II can use. By cross referencing each players strategy the pay out to each player can be seen Utility The notion of utility is used when we are talking about a pay off to all the participants in the game. When we do this we assign a numerical value for the pay off. The problem here is that how can we assign a value if the pay off is not monetary? The following discussion is based on Neumann & Morgenstern[1944] where there is a section devoted to this idea. Is it possible to assign a value to a certain situation? Part of the problem is how can we compare one situation to another. For example let us compare heat. We could intuitively say whether one object is warmer than another one, but there would be no accurate way to express by how much. It can be assumed that an individual will have preferences of one outcome over another. It can also be assumed that an individual will have a preference of a combination of events that lead to an outcome. Therefore given outcomes A, B and C that person will be able to put them in order of preference. If we allow this assumption then we could assign a numerical value to the outcome. This assumption will be used for the rest of the dissertation. 2.2 Strategies Pure Strategies A game G consists of N players. Each of these players can choose from a finite set of n strategies. S = α 1, α 2...α n. (2.1) Player I can then choose strategy α i from S. Player II would choose strategy β j from S and so on. This idea of each player having a set of strategies at the beginning of the game does not really relate well to real life. In real life it would be impossible to plan for every outcome of the game from the start. For this project a theoretical approach will be used where it will be assumed that all the players already have a complete set 5

16 of strategies. This is known as the set of pure strategies. At the end of the game there is a payoff function π that calculates the pay out to each of the players. So for n players the payoff function is given by π(σ 1...σ n ). (2.2) Where σ n represents the pay out to player n. It is possible to represent this function so that for any combination of strategies an n-dimensional array can be created. In the case when n = 2, a matrix can be created. This form of representation of payoff is known as the normal form as was introduced in section Equilibrium Points A game is said to be in equilibrium if there is no reason for any player in the game to change their strategy. In other words it is an optimal solution for every player. The following example from Owen [1982] is an example of a game in equilibrium. In this β 1 β 2 α 1 (2,1) (0,0) α 2 (0,0) (1,2) Figure 2.3: A Game in Equilibrium example both (α 1, β 1 ) and (α 2, β 2 ) are equilibrium points. Neither player can improve their pay out by changing their strategy. Not all games have equilibrium points. The matching pennies game in figure 2.2 is a game where there are no equilibrium points. John Nash was one of the pioneers in the study of game theory. He proposed a solution for all finite games so that for every game no player would ever regret the strategy they took. This equilibrium is now known as the Nash equilibrium and is formally defined as follows....a set of pure strategies S = (σ1, σ 2,..., σ i,..., σ N ) constitutes a Nash equilibrium if and only if pure strategy σi is a best reply to the combination of the strategies for all other players in S for all i = 1,..., N. (Definition from Hargreaves and Varoufakis [2004]) Game theory is a very broad subject so at this point we will impose some restrictions 6

17 on the games that will be studied. From now on only games in normal form will be considered. In addition only zero sum games will be considered. A game is said to be zero sum if the sum of the pay outs to each player is zero. i.e. if the payoff function (σ 1,..., σ n ) satisfies n σ i = 0. (2.3) i=1 This form of game represents a closed system. Everything that one player wins must be lost by the other. Two player zero-sum games are called strictly competitive. These games are represented in a very similar way to normal form as discussed. The only difference now is that there is only one pay out. This indicates the pay out to player I. The pay out to player II is just the inverse. i.e. σ 1 (α i, β j ) = σ 2 (α i, β j ). The elements a ij indicate the pay out based on the strategies (α i, β j ). You can see that there is β 1 β 2 β 3 α α α Figure 2.4: Two Player Zero Sum Game Figure 2.5: Zero Sum Game in Matrix Form only one payoff for each pair of strategies. Figure 2.5 shows the same game in matrix form. This is the representation that will be used for the rest of this document. A strategy pair a ij will be in equilibrium if it is both the smallest in its row and the largest in its column. If such an element exists it is known as a saddle point. In figure 2.5 above, the element a 22 is a saddle point and is the optimal solution for both players. It is trivial to find solutions for these games. In most games however there are no saddle points. If you consider the matching pennies game described in figure 2.2 there are no saddle points. If we now consider the following example from Owen[1982]. This game has no saddle [ ] Figure 2.6: A Game Without Saddle Points points. Let us assume that the other player will guess correctly whichever strategy we choose. Therefore we will try and maximise our winnings. In this situation we choose 7

18 the first strategy. With this strategy we are guaranteed to win at least two. If we were to use the other strategy there is a chance that we could only win one. This guarantee is known as the gain-floor and is denoted as v I. v I = max i min j a ij. (2.4) If we were player II and our thinking went along the same lines then we would choose the second strategy as this would guarantee the most that we would pay out would be three. The other strategy could result in us paying out four. This is known as the loss-ceiling and is denoted as v II. v II = min j max i a ij. (2.5) It is obvious from the above that v I will never be greater than v II. If the two are ever equal then we have a saddle point. If we consider two players competing in a matrix game. Player I will choose row i whereas player II will choose column j. The pay out to each player will be the element a ij where player II will pay player I this value. It is obvious from this that player I will be trying to maximise this value while at the same time player II will be trying to minimise it. From the definitions of v I and v II above we can see that player I will never win less than the gain-floor whereas player II will never lose more than the loss-ceiling. In this scenario each player will be trying to guess the strategy of the other player so that they can choose the best strategy for themselves. However the other player will be doing the same and will try and alter their strategy accordingly. Therefore the outcome of the game will be uncertain. If either player were to improve their outcome they would have to keep their strategy secret. However this is impossible as the other player would be able to reconstruct their reasoning. Therefore it would be better to choose a strategy randomly and then stick to it. By doing this, the other player will have no way of guessing the strategy. As an example consider the game paper, scissors, stone. In this game each player has to choose one of paper, scissors or stone. Each of these strategies loses against one and wins against the other. If you were to play the game and try and figure out a strategy then the other player could also work out your reasoning and you would never win. In this situation it would be preferable to choose your strategy randomly with each strategy having probability 1 3. By doing this you are guaranteed to win at least half the time. 8

19 The set of probabilities to choose one of the strategies randomly is known as a mixed strategy. In the preceding example the probability set would be 1 3, 1 3, Mixed Strategies The idea behind mixed strategies is that the strategy should be chosen irrationally but the randomisation scheme should be chosen rationally. A mixed strategy is a probability distribution over all the pure strategies. A mixed strategy for player I with a set of m strategies is an m-vector, x = (x 1,..., x m ) such that the probability of choosing a strategy a ij is x i. The following conditions must be met for a mixed strategy x i 0. (2.6) m x i = 1. (2.7) i=1 Conversely, a mixed strategy for player II with n strategies is an n-vector, y = (y 1,...y n ) such that the following conditions hold y j 0. (2.8) n y j = 1. (2.9) j=1 The payoff for player I can be written as 1 i n a ij x i = a 1j x 1 + a 2j x a nj x n. (2.10) Suppose the players are playing a matrix game A. The payoff function for both players is then A(x, y) = 1 i n 1 j m a ij x i y j. (2.11) This function can also be described in matrix notation a 1,1 a 1,2... a 1,j y 1 A(x, y) = [x 1, x 2..., x n ] a 2,1 a 2,2... a 2,j y (2.12) a i,1 a i,2... a i,j y m This matrix multiplication can be simplified to XAY T where X is the set of player I s strategies and Y is the set of player II s strategies. 9

20 Just as before each player will fear that the other will guess their strategy and play accordingly. For example, if player I were to use their mixed strategy x then player II would choose a y to minimise the amount that they would lose. As documented earlier this is known as player I s gain-floor. The following steps are based on Owen[1982]. v(x) = min y Y xayt. (2.13) This equation can be thought of as an average of all the expected pay offs based on all of player II s pure strategies. Therefore the minimum can be written as v(x) = min xa j. (2.14) where A j is the jth column of the matrix A. Therefore player I should now choose x so as to maximise v(x) so that they may obtain v I = max x X min j xay j. (2.15) Such an x is known as I s maximin strategy. If we do the same for player II we get v II = min y Y max i A i y T. (2.16) Where y is known as II s minimax strategy. As before we have two values v I and v II. It is again obvious that v I v II We showed that there is only an equality when there is a saddle point. Neumann & Morgenstern[1944] showed that there is an equilibrium for every finite zero sum matrix game. v I = v II. (2.17) This is known as the minimax theorem. Davis [1983] states that the minimax theorem is that...one can assign to every finite, two person, zero sum game a value V : the average amount player I can expect to win from player II if both players act sensibly. This concept along with mixed strategies makes the study of this topic far simpler. In their absence, except for trivial games that have saddle points, it would be impossible to choose a sensible strategy or to predict the outcome of the game. With the inclusion of the minimax theorem you can effectively treat all games as though they have equilib- 10

21 rium points. This means that every game can have a clear value, and that every player can enforce this value by selecting the appropriate strategy. The difference between these games and those that have saddle points is that you use a pure strategy whereas with games without saddle points, you use a mixed strategy and obtain the value of the game on average. The maximin value for player I combined with the minimax value for player II is said to be an equilibrium pair. Like the equilibrium pairs discussed, earlier for pure strategies, these form a unique outcome v. This is said to be the value of a zero sum two person game. The proof of this theorem will not be discussed here, but a full proof can be found in Owen[1982]. 2.3 Calculating Optimal Mixed Strategies The minimax theorem in the previous section guarantees that every two person game will have an optimal strategy. The solution is trivial if there are saddle points, in which case the solution is the strategy α i and β j that corresponds to this saddle point, or if we are using mixed strategies the elements x i and y j both equal one and everything else is equal to zero. As mentioned before, except for the most trivial of games, most do not have saddle points. In these cases we need another way to compute the optimal strategy. To start we try and make the problem simpler. This leads us to the concept of domination. Domination By careful inspection of the payoff matrix and a little intuition it is possible to remove unnecessary rows and columns. Strictly dominant/dominated strategy are defined by Hargreaves and Varoufakis [2004] as follows A strategy is strictly dominant if it guarantees a player higher pay-offs than she or he would receive playing any other strategy, against all possible strategies of the opposition. Reversing this definition, a strategy is strictly dominated if it guarantees a player lower pay-offs than she or he would receive playing some other strategy, against all possible strategies of the opposition 11

22 A strategy is dominant if it guarantees a player an equal or higher payoff than she or he would receive playing any other strategy against all possible strategies of the opposition. To make this more formal we can say that given a matrix A, the ith row dominates the kth row if a ij a kj for all j. (2.18) and a ij > a kj for at least one j. (2.19) We can also say that the jth column dominates the lth column if a ij a il for all i. (2.20) and a ij < a il for at least one i. (2.21) We will now work through an example to demonstrate this. Consider the matrix game It is clear that the second column dominates the fourth column. Therefore player II will never use their fourth strategy. As a result we can now ignore this column and the matrix becomes We can now see that the second row dominates the first, so we are left with the matrix [ 0 1 ] We can now see that the third column is dominated by the second. Therefore we will be left with the matrix [ ] This is a much simpler matrix. Now we only have to compute an optimal strategy for a 2 2 matrix. 12

23 2 2 Matrix Games Computing optimal strategies for these matrices is quite simple. In fact it can be shown that the solution is given by x = JA JA J T. (2.22) y = A J T JA J T. (2.23) v = A JA J T. (2.24) Where A is the adjoint of A, A the determinant of A, and J the vector (1, 1). The derivation of these equations are given in Owen [1982]. We now discuss an example. Consider the matrix that we were left with after removing the dominated strategies. [ 0 1 ] 1 2 This game has no saddle point, therefore we can use the equations given above. The adjoint of A is [ 2 ] and A is -1, JA = ( 3, 1), A J T = ( 3, 1) and JA J T = 4. Using these and the equations given we have x = ( 3 4, 1 4 ) y = (3 4, 1 4 ) v = 1 4. (2.25) It can be checked that by using these values for x and y it does guarantee a pay off value of n and m 2 Matrix Games After 2 2 games the next simple form of games are those where one of the players only has two strategies to choose from. Lets first discuss the case when player I only has two strategies to play whereas player II has n strategies. When the game is played, player I will play their mixed strategy X against player IIs pure strategies β j. Player 13

24 Is mixed strategy is (x 1, x 2 ). Therefore we will have some payoff matrix [ a 1,1 a 1,2... a 1,j a 2,1 a 2,2... a 2,j ] (2.26) with player I choosing strategies α 1 and α 2 with probabilities x 1 and x 2 respectively. Therefore player I will want to maximise v(x) = mina 1j x 1 + a 2j x 2. (2.27) We can see from equation 2.7 that the sum of the probabilities must be 1. Therefore we can say that x 1 = 1 x 2 and we can then write v(x) = min(a 2j a 1j )x 2 + a 1j. (2.28) We now have a several of equations that form a set of straight lines with gradients a 2j a 1j that intercept the vertical axis at a 1j. We can easily see that the lines must go through the point (1, a 2j ). Figure 2.7 is an example of a graph that could be drawn to display all the equations. The value of x 2 is displayed along the horizontal axis and the value of v(x) is shown on the vertical axis. The piecewise line at the bottom correspond to the minimum values for v(x) from all of player II s pure strategies. The peak of this line is the maximum among these minimums and is therefore the optimal strategy for player I. The value of x 1 can easily be calculated from x 2. It is worth noting that this point does not always have to be a peak. It is possible for the optimal strategy to be a range of values. This would show on the graph as a horizontal line at the maximum point. Figure 2.7: A 2 n game 14

25 It is a very similar situation when player I has m pure strategies and player II has 2 mixed strategies. In this case there would be m equations that would relate to straight lines on a graph. These could then be plotted and the solution read off as before. The only difference this time is that the equations would be maximum values for v(y) and the solution would be the minimum of all these values. Geometrically the graph would be the inverse of the one in figure 2.7. In this case when one player has three strategies, the graph would then be three dimensional and the solution would be the intersection of three planes. This is a lot harder to visualise. The same is true when a player has even more strategies. It is possible to adapt the algorithm to cope, but it it generally too complicated. In these cases we would need another way to find the optimal strategy. One algorithm that we may utilise is the simplex method Linear Programming According to Owen[1982] a linear program is the problem of maximizing (or minimizing) a linear function (called the objective function) subject to linear constraints. In the most common case, the constraints are loose inequalities, and the variables are restricted to non-negative values. The Simplex Method The simplex method was devised by Dantzig[1963]. The idea behind it, is to try and maximise (or minimise) a linear function according to a set of inequalities. Geometrically these constraints would intercept at different points on a graph. These vertices would be the basic solutions of the problem. The linear function that we are trying to find the solution for would then be moved through the solutions until we come to the last possible point. This vertex would be the optimal solution. The arrow in this diagram indicates the gradient of the objective function. It is clear that in this case the last point that the function would pass through would be at C. This would then be the optimal solution. The algorithm is described in Williams[1966] and Ferguson[2004]. The algorithm runs as follows: 15

26 Figure 2.8: A Graphical Representation of a Linear Programming Problem 1. Search the game for negative numbers. If there are then add a constant to all the values to remove these numbers. Note here that adding a constant to the matrix affects the value of the game and that this addition must be reversed at the end of the computation. 2. The next step is to augment the game matrix with a border of 1 s, -1 s and a 0 such that there is a row of -1 s at the bottom and a column of 1 s on the right and a 0 in the bottom right corner. This is known as a schema and in particular, the First Schema. The final Schema will be the solution to the problem. 3. The third step is to choose one of the elements from the original game matrix that will be known as the pivot element. In order to find the pivot we must consider several trios of numbers: the potential pivot itself p and the border elements in the row and column of the potential pivot r and c. The rules for choosing the p.. r.. c Figure 2.9: A potential pivot p with corresponding values of r and c pivot are as follows: (a) The potential pivot p must be positive (b) The number at the foot of the column c must be negative (c) Compute for each potential pivot the value of 16 r c p. (2.29)

27 This value is known as the pivot criterion (d) Highlight the smallest value of the criterion in each column (e) Of the highlighted values select the largest value. This element corresponds to the pivot p 4. We now pivot as follows (a) Replace the pivot with its reciprocal (b) Replace all values in the pivot row, r, except the pivot with its value divided by the pivot p (c) Replace all values in the pivot column, c, except the pivot with its negative divided by the pivot p (d) Replace all values that are not in the pivot row or column, q, by its value subtracted by r c p where p is the pivot, r is the value in the pivot row and c is the value in the pivot column. This transformation can be shown graphically as Figure 2.10: A General Transformation 5. Exchange the label on the pivot row with the one on the pivot column 6. If there are any negative numbers remaining in the lower border row, go back to step 3 7. Otherwise the solution maybe read out (a) The value of the game, v, is the reciprocal of the number in the lower right corner (b) I s optimal strategy is constructed as follows. Those variables that end up on the left side receive probability zero. Those that end up on the top receive the value of the right edge in the same column divided by the lower right corner (c) II s optimal strategy is constructed as follows. Those variables of Player II that end up on the top receive probability zero. Those that end up on the left receive the value of the right edge in the same row divided by the lower right corner. We will now work through an example of the simplex method from Ferguson [2004]. Given the matrix 17

28 B = We start by looking for saddle points. In this example there are none. We also look for domination. Again there is none. We now add 2 to all the elements to ensure that they are all positive. We note here that this will affect the value of the game v. Therefore we have to remember to subtract 2 at the end to get the true value of the game. The matrix is now B = We can now set up the tableau as follows y 1 y 2 y 3 x x x We must now choose a pivot. All the border values in the columns are negative, so we can choose any column. We now have to highlight the smallest criterion in each row where the potential pivot is positive. We get 1 4, 1 4 and 1 8. We now choose the largest of these values. We could choose either of the 1 4 values. We will choose the 4 in the top left. We now pivot according to the instructions in step 4. The pivot gets replaced by its reciprocal. The rest of the values in the pivot row are divided by the pivot. The rest of the values in the column get divided by the pivot and then the sign is reversed. The rest of the values in the matrix are modified by subtracting r c p. For example for the 3 in the second row, second column we subtract = 1 2, leaving 5 2. After all the transformations, the tableau looks like y 1 y 2 y 3 x 1 4* x x x 1 y 2 y 3 y 1 1/4 1/4 2 1/4 x 2 1/2 5/2-3 1/2 x /4 3/4 1 1/4 Notice here that the labels on the pivot row and column have been interchanged. We now check for negative values in the lower edge. As there is still one we return to step 3. The pivot this time is the 5 2 in the second row, second column. Completing the transformation yields the following 18

29 x 1 y 2 y 3 y 1 1/4 1/4 2 1/4 x 2 1/2 5/2-3 1/2 x /4 3/4 1 1/4 x 1 x 2 y 3 y y x Notice now that all the values on the lower edge are non-negative so we can now read the solution off. The value v of the game is the reciprocal of 0.4, namely 5/2. We have to remember to subtract 2 since we added it on at the beginning. This means the value is 1/2. Since the x 3 is on the left, the optimal p 3 is zero. p 1 and p 2 are the ratios.1/.4 and.3/.4. i.e. 1/4 and 3/4. Therefore I s optimal strategy is (1/4, 3/4, 0). By the same method II s optimal mixed strategy is (1/2, 1/2, 0) Existing Solutions There are a number of packages that have already been created to solve these kinds of games. The mathematics package Maple has a simplex function that can compute the optimal return given an objective function and a set of constraints. A quick search on the internet returns many online applets that solve the games that have been discussed so far. Warner and Costenoble [1996] is an example of one of these applets. With this example you simply enter the normal form of a two person zero sum game and the optimal strategies are returned. This particular applet uses the simplex method that has been introduced in this document. 19

30 Chapter 3 Requirements 3.1 Introduction The aim of this project is to develop an application that solves matrix games that are supplied by the user. The user will input the games at the command line and the application will return the optimal solution for both of the players. The literature review in the previous chapter gave an insight into the problem and gave a brief history of the topic. A number of existing solutions were mentioned along with algorithms that could be used in the final implementation of the application. This chapter aims to strictly define required functionality of the application as well as any constraints on the system. The aim of this project is to design a system capable of solving matrix games. As mentioned above the input to this system is likely to be from the command line. There will be no additional user interface. A discussion about this topic maybe included, but is likely to be beyond the scope of this project. The literature review gave brief details of existing solutions to the problem. These could be used to solve the problems described in this dissertation, but these have been deemed unsuitable to this project as a greater understanding of the topic will be gained from developing the application from the beginning. 20

31 3.2 Project Requirements As with most projects this one is constrained by time. There are only a few months available to design, implement and test the final system as well as writing the final report (this dissertation). As a result the system must be designed so that it can be realistically finished in the allotted time. Time must also be given for other academic commitments of the author. 1. The project consisting of the final implementation and write-up is to be finished and handed in by the 8 th May The system should run efficiently and reliably on the BUCS machines. It may be adapted to run on other systems if time permits. 3.3 System Requirements Sommerville[2001] breaks the system requirements into three sections Functional requirements - These are statements that describe what services the software will provide. They may describe how the system reacts to the user. They may also state what the system will not do. Non-functional requirements - These are constraints on the functionality offered by the system. They include timing constraints standards etc. Domain requirements - These are requirements that come from the application domain of the system. They may be functional or non-functional requirements The system will be developed as an iterative process. This will allow the core functionality to be developed first then as time goes on the rest of the system can be completed. This means that to start, the application will be developed for two players. Each of these players will only have a maximum of two strategies. When this is complete, the system will be enhanced so that one of the players will be able to choose from m strategies. Finally when this is complete the final system will be written to allow player I and II to have m and n strategies respectively. The requirements will be written in the following format. Essential requirements will be identified by the word must whereas non-essential requirements will be identified 21

32 by the word should. Although they are non-essential they are still intended to be included in the final application. Finally requirements identified with may denote desirable functionality, but these should only be considered when the core functionality is complete Functional Requirements 1. User Input 1.1. The user must be able to enter the pay off matrix from the command line prompt in the form [[a 1,1, a 1,2 ], [a 2,1, a 2,2 ],... ] The system should ignore white so that the user can enter the matrix in a manner that makes it more readable The user should be able to enter a pay off matrix to be solved from an input file The pay off matrix must be in the same format as it would be if it were entered from the command line The user must be able to enter any m n matrix 1.4. The user must be able to enter either positive or negative numbers in the pay off matrix The user must be able to enter floating point numbers in the pay off matrix The user may be able to enter fractions in the pay off matrix The system may provide a graphical user interface for the user. 2. System Output 2.1. If the user enters the pay off matrix at the command line then the output will be to the command line If the user enters the pay off matrix from a file, then the output will be to a file The solution will be appended to the input file The system will output the optimal mixed strategy for both players as well as the value of the game v The output must be in the form Player I : [p 1, p 2,... p n ] Player II : [q 1, q 2,... q m ] Value = v 22

33 If the pay off matrix is either a 2 m or m 2 matrix, then the system may output a graphical representation of the solution The value of the game may be an integer, floating point number or fraction The value of the game may be positive, negative or zero Each value of p i and q i must be either an integer or a floating point number in the range [0,1] 2.7. The following must be true for all mixed strategies p and q n p i = 1 i=1 and m q i = 1 j= Non-Functional Requirements 3. The system must be written in C. 4. The system must run on the BUCS machines The system may run on other platforms. 5. The system must run efficiently The system must allocate and deallocate memory as it runs The output should be returned to the user within 3 seconds. 6. The system must be easy to use Users should need little or no training to use the system. 7. The system must be reliable The application should not crash unless there is a problem with the system that it is running on. 8. The system must be robust The system must be able to handle any errors by reporting them to the user and then terminating System errors such as file handling errors or memory allocation errors should be reported to the screen so that the user is aware of them Errors resulting from incorrect user input should be reported to the screen or a file depending on what input method the user chooses. 23

34 3.3.3 Domain Requirements 9. The system must look for saddle points in the pay off matrix 10. The system must look for dominated strategies in the pay off matrix 24

35 Chapter 4 Design 4.1 Introduction So far we have looked at the various algorithms that can be used to find the optimal strategies of matrix games. The aim of this section is to describe them in more detail and discuss how to implement them along with any problems that may arise as well as solutions. 4.2 Structure of the System The system will accept a variety of different games. It must know what to do in each of these different situations. In games where one of the players only has one strategy to choose from. i.e. a game where the pay off matrix is 1 n; the solution is trivial. The other player would simply choose the strategy that yields the best pay off for them. The system will return the best strategy for the player who has more than one strategy. In all games that are solvable, the system will check for saddle points first to see if there is an obvious solution. It will then check for domination so that the matrix maybe simplified. Once this is complete, the system will check for saddle points again. The methods used for finding saddle points are discussed in section 4.3. The techniques used for finding dominated strategies are discussed in section

36 If after the matrix has been checked for domination and saddle points, the resulting matrix is a 2 2 matrix then the system will find the optimal mixed strategy for the game. The method for this is explained in section 4.5. If the resulting matrix is 2 n then the system will find the optimal mixed strategy. If possible the system will also output a graphical representation of this solution. The method for implementing this algorithm is discussed in section 4.6. Finally, if the matrix is such that m > 2 and n > 2 then the simplex method will be used. This is discussed in section 4.8. The system will be implemented incrementally with each of the different scenarios being coded separately. This will allow the system to be in a state where it can be finished at different times, depending on the time constraints. 4.3 Saddle Points An element a ij in the pay off matrix is a saddle point if it happens to be the smallest in a row and the largest in a column. A strategy that the system could use to find saddle points would be to cycle through each of the rows looking for the smallest value. Once this value has been found, it would then search that column for the largest value. If the two elements are the same then a saddle point will have been found. If there is no saddle point then a different method would be needed to find the optimal strategies for each player. Figure 4.1(a) is a game that does not have a saddle point. The minimum [ ] [ 0 1 ] [ 1 ] (a) (b) (c) (d) Figure 4.1: Examples of games with and without saddle points value in the first row is a 2, whereas the largest value in that column is a 3. The minimum value in the second row is a 1 whereas the largest value in that column is a 4. Figure 4.1(c) is another game that does not have any saddle points. Figure 4.1(b) is an example of a matrix game that has a saddle point. The element a 22 is the smallest in that row and the largest in the column. The same is true for the element a 21 in figure 4.1(d). 26

37 If a saddle point is found then this is the pure strategy for both players. The mixed strategy in this situation is 1 for the element that is the saddle point and 0 for everything else. For example, the mixed strategy for the saddle point in figure 4.1(b) would be (0, 1, 0) for both players. If a game does not have a saddle point then it is necessary to find the optimal mixed strategy by other means. 4.4 Domination It is possible, by removing strategies that are dominated, to make the pay off matrix a lot simpler. A dominated strategy is one that is at best equal to another. The system will do this by cycling through the rows and columns in turn, checking for dominant strategies. If any are found, then the dominated strategy will be removed. An example of how to remove dominated strategies can be found in section 2.3. Once the system is sure that there are no remaining dominated strategies then it will find the solution to the game using the methods described below Games The system will be asked to find optimal mixed strategies of games that are in the form [ a11 a 12 ] a 21 a 22 Games of this nature are solved using methods described in Owen[1982]. If the game has no saddle points then the mixed strategies x = (x 1, x 2 ) and y = (y 1, y 2 ) must have positive components. Therefore the value v is given by a 11 x 1 y 1 + a 12 x 1 y 2 + a 21 x 2 y 1 + a 22 x 2 y 2 = v, (4.1) or x 1 (a 11 y 1 + a 12 y 2 ) + x 2 (a 21 y 1 + a 22 y 2 ) = v. (4.2) The two terms in brackets in equation 4.2 are both less than or equal to v, since y is 27

38 by hypothesis an optimal strategy. Therefore we can write a 11 y 1 + a 12 y 2 < v, (4.3) a 21 y 1 + a 22 y 2 v. (4.4) Since x i > 0 and x 1 + x 2 = 1 the left term in equation 4.2 would be strictly smaller than v. Thus it follows that both of the terms in equation 4.2 must equal v. Hence a 11 y 1 + a 12 y 2 = v, (4.5) a 21 y 1 + a 22 y 2 = v. (4.6) It can also be seen that a 11 x 1 + a 21 x 2 = v, (4.7) a 12 x 1 + a 22 x 2 = v. (4.8) or in matrix form, [ ] Ay T v =, (4.9) v xa = (v, v). (4.10) Where y T is the transpose of y. These equations, along with the equations x 1 + x 2 = 1. (4.11) y 1 + y 2 = 1. (4.12) allow us to solve for x, y and v. In fact, if A is nonsingular, we will have x = vja 1, (4.13) where J is the vector (1, 1). We can now eliminate v by noticing that the sum of the components of x, xj T = 1. This gives us vja 1 J T = 1, (4.14) or and We also find that v = x = 1 JA 1 J T. (4.15) JA 1 JA 1 J T. (4.16) y = A 1 J T JA 1 J T. (4.17) 28

39 In the case that A is singular, this is meaningless. It can be written then that and x = JA JA J T. (4.18) y = A J T JA J T. (4.19) v = A JA J T. (4.20) where A is the adjoint of A and A is the determinant of A. These equations therefore give us the optimal mixed strategies in a 2 2 matrix game n Games 2 n games are those that are in the form [ ] a 11 a a 1n. (4.21) a 21 a a 2n where player I has two strategies and player II has multiple strategies (but more than 2). These games could be solved by the simplex method or a trial and error version of the 2 2 method(williams[1966]), however after discussion with the supervisor it was decided that these games could be solved using graphical methods. This will encourage a deeper understanding of the subject, and give the system another end point. The first player s problem is to maximise v(x) = mina 1j x 1 + a 2j x 2. (4.22) We know that x 1 = 1 x 2. Hence v(x) = min(a 2j a 1j )x 2 + a 1j. (4.23) We can see here that v(x) is the minimum of n linear equations in x 2. These equations can be plotted and the maximum found using graphical methods. The functions all pass through the points (0, a 1j ) and (1, a 2j ) with gradient a 2j a 1j. As mentioned, the value of v can be found by finding the maximum point of the minimums of the n functions. This maximum point will be the interception of two (or 29