Solutions of Stochastic Coalitional Games

Transcription

1 Applied Mathematical Sciences, Vol. 8, 2014, no. 169, HIKARI Ltd, Solutions of Stochastic Coalitional Games Xeniya Grigorieva St.Petersburg State University Faculty of Applied Mathematics and Control Processes University pr. 35, St.Petersburg, , Russia Copyright c 2014 Xeniya Grigorieva. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract The stochastic game Γ under consideration is repetition of the same stage game G which is played on each stage with different coalitional partitions. The probability distribution over the coalitional structures of each stage game depends on the initial stage game G and the n-tuple of strategies realized in this game. The payoffs in stage games (which is a simultaneous game with a given coalitional structure) are computed as components of the generalized PMS-vector [1], [2]. The total payoff of each player in game Γ is equal to the mathematical expectation of payoffs in different stage games G (mathematical expectation of the components of PMS-vector). The concept of solution for such class of stochastic game is proposed and the existence of this solution is proved. Mathematics Subject Classification: 91Axx Keywords: stochastic games, coalitional partition, Nash equilibrium, Shapley value, PMS-vector 1 Introduction This paper belongs to the well investigated direction in management theory - game theory which deals with existence and finding optimal solution problems of management in a collision of parties, when each party tries to influence development of the conflict in accordance with its own interest. Problems that arise in different practical spheres are solved according to the game theory

2 8444 Xeniya Grigorieva models. In particular, these are the problems of management, economics, decision theory. In the paper a class of multistage stochastic games with different coalitional partitions is examined. A new mathematical method for solving stochastic coalitional games, based on calculation of the generalized PMS-value introduced in [1], [2] for the first time, is proposed along with the proof of the solution existence. The probability distribution over the coalitional structures of each stage game depends on the initial stage coalitional game and the n- tuple of strategies realized in this game. The theory is illustrated by 3-person 3-stage stochastic game with changing coalitional structure. Remind that coalitional game is a game where players are united in fixed coalitions in order to obtain the maximum possible payoff, and stochastic game is a multistage game with random transitions from state to state, which is played by one or more players. 2 Statement of the problem Suppose finite graph tree Γ = (Z, L) where Z is the set of vertices in the graph and L is point-to-set mapping, defined on the set Z: L (z) Z, z Z. Finite graph tree with the initial vertex z 0 will be denoted by Γ (z 0 ). In each vertex z Z of the graph Γ (z 0 ) simultaneous N-person game is defined in a normal form G (z) = N, X 1,..., X n, K 1,..., K n, (1) where N = {1,..., n} is the set of players identical for all vertices z Z; X j = { } x z j x z j = k, k = 1, m j is the set of pure strategies of player j N identical for all vertices z Z; m j is the number of pure strategies of player j N in the set X j ; x z j is the pure strategy of player j N at the vertex z Z; µ z j = { µ j } k Σ j is the mixed strategy of player j N at the k=1,m j vertex z Z where µ j k is a choice probability for the j-th player to pick k-th pure strategy: µ j k 0, j = 1, n, k = 1, m m j j, µ j k = 1; Σ j is the set of all mixed strategies of j-th player; x z = (x z 1,..., x z n) X, x z j X j, j = 1, n, is the n-tuple of pure strategies in the game G (z); X = X i is the set of n-tuples identical for all vertices z Z; i=1, n µ z = (µ z 1,..., µ z n) Σ, µ z j Σ j, j = 1, n, is the n-tuple of mixed strategies in the game G (z) in the mixed strategies at the vertex z Z; k=1

3 Solutions of stochastic coalitional games 8445 Σ = j=1, n Σ j is the set of n-tuple in the mixed strategies identical for all vertices z Z; K j (x z ), x z X, is the payoff function of the player j, identical for all vertices z Z ; it is proposed that K j (x z ) 0 x z X and j N. Furthermore, let in each vertex z Z of the graph Γ (z 0 ) the coalitional partition of the set N be defined l Σ z = {S 1,..., S l }, l n, S i S j = i j S i = N, i=1 i. e. the set of players N is divided into l coalitions each acting as one player. Coalitional partitions can be different for different vertices z, i. e. l = l (z). Then in each vertex z Z we have the simultaneous l-person coalitional game in a normal form associating with game G (z) G (z, Σ z ) = N, X z S 1,..., X z S l, H z S 1,..., H z S l, (2) where X S z i = X j is the set of strategies x z S i of coalition S i, i = 1, l, where the strategy x z S i X S z i of coalition S i is n-tuple strategies of players from coalition S i, i. e. x z S i = { x j X j j S i }; x z = ( x ) z S 1,..., x z S l Xz, x z S i X S z i, i = 1, l, is n-tuple of strategies in the game G (z, Σ z ); X z = X S z i is the set of n-tuples in the simultaneous game G (z, Σ z ); i=1, l µ z i is the mixed strategy of coalition S i, i = 1, l, at the vertex z Z; Σ z i is the set of all mixed strategies of coalition S i, i = 1, l, at the vertex z Z; µ z = ( µ z 1,..., µ z l ) Σ z, µ z i Σ z i, i = 1, l, is the n-tuple of mixed strategies in the game G (z) in the mixed strategies at the vertex z Z; Σ z = is the set of n-tuple in the mixed strategies at the vertex i=1, l Σ z i z Z; the payoff of coalition S i is defined as a sum of payoffs of players from coalition S i, i. e. ( x z ) = K j (x), i = 1, l, where x z = (x z 1,..., x z n) and x z = ( x z S 1,..., x z S l ) are the same n-tuples in the games G (z) and G (z, Σ z ) correspondingly such that for each component x z j, j = 1, n, from the n-tuple x z it follows that the component x z, j S i, is included in the strategy x z S i from the n-tuple x z.

4 8446 Xeniya Grigorieva Denote by only x z the n-tuple in the games G (z) and G (z, Σ z ). However it does not lead to µ z = µ z. After that for each vertex z Z of the graph Γ (z 0 ) the transition probabilities p (z, y ; x z ) to the next vertices y L (z) of the graph Γ (z 0 ) are defined. The probabilities p (z, y ; x z ) depend on n-tuple of strategies x z realized in the game G (z, Σ z ) with fixed coalitional partition: p (z, y ; x z ) 0, p (z, y ; x z ) = 1. Definition 2.1 The game defined on the finite graph tree Γ (z 0 ) with initial vertex z 0 will be called the finite step coalitional stochastic game Γ (z 0 ): Γ (z 0 ) = N, Γ (z 0 ), {G (z, Σ z )} z Z, {p (z, y ; x z )} z Z,, x z X, k Γ, Z (3) where N = {1,..., n} is the set of players identical for all vertices z Z; Γ (z 0 ) is the graph tree with initial vertex z 0 ; {G (z, Σ z )} z Z is the simultaneous coalitional l-person game defined in a normal form in each vertex z Z of the graph Γ (z 0 ) ; {p (z, y; x z )} z Z,, x z X is the realization probability of the coalitional game G (y, Σ y ) at the vertex y L (z) under condition that n-tuple x z Z was realized at the previous step in the simultaneous game G (z, Σ z ); k Γ is the finite and fixed number of steps in the stochastic game Γ (z 0 ); the k-th step in the vertex z k Z is defined according to the condition of z k (L (z 0 )) k, i. e. the vertex z k is reached from the vertex z 0 in k steps. States in the multistage stochastic game Γ are the vertices of graph tree z Z with the defined coalitional partitions Σ z in each vertex, i. e. pair (z, Σ z ). Game Γ is stochastic, because transition from state (z, Σ z ) to the state (y, Σ y ), y L (z), is defined by the given probability p (z, y ; x z ). Game Γ (z 0 ) is realized as follows. Game Γ (z 0 ) starts at the vertex z 0, where the game G (z 0, Σ z0 ) with a certain coalitional partition Σ z0 is realized. Players choose their strategies, thus n-tuple of game x z 0 is formed. Then with given probabilities p (z 0, z 1 ; x z 0 ) depending on n-tuple x z 0 the transition from vertex z 0 on the graph tree Γ (z 0 ) to the game G (z 1, Σ z1 ), z 1 L (z 0 ), is realized. In the game G (z 1, Σ z1 ) players choose their strategies again, n- tuple of game x z 1 is formed. Then from vertex z 1 L (z 0 ) the transition to the vertex z 2 (L (z 0 )) 2 is made, again n-tuple game x z 2 is formed. This process continues until vertex z k Γ (L (z 0 )) k Γ, L ( z k Γ) = is reached. Denote by Γ (z) the subgame of game Γ (z 0 ), starting at the vertex z Z of the graph Γ (z 0 ), i. e. at coalitional game G (z, Σ z ). Obviously the subgame Γ (z) is also a stochastic game.

5 Solutions of stochastic coalitional games 8447 Denote by: u z j ( ) is the strategy of player j, j = 1, n, in the subgame Γ (z) which to each vertex y Z assigns the strategy x y j of player j in each simultaneous game G (y, Σ y ) at all vertices y Γ (z) i. e. u z j ( ) = { x y j y Γ (z) } ; u z S i ( ) is the strategy of coalition S i in the subgame Γ (z) which is a set of strategies u z j ( ), j S i ; u z ( ) = (u z 1 ( ),..., u z n ( )) = ( u z S 1 ( ),..., u z S n ( ) ) is the n-tuple in the game Γ (z). It s easy to show that the payoff E z j (u z ( )) of player j, j = 1, n, in any game Γ (z) is defined by the mathematical expectation of payoffs of player j in all its subgames, i. e. by the following formula ([3], p. 158): E z j (u z ( )) = K j (x z ) + [ p (z, y ; x z ) E y j (u y ( )) ]. (4) Thus, a coalitional stochastic game Γ (z 0 ) can be written as a game in normal form Γ (z 0 ) = N, Γ (z 0 ), {G (z, Σ z )} z Z, {p (z, y; x z )} z Z,, { Uj z, { E z j=1,n j, j=1,n k Γ, (5) where Uj z is the set of the strategies u z j ( ) of the player j, j = 1, n. The payoff HS z i (x z ) of coalition S i Σ z, i = 1, l, in each coalitional game G (z, Σ z ) of game Γ (z 0 ) at the vertex z Z in all n-tuple x z is defined as the sum of payoffs of players from the coalition S i : (x z ) = } K j (x z ). (6) The payoff HS z i (u z ( )), S i Σ z, i = 1, l, in the subgame Γ (z) of the game Γ (z 0 ) at the vertex z Z is defined as the sum of payoffs of players from the coalition S i in the subgame Γ (z) at the vertex z Z: (u z ( )) = E z j (u z ( )) = K j (x z ) + } [ p (z, y ; x z ) E y j (u y ( )) ]. It s clearly, that in any vertex z Z under the coalitional partition Σ z the game Γ (z) with payoffs Ej z of players j = 1, n defined by (4), is a noncoalitional game between coalitions with payoffs HS z i (u z ( )) defined by (7). For (7)

6 8448 Xeniya Grigorieva non-coalitional games the existence of the NE ([4], p. 137) in mixed strategies is proved. Remind that the Nash equilibrium (NE) is n-tuple ū z ( ): (ū z ( )) (ūz ( ) u z S i ( ) ) u z S i ( ) U z S i, S i Σ z, i = 1, l, where US z i is the set of the strategies u z S i ( ) of coalition S i Σ z, i = 1, l, and (ūz ( ) u z S i ( ) ) means that the coalition S i deviates from the n-tuple ū z ( ) choosing a strategy u z S i ( ) instead of strategies ū z S i ( ) ū z ( ). However, as the payoffs of players j, j = 1, n, are not selected from the payoff of coalition in the subgame Γ (z), it may occur at the next step in the subgame Γ (y), y L (z), with another coalitional partition at the vertex y, the choice of player j is not trivial and is different from the corresponding choice of entering into an equilibrium strategy ū z j ( ) in the subgame Γ (z). So, solving the coalitional stochastic subgame Γ (z) means forming the NE ū z ( ) in the subgame Γ (z) taking into account the presence of coalition structures in the subgames included in the subgame Γ (z) in particular, by calculating the PMS-vector of payoffs of players in all subgames included in the subgame Γ (z). Formulate follow problem: it s required to solve coalitional stochastic game Γ (z 0 ), i. e. to form n-tuple of NE ū z ( ) in the game Γ (z 0 ) by using the generalized PMS-vector as the optimal solution of coalitional games. 3 Nash Equilibrium in a multistage stochastic game Remind the algorithm of constructing the generalized PMS-value in a coalitional game. Calculate the values of payoff (x z ) for all coalitions S i Σ z, i = 1, l, for each coalitional game G (z, Σ z ) by formula (6): (x z ) = K j (x z ). In the game G (z, Σ z ) find n-tuple NE x z = ( x z S 1,..., x z S l ) or µ z = ( µ z S 1,..., µ z S l ). In case of l = 1 the problem is the problem of finding the maximal total payoff of players from the coalition S 1, in case of l = 2 it is the problem of finding of NE in bimatrix game, in other cases it is the problem of finding NE n-tuple in a non-coalitional game. In the case of multiple NE [5] the solution of the corresponding coalitional game will be not unique. The payoff of each coalition in NE n-tuple ( µ z ) is divided according to

7 Solutions of stochastic coalitional games 8449 Shapley s value [6] Sh (S i ) = (Sh (S i : 1),..., Sh (S i : s)): Sh (S i : j) = S S i S j (s 1)! (s s )! s! [w (S ) w (S \ {j})] j = 1, s, (8) where s = S i (s = S ) is the number of elements of set S i (S ) and v (S ) is the total maximal guaranteed payoff of subcoalition S S i. We have s v (S i ) = Sh (S i : j). j=1 Then PMS-vector in the NE in mixed strategies in the game G (z, Σ z ) is defined as PMS ( µ z ) = (PMS 1 ( µ z ),..., PMS n ( µ z )), where PMS j ( µ z ) = Sh (S i : j), j S i, i = 1, l. We proceed to the construction of solutions in the game Γ (z 0 ). Step 1. Calculate PMS-vector in NE in the mixed strategies for all coalitions S i Σ z, i = 1, l, of each coalitional game G (z, Σ z ), L (z) = : PMS (z) = (PMS 1 (z),..., PMS n (z)), where PMS (z) := PMS ( µ z ) and PMS j (z) := PMS j ( µ z ) are PMS-vector and components of PMS-vector accordingly in one step coalitional game G (z, Σ z ), L (z) =. Step 2. Consider from the end of game Γ (z 0 ) all possible two stage subgames Γ (z), y L (z), L (y) =, with payoffs of coalitions (u z ( )) = K j (x z ) + [p (z, y ; x z ) PMS j (y)] S i Σ z, i = 1, l. Find the NE x z or µ z and ū z ( ). Calculate PMS-vector in NE for all coalitions S i Σ z, i = 1, l, of each coalitional stochastic subgame Γ (z), y L (z), L (y) = : PMS (z) = ( PMS 1 (z),..., PMS n (z) ), where PMS (z) := PMS (ū z ( )) and PMS j (z) := PMS j (ū z ( )) are PMSvector and its components accordingly in the coalitional stochastic game Γ (z), L (z).

8 8450 Xeniya Grigorieva Step k. Define the operator PMS as PMS-vector, which for each player j = 1, n in any coalitional game G (z, Σ z ) of subgame Γ (z), y [L (z)] k 1, L (y) =, correspondingly PMS-components in the NE ū z ( ) : PMS j (z ) = PMS K j (x z ) + z L(z) [ p (z, z ; x z ) PMS j (z ) ] where PMS j (z ) := PMS j (ū z ( )), PMS j (z ) := PMS j (ūz ( ) ) etc. 4 Conclusion, j = 1, n, In this paper the new algorithm of solving of finite step coalitional stochastic game is proposed. A mathematical method for solving stochastic coalitional games is based on calculation of the generalized PMS-value. (9) References [1] X. Grigorieva, Solutions of Bimatrix Coalitional Games, Applied Mathematical Sciences, vol. 8, 2014, no. 169, [2] L. Petrosjan, S. Mamkina, Dynamic Games with Coalitional Structures, International Game Theory Review, 8(2) (2006), [3] N. Zenkevich, L. Petrosjan, D. Young, Dynamic Games and their applications in management. - SPb.: Graduate School of Management, [4] L. Petrosjan, N. Zenkevich, E. Semina, The Game Theory. - M.: High School, [5] J. Nash, Non-cooperative Games, Ann. Mathematics 54 (1951), [6] L. S. Shapley, A Value for n-person Games. In: Contributions to the Theory of Games (Kuhn, H. W. and A. W. Tucker, eds.) (1953), Princeton University Press. Received: November 10, 2014; Published: November 27, 2014