Stochastic Coalitional Games with Constant Matrix of Transition Probabilities

Transcription

1 Applied Mathematical Sciences, Vol. 8, 2014, no. 170, HIKARI Ltd, Stochastic Coalitional Games with Constant Matrix of Transition Probabilities Xeniya Grigorieva St.Petersburg State University Faculty of Applied Mathematics and Control Processes University pr. 35, St.Petersburg, , Russia Copyright c 2014 Xeniya Grigorieva. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract The stochastic game Γ under consideration is repetition of the same stage game G which is played on each stage with different coalitional partitions. The transition probabilities over the coalitional structures of stage game depends on the initial stage game G in game Γ. The payoffs in stage games (which is a simultaneous game with a given coalitional structure) are computed as components of the generalized PMS-vector [2], [3]. The total payoff of each player in game Γ is equal to the mathematical expectation of payoffs in different stage games G (mathematical expectation of the components of PMS-vector). The concept of solution for such class of stochastic game is proposed and the existence of this solution is proved. Mathematics Subject Classification: 90Axx Keywords: stochastic games, coalitional partition, Nash equilibrium, Shapley value, PMS-vector 1 Introduction In the papers [1] a class of multistage stochastic games with different coalitional partitions where the transition probability from some coalitional game to another depends from coalitional partition in the initial game and from the

2 8460 Xeniya Grigorieva n-tuple of strategies which realizes in initial game is examined. A new mathematical method for solving stochastic coalitional games, based on constructing Nash equilibrium (NE) in a stochastic game similarly scheme of constructing of absolute NE in a multistage game with perfect information [4], [5], and based on calculation of the generalized PMS-value introduced in [2], [3], for the first time, is proposed along with the proof of the solution existence. In this paper transition probability from some coalitional game to another depends only from coalitional partition in the initial game. So the matrix transition probabilities which is constant during of whole multistage game is form. Remind that coalitional game is a game where players are united in fixed coalitions in order to obtain the maximum possible payoff, and stochastic game is a multistage game with random transitions from state to state, which is played by one or more players. 2 State of problem Suppose finite graph tree Γ = (Z, L), where Z is the set of vertices in the graph and L is point-to-set mapping, defined on the set Z: L (z) Z, z Z. Finite graph tree with the initial vertex z 0 will be denoted by Γ (z 0 ). In each vertex z Z of the graph Γ (z 0 ) simultaneous N-person game is defined in a normal form G (z) = N, X 1,..., X n, K 1,..., K n, where N = {1,..., n} is the set of players identical for all vertices z Z; X j is the set of pure strategies x z j of player j N, identical for all vertices z Z; x z = (x z 1,..., x z n), x z j X j, j = 1, n, is the n-tuple of pure strategies in the game G (z) at vertex z Z; µ z = (µ z 1,..., µ z n), j = 1, n, is n-tuple of mixed strategies in the game G (z) in mixed strategies at vertex z Z; K j (x z ), is the payoff function of the player j identical for all vertices z Z; it is supposed that K j (x z ) 0 x z X and j N. Furthermore, let in each vertex z Z of the graph Γ (z 0 ) the coalitional partition of the set N be defined Σ z = {S 1,..., S l }, l n, S i S j = i j, l S i = N, i=1 i. e. the set of players N is divided into l coalitions each acting as one player. Coalitional partitions can be different for different vertices z. Then in each vertex z Z we have the simultaneous l-person coalitional game in a normal form associating with the game G (z) G (z, Σ z ) = N, X z S 1,..., X z S l, H z S 1,..., H z S l,

3 Stochastic coalitional games with constant matrix of transition probabilities 8461 where X S z i = X j is the set of strategies x z S i of coalition S i, i = 1, l, the strategy x z S i X S z i of coalition S i is n-tuple of strategies of players from coalition S i, i. e. x z S i = { x j X j j S i }; x z = ( x ) z S 1,..., x z S l, x z Si X S z i, i = 1, l, is n-tuple of strategies in the game G (z, Σ z ); µ z = ( µ z 1,..., µ z l ), i = 1, l, is n-tuple of mixed strategies in the game G (z) in mixed strategies at the vertex z Z; however notice that µ z µ z ; the payoff of coalition S i is defined as a sum of payoffs of players from coalition S i, i. e. H z S i ( x z ) = K j (x), i = 1, l. (1) For each vertex z Z of the graph Γ (z 0 ) by matrix of transition probabilities the probabilities p (z, y) of transition to the next vertices y L (z) of the graph Γ (z 0 ) are defined: p (z, y) 0, p (z, y) = 1. Definition 2.1 The game defined on the finite graph tree Γ (z 0 ) with initial vertex z 0 is called the finite step coalitional stochastic game Γ (z 0 ) with constant matrix of transition probabilities: Γ (z 0 ) = N, Γ (z 0 ), {G (z, Σ z )} z Z, {p (z, y)} z Z,, k Γ, where N = {1,..., n} is the set of players identical for all vertices z Z; Γ (z 0 ) is the graph tree with initial vertex z 0 ; {G (z, Σ z )} z Z is the simultaneous coalitional l-person game defined in a normal form in each vertex z Z of the graph Γ (z 0 ); {p (z, y)} z Z,, is the realization probability of the coalitional game G (y, Σ y ) at the vertex y L (z) under condition that the simultaneous game G (z, Σ z ) was realized at the previous step at vertex z; k Γ is the finite and fixed number of steps in the stochastic game Γ (z 0 ); the step k, k = 0, k Γ at the vertex z k Z is defined according to the condition of z k (L (z 0 )) k, i. e. the vertex z k is reached from the vertex z 0 in k stages. States in the multistage stochastic game Γ are vertices of graph tree z Z with the defined coalitional partitions in each vertex Σ z, i. e. pair (z, Σ z ). Game Γ is stochastic, because transition from state (z, Σ z ) to state (y, Σ y ), y L (z), is defined by the given probability p (z, y). Multistage stochastic coalitional game Γ (z 0 ) is realized as follows. At moment t 0 the game Γ (z 0 ) starts at the vertex z 0, where the game G (z 0, Σ z0 ) with a certain coalitional partition Σ z0 is realized. Players choose their strategies, thus n-tuple of strategies x z 0 is formed. Then on the next stage with given

4 8462 Xeniya Grigorieva probabilities p (z 0, z 1 ) the transition from vertex z 0 on the graph tree Γ (z 0 ) to the game G (z 1, Σ z1 ), z 1 L (z 0 ) is realized. In the game G (z 1, Σ z1 ) players choose their strategies again, n-tuple of strategies x z 1 is formed. Then from vertex z 1 L (z 0 ) the transition to the vertex z 2 (L (z 0 )) 2 is made, again n-tuple of strategies x z 2 is formed. This process continues until at the end of the game the vertex z k Γ (L (z 0 )) k Γ, L ( z k Γ) = is reached. Denote by Γ (z) the subgame of game Γ (z 0 ), starting at the vertex z Z of the graph Γ (z 0 ), i. e. at coalitional game G (z, Σ z ). Obviously the subgame Γ (z) is a stochastic game as well. Denote by: u z j ( ) the strategy of player j, j = 1, n, in the subgame Γ (z), which to each vertex y Z assigns the strategy x y j of player j in each simultaneous game G (y, Σ y ) at all vertices y Γ (z), i. e. u z j (y) = { x y j y Γ (z) } ; u z S i ( ) the strategy of coalition S i in the subgame Γ (z), which is a set of strategies u z j ( ), j S i ; u z ( ) = (u z 1 ( ),..., u z n ( )) = ( u z S 1 ( ),..., u z S n ( ) ) the n-tuple in the game Γ (z). It s easy to show that the payoff E z j (u z ( )) of player j, j = 1, n, in any game Γ (z) is defined as the mathematical expectation of payoffs of player j in all its subgames, i. e. by the following formula ([4], p. 158): E z j (u z ( )) = K j (x z ) + [ p (z, y) E y j (u y ( )) ]. (2) Thus, a coalitional stochastic game Γ (z 0 ) with constant matrix of transition probabilities can be written as a game in normal form Γ (z 0 ) = = N, Γ (z 0 ), {G (z, Σ z )} z Z, {p (z, y)} z Z,, { Uj z } j=1,n, { E z j } j=1,n, k Γ where Uj z is the set of the strategies u z j ( ) of the player j, j = 1, n. The payoff HS z i (x z ) of coalition S i Σ z, i = 1, l, in each coalitional game G (z, Σ z ) at the vertex z Z is defined as the sum of payoffs of players from the coalition S i, see formula (1). The payoff HS z i (u z ( )), S i Σ z, i = 1, l, in the subgame Γ (z) of the game Γ (z 0 ) at the vertex z Z is defined as the sum of payoffs of players from,

5 Stochastic coalitional games with constant matrix of transition probabilities 8463 the coalition S i in the subgame Γ (z) at the vertex z Z: HS z i (u z ( )) = Ej z (u z ( )) = K j (x z ) + [ p (z, y) E y j (u y ( )) ] = = K j (x z ) + = H z S i (x z ) + p (z, y) E y j (u y ( )) = [ p (z, y) H y S i (u y ( )) ]. (3) It s clear, that in any vertex z Z under the coalitional partition Σ z the game Γ (z) with payoffs Ej z of players j = 1, n defined by (2), is a noncoalitional game between coalitions with payoffs HS z i (u z ( )) defined by (3). For finite non-coalitional games the existence of the NE ([5], p. 137) in mixed strategies is proved. However, as the payoffs of players j, j = 1, n, are not partitioned from the payoff of coalition in the subgame Γ (z), it may occur at the next step in the subgame Γ (y), y L (z), with another coalitional partition at the vertex y, the choice of player j will be not trivial and will be different from the corresponding choice of equilibrium strategy ū z j ( ) in the subgame Γ (z). 3 Nash Equilibrium in a multistage stochastic game with constant matrix of transition probabilities Remind the algorithm of constructing the generalized PMS-value in a coalitional game. Calculate the values of payoff H z S i (x z ) for all coalitions S i Σ z, i = 1, l, for each coalitional game G (z, Σ z ) by formula (1). In the game G (z, Σ z ) find n-tuple NE x z = ( x z S 1,..., x z S l ) or µ z = ( µ z S 1,..., µ z S l ). In case of l = 1 the problem is the problem of finding the maximal total payoff of players from the coalition S 1, in case of l = 2 it is the problem of finding of NE in bimatrix game, in other cases it is the problem of finding NE n-tuple in a non-coalitional game. In the case of multiple NE [6] the solution of the corresponding coalitional game will be not unique. The payoff of each coalition in NE n-tuple H z S i ( µ z ) is divided according to Shapley s value [7] Sh (S i ) = (Sh (S i : 1),..., Sh (S i : s)): Sh (S i : j) = S S i S j (s 1)! (s s )! s! [v (S ) v (S \ {j})] j = 1, s, (4)

6 8464 Xeniya Grigorieva where s = S i (s = S ) is the number of elements of set S i (S ) and v (S ) is the total maximal guaranteed payoff of subcoalition S S i. We have s v (S i ) = Sh (S i : j). j=1 Then PMS-vector in the NE in mixed strategies in the game G (z, Σ z ) is defined as PMS ( µ z ) = (PMS 1 ( µ z ),..., PMS n ( µ z )), where PMS j ( µ z ) = Sh (S i : j), j S i, i = 1, l. Remark. If the calculation of PMS-vector is difficult, then any other optimal solution can be proposed to be used as a PMS-solution, for example, Pareto-optimality or Nash arbitration scheme [6]. We apply the known algorithm of constructing NE n-tuple in a stochastic coalitional game to the stochastic coalitional game Γ(z 0 ) with constant matrix of transition probabilities [1]. 4 Conclusion In this paper the new algorithm of solving of finite step coalitional stochastic game with constant matrix of transition probabilities is proposed. A mathematical method for solving stochastic coalitional games with constant matrix of transition probabilities is based on calculation of the generalized PMS-value. References [1] X. Grigorieva, Solutions of stochastic coalitional games, Applied Mathematical Sciences, vol. 8, 2014, no. 169, [2] X. Grigorieva, Solutions of Bimatrix Coalitional Games, Applied Mathematical Sciences, vol. 8, 2014, no. 169, [3] L. Petrosjan, S. Mamkina, Dynamic Games with Coalitional Structures, International Game Theory Review, 8(2) (2006), [4] N. Zenkevich, L. Petrosjan, D. Young, Dynamic Games and their applications in management. - SPb.: Graduate School of Management, [5] L. Petrosjan, N. Zenkevich, E. Semina, The Game Theory. - M.: High School, 1998.

7 Stochastic coalitional games with constant matrix of transition probabilities 8465 [6] J. Nash, Non-cooperative Games, Ann. Mathematics 54 (1951), [7] L. S. Shapley, A Value for n-person Games. In: Contributions to the Theory of Games (Kuhn, H. W. and A. W. Tucker, eds.) (1953), Princeton University Press. Received: November 15, 2014; Published: November 27, 2014