Game theory and non-linear dynamics: the Parrondo Paradox case study

Transcription

1 Chaos, Solitons and Fractals 17 (2003) Game theory and non-linear dynamics: the Parrondo Paradox case study P. Arena, S. Fazzino, L. Fortuna *, P. Maniscalco Dipartimento Elettrico, Elettronico e Sistemistico, System and Control Group, Universita degli Studi di Catania, Viale A. Doria 6, Catania, Italy Abstract In this paper a newresearch topic is explored on the role of chaos in a particular game problem: the Parrondo Paradox. In the original formulation of this paradox, it has been proved that two separate losing games can be combined following a random or periodic strategy in order to have a resulting winning game. In this paper, three key points will be dealt with. The first one regards the introduction of a strategy based on various chaotic time series: this could improve the gain in the classical two games Parrondo problem. The second one concerns with the introduction of a third loosing game based on the history of the game and not on the capital as in the classical Parrondo two games Problem. Finally, the Parrondo Paradox has been generalized for N games and an algorithm has been proposed in order to investigate through an optimization approach the region of probability parameter space in which Parrondo Paradox can occur. Ó 2002 Elsevier Science Ltd. All rights reserved. 1. Introduction ParrondoÕs Paradox [1 4] is a newresearch topic in Game Theory devised by Parrondo as a pedagogical illustration of the Brownian ratchet. It occurs when two statistically losing games of chance, say game A and game B, are combined following a random or periodic strategy in order to have a resulting winning game. This is best demonstrated in the original version of ParrondoÕs paradox by tossing coins where the coins are biased towards winning or losing. In particular: game A consists of a biased coin, say coin 0, which has probability p of winning; game B can be described by the following statement. If the present capital is a multiple of M then the chance of winning is p 1, if it is not a multiple of M the chance of winning is p 2. It has been proved [2] that game A results a loosing game when the following condition is met: 1 p > 1 ð1þ p while game B is loosing when: ð1 p 1 Þð1 p 2 Þ M 1 > 1 ð2þ p 1 p2 M 1 * Corresponding author. Tel.: ; fax: address: lfortuna@dees.unict.it (L. Fortuna) /03/$ - see front matter Ó 2002 Elsevier Science Ltd. All rights reserved. PII: S (02)

2 546 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) A resulting game can be constructed by randomly switching between games A and B with probability c. This game is a capital-dependent (CD) game. It was established in [1] that there are choices of p, p 1 and p 2 such that games A and B are both losing, but the resulting game is winning. This behavior has been termed Parrondo s Paradox. By regarding the current capital as the state of a discrete-time Markov chain, it has been shown in [1] that the paradox exists if the following condition is met: ð1 q 1 Þð1 q 2 Þ M 1 < 1 ð3þ q 1 q M 1 2 where q 1 ¼ cp þð1 cþp 1 and q 2 ¼ cp þð1 cþp 2. A convenient parameterization can be introduced if we require to control the three probabilities p, p 1 and p 2 via a biasing parameter e. Thus, we can consider the following transformation: p ¼ p 0 e, p 1 ¼ p1 0 e and p 2 ¼ p2 0 e. The ParrondoÕs original values for these parameters are: p 0 ¼ 1=2, p1 0 ¼ 1=10, p0 2 ¼ 3=4, M ¼ 3 and e ¼ 0:005 [1]. A simulation result using these values is shown in Fig. 1 where it is represented the progress when playing games A and B, as well the effect of switching periodically and randomly between the games. In this simulation, each game was played individually 100 times and the outcomes were averaged over 50,000 trials. In this paper, through a simulation tool, it is investigated howthe introduction of a strategy based on various chaotic time series can improve the final gain in the classical two games ParrondoÕs problem. However, one of the limitations of the applicability of the original game paradox in real world problems such as genetics, evolution and economics [4] is that it involves only two games and one of them is a capital dependent game. Thus, in this paper a third game is defined, based on the recent history of wins and losses and the role of chaos in the resulting game obtained by playing together all the three games is discussed. Finally, facing with the problem of playing N games, an algorithm is presented in order to investigate, through the resolution of a constrained optimization problem, the region of probability parameter space in which the ParrondoÕs Paradox occur. The proposed approach is based on Genetic algorithms (GAs). They represent [5 10] a class of stochastic search optimization algorithms based on the metaphors of natural evolution and they are part of the larger class of Fig. 1. Plot of the effect of playing games A and B individually and of switching between games A and B with ParrondoÕs original parameters.the simulation was performed with e ¼ 0:005 playing game A a times, game B b times and so on until 100 games were played. Each point was obtained as the average over 50,000 trials. The values of a and b are shown by the vectors [a; b].

3 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) evolutionary computation which include evolutionary programming, evolutionary strategy and genetic programming. The basic algorithm was adapted in order to manage the high number of constraints in the N games problem. 2. Complex dynamics and the two games Parrondos problem While in the original ParrondoÕs Paradox, the strategy is based on randomly choosing one of the two games with probability c, in this work the role of a strategy based on various chaotic time series is investigated through simulation results. The chosen strategy consists of switching between the two games A and B following the numeric sequence generated by different chaotic systems. From a time-series point of view, a chaotic system is characterized by signals with a broad band spectrum that strongly depend on the initial conditions. The chaotic time series sequence generators used in our experiments are the Logistic map, the Sinusoidal map and the Henon map. the Logistic map [12] is one of the simplest dynamic systems showing chaotic behavior and its equation is: x kþ1 ¼ ax k ð1 x k Þ ð4þ the Sinusoidal map [13] is defined by: x kþ1 ¼ ax 2 sinðpx k Þ the Henon map [13] is defined by: x kþ1 ¼ y k þ 1 ax 2 k y kþ1 ¼ bx k the Tent map [13] is defined by: x kþ1 ¼ ax k if x k 6 0:5 að1 x k Þ if x k > 0:5 ð5þ ð6þ ð7þ Figs. 2 and 3 showthe simulation results obtained by playing game A and B following a strategy based on the previous described generators with the parameter sets reported respectively in Tables 1 and The Parrondo Paradox with three games A third game is now introduced called game C, described by the following rules: depending on whether in the previous game history a win or loss took place, one of the two coins to toss is chosen (see Table 3). Thus, this game can be considered as history dependent (HD) because the choice of the next coin to play depends on the results of the previous game. It is assumed that Coin 3 has winning probability p 3 and Coin 4 has winning probability p 4 and X ðtþ, t ¼ 0; 1; 2;... indicate the capital of player. The variable Y ðtþ is defined as follows: Y ðtþ ¼X ðtþ X ðt 1Þ ð8þ which can take on two values (þ1,)1) and forms the state of a Markov chain. The transition probabilities can be easily obtained from the rules of game C. Let p 1 ðtþ and p 1 ðtþ be the probability that Y ðtþ is 1 and þ1, respectively. The probability distribution vector pðtþ verifies the evolution equation pðt þ 1Þ ¼ApðtÞ, where A is the transition probabilities matrix: A ¼ 1 p 3 p 3 ð9þ 1 p 4 p 4 The stationary distribution p ¼ðp 1 p 1 Þ of this Markov chain must satisfy the following equations: p ¼ pa p 1 þ p 1 ¼ 1 ð10þ

4 548 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) Fig. 2. Plot of the effect of playing A and B individually and of switching between games A and B randomly and using various chaotic generators with settings reported in Table 1. Resolving the system Eq. (10) the following solution is obtained p 1 ¼ 1 p 4 1 p 3 p 4 ð11þ p 3 p 1 ¼ ð12þ 1 p 3 p 4 In the stationary regime, the probability that game C is winning is P C;win ¼ p 3 p 1 þ p 4 p 1, that is: p 3 P C;win ¼ ð13þ 1 p 3 p 4 This implies that game C is a losing game when P C;win < 1=2 and so when p 3 < 1 p 4 ð14þ The original ParrondoÕs games A and B can be combined with the game C in order to obtain a resulting game. This game is defined by the value of the following parameters: p, p 1, p 2, p 3, p 4 and M. The conditions expressed by Eq. (3) are no longer valid in order to verify the existence of the ParrondoÕs Paradox in this three games problem. However, it has been possible to find heuristically a set of parameters for which the ParrondoÕs Paradox is yet verified. Fig. 4 shows the simulation results obtained using the parameters of Table 4. Again, as in the previous case, the introduction of a strategy based on various chaotic time series improves the final gain of the capital. 4. Formulation of N games problem A key observation about ParrondoÕs games A and B (with M ¼ 3) is that they depend on the three parameters p, p 1 and p 2. Considering also the game C, two other parameters p 3 and p 4 are introduced. Thus, to demonstrate the existence

5 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) Fig. 3. Plot of the effect of playing A and B individually and of switching between games A and B randomly and using various chaotic generators with settings reported in Table 2. Table 1 Parameter set 1 used in chaotic generators Iterator Parameters Initial conditions Logistic a ¼ 3:74 x 0 ¼ 0:1 Sinusoidal a ¼ 2:27 x 0 ¼ 0:7 Henon a ¼ 1:4, b ¼ 0:3 x 0 ¼ 0, y 0 ¼ 0 Table 2 Parameter set 2 used in chaotic generators Iterator Parameters Initial conditions Logistic a ¼ 4:0 x 0 ¼ 0:1 Sinusoidal a ¼ 2:3 x 0 ¼ 0:7 Henon a ¼ 1:32, b ¼ 0:3 x 0 ¼ 0, y 0 ¼ 0 Table 3 Rules of game C Game n 1 Coin played Loss Coin 3 Win Coin 4

6 550 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) Fig. 4. Plot of the effect of playing games A, B and C individually and of switching between them randomly and playing game A a times, game B b times and game C c times. Each point was obtained as the average over 50,000 trials. The simulation was performed by adopting the parameters of Table 4. Table 4 Parameters for three games problem Game Parameters A p ¼ 0:495 B p 1 ¼ 0:095, p 2 ¼ 0:745, M ¼ 3 C p 3 ¼ 0:376, p 4 ¼ 0:620 of ParrondoÕs Paradox, a particular set of these parameters must be found inside the region defined by Eqs. (1), (2) and (14), as for hypothesis each game has to be loosing. The above observations suggest a generalization. In fact, three sets A, B and C representing the set of loosing games with the same rules, respectively, of game A, B and C can be considered. All games belonging to one set followthe same rules but are characterized by different values of probabilities. In particular, a system of n games can be defined: a 1 ; a 2 ;...; a a 2 A b 1 ; b 2 ;...; b b 2 B c 1 ; c 2 ;...; c c 2 C a þ b þ c ¼ n ð15þ For each game a i, b i and c i, let p ai, p bi and p ci be their vectors of probabilities, respectively. These vectors should verify the constraint equations (1), (2) and (14) and if V gi < 0 denotes the constrain equation for a generic game g i, all the constraints can be expressed with the system

7 8 V a1 ðp a1 Þ < 0 V a2 ðp a2 Þ < 0 ><... V b1 ðp b1 Þ < 0... >: V c1 ðp c1 Þ < 0 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) ð16þ The resulting game is obtained by randomly switching between them. The n games are mapped one-to-one into n contiguous intervals in the range [0,1] of equal sizes. To select a game, a random number f k is generated in the interval [0,1] and the game whose segment spans the random number is selected. The resulting game is defined as winning if the capital hx ðtþi, t ¼ 0; 1; 2;...; N is an increasing function of t and the final capital hx ðnþi is greater than hx ð0þi. Thus, if it exists, the set of parameters such that the resulting game is winning must be a solution of the non-linear optimization problem: 8 maxhx ðnþi V a1 ðp a1 Þ < 0 >< V a2 ðp a2 Þ < 0... ð17þ V b1 ðp b1 Þ < 0... >: V c1 ðp c1 Þ < 0 It is a challenging optimization problem, due to the high number of variables and the existence of the solution is not known a priori. So, an approach based on GAs has been proposed in this article in order to investigate the solutions of Eq. (17) Genetic algorithm Our simulations were performed using steady state genetic algorithms [11]. This genetic algorithm uses overlapping populations. Each generation the algorithm creates a temporary population of individuals, adds these ones to the previous population, then removes the worst individuals in order to return the population to its original size Coding of chromosomes Each individual of the population, called chromosome, represents a solution and is coded as a sequence of bits. One chromosome is composed by m phenotypes of 16 bits, representing the m probabilities that define a solution. As it is a priori known that some coins are winning and other ones are loosing, these probabilities are respectively limited in the range ½0; 0:5Š or ½0:5; 1Š Fitness function The objective function is calculated from q N, the final capital hx ðtþi at t ¼ N and calculated as the mean value over T trials. In order to ensure that q N is a positive number, a weight C is introduced in the fitness function. Then, the fitness function is: f ¼ q N if q N > 0 ð18þ Cq N if q N < Constraints and genetic operators In order to take into account the constraints expressed in Eq. (16), the standard initial population generation, crossover and mutation algorithms have been changed in such a way that these operators produce feasible solutions.

8 552 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) Table 5 Results for six games Game Parameter Value a 1 p b 1 p p c 1 p p a 2 p b 2 p p c 2 p p Table 6 Six game problem: final gain after 100 iterations Generator and parameters Capital Random Logistic map (a ¼ 4:0, x 0 ¼ 0:27) Tent map (a ¼ 1:9, x 0 ¼ 0:1) Henon map (a ¼ 1:4, b ¼ 0:3, x 0 ¼ 0:1, y 0 ¼ 0:1) Table 7 Results for nine games Game Parameter Value a 1 p b 1 p p c 1 p p a 2 p b 2 p p c 2 p p a 3 p b 3 p p c 3 p p Table 8 Nine games problem: final gain after 100 iterations Generator and parameters Capital Random Logistic map (a ¼ 4:0, x 0 ¼ 0:27) Tent map (a ¼ 1:9, x 0 ¼ 0:1) Henon map (a ¼ 1:4, b ¼ 0:3, x 0 ¼ 0:1, y 0 ¼ 0:1)

9 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) Table 9 Results for twelve games Game Parameter Value a 1 p b 1 p p c 1 p p a 2 p b 2 p p c 2 p p a 3 p b 3 p p c 3 p p a 4 p b 4 p p c 4 p p Table 10 Twelve games problem: final gain after 100 iterations Generator and parameters Capital Random Logistic map (a ¼ 4:0, x 0 ¼ 0:27) Tent map (a ¼ 1:9, x 0 ¼ 0:1) Henon map (a ¼ 1:4, b ¼ 0:3, x 0 ¼ 0:1, y 0 ¼ 0:1) Numerical results The proposed algorithm to solve some test problems was implemented and their results were compared. In particular, problems with six, nine and twelve games were considered. For each problem, the set of probabilities found by the genetic algorithm are reported (Tables 5, 7 and 9). Even in these problems, the resulting game was simulated using various chaotic generators: the corresponding results are reported in Tables 6, 8 and 10. Besides, Figs. 5 7 graphically showhowgain changes across iterations in various cases. 5. Conclusions In this work a novel application of chaotic maps is explored in the strategy of ParrondoÕs two games problem: through simulation the benefits of this approach compared to the strategy proposed in literature were verified. This approach has been applied to a three game problem and to a more general N game problem. In fact, a newfamily of games was introduced, based upon the ParrondoÕs strategy of alternating non-winning games to generate a winning result. The definition of the parameters for this generalized game has been studied as a non-linear optimization problem solved through a genetic algorithm. Even in these cases the initial conjecture about the application of chaotic dynamics in this problem of game theory was verified. This approach could be applied for the study of social and biological models inspired to the ParrondoÕs games.

10 554 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) Fig. 5. Plot of the effect of playing the six games of Table 5 individually and of switching between them with random and chaotic strategy. Each game was played 100 times and each point was obtained as the average over 50,000 trials. Fig. 6. Plot of the effect of playing nine games of Table 7 individually and of switching between them with random and chaotic strategy. Each game was played 100 times and each point was obtained as the average over 50,000 trials.

11 P. Arena et al. / Chaos, Solitons and Fractals 17 (2003) Fig. 7. Plot of the effect of playing twelve games of Table 9 individually and of switching between them with random and chaotic strategy. Each game was played 100 times and each point was obtained as the average over 50,000 trials. Acknowledgement This work has been partially supported by the Italian Ministero dellõistruzione, dellõuniversita e della Ricerca (MIUR) under the Firb project RBNE01CW3M. References [1] Harmer GP, Abbott D. ParrondoÕs Paradox. Statistical Science 1999;14: [2] Harmer GP, Abbott D, Taylor PG, Parrondo JM. ParrondoÕs paradoxical games and the discrete Brownian ratchet, Proc. UponÕ99, University of Adelaide, July , p [3] Harmer GP, Abbott D, Taylor PG, Pearce CE, Parrondo JM. Information entropy and ParrondoÕs discrete-time ratchet stochastic and chaotic dynamics in the lakes (stochaos). USA: American Institute of Physics; p. 502, [4] Harmer GP, Abbott D. ParrondoÕs paradox: losing strategies cooperate to win. Nature 1999;402:864. [5] Holland JH. Adaptation in natural and artificial system. Ann Arbor, MI: University of Michigan Press; [6] Goldberg DE. Genetic algorithm in search optimization and machine learning. Addison Wesley; [7] Michalewicz Z. Genetic algorithm þ data structure ¼ evolution program. Springer Verlag; [8] Vose MD. The simple genetic algorithm. Foundation and theory. MIT press; [9] Gen M, Cheng R. Genetic algorithms and engineering optimization. Wiley; [10] Zalzala M, Fleming P, editors. Genetic algorithm in engineering systems. IEE Control Engineering Series, [11] Whitley D. The GENITOR algorithm and selection pressure: why rank-based allocations of reproductive trials is best, Proc. 3rd Int. Conf. on Genetic Algorithms. 1989, p [12] Parker TS, Chua LO. Practical numerical algorithms for chaotic system. Springer Verlag; [13] Peitgen H, Jurgens H, Saupe D. Chaos and fractals. Springer-Verlag; 1992.