DECISION SCIENCES INSTITUTE. Exponentially Derived Antithetic Random Numbers. (Full paper submission)

Transcription

1 DECISION SCIENCES INSTITUTE (Full paper submission) Dennis Ridley, Ph.D. SBI, Florida A&M University and Scientific Computing, Florida State University Pierre Ngnepieba, Ph.D. Department of Mathematics, Florida A&M University, Tallahassee, Florida. ABSTRACT Serial correlation in pseudo random numbers that are used in Monte Carlo simulations can create hidden errors in the response variable. The estimate of the response variable becomes biased, does not converge to the true value, and its variance is larger than the true value. A pair of antithetic exponential random numbers is created by power transformation of uniformly distributed random [0,1] numbers. This pair of antithetic numbers is used in two separate computer simulation experiments and response variables from the simulations combined. The combined response has bias and variance that are smaller than those of the individual responses. KEYWORDS: Exponential antithetic random variates, Simulation model bias, Variance reduction, Bias reduction, Inverse correlation.

2 DECISION SCIENCES INSTITUTE (Full paper submission) Dennis Ridley, Ph.D. SBI, Florida A&M University and Scientific Computing, Florida State University Pierre Ngnepieba, Ph.D. Department of Mathematics, Florida A&M University, Tallahassee, Florida. ABSTRACT Serial correlation in pseudo random numbers that are used in Monte Carlo simulations can create hidden errors in the response variable. The estimate of the response variable becomes biased, does not converge to the true value, and its variance is larger than the true value. A pair of antithetic exponential random numbers is created by power transformation of uniformly distributed random [0,1] numbers. This pair of antithetic numbers is used in two separate computer simulation experiments and response variables from the simulations combined. The combined response has bias and variance that are smaller than those of the individual responses. KEYWORDS: Exponential antithetic random variates, Simulation model bias, Variance reduction, Bias reduction, Inverse correlation. INTRODUCTION Background Hammersley & Morton (1956) suggested a very simple method for obtaining inversely correlated random numbers for use in Monte Carlo computer simulation experiments. A computer simulation experiment is conducted with a uniformly distributed random number r ~ U(0,1). The simulation is repeated with the inversely correlated complementary random number 1-r. Finally, the response variables from the two simulations are averaged. This can result in a variance in the response variable that is smaller than that for either simulation. However, there is no guarantee that the variance will decrease. In practice the variance can actually increase. See also Kleijnen (1975) Antithetic random variables by power transformation was introduced by Ridley(1999). The Ridley (1999) antithetic time series theorem states that if XX tt > 0, tt = 1,2,3, is a discrete realization of a lognormal stochastic process, such that ln XX tt ~NN(µ, σσ), then if the correlation between XX tt and XX tt pp is ρρ XXXX pp, then lim pp 0,σσ 0 ρρ XXXX pp = 1. This resulted in large reduction in time series model fitted and forecast values. See also Ridley (1995, 1997, 2013 & 2014) for applications in time series analysis and forecasting. In this paper we are interested in applications in computer simulation where exponential random variates are called for.

3 Proposed research. The common practice in computer simulation is to generate uniformly distributed pseudo random numbers on the interval from 0 to 1. These rr~u(0,1) numbers are then used to generate random variates from the cumulative distribution that correspond to the actual random variable under study. The original random numbers and therefore the random variates are assumed to be identically and independently distributed. In practice however, the random numbers are generated from a mathematical formula. One simple method is the linear congruential generator. Random numbers generated from a mathematical formula have the advantage that very large sets can be reproduced for repetition and comparison of the outcomes from different computer simulations under different conditions. However, if the random numbers contain any traces of serial correlations, they can introduce hidden errors into the response variable. For an example see Ferrenberg, Landau & Wong (1992) where so called good random numbers were used and inaccurate results were obtained, but where better results were obtained using simple methods. The objectives of this research are two-fold. One objective is to correct for the bad effects whether they are caused by good or bad random numbers. The other objective is to reduce the variance of the estimate of the response function. This is particularly important when very long simulations are required. Some physics experiments utilize as much as in length. Variance reduction can reduce the length that is necessary for the same level of accuracy. The paper is organized as follows. In section 2 we investigate the correlation between rr and rr pp as p 0. In section 3 we investigate the correlation between rr pp and rr pp as p 0. In section 4 we investigate the distributions of rr, rr pp and rr pp. In section 5 we carry out computer simulation experiments using exponential antithetic random variates and compare the results with those obtained by traditional random numbers. Section 6 contains conclusions and suggestions for future research. CORRELATION BETWEEN rr AND rr pp vs p. First we consider the correlation between rr and rr pp. Correlations for different values of p as p approaches zero from the left are listed in Table 1 and drawn in Figure 1. The numbers were generated with Matlab (2008) and calculated as the average of 1000 samples of 500 numbers each. The correlation is seen to approach recurring, or - 3/2. Therefore rr and rr pp are antithetic in the sense that they are negatively correlated. Table 1: Correlation between rr and rr pp vs p p Corr(r,rr pp ) Figure 1: Correlation between rr and rr pp vs p

4 CORRELATION BETWEEN rr pp AND rr pp vs p. Next, we consider the correlation between rr pp and rr pp. Correlations for different values of p as p approaches zero from the left are listed in Table 2 and drawn in Figure 2. In this case, the sample correlations converge very rapidly in just one sample of 100 random numbers. The correlation is seen to approach -1. Therefore, rr pp and rr pp, are good candidates for use as antithetic random variates. Table 2: Correlation between rr pp and rr pp vs p p Corr(rr pp,rr pp ) Figure 2: Correlation between rr pp and rr pp vs p DISTRIBUTION r, rr pp AND rr pp. Before we can proceed, we need to understand the distributional characteristics of rr, rr pp and rr pp. The histograms for rr, rr pp and rr pp are given in Figure 3. The histograms are based on 300 random numbers rr~u(0,1). As illustrated, in the limit as pp 0, the distributions of rr pp and rr pp are similar to negative and positive exponential distributions, respectively. Figure 3: Distributional characteristics pp 0 a. Uniform rr~u(0,1) b. rr pp c. rr pp

5 Source of serial correlation Any sequence of random numbers that in its totality is serially perfectly uncorrelated, can contain what appears to be subsets of correlated sequences. The smallest trace of serial correlation in any such subset can create a trend that biases the random numbers to the high or low side of the true value. Even if an opposite trend develops subsequently, a very large number of random numbers can occur before the net bias is corrected. Either way, apparent subsets of traces of serial correlation introduce unreliability into the simulation. Positively Correlated Random Numbers Approach Consider the regression of rr pp on rr pp rr pp = ff(rr pp ) + εε = αα + ββrr pp + εε. Since lim pp 0 Corr(rrpp, rr pp ) = 1, that is rr pp and rr pp are perfectly correlated, it follows that lim pp 0 rrpp = αα + ββrr pp + εε wwheeeeee εε 0. That is, rr pp αα + ββrr pp. Let the fitted values of rr pp be denoted by rr pp, let αα and ββ denote least squares estimates and let s denote standard deviation. Then, rr pp = αα + ββ rr pp = rr pp + Corr(rr pp, rr pp )(ss rr pp/ss rr pp)(rr pp ). rr pp This random variable is negatively correlated with rr. However, unlike rr, the probability pp pp distribution of rr is not uniform. For purpose of comparison and uniformity, rr can be converted back to the original scale of and distribution of rr by, rr = (rr pp ) pp. Based on this formulation of the pair of random numbers rr and rr, p=-0.01, the combined response from two separate computer simulations based on rr and rr was calculated. As it turns out, with rr and rr positively correlated, the result was two identical values for the response variables and no gain from combining. Antithetic Random Numbers Random Variates Approach Consider the negatively correlated random number random variate pair rr and rr pp. This suggest the combination of responses from two computer simulations based on rr and rr pp. Consider the simple queueing system shown in Figure 4. The time between arrivals are exponentially distributed with mean (1/ λ). The mean arrival rate= λ. The exponential probability density function is given by f(x)= λλee λλλλ for XX 0 0 eeeeeeeeeeheeeeee The corresponding cumulative distribution is given by 1 F(x)=P(X<x)= 1 ee λλλλ for xx 0 0 eeeeeeeeeeheeeeee

6 The inter arrival times are calculated by setting rr equal to the cumulative distribution rr = 1 ee λλλλ The inter arrival times are given by x=-(1/ λ)ln(1-r) For the antithetic variates rr pp, the inter arrival times are found by converting the mean and standard deviation. This is accomplished by subtracting its sample average rr pp and dividing by the standard deviation (ss rr pp), then multiply by the population standard deviation (1/ λ) and adding the population mean (1/ λ) time of interest as follows. Antithetic Random Variates Approach xx = rrpp rr pp ss rr ppλλ + 1 λλ Consider the perfectly negatively correlated antithetic random variates rr pp and rr pp. Based on this formulation of 300 pairs of random variates rr pp, rr pp, p=-0.01, the following were calculated: rr pp = , rr pp = , Corr rr pp, rr pp ii = , ss rr pp = , ss rr pp = , rr pp = The following averages were calculated: rr = , rr = , 0.5rr +0.5rr = Therefore, the average of the equally weighted combined random number is closer to the true value of 0.5 for the uniform distribution than either individual average. The corresponding standard deviation deviations were calculated: ss rr = , ss rr = , (0.5ss 2 rr + 0.5ss 2 rr ) 1/2 = Therefore, the standard deviation of the equally weighted combined random number is closer to the true value of 1/ 12= for the uniform distribution than either individual standard deviation. This suggests the combination of responses from two computer simulations based rr pp and rr pp. However, this experiment is beyond the scope of this paper. COMPUTER SIMULATION EXPERIMENT To explore the possibility of reducing bias and variance by combining the estimates of the responses from two Monte Carlo experiments, one based on xx and one based on xx, a simple queue will be analyzed as shown in Figure 4. In the simple queue, the source population for arriving entities is infinite. The queue is not capacitated. There is one server. The entities are processed on a first come served basis. The time between arrivals and the service times are exponentially distributed. The entity arrival rate is λ. The service rate is μ.

7 Figure 4. Simple queueing system. Arriving entities Queue Server Departing entities Arrival rate λ xx =-(1/ λ)ln(1-r) Service rate μ TT Arriving entities Queue Server Departing entities Combined response ωωωω + (1 ωω)tt Arrival rate λ xx = rrpp rr pp ss rr ppλλ + 1 λλ Service rate μ TT The response variable chosen is the average time that an entity spends in the system. The average time that entities spend in the system is well known to be precisely TT ttheeeeeeeeeeeeee =1/( μ - λ). Consider two Monte Carlo simulations. One simulation uses the exponential random numbers xx =-(1/ λ)ln(1-r), rr~uu(0,1) with response TT, and the other simulation uses the exponential random numbers xx = (rr pp rr )/ss pp rr ppλλ + 1 λλ, rr~uu(0,1), pp = 0.01 with response TT. The combined response is ωωωω + (1 ωω)tt where ωω and (1 ωω) are combining weights. The results in Table 3 are based on ωω = 0.5. The length of the simulation is 40,000 arrivals. Before antithetic combining, the average time in the system shows a bias of 1.8 minutes. After combining, the average time in the system has a smaller bias of 1.1. Before antithetic combining, the standard deviation of the time in the system is 14.4 minutes. After combining, the standard deviation of time in the system is 13.5 minutes. Table 3: Computer simulation results Simulation step number 16k 20k 24k 28k 32k 36k 40k TT Bias Standard Deviation of TT TT Standard Deviation of TT ωωωω + (1 ωω)tt Bias Standard Deviation of ωωωω + (1 ωω)tt

8 CONCLUSIONS Antithetic random number random variate pairs were created. These antithetic variables were used to perform two separate Monte Carlo computer simulations. The two estimates of the response variable were combined. The combined estimate exhibited better statistical properties than those of the uncombined estimates. The combined average of the estimated response variable had smaller bias. REFERENCES Ferrenberg, A.M., Landau, D.P., & Wong, Y. J. (1992). Monte Carlo Simulations: Hidden errors from good random number generators, Physical Review Letters, 69(23), Hammersley, J.M., & Morton, K.W. (1956). A new Monte Carlo technique: antithetic variates, Mathematical Proceedings of the Cambridge Philosophical Society, 52(3), Kleijnen, J. P. C. (1975). Antithetic variates, common random numbers and optimal computer time allocation in simulation, Management Science, 21(10), MATLAB (2008). Application Program Interface Reference, Version 8, The MathWorks, Inc. Ridley, A.D. (1995). Combining Global Antithetic Forecasts, International Transactions in Operational Research, 2(4), Ridley, A.D. (1997). Optimal Weights for Combining Antithetic Forecasts, Computers & Industrial Engineering, 32(2), Ridley, A.D. (1999). Optimal Antithetic Weights for Lognormal Time Series Forecasting, Computers and Operations Research, 26(3), Ridley, A.D., Ngnepieba, P., & Duke, D. (2013). Parameter Optimization for Combining Lognormal Antithetic Time Series, European Journal of Mathematical Sciences, 2(2), Ridley, A.D., & Ngnepieba, P. (2014). Antithetic time series analysis and the CompanyX data, Journal of the Royal Statistical Society, A, 177(1),