Simulating Correlated Multivariate Pseudorandom Numbers

Transcription

1 Simulating Correlated ultivariate Pseudorandom Numbers Ali A. Al-Subaihi Institute of Public Administration Riadh 4, Saudi Arabia

2 Abstract A modification of the Kaiser and Dichman (96) procedure of generating multivariate random numbers with specified intercorrelation is proposed. The procedure wors with positive and non-positive definite population correlation matri. A SAS module is also provided to run the procedure. Ke words: Random Numbers, ultivariate Random Numbers, Random Number Generation.

3 . INTRODUCTION A fundamental tas of quantitative researches is to mae probabilit-based inferences about population characteristics, θ, based on an estimator, θ ), using a "representative" sample drawn from that population. However, the statistics of classical parametric inference inform us about how the population wors to the etent necessar mathematical assumptions are met, and violation of an of the mathematical assumptions ma harm the accurac of the research conclusion(s). In regression, for instance, when the slope of the independent variable () is statisticall significant and BUE (best linear unbiased estimator), a researcher can clearl epect what happens to the dependent variable () when changes one unit provided that the usual mathematical assumptions of the regression (independence, homogeneit of variance, and normalit) are met. If an of these assumptions is violated, the ris of inference from the ordinar lease squares (OS) is seriousl off the mar (oon, 997). In real world, however, there are numerous situations where data violates at least one of the mathematical assumptions of the inferential statistic that is going to be used in analsis. Therefore, statisticians as well as researchers are interested to now how the mathematical assumptoin(s) violation affects the inferntial statistics power (the probabilit of correctl rejecting a false null hpothesis) and/or the Tpe I error rate (the probabilit of incorrectl rejecting a true null hpothesis). Fortunatel, meaningful investigation of man problems in statistics can be done through onte Carlo simulations (Broos et al., 999).. Statement of the Problem onte Carlo simulation is a computer-based technique that enables one to generate artificiall random samples from populations with nown characteristics in order to control some variables and manipulate others to investigate the effect of the manipulation on the robustness of a statistic. For more details about onte Carlo simulation, the reader is referred to Broos et al., 999 and oone,

4 onte Carlo simulations that require correlated data from normal and nonnormal populations are frequentl used to investigate the small sample properties of competing statistics, or the comparison of estimation techniques (Headric & Sawilows, 999). And, the widel used technique to generate correlated data from normal population (which is the paper's concentration) is the Kaiser and Dichman (96) procedure. The procedure uses a component analsis (e.g., principal components, square-root components) of a positive definite population correlation matri (R) for generating multivariate random numbers with specified intercorrelations. The procedure depends mainl on the decomposition of R and wors onl with a positive definite R. This limitation is an obstacle often faces users because not all real world situations have a positive definite R. Thus, a modification for this procedure (or development of a new one) is highl desired.. Purpose of the Stud The stud proposes a modification of the Kaiser and Dichman procedure in order to decrease its limitation effect. That is, the paper presents techniques that enable the Kaiser and Dichman procedure users to generate multivariate correlated pseudorandom numbers using a nonpositive definite R. 4

5 . REVIEW OF THE ITERATURE The importance of the problem of generating a correlated pseudorandom numbers comes up from the considerable attention that has been given to the onte Carlo studies; onte Carlo studies have been of interest to applied statisticians and continue to receive large focus in the recent statistical literature. The first to introduce a method for generating correlated pseudorandom numbers was Hoffman in 94. He presented a simple technique that is used onl to simulate two correlated variables. Then, Kaiser and Dicman (96) etended the Hoffman s procedure for m, where m is the number of the correlated variables. The proposed a method for generating sample and population score matrices and sample correlation matrices from a given R. The procedure utilizes component analsis for generating multivariate random numbers. The disadvantage of the Kaiser and Dicman procedure is that it depends on matri factorization that requires positive definiteness, and this condition does not frequentl hold. Fleishman (978) noted that real-world distributions of variables are tpicall characterized b their first four moments (i.e., mean, variance, sew, and urtosis) and presented a procedure for generating normal as well as nonnormal random numbers with these moments specified. The procedure accomplishes this b simpl taing a linear combination of a random number drawn from a normal distribution and its square and cube. Tadiamalla (980) criticized the Fleishman's procedure for producing distributions of variables for which the eact distribution was nown and thus laced probabilit-densit and cumulative-distribution functions and which, furthermore, could not produce distributions with all possible combinations of sew and urtosis. Tadiamalla (980) proposed five alternative procedures for generating nonnormal random numbers and compared them all with Fleishman power method for speed of 5

6 eecution, simplicit of implementation, and generalit of their abilit to generate nonnormal distributions. Vale and aurelli (983) etended the Fleishman (978) and Kaiser and Dicman (96) procedures to the multivariate case. The proposed a method for generating multivariate normal and nonnormal distributions with specified intercorrelations and marginal means, variances, sews, and urtoses. However, lie Kaiser and Dicman (96) procedure, the procedure fails to wor when R is not positive definite. Not onl that, but also the procedure fails to generate desired intercorrelations when the conditional distributions possess etreme sew and/or heav tails, even for sample sizes as large as N = 00 (Headric and Sawilows, 999). Headric and Sawilows (999) proposed a method that combines a generalization of Theorem and from Knapp and Swoer (967) with the Fleishman (978) procedure. The procedure generates multivariate nonnormal distributions with average values of intercorrelations closer to the population parameters for sewed and/or heav tailed distributions and for small sample sizes. Although the procedure eliminates the necessit of conducting a factorization procedure on R that underlies the random deviate, et it still requires the positive definiteness of R. Briefl, all procedures that can be used to generate multivariate pseudorandom numbers require either directl or indirectl positive definite R and do not wor without it, but the Fleishman s (978). This stud will present a modified technique that minimizes the reliance on positive definiteness condition for R. 6

7 3. DESCRIPTION OF THE PROCEDURES 3- Kaiser and Dichman procedure The Kaiser and Dichman (96) procedure is generall applied to generate multivariate normal random numbers, and uses a matri decomposition procedure. A Choles factorization (or an factorization, for that matter) is performed on R that is to underlie the random numbers. To generate a multivariate random number, one random number is generated for each component, and each random variable is defined as the sum of the products of the variable s component loadings and the random number corresponding to each of the components. When the random numbers input are normal, the resulting random numbers are multivariate normal with population intercorrelations equal to those of the matri originall decomposed. 3- The odified Procedure The modified procedure is also used to generate multivariate normal pseudorandom numbers utilizing R. However, unlie other procedures, it wors with a positive as well as a nonpositive definite R. n m n q n To generate a correlated multivariate data matri D = [ Y X ] with specific R, we rewrite R as a bloc matri such as A R = B B C q q q where A and C are the correlation matrices of Y and X, respectivel, and the B is the correlation matri between Y and X. The division is assumed to occur intentionall to ensure q q positive definiteness of both A and C. This is a necessar condition (i.e., the method dose not wor without it). Net, one of the following techniques ma be applied. 7

8 3-- When A q q and q = This technique is applied to generate multivariate pseudorandom numbers when R can be partitioned into three matrices: A, B, and C, where is the number of 's and > such as R = B B = C O O O The technique is performed as follows: A total of correlation matrices that contain the correlation between and each individuall are created such as B i = i i i =,,,. Then, a multivariate normal data matri ( X n ) is generated through the equation X =µ + XU ˆ where X is the multivariate normal data matri, µ is the vector containing the variable means, Xˆ contains vectors of independent and standard normal variates, U is the Choles upper triangular matri of C. The factorization C = UU T is nown as the Choles factorization. A data vector ( n ) of standard normal variates is generated and concatenated horizontall with each column of the matri X individuall such as T i =[ X i ] i =,,, ). This will gives us a total of matrices each of size (n ). A total of vectors Z i of size (n ) are generated independentl through the equation Z i = µ + T i U ic i =,,, 8

9 where U ic is the nd column of the Choles upper triangular matri of B i which was created in the beginning. The vector is then concatenated horizontall with Z i i =,,, to create data matri of size (n +) such as D n + = [ Z Z Z ]. The resulting data matri has a correlation matri that most liel varies from the given population matri (R) and the change taes place mainl in C. Thus, to get data matri that has a correlation matri close to (i.e. with average intercorrelation among the s around) the desired R, we manipulate the actual correlation value among the s, and repeat the process. The average intercorrelation among the s is assessed using the measure proposed b Kaiser (968) λ γ = O (3.) where λ is the largest eigenvalue of the correlation matri, and O is the number of variables (which is when measuring the average intercorrelation among the s). The larger the value of γ, the greater the correlation; if γ = 0, there is no correlation. γ = when the correlations among the variables in the set are quite high. Accordingl, one should epect that γ taes values near the values of. Simulation Eample Suppose a data matri of size (0 5) with specific intercorrelations need to be generated, and suppose 0.5 R =

10 The Kaiser and Dicman procedure cannot be used here to generate the required data because the provided R is not positive definite. Thus, the new procedure is going to be implemented instead (the SAS module in the Appendi A can be utilized to simulate the data using the modified procedure). Table shows the correlation matri (R i ) used in the procedure and the correlation matri of the simulated data (Re i ) at attempt i. At the second attempt (when the correlation among the s was replaced b 0.84 in the original R), we had data with correlation matri (Re ) that is closer to the original R (which equals R ) than those of the first (Re ) and the third (Re 3 ) attempts. Notice that Re i in the Table are full ran (i.e., Ran(Re i ) = 5), and usuall compared with the original R. R, R 3, etc. are onl used in the method to simulate the needed data. 0

11 TABE The Theoretical and Empirical Correlations Values The Correlation matri used The Correlation matri gotten R E Re E γ = R Re γ = R Re γ = Note: R i is the correlation matri used to simulate the required data; Re i is the average correlation matri of the data simulated 000 times; γ is the Kaiser s Gamma. The real and cpu times needed for 000 replications = 0.0 and 0.07 seconds, respectivel when a hp with.70ghz processor and 56 KB RA is used. The standard error matri of Re is as follows:

12 Re When A q q and q This technique is applied when R can be partitioned into three matrices:, B, and, where q > and > are numbers of the 's and the 's, respectivel, such as q q A q C = = C B B A q q q q q q O O O O O O R In this case, the technique is performed in this wa: A total of (q ) correlation matrices that contain the correlation between each and each are created such as i =,,, q and j =,,,. Then a multivariate normal data matri ( ) is generated through the equation = i j j i ij B n X XU X ˆ + =µ where X is the multivariate normal data matri, µ is the vector containing the variable means, contains vectors of independent and standard normal variates, U is the Choles upper triangular matri of C. Xˆ

13 Similarl, a multivariate normal data matri ( Y Choles upper triangular matri of q q A n q ) is generated using the. Net, each column of the matri Y is concatenate horizontall with each column of the matri X individuall (i.e., T ij =[ Y i X j ] i =,,, q and j =,,, ), which gives a total of (q ) matrices each of size (n ). A total of (q ) vectors Z ij of size (n ) are generated through the equation Z ij = µ + T ij U ijc i =,,, q and j =,,, where U ijc is the nd column of the Choles upper triangular of the matri B ij which was created at first. Afterward, the vectors Z j j =,,, are concatenate horizontall to ~ n create data matri of size (n ) for each Y i individuall such as X = [ Z Z... Z ] i i i i. The matrices X ~ n j i =,,, q are then summed together to create data matri ( q i= ~ X n i ), and then the matri Y is concatenate horizontall with the matri q ~ n Xi ~ i= n q+ X = to create the required data matri ( D ). q Similar to the first case, the resulting data matri ( D n q+ ) has a correlation matri that most liel varies from the population matri and the change, once again, taes place mainl in C. Thus, to get data matri that has a correlation matri close to (i.e. with average intercorrelation among the s around) the desired population correlation matri, we manipulate the actual correlation value among the s, and repeat the process. Simulation Eample Suppose a data matri of size (0 6) with specific intercorrelation need to be generated and suppose 3

14 R = Again, the Kaiser and Dicman procedure does not wor in this situation because the provided R is not positive definite. Consequentl, the modified procedure is going to be implemented as an alternative (the SAS module in Appendi A also can be utilized to simulate the data using the modified procedure). Table shows the correlation matri (R i ) used in the procedure and the correlation matri of the simulated data (Re i ) at attempt i. At the third attempt (when the correlation among the s was replaced b 0.84 in the original population correlation matri), we had data with correlation matri (Re 3 ) that is closer to the original population correlation matri (R ) than those of the first (Re ) and the second (Re ) attempts. 4

15 TABE The Theoretical and Empirical Correlations Values The Correlation matri used The Correlation matri gotten R Re γ = R Re γ = R Re γ = Note: R i is the correlation matri used to simulate the required data; Re i is the average correlation matri of the data simulated 000 times; γ is the Kaiser s Gamma; the real and cpu times needed for 000 replications = 0.57 and 0.0 seconds, respectivel. The real and cpu times needed for 000 replications = 0.57 and 0.0 seconds, respectivel when a hp with.70ghz processor and 56 KB RA is used. The standard error matri of Re is as follows: 5

16 Se =

17 4. CONCUSION The paper has suggested a modification to the Kaiser and Dicman (96) procedure of generating multivariate normal pseudorandom numbers. As demonstrated in the numerical eamples, the procedure can produce random numbers with intercorrelation near the population correlation matri when other procedures do not wor at all. Although there are few situations (especiall when A or C are not positive definite) where the procedure does not wor, still it might be used to generate numbers close to the preferred intercorrelations b manipulating the correlations among the s or the s beside the correlations between the s and the s. 7

18 proc iml; APPENDIX (A) OPTIONS ls=00 ps=60 nodate nonumber; /********* The Parameters **************/ NS=0; /* No. of subjects */ /* The population correlation matri is entered as YY YX, XY XX */ PopCor={ , , , , ,..... }; %et NY=3; /* No. of the 's */ %et NX=3; /* No. of the 's */ %et NPC=9; /* No. of correlations = NY*NX */ /***************************************************/ NV=&NY+&NX; /* No. of Variables */ CorY= PopCor[:&NY,:&NY]; /* Corr. among the 's */ CorX= PopCor[&NY+:NV,&NY+:NV]; /* Corr. among the 's */ CorYX= PopCor[&NY+:NV,:&NY]; /* Corr. betw. the 's & the 's */ do i= to ncol(coryx); /* Corr. betw. the 's & the 's as a column*/ CorYXs=CorYXs//CorYX[,i]; end; %macro loop(npc); %Do i= %to &NPC; /* Bi's Correlation matrices */ Cr&i=I(); Cr&i[,]=CorYXs[&i,]; Cr&i[,]=CorYXs[&i,]; %end; %mend loop; %loop (&npc); X=Rannor(Repeat(0,NS,&NX))*root(CorX); /* The X data matri */ =Rannor(Repeat(0,NS,&NY))*root(CorY); /* The Y data matri */ DaXs=0*j(ns,&NX); %macro loop (NY); %et =0; %do j= %to &NY; %do i= %to &NX; %et c=%eval(&i+&k); %put c=&c; dat=(y[,&j] X[,&i])*(root(CrYX&c)); dat=dat dat[,]; %end; %et =&c; daxs=daxs+dat; free dat; %end; 8

19 %mend loop; %loop (&NY ); daxs=daxs*(/&ny); data=y daxs; /* The final data matri */ eg=eigval(corr(daxs)); CXs=(eg[<>,]-)/(&NX-); /* The average Correlations among all 's */ eg=eigval(corr(y)); CYs=(eg[<>,]-)/(&NY-); /* The average Correlations among all 's */ Call=corr(data); /* Correlations among all data */ ca=call[:&ny,(&ny+):(&ny+&nx)]; print 'The Correlations between Xs and Ys',ca, 'The average Correlations among all Xs = ' CXs, 'The average Correlations among all Xs = 'CYs, 'The total correlation matri of the data', call; quit; 9

20 REFERENCES [] Broos, G. P.,& Robe, R. R. (999), "onte Carlo Simulation for Perusal and Practice", Paper presented at the meeting of the American Educational Research Association, otereal, Quebec, Canada. [] Fleishman, A. (978), "A ethod for Simulating Non-Normal Distributions", Pschomeria, 43 (4), [3] Headric, T. C., & Sawilows, S. S. (999), Simulating Correlated ultivariate Nonnormal Distributions Etending the Fleishman Power ethod, Pschomeria, 64 (), [4] Hoffman, P. J. (94), "Generating Variables with Arbitrar Parameters" Pschomeria, 4, [5] Kaiser, H. F., & Dicman, K. (96), "Sample and Population Score atrices and Sample Correlation atrices From an Arbitrar Population Correlation atri", Pschomeria, 7 (), [6] oon, C. Z. (997), "onte Carlo Simulation", Sage Publications, Thousand Oas, CA. [7] Tadiamalla, P. R. (980), "On Simulating Non-Normal Distributions", Pschomeria, 45 (), [8] Vale, C. D., & aurelli, V. A. (983), "Simulating ultivariate Nonnormal Distributions", Pschomeria, 48 (3), [9] Knapp, T.R., & Swoer, V. H. (967), "Some Empirical Results Concerning the Power of Bartlett's Test of the Significance of a Correlation atri", American Educational Research Journal, 4 (),