Testing Primitive Polynomials for Generalized Feedback Shift Register Random Number Generators

Transcription

1 Brigham Young University BYU ScholarsArchive All Theses and Dissertations Testing Primitive Polynomials for Generalized Feedback Shift Register Random Number Generators Guinan Lian Brigham Young University - Provo Follow this and additional works at: Part of the Statistics and Probability Commons BYU ScholarsArchive Citation Lian, Guinan, "Testing Primitive Polynomials for Generalized Feedback Shift Register Random Number Generators" (2005). All Theses and Dissertations This Selected Project is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in All Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact scholarsarchive@byu.edu.

2 TESTING PRIMITIVE POLYNOMIALS FOR GENERALIZED FEEDBACK SHIFT REGISTER RANDOM NUMBER GENERATORS by Guinan Lian A project submitted to the faculty of Brigham Young University In partial fulfillment of the requirements for the degree of Master of Science Department of Statistics Brigham Young University December 2005

3 Copyright 2005 Guinan Lian All Right Reserved

4 BRIGHAM YOUNG UNIVERSITY GRADUATE COMMITTEE APPROVAL of a project submitted by Guinan Lian This project has been read by each member of the following graduate committee and by majority vote has been found to be satisfactory. Date Date Bruce J. Collings, Chair Del T. Scott Date William F. Christensen

5 BRIGHAM YOUNG UNIVERSITY As chair of the candidate s graduate committee, I have read the project of Guinan Lian in its final form and found that (1) its format, citations, and bibliographical style are consistent and acceptable and fulfill university and department style requirements; (2) its illustrative materials including figures, tables, and charts are in place; and (3) the final manuscript is satisfactory to the graduate committee and is ready for submission to the university library. Date Bruce J. Collings Chair, Graduate Committee Accepted for the department Bruce Schaalje Graduate Coordinator Accepted for the College G. Rex Bryce Associate Dean, College of Physical and Mathematical Sciences

6 ABSTRACT TESTING PRIMITIVE POLYNOMIALS FOR GENERALIZED FEEDBACK SHIFT REGISTER RANDOM NUMBER GENERATORS Guinan Lian Department of Statistics Master of Science The class of generalized feedback shift register (GFSR) random number generators was a promising method for random number generation in the 1980 s, but was abandoned because of some flaws such as poor performance on certain tests for randomness. The poor performance may be due to the choice of primitive polynomials used in the generators, rather than inherent flaws in the method. The original GFSR generators were all based on primitive trinomials. This project examines several alternative choices of primitive polynomials with more than one "interior" term to address this problem and hopefully provide access to good random number generators.

7 ACKNOWLEDGMENTS I have to acknowledge my advisor Dr. Bruce J. Collings. This paper is an extension of his research and I could not have done it without him. I also want to thank the members of my graduate committee, and the Department of Statistics for all the continual help and support I received throughout this entire process.

8 Contents Chapter List of Figures List of Tables viii x Chapter 1: Introduction 1 Chapter 2: Literature Review The Generators Testing Random number Generators 9 Chapter 3: Methods Generators to be investigated Tests to be used 12 Chapter 4: Results Linear Congruential Generator G Combined Generator G Primitive trinomial Generalized Feedback Shift Generators G 3 -G Primitive Pentanomial Generalized Feedback Shift Generators (3 interior terms) G 7 -G Primitive Polynomial Generalized Feedback Shift Generators (more than 3 interior terms) G 13 -G Chapter 5: Conclusion and Discussion 80 Bibliography 82 vi

9 Appendix A Fortran codes A.1 Initialization 86 A.2 Programs for generators 89 A.3 mtuple results 96 A.4 Resort the data 98 A.5 OPSO results 99 Appendix B R codes 101 Appendix C Graphs (QQ-plots) for each generator C.1 graphs for G 1 (LCG) 102 C.2 Graphs for Combined Generator G C.3 Primitive Trinomial GFSRs G 3 G C.4 Primitive Pentanomial GFSRs G 7 G 12 (3 interior terms) 110 C.5 Primitive Polynomial GFSRs G 13 G 32 (more than 3 interior terms) 118 vii

10 Figures Figure Figure 2. 1: the first shift of a GFSR 6 Figure 2. 2: the second shift of a GFSR 6 Figure 4.1: Chi-square QQ-plots for Generator 1 17 Figure 4.2: Chi-square QQ-plots for Generator 2 19 Figure 4.3: Chi-square QQ-plots for Generator 3 21 Figure 4.4: Chi-square QQ-plots for Generator 4 23 Figure 4.5: Chi-square QQ-plots for Generator 5 25 Figure 4.6: Chi-square QQ-plots for Generator 6 27 Figure 4.7: Chi-square QQ-plots for Generator 7 29 Figure 4.8: Chi-square QQ-plots for Generator 8 31 Figure 4.9: Chi-square QQ-plots for Generator 9 33 Figure 4.10: Chi-square QQ-plots for Generator Figure 4.11: Chi-square QQ-plots for Generator Figure 4.12: Chi-square QQ-plots for Generator Figure 4.13: Chi-square QQ-plots for Generator Figure 4.14: Chi-square QQ-plots for Generator Figure 4.15: Chi-square QQ-plots for Generator Figure 4.16: Chi-square QQ-plots for Generator viii

11 Figure 4.17: Chi-square QQ-plots for Generator Figure 4.18: Chi-square QQ-plots for Generator Figure 4.19: Chi-square QQ-plots for Generator Figure 4.20: Chi-square QQ-plots for Generator Figure 4.21: Chi-square QQ-plots for Generator Figure 4.22: Chi-square QQ-plots for Generator Figure 4.23: Chi-square QQ-plots for Generator Figure 4.24: Chi-square QQ-plots for Generator Figure 4.25: Chi-square QQ-plots for Generator Figure 4.26: Chi-square QQ-plots for Generator Figure 4.27: Chi-square QQ-plots for Generator Figure 4.28: Chi-square QQ-plots for Generator Figure 4.29: Chi-square QQ-plots for Generator Figure 4.30: Chi-square QQ-plots for Generator Figure 4.31: Chi-square QQ-plots for Generator Figure 4.32: Chi-square QQ-plots for Generator ix

12 Tables Table Table : Generator G 1 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 16 Table 4.1.2: Z-scores for four repetition OPSO tests of Generator 1 17 Table 4.2.1: Generator G 2 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 18 Table 4.2.2: Z-scores for four repetition OPSO tests of Generator 2 19 Table 4.3.1: Generator G 3 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 20 Table 4.3.2: Z-scores for four repetition OPSO tests of Generator 3 21 Table 4.4.1: Generator G 4 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 22 Table 4.4.2: Z-scores for four repetition OPSO tests of Generator 4 23 Table 4.5.1: Generator G 5 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 24 Table 4.5.2: Z-scores for four repetition OPSO tests of Generator 5 25 Table 4.6.1: Generator G 6 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 26 Table 4.6.2: Z-scores for four repetition OPSO tests of Generator 6 27 x

13 Table 4.7.1: Generator G 7 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 28 Table 4.7.2: Z-scores for four repetition OPSO tests of Generator 7 29 Table 4.8.1: Generator G 8 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 30 Table 4.8.2: Z-scores for four repetition OPSO tests of Generator 8 31 Table 4.9.1: Generator G 9 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 32 Table 4.9.2: Z-scores for four repetition OPSO tests of Generator 9 33 Table : Generator G 10 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 34 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 11 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 36 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 12 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 38 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 13 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 40 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 14 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 42 xi

14 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 15 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 44 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 16 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 46 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 17 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 48 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 18 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 50 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 19 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 52 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 20 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 54 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 21 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 56 Table : Z-scores for four repetition OPSO tests of Generator xii

15 Table : Generator G 22 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 58 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 23 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 60 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 24 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 62 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 25 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 64 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 26 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 66 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 27 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 68 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 28 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 70 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 29 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 72 xiii

16 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 30 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 74 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 31 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 76 Table : Z-scores for four repetition OPSO tests of Generator Table : Generator G 32 mtuple p-values for KS test of whether values are χ2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) 78 Table : Z-scores for four repetition OPSO tests of Generator xiv

17 Chapter 1: Introduction Random numbers are numbers chosen at random; they should appear to be independent and uniformly distributed. Random numbers are useful in many different kinds of applications, such as simulation, sampling, numerical analysis, computer programming, decision making and recreation (see Knuth, 1973). Those applications are including: job-shop scheduling and marketing in business; psychoanalysis in humanities; air pollution, chemistry, and genetics in science; social conflict and housing policies in social science. (Dudewicz and Ralley, 1981). If the numbers chosen are not random, in another words, not independent and uniformly distributed, they can invalidate the results of the study, so good random number generators are important to researchers. There are many kinds of random number generators: linear congruential generators, multiple recursive generators, Tausworthe generators, generalized feedback shift register, combining generators and some of their variations. This paper focuses on generalized feedback shift register (GFSR) generators which have some very attractive properties and certain advantages over other classes of generators. But they also have some drawbacks such as initialization and partitioning problems, and poor performance on some tests for randomness. The problem of initialization and partitioning were solved by Collings and Hembree (1986). This project examines some alternative choices of primitive polynomials with more than one interior" terms in an attempt to solve the randomness problems. Two tests for randomness are conducted on a variety of random number generators, some proposed in the literature and some new ones. 1

18 Chapter 2: Literature Review 2.1 The Generators When first introduced to random number generation, people usually think about what constitutes a random number. Most consider longer and longer sequences, looking for patterns or a lack thereof. All practical random number generators produce only a finite sequence which is then repeated over and over. These periodic sequences are clearly not random. However, a pseudorandom number generator that produces an apparently random number sequence is in many ways better than a truly random number generator. (Bratley, Fox and Schrage, 1987) An ideal random number generator would produce a sequence of numbers that are independent and identically distribute (i. i. d.) uniformly either on the interval (0, 1) or on the set {1, 2, 3,, n} for some (large) integer n. A generator is accepted as sufficiently random for practical purposes if the sequence of numbers produced appear to be i. i. d. uniform. (See Bratley, Fox and Schrage, 1987.) Some early suggestions for possible random number generators include: Random devices such as cosmic ray counters, the least significant bits of a digital clock, or just flipping a coin, etc. Random digit tables such as the one produced by RAND Corporation in The midsquare method illustrated as follows. Suppose we want to generate four digit integers and the last number was 1234, to obtain the next number in the sequence we square the last one and use the middle four digits of the product. In this case the product is so the next pseudorandom number is The next few numbers in the sequence are 3215, etc. These particular generators are of historical interest only (none of them are well suited to current demands for random numbers). With the advent and subsequent widespread use of computers, it is typically easier to generate random number by some arithmetic process 2

19 as implemented in various software packages, or by an individual computer program. For any such generator there is ultimately a cycle of numbers that is repeated endlessly. The length of this repeated cycle is called the period of the generator. A useful generator will of course have a relatively long period. However, a long period is only one desirable criterion for a random number generator. The most common random numbers are discussed briefly below. Linear Congruential Generator (LCG) A linear congruential generator is defined by an equation of the form: x i = xi 1 a + c mod m. In this equation, a is called the multiplier ; c is called the increment ; and m is called the modulus of the generator. (In the most popular case, in fact, the only case currently used in the literature, c = 0 and this generator is then sometimes called a multiplicative congruential generator.) We want to choose a multiplier that will produce a sequence with the maximum length period. Since only m different values are possible, the period surely cannot be longer than m. Moreover, when c = 0, if any x i = 0 then all subsequent terms of the sequence will also be 0. Hence, the maximum possible period of this generator is clearly m-1. As an example, suppose a = 3 and m = 31 with X 0 = 9, then we have X i = 3 X i-1 mod 31, with 0 < X i < m. We can get X 1 = 3. 9 = 27; X 2 = = 81, since m = 31, = 19; X 3 = = 57, = 26; and so on. Thus the sequence is 27, 19, 26, 16, 17, 20, 29, 25, 13, 8, 24, 10, 30, 28, 22, 4, 12, 5, 15, 14, 11, 2, 6, 18, 23, 7, 21, 1, 3, 9, and of course, at this point the sequence begins the repeat. The period is m-1 = 30; we know 3 is a primitive root modulo 31(See James E. Gentle, 1998). The numbers (X i ) could be converted to numbers between 0 and 1 by dividing them by the period ( i. e. X i /(m-1) ). 3

20 1973). To achieve the maximum period, we have the following theorem (See Knuth, Theorem: The maximum possible period when c = 0 is achieved if i) x 0 is relative prime to m; ii) a is a prime element (primitive root) modulo m. The number is a prime element modulo m (or a is a primitive root of m) if and only if a 0 (mod m), and a (m-1/q) 1 (mod m), for any prime divisor q of m-1 (Wu, 1997). Much of the research in the literature is based on the modulus , since this is a prime number and computations modulo are fairly fast with 32 bit arithmetic. However, for fast present day computers a period of is short enough that a fast desktop computer now cycles through the whole period in under 30 minutes. Much current research involving LCM generators basically amounts to starting over looking for suitable larger modulii and multipliers. A simple extension of the multiplicative congruential generator consists of using multiples of the previous k values to generate the next one: x = (a i 1 x i x a i 2 + a k x ) mod m. i k This is sometimes called a multiple recursive generator (Gentle, 1998). A multiple recursive generator can have a much longer period than a simple multiplicative generator. Although these congruential generators have been, and still are, widely used despite some potential failings. It has long been known that the points in the unit n-cube generated by these methods lie in a relatively small number of parallel hyperplanes (Marsaglia, 1968). Another way of saying this is that LCGs have a lattice structure. It is well accepted that generators with bad, that is, too coarse, lattice structure must be avoided (L Ecuyer, 1990). Because of these cautions, some researchers have shifted their attention to other possible types of generators. 4

21 Tausworthe (Shift Register) Generators A Tausworthe sequence, { t i }, is any sequence of zeros and ones which satisfies the recurrence relation t k = c0 tk p + c1 t k p cp 1 t k 1 (mod2) for some set of integers { c 0,, c p 1 }each taking the value zero or one. ( See Tauseworthe 1965, or Collings, 1987a.) The sequence { t i } will have maximal period p 2 1 if, and only if, the polynomial f (x) = c 0 + c 1 x + c 2 2x p c x + x p 2 is primitive over GF (2), the field of integers modulo2. f (x) = c 0 + c 1 x + c 2x + p 1 + c x + x p is a primitive polynomial of order p over the integers modulo 2 if the p 1 smallest value of k for which x k x (modulo f (x) ) is k = 2 p. That is, all the powers of x; x, x 2 x 2^p-1, reduced modulo f (x) are distinct. The advantage of this method lies in the fact that for a suitable choice of parameter values, the generated sequences are theoretically guaranteed to have several desirable properties including the k-distribution property, that is, the full sequence is uniformly distributed over all dimensions up to dimension k (Tesuka, 1983). One shortcoming of the Tausworthe generator is that it is time-consuming compared to congruential generators in that it produces only one bit at a time. Also Tausworthe sequences may have problems with long strings of like bits (i.e. all zeros or all ones) clumping together. This latter problem can be alleviated by using judiciously selected primitive polynomials that have more non-zero terms than the primitive trinomials considered by Collings (1987a). p 1 5

22 Generalized Feedback Shift Register (GFSR) Generators Generalized Feedback Shift Register (GFSR) generators are a variant of the Tausworthe generators. A GFSR sequence is defined by Lewis and Payne (1973) as a sequence of words, {W i }, satisfying the equation W k+p = c 0 W k c 1 W k+1 c p-1 W k+p-1 for all k 0, where {c 0, c 1,, c p-1 } is some set of zeros and ones with c 0 = 1 and denotes the exclusive-or operator (i.e. bitwise addition modulo 2). If the W i are L-bit words, then p must be at least L or there will be a linear dependence among the columns of the W i. If the polynomial f (x) = c 0 + c 1 x + c 2 2x p c x + x p p 1 is a primitive polynomial over GF(2), the {W i } sequence will have maximal period 2 p -1. (Alternate formulations exist. See, for example, Bright and Enison, 1979; Simmons, 1979; or Tesuka, 1991.) To illustrate how GFSR produces random numbers, consider the trinomial 1 + x + x 4, and begin with the bit sequence 1, 0, 1, 0 (the initial set of p numbers). For this polynomial, we have p= 4 and c 0 = 1, c 1 = 1, c 2 = 0 and c 4 = 0, so the recurrence is W k+4 = W k W k+1. Operating the generator, we get Figure 2. 1 the first shift of a GFSR Figure 2. 2 the second shift of a GFSR 6

23 and so on. By this way we obtain 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1 at which point the sequence repeats; its period is Forming L= 4 bit words by putting the bits into a fixed binary position with a delay of 3 between binary positions, we have 1010 = 10, 1111 = 15, 0001 = 1, 0011=3, = 0101 = 5, = 0010 = 2, and so on. So a 4-wise decimation using the recurrence yields the numbers 10, 15, 1, 3, 5, 2, (in which the 3 required an additional bit in the sequence above). We could continue in this way to get = 15 integers between 1 and 15 before the sequence begins to repeat (See James E. Gentle, 1998). All the integer numbers above could be converted to numbers between 0 and 1 by dividing the words W i by 2 L (i.e. W i /2 L, L p). GFSR generators produce multidimensional pseudorandom numbers, which is an advantage over other pseudorandom number generators; the period can be arbitrarily long, independent of the word size of the computer on which the generator is implemented; the same floating-point sequence is obtained on any machines; they are easily portable; and, they can be faster. But GFSR generators also have some potential drawbacks including initialization and partitioning problems, and poor performance on some tests for randomness. The problems of initialization and partitioning were solved by Collings and Hembree (1986). This project investigates the use of primitive polynomials larger than trinomials (i.e. polynomials with more than three non-zero terms) to improve the performance of the generators on tests for randomness. Some research (See Collings, 1989) has indicated that increasing the number of non-zero terms in the underlying primitive polynomial generally improves the randomness performance of the associated sequence. For primitive trinomials, long strings of like bits tend to be followed by other long strings of like bits. For example, for the generator used in Figure 2.1 and 2.2, the string of 4 ones is followed by a string of 3 zeros (both long relative to the generator s period of 15). The more complicated recurrence structure for primitive polynomials with more than 3 non-zero terms means that long strings of like bits are followed by relatively shorter strings of like bits. This appears to improve the randomness of the resulting sequence of generated numbers. 7

24 A modification of the GFSR was proposed by twisting the bit pattern in the W i s before the exclusive-or operations. This method has only been applied in the case of primitive trinomials, 1 + x q + x p. In this case, the recursion becomes W k = W k-q AW k-p, where A is some LΗL permutation matrix. This is called a twisted GFSR generator (See Matsumoto and Kurita, 1992). Another class of modified GFSR generators is the tempered GFSR proposed by Matsumoto and Kurita (1994). This is an efficient method to attain the order of equidistribution with a tight upper bound on the order. Tempering can be easily implemented by adding two transformed lines to the original programs. Combination Generators Both the period and apparent randomness of random number generators can often be improved by combining more than one generator. Some methods of this type can be found in Maclaren and Marsaglia (1965), Bays and Durham (1976) and Collings (1987b). The generators that are combined can be of any type. For example, the Super-Duper generator proposed by Masaglia used a linear congruential generator and a Tausworthe generator; Wichmann and Hill (1982 and corrigendum, 1984) combined three linear congruential generators; L Ecuyer (1988) combined k multiplicative congruential generators; and Collings (1987b) used a linear congruential generator to interweave the outputs from several other linear congruential generators. Other Variations There are a number of other types of generators that have been proposed. These include non-linear (quadratic) generators (Gentle, 1998), inversive generators, lagged Fibonacci generators (L Ecuyer, 1990), and add-with-carry, subtract-with borrow, and multiple-with-carry generators (mostly proposed by Marsaglia in [1991]). 8

25 2.2 Testing Random Number Generators We need to decide whether a sequence of numbers is sufficiently random-looking. Because the random number generators (RNGs) on a computer are deterministic functions, the question becomes Does the sequence of numbers mimic a sample of i. i. d. U (0, 1) random variables? Two classes of tests are commonly applied to such generators. (L Ecujer, 1992) Theoretical tests Theoretical tests look at the intrinsic structure of the generator to derive behavioral properties of the sequence of points, usually over the whole period. These theoretical tests are generally specific to a particular class of generators. Included in the class of tests are the lattice and spectral tests, discrepancy bound calculations, period length calculations, etc.. (See L Ecujer, 1992) Empirical tests The empirical goodness-of-fit tests try to find statistical evidence against the null hypothesis that the sequence is a random sample from a population of uniformly distributed random variables on the interval (0, 1) or on the set {1, 2,, N}. Many empirical tests of various kinds have been proposed over the years. These include the maximum-of-t test, various Kolmogorov-Smirnov (KS) tests, the runs-up test, the overlapping-pairs-sparse-occupancy (OPSO) test, the birthday spacing test, the poker test and the monkey test. There are also a number of other proposed tests that are not included in this project. These include several chi-square goodness-of-fit tests (see, for example, Bratley, Fox and Schrage, 1987), the spectral test (see Knuth, 1973), the nearest pair test, (see L Ecujer, 1992), etc. 9

26 Chapter 3: Methods This project will compare a number of generators proposed in the literature, plus several GFSR generators based on primitive polynomials with more than 5 non-zero terms. The quality of the generators will be measured by their performance on some standard tests for randomness. The specifics and tests are listed and described below. 3.1 Generators to be Investigated Linear Congruential Generator The first generator G 1 is the linear congruential generator commonly known as randu. This generator is generally recognized as being poor. It has multiplier and modulus The period is Combined Generator of L Ecuyer (1988) The second generator considered, G 2 is the combined generator proposed by L Ecuyer (1988). This generator combines the output from two LCGs. The first component has multiplier and modulus , while the second has multiplier and modulus The period of this combined generator is approximately 2.3 x Each individual generator produces uniform values by dividing the current seed by the modulus. These two values are then combined by adding them together and reducing the result modulo 1. Trinomial GFSR Generators As mentioned before, primitive trinomials that have three non-zero terms generally do not perform well. They tend to take long strings of like bits and break them into two or three pieces. We test four trinomial GFSR Generators as following: G 3 : 1 + x 3 + x 31, G 4 : 1 + x 6 + x 31, G 5 : 1 + x 7 + x 31, and 10

27 G 6 : 1 + x 13 + x 31. They all have a period of Pentanomial GFSR Generators of Wu (2001) Wu (2001) proposed using GFSR generators based on the special form of primitive polynomials, 1 + x q-1 + x q + x p-1 + x p. This particular form can be efficiently computed because of the adjacent pairs of exponents. The specific generators of this type used in this project, along with the corresponding primitive polynomials, are G 7 : 1 + x 79 + x 80 + x x 521, with period , G 8 : 1 + x x x x 607, with period , and G 9 : 1 + x x x x 1279, with period Pentanomial GFSR Generators of Matsumoto and Kurita (1991) The next three generators, suggested by Matsumoto and Kurita (1991), are based on primitive pentanomials of the form 1 + x q1 + x q2 + x q3 + x p, where p is a Mersenne prime and the q s satisfy 0 < q1 < q2 < q3 < p, with qk chosen from the interval [p(2k-1)/8]:[p(2k+1)/8] to provide some separation between the exponents. The specific generators used in this project along with the corresponding primitive polynomials are G 10 has primitive polynomial 1 + x 86 + x x x 521, with period ; G 11 has primitive polynomial 1 + x x x x 607, with period , and G 12 has primitive polynomial 1 + x x x x 1279, with period New GFSR Generators The final twenty generators, G 13,..., G 32, are GFSR generators based on randomly selected primitive polynomials with degree 31. These generators each have period The specific generators along with the exponents of the interior non-zero terms of their primitive polynomials are G 13 : 14, 19, 22, 24, 29 G 14 : 1, 5, 7, 9, 13, 15, 18 G 15 : 4, 5, 7, 13, 21, 25, 27, 29, 30 G 16 : 2, 3, 5, 7, 11, 16, 20, 23, 26, 27, 28 11

28 G 17 : 3, 8, 10, 14, 17, 21, 23, 25, 26, 28, 30 G 18 : 1, 2, 3, 4, 5, 10, 14, 18, 19, 22, 23, 24, 27 G 19 : 2, 3, 4, 5, 10, 13, 17, 20, 21, 23, 25, 26, 28 G 20 : 2, 6, 10, 11, 12, 13, 15, 19, 24, 26, 28, 29, 30 G 21 : 1, 2, 4, 5, 6, 8, 11, 12, 14, 15, 16, 22, 23, 25, 29 G 22 : 2, 3, 4, 5, 6, 8, 9, 11, 15, 19, 20, 23, 24, 25, 30 G 23 : 2, 3, 4, 10, 13, 15, 17, 19, 21, 24, 25, 26, 27, 29, 30 G 24 : 5, 7, 8, 10, 11, 13, 14, 16, 21, 24, 25, 26, 27, 29, 30 G 25 : 1, 4, 5, 6, 7, 8, 9, 11, 12, 16, 18, 19, 20, 21, 24, 26, 29 G 26 : 1, 3, 5, 8, 9, 10, 13, 14, 16, 17, 20, 23, 24, 26, 27, 28, 29 G 27 : 1, 2, 3, 4, 6, 7, 11, 12, 14, 15, 18, 19, 22, 24, 26, 27, 29 G 28 : 1, 4, 6, 7, 8, 10, 12, 13, 16, 17, 18, 19, 20, 21, 23, 25, 27, 28, 30 G 29 : 1, 2, 3, 7, 8, 9, 13, 14, 16, 17, 19, 20, 21, 22, 23, 24, 25, 27, 28 G 30 : 2, 3, 4, 7, 8, 12, 13, 14, 15, 16, 17, 18, 20, 22, 23, 24, 25, 27, 28, 29, 30 G 31 : 1, 3, 4, 5, 6, 7, 9, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 25, 26, 27, 28, 30 G 32 : 1, 2, 3, 4, 6, 7, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 24, 26, 27, 28, 29, 30 The above polynomials were chosen at random from the list of all k-term primitive polynomials of order 31 with k = 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27. Because of the number of possibilities, more than one generator was tested for k = 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, Tests to be Used mtuple Test (G. Marsaglia, 1985) 2 This test uses Pearson chi-square, ( OBS EXP) / EXP, on the individual digits to see if the observed were satisfactorily close to the expected frequencies. Let v 1,v 2, v 3, be a sequence of independence random variables taking values 1, 2,,b with probabilities p 1, p 2,, p b. If W ijk and W ij are, respectively the number of times the triple i, j, k and the double i, j appear in a circular sequence of n v s, then Q 3 Q 2 = 12

29 2 ( ijk U ijk ) / U ijk ( Wij W U ) / U 2 ijk ij ij ij is asymptotically chi-square distributed with b 3 - b 2 degrees of freedom, and is equivalent to the likelihood-ratio test for jointly normal variables with covariance matrix that of the b 3 random variables W ijk. For 4- tuples, the quadratic from Q 4 Q 3 should be chi-square with b 4 b 3 degrees of freedom, 2 where Q 4 is the ( OBS EXP) / EXP value one would get by naive application of Pearson s test to the overlapping 4-tuple counts. The test results of mtuple are based on overlapping 4-tuples for bits 1, 2, 3 then for 2, 3, 4, then for 3, 4, 5 and so on, for each successive three bits of the computer words produced by the RNG being tested. OPSO (Overlapping Pairs Sparse Occupancy) Test Marsaglia (1984) proposed taking n overlapping pairs of successive U (0, 1) variates. For each pair, juxtapose the bits r+1 to r+b of the binary expansion of each variate, to obtain a 2b-bit number. Then compute the number of distinct 2b-bit numbers thus obtained and compare the results with the theoretical expectation. We expect the count is close to normally distributed with mean 141,909 and standard deviation 290. This is called the Overlapping-Pairs-Sparse-Occupancy test. For this project we used n = 2 21, r = 0, and b = 10. (See also Marsaglia, 1996.) 13

30 Chapter 4: Results For the mtuple test, suppose there are base-8 digits produced by the first 3 bits of calls to a RNG (See G. Marsaglia, 1985). Marsaglia defines four quadratic forms Q 1, Q 2, Q 3, Q 4 such that Q 2 Q 1, Q 3 Q 2 and Q 4 Q 3 have (approximate) χ 2 distributions if the values are independent and identically distributed uniform (0, 1) variables. These three χ 2 variables test both independence and uniformity in dimensions 2 through 4. These χ 2 variables should have degrees of freedom 56, 448 and 3584 respectively. These three differences have a chi-square distribution with the above degrees of freedom. A Kolmogorov-Smirnov (KS) goodness-of-fit test for all twenty bit sets of each dimension for each generator was used to test the assumption that the statistic appears to be chi-square distributed if the p-value for a test is not significant (> 0.05). In other word the values produced by the generator appear to be uniformly and independently distributed in these four dimensions. By this, it cannot be proved to be good, but it appears to be Okay. The KS p-values for each of the 32 generators are shown in tables. The label p-value 1 is the difference between dimensions one and two; the label p-value 2 is the difference between dimensions two and three; and the label p-value 3 is the difference between dimensions three and four. QQ-plots are used to summarize the outcomes of the test statistics for each set of bits for each generator. Three plots allow the comparison of the three differences of dimensions (Chi1, Chi2 and Chi3) in a plane graph. If the points follow a straight line in the graph, then a chi-square is an acceptably assumption, if the p-value is also non-significant. All the plots in chapter 4 represent 5000 tests combined over all bit combinations. All the Fortran code is provided in Appendix A and the R code is provided in Appendix B, while all the graphs for all bits sets for each generator (each plot is 250 tests for each bit set) are showed on Appendix C. The OPSO test takes the first 10 bits from each of 2 21 calls to each generator to produce a circular list of 2 21 integers in the range 1 to Successive overlapping pairs are placed in the appropriate cells of a 1024 x 1024 grid, and then the numbers of empty cells are counted. Subtracting the mean, µ = 141,909, and dividing by σ = 290 produce what should be a standard normal variate value (See G. Marsaglia, 1985). This test 14

31 yielded four z-score values for each generator. These values are satisfactory if they are between -2 and 2. The Fortran code for OPSO is given in Appendix A. 15

32 4.1 Linear Congruential Generator G 1 : X i = *X i-1 mod mtuple results: Table shows that G 1 has acceptable performance(i.e. p- value>0.05) in mtuple χ 2 test for each set of three bits from bits 1-3 till bits except those in shadows. These sixty p-values can be visualized by sixty graphs in Appendix C. Table Generator G 1 mtuple p-values for KS test of whether values are χ 2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) p-value 1 p-value 2 p-value 3 for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits graphs: These three QQ-plots show that G 1 produces roughly chi-square distributed values for each of the three differences of dimensions (Chi1, Chi2 and Chi3). They agree with the p-values in the mtuple test. For graphs of each bit set for this generator, refer to Appendix C. 16

33 Figure 4.1 Chi-square QQ-plots for Generator 1 OPSO results: Table shows the z-scores and the number of empty cells G 1 yielded for each of the four runs of OPSO. It seems acceptable since all z-scores are between -2 and 2. Table Z-scores for four repetition OPSO tests of Generator 1 LCG 1 zero cells z-score= LCG 1 zero cells z-score= LCG 1 zero cells z-score= LCG 1 zero cells z-score=

34 4.2 Combined Generator G 2 : X i = *X i-1 mod , is combined with X i = *X i-1 mod mtuple results: Table shows that G 2 has acceptable performance(i.e. p- value>0.05) in mtuple χ 2 test for each set of three bits from bits 1-3 till bits except those in shadows. These sixty p-values can be visualized by sixty graphs in Appendix C. Table Generator G 2 mtuple p-values for KS test of whether values are χ 2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) p-value 1 p-value 2 p-value 3 for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits graphs: These three QQ-plots show that G 2 produces roughly chi-square distributed values for each of the three differences of dimensions (Chi1, Chi2 and Chi3). They agree with the p-values in the mtuple test. For graphs of each bit set for this generator, refer to Appendix C. 18

35 Figure 4.2 Chi-square QQ-plots for Generator 2 OPSO results: Table shows the z-scores and the number of empty cells G 2 yielded for each of the four runs of OPSO. It seems acceptable since all z-scores are between -2 and 2. Table Z-scores for four repetition OPSO tests of Generator 2 Combined 1 zero cells z-score= Combined 1 zero cells z-score= Combined 1 zero cells z-score= Combined 1 zero cells z-score=

36 4.3 Primitive trinomial Generalized Feedback Shift Generators G 3 -G Generator G 3 based on 1 + x 3 + x 31 mtuple results: Table shows that G 3 has bad performance(i.e. most p- values<0.05) in mtuple χ 2 test for each set of three bits from bits 1-3 till bits except those not in shadows. These sixty p-values can be visualized by sixty graphs in Appendix C. Table Generator G 3 mtuple p-values for KS test of whether values are χ 2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) p-value 1 p-value 2 p-value 3 for bits E E E-07 for bits E E E-03 for bits E E E-02 for bits E E E-03 for bits E E E-04 for bits E E E-04 for bits E E E-04 for bits E E E-02 for bits E E E-02 for bits E E E-02 for bits E E E-03 for bits E E E-02 for bits E E E-03 for bits E E E-02 for bits E E E-02 for bits E E E-06 for bits E E E+00 for bits E E E+00 for bits E E E+00 for bits E E E+00 graphs: These three QQ-plots show that G 4 does not produce chi-square distributed values for each of the three differences of dimensions (Chi1, Chi2 and Chi3). The points don t follow a straight line. For graphs of each bit set for this generator, refer to Appendix C. 20

37 Figure 4.3 Chi-square QQ-plots for Generator 3 OPSO results: Table shows the z-scores and the number of empty cells G 3 yielded for each of the four runs of OPSO. It seems acceptable since all z-scores are between -2 and 2. Table Z-scores for four repetition OPSO tests of Generator 3 Trinomioal 1 zero cells z-score= Trinomioal 1 zero cells z-score= Trinomioal 1 zero cells z-score= Trinomioal 1 zero cells z-score=

38 4.3.2 Generator G 4 based on 1 + x 6 + x 31 mtuple results: Table shows that G 4 has acceptable performance(i.e. p- value>0.05) in mtuple χ 2 test for each set of three bits from bits 1-3 till bits except those in shadow. The number of small p-values in the p-value 1 column indicates this generator has some problems. These sixty p-values can be visualized by sixty graphs in Appendix C. Table Generator G 4 mtuple p-values for KS test of whether values are χ 2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) p-value 1 p-value 2 p-value 3 for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits graphs: These three QQ-plots show that G 4 produces roughly chi-square distributed values for each of the three differences of dimensions (Chi1, Chi2 and Chi3). They agree with the p-values in the mtuple test. For graphs of each bit set for this generator, refer to Appendix C. 22

39 Figure 4.4 Chi-square QQ-plots for Generator 4 OPSO results: Table shows the z-scores and the number of empty cells G 4 yielded for each of the four runs of OPSO. It seems acceptable since all z-scores are between -2 and 2. Table Z-scores for four repetition OPSO tests of Generator 4 Trinomioal 2 zero cells z-score= Trinomioal 2 zero cells z-score= Trinomioal 2 zero cells z-score= Trinomioal 2 zero cells z-score=

40 4.3.3 Generator G 5 based on 1 + x 7 + x 31 mtuple results: Table shows that G 5 has bad performance(i.e. most p- value<0.05) in mtuple χ 2 test for each set of three bits from bits 1-3 till bits except those not in shadows. These sixty p-values can be visualized by sixty graphs in Appendix C. Table Generator G 5 mtuple p-values for KS test of whether values are χ 2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) p-value 1 p-value 2 p-value 3 for bits E E E-11 for bits E E E-12 for bits E E E-13 for bits E E E-16 for bits E E E-15 for bits E E E-15 for bits E E E-10 for bits E E E-08 for bits E E E-09 for bits E E E-12 for bits E E E-12 for bits E E E-13 for bits E E E-13 for bits E E E-09 for bits E E E-07 for bits E E E-08 for bits E E E-09 for bits E E E-05 for bits E E E-03 for bits E E E-02 graphs: These three QQ-plots show that G 5 does not produce chi-square distributed values for each of the three differences of dimensions (Chi1, Chi2 and Chi3). The points don t follow a straight line. For graphs of each bit set for this generator, refer to Appendix C. 24

41 Figure 4.5 Chi-square QQ-plots for Generator 5 OPSO results: Table shows the z-scores and the number of empty cells G 5 yielded for each of the four runs of OPSO. It seems not so good since there are three out of -2 <z< 2. Table Z-scores for four repetition OPSO tests of Generator 5 Trinomioal 3 zero cells z-score= Trinomioal 3 zero cells z-score= Trinomioal 3 zero cells z-score= Trinomioal 3 zero cells z-score=

42 4.3.4 Generator G 6 based on 1 + x 13 + x 31 mtuple results: Table shows that G 6 has acceptable performance(i.e. p- value>0.05) in mtuple χ 2 test for each set of three bits from bits 1-3 till bits except those in shadows. The number of small p-values in the p-value 2 column and p-value 3 column indicates this generator has some problems. These sixty p- values can be visualized by sixty graphs in Appendix C. Table Generator G 6 mtuple p-values for KS test of whether values are χ 2 distributed (p-value i compares dimension i with dimension i+1, i = 1, 2, 3.) p-value 1 p-value 2 p-value 3 for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits for bits \ Graphs: These three QQ-plots show that G 6 produces roughly chi-square distributed values for a subset of the test for each of the three differences of dimensions (Chi1, Chi2 and Chi3), the pattern is broken for Chi1 and Chi2. For graphs of each bit set for this generator, refer to Appendix C. 26