Testing unrolling optimization technique for quasi random numbers

Transcription

1 Testing unrolling optimization technique for quasi random numbers Romain Reuillon David R.C. Hill LIMOS, UMR CNRS 6158 LIMOS, UMR CNRS 6158 Blaise Pascal University Blaise Pascal University ISIMA, Campus des Cézeaux, BP ISIMA, Campus des Cézeaux, BP Aubière Cedex, France Aubière Cedex, France ABSTRACT Quasi Monte Carlo simulations are built on quasi random number generator (QRNG). Even if the use of a QRNG might lead to a faster convergence, for example with Monte Carlo integral computation, most of the simulation time can be taken by the number generation part. This paper present an optimization technique, called unrolling applied to QRNG. It consists in the generation and storage of random numbers in a compiled binary object file. After that, the library is used at the execution time by the simulator. We obtained good gains with Unrolling technique and QRNG, allowing a minimum gain factor of 4 on different computer architectures compared to the Sobol algorithm. KEYWORDS Quasi random numbers, unrolling, optimization, stochastic, simulation 1. Introduction Monte Carlo simulations are stochastic simulations and are always based on a Random Number Generator (RNG). Running the RNG constitutes often a non negligible part of the simulation time. The Unrolling technique is a way to optimize the access to random numbers. This method consists in a pre-generation and storage of random numbers in an array of values. This array is directly included into the binary code during the compilation process. The convoluted RNG algorithm is so bypassed and the numbers are directly picked in sequence in the array. The Unrolling optimization is fully portable and is usable with generators written in any language. This method has been proved to be pretty efficient with small series of pseudo random numbers [Hill 2003]. Another type of generator is very interesting for stochastic simulations : Quasi Random Number Generators (QRNG). QRNG gain time by improving the convergence in numerous simulations [Papageorgiou 2001]. Pseudo-random numbers are well known and some good definitions were known since 1951 (see [Lelhmer 1951] ). Quasi random numbers are a bit different and are also known as low discrepancy sequences, where the discrepancy is considered at the distance to perfect uniformity. This main characteristic helps in fastening convergence and with the lack of replication, quasi Monte Carlo simulations are naturally inclined to be unrolled in arrays stored in random access memory. Since the Van der Corput Sequence in 1935, many other sequences have been studied. The Halton sequence helped in obtaining results for various dimensions [Halton 1960]. In addition to these pioneering quasi-random sequence, the other best known sequences were given in 1960 by Hammersley [Hammersley 1960], in 1967 by Sobol [Sobol 1967, 1976], in 1980 by Faure in his thesis (see [Faure 1993] for details and in 1987 Niederreiter (see [Niederreter 1992]). In this paper we have mainly tested the unrolling technique on the GNU Scientific Library implementation of the Sobol algorithm. This is the fastest known implementation of the Sobol algorithm [Antonov and Saleev 1979], the latter being itself recognized as very interesting to fasten convergence. This implementation is valid up to 40 dimensions but is limited to 1 million drawings. Blaise Pascal University, France 355 OICMS 2005

2 2. Convergence Due to their good covering in N dimensional spaces, quasi random number generators are pretty efficient for integral computation [Jank 2005]. The PI computation is a good example to illustrate the astonish gain in term of convergence speed. In figure 1, QnD, ShNR and MT stand for three different types of Pseudo Random Number Generators (PRNG). They are respectively the Quick and Dirty [Numerical Recipes 1992], a linear congruential generator. The shuffled version of ran2 generator described in Numerical Recipes [Numerical Recipes 1992] and the Mersenne Twister generator of Makoto Matsumoto and Takuji Nishimura [Matsumoto and Nishimura 1997]. We have chosen the quick and dirty because it is a very fast but not very good generator. The two other ones are a lot slower but they provide a pretty good randomness quality. They both have passed the wall DIE HARD battery of tests from George Marsaglia [Marsaglia 1995]. More of that, the Mersenne Twister has a good space covering in 623 dimensions [Matsumoto and Nishimura 1997]. Figure 1 shows clearly that the QRNG allows a quick convergence in comparison to PRNG. It reaches, with only a few hundred million numbers, a precision of 10-7 that any of the PRNG never obtains even with two trillion generated numbers. 0,00E+00 5,00E+08 1,00E+09 1,50E+09 2,00E ,1 Generated numbers 0,01 0,001 0,0001 1E-05 1E-06 1E-07 1E-08 1E-09 QRNG QnD MT ShNR 1E-10 Variation to PI Figure 1 : Convergence comparison 3. Computing Effort QRNGs increase the convergence for integral calculation doesn t necessarily mean that it reduces the computation time. We need to take into account the calculation effort taken to generate low discrepancy sequences and compare it with the one taken to generate pseudo random numbers. The results of figure 2 have been obtained using an Intel Xeon 64 bits 3 GHz with 800 MHz internal bus and with DDR2 400 RAM. Two QRNG have been tested the Sobol [Sobol 1976] and the Neiderreiter [Neiderreiter, Bratley and Fox 1992]. The test consists in measuring the time needed to draw a million numbers with the different generators. Quasi random number generation is not way more resource consuming compared to pseudo random number generation. On our architecture the Sobol QRNG is as fast as the Mesrsenne Twister witch is reputed to be fast for a good PRNG. Blaise Pascal University, France 356 OICMS 2005

3 4. Optimization Figure 2 : Generation time in millisecond for 10 6 numbers on an Intel Xeon 3 GHz The unrolling technique is an optimization method for number generators [Hill 2003]. It consists in calculating in advance a sequence of number and storing it in a file formatted as a static array of a targeted language source code. Next step is to compile this array of numbers to obtain a binary object file, ready to be linked. Finally, the end-user code is linked to the binary file and directly access to the array instead of using a generator. The computation time is so decreased by bypassing the complex calculation of each value by the generator. The full Unrolling optimization process is described in the figure 3. The generation of the optimized generator source code can be done in any target language, independently from the generator language. In order to evaluate the performances of the unrolling method applied to QRNG, we have written several programs. The tests we achieved were done with the C language but the technique is fully portable. To generate the source code with static arrays of random numbers, we have developed a program: QRNGUnroll. This program is able to unroll quasi random number sequences up to 40 dimensions using the Sobol algorithm. The program generates two source files: a header (UQRNG_1.h) and its corresponding implementation (UQRNG_1.c) of the static array of random numbers (the _1 at the end of file names means one dimension). Both files are ready for separate compilation and this compilation is done automatically by the QRNGUnroll program. The result of the separate compilation (UQRNG_1.o) is then linked with the tests or application programs whose source code only includes the header file UQRNG_1.h containing: the declaration of the static array and the macroinstruction designed for accessing quasi random numbers (see appendix). Blaise Pascal University, France 357 OICMS 2005

4 Figure 3: Unrolling optimization process Due to the obligation to compile a big array, the unrolling technique is generally applicable for short and middle size series of number, depending on the amount of memory we own. According to figure 4, a source file containing 5 millions numbers, need 2 Go of random access memory to be compiled with gcc 3.4. Size of the sequence Size of the source code RAM needed to compile / compilation time Size of the binary executable ,3 Mo 24 Mo / 0,6 s 392 Ko Mo 210 Mo / 6 s 3,9 Mo Mo 408 Mo / 12,5 s 7,7 Mo Mo 1,15 Go / half an hour 12 Mo Mo 1,95 Go / several hours 20 Mo Figure 4: Unrolling and Memory Space Occupancy of the Compilation Process Time given on an Intel Xeon 64 bits 3 GHz with 800 MHz internal bus and with DDR2 400 RAM running a Linux Fedora Core 3. For some simulations, using QRNG allows a fast convergence and by consequence requires short sequences, short enough for the unrolling technique to be usable. Furthermore, the quasi Monte Carlo simulations don t imply the use of replications and thus it avoids the use of several uncorrelated sequences. With the unrolling technique we can compile, once for all, a binary object file containing a huge sequence of quasi random numbers. For the two reasons above, we can consider unrolling optimization technique as particularly well adapted for quasi random generation. On the figure 5 the computing time for drawing a ten million numbers with several generators have been measured on different recent computer architectures and compared with an unrolled generator. The precise time is mentioned in millisecond under the graphical view. Each program has been compiled with the GNU C Compiler (gcc 3.4), and with the maximum optimisation level (option O3). Blaise Pascal University, France 358 OICMS 2005

5 Generation Time (ms) Athlon P4 2,8GHz Xeon 3 GHz Sobol Shuffled NR Quick and Dirty Unrolled Figure 5: Generation time of 10 6 numbers Four generators have been tested: the Sobol QRNG, the Shuffled Ran2 and the Quick and Dirty of Numerical Recipes and an unrolled generator. The computer architectures used for the tests are, an AMD Athlon at 900 MHz, a Intel Pentium 4 at 2,8 GHz and an Intel Xeon 64 bits at 3 GHz. The results are pretty convincing. The unrolled generator is always faster than the others. The unrolling technique allows a gain of a 5,5 factor for the Sobol QRNG, 15,8 factor for the Shuffled NR and 2 for the Quick and Dirty on the fastest and newest computer. The Pentium 4 is a bit slower in floating point calculation, that s why it obtains bigger gain factors. In the computer improvement process, the memory speed increase with the CPU efficiency, that why the gain factor tends to be pretty independent of the architecture of the computer. A factor above 5 for the Sobol algorithm is pretty decent and shows how the unrolling is a way of decreasing significantly the computation time of quasi random simulations. 5. Conclusion QRNG allow a fast convergence for integral calculation and thus they also reduce the global calculation time of simulations. In addition, QRNG are as fast as some statistically sound RNG. By decreasing by 5 the generation time for the Sobol algorithm in one dimension, the unrolling optimization technique we proposed, turned out to be pretty efficient on QRNG. The current implementation of the Sobol algorithm, in the GNU Scientific Library, only produces 1 million drawings up to 40 dimensions. The unrolling technique is interesting up to 5 millions drawings on a regular desktop computer. Moreover the unrolling optimization technique is particularly well adapted for Monte Carlo simulations since instead of storing less "efficient" (in terms of convergence) pseudo-random numbers, we can obtain a better precision with the same amount of stored quasi-random numbers. However, for multiple dimensions, QRNG will be harder to optimize with this technique and it is precisely to evaluate multidimensional integrals that such sequences are used at their best (see [Halton 1960] and [L'Ecuyer 2003]). Still dealing with the limits of the unrolling technique, a user has to be aware that it implies: the creation of huge source files, the compilation time can be very long if the compiler has to swap and of course the efficiency of this technique depends on the available Blaise Pascal University, France 359 OICMS 2005

6 RAM and of its access bandwidth. Another point is that the time needed to generate a 40 dimensions quasi randomized n- uple with the Sobol algorithm is only ten times as important as the one needed to generate a unique quasi-random number (for only 1 dimension); thus it is probable that very few gains will be obtained in this case. Solutions have to be explored to address these issues, our next focus will be given to the memory mapping optimization technique. REFERENCES [Antonov and Saleev 1980] I.A Antonov and V.M. Saleev, An economic method of computing LP τ - Sequences, USSR Comput. Maths. Math. Phys. 19 : 252, [Faure 1993] H. Faure, Suggestions for quasi-monte-carlo Users, Lecture Notes in Pure and Applied Math., Marcel Dekker, New York, 141 : , [Halton 1960] J.H. Halton, On the efficiency of certain quasi-monte Carlo sequences of points in evaluating multidimensional integrals, Nummer. Math. 2 : 84-90, [Hammersley 1963] J.M. Hammersley, Monte Carlo methods for solving multivariate problems, Annals of the New-York Academy of Science, 86 : , [Hill 2003] D. R.C. Hill, URNG: A portable optimization technique for software applications requiring pseudo-random numbers, Simulation Modelling Practice and Theory, 11-7,8 : , [Jank 2005] W. Jank: Quasi-Monte Carlo sampling to improve the efficiency of Monte Carlo EM, Computational Statistics & Data Analysis, 48-4 : , [Lehmer 1951] D.H. Lehmer, Good parameters and implementations for combined multiple recursive random number generators, Operation Research, 47-1 : , [L Ecuyer 2003] P. L'Ecuyer, Quasi-monte carlo methods in practice: quasi-monte carlo methods for simulation, Proceedings of the 35th conference on Winter simulation: driving innovation, 2003, pp [Marsaglia 1995] G. Marsaglia, DIEHARD Battery of Tests of Randomness, [Matsumoto and Nishimura 1997] M. Matsumoto and T. Nishimura: Mersenne Twister: A 623-dimensionally equidistributed uniform pseudorandom number generator, Proceedings of the 29th conference on Winter simulation, 1997, pp [Niederreiter 1992] H. Neiderreiter : "Random Number Generation and Quasi-Monte Carlo Methods" SIAM, CBMS 63, 1992, 241 pp. [Neiderreiter, Bratley and Fox 1992] H. Neiderreiter, P. Bratley and B. L. Fox: Implementation and tests of low-discrepancy sequences, ACM Trans. on Modeling and Computer Simulation, 2 : , 1992 [Numerical Recipes 1992] W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery: Numerical Recipes in C, pp , ISBN , [Papageorgiou 2001] A. Papageorgiou, Fast Convergence of Quasi-Monte Carlo for a Class of Isotropic Integrals, Mathematics of Computation, : , January [Sobol 1967] I. M. Sobol, On the distribution of points in a cube and the approximate evaluation of integrals, USSR Computational Mathematics and Mathematical Physics, 7-4 : , [Sobol 1976] I. M. Sobol, Uniformly distributed sequences with an additional uniform property, USSR Computational Mathematics and Mathematical Physics, 16-5 : , Blaise Pascal University, France 360 OICMS 2005

7 APPENDIX : FILES GENERATED BY THE QRNGUNROLL PROGRAM The following paragraphs show the skeleton of the files (header and source) generated by the QRNGUnroll program with an array of 10 random values with a 1 dimension Sobol QRNG. 1 Header file /************************************************************************** * UQRNGSob_1_10.h * * Unroling method for QRNG * **************************************************************************/ /* Increment the pointer of the size of N doubles, N is equal to the number of dimensions, and return it */ #define NEXT() ( cur += 1 ) #define GET_NEXT_QRN_AT_DIM(d) ( * ( cur + ( d ) ) ) float * cur; void inituqrng(); Fig. A.1: A header file generated by the QRNGUnroll program Blaise Pascal University, France 361 OICMS 2005

8 2 Source file /************************************************************************** * UQRNGSob_1_10.c * **************************************************************************/ #include "UQRNGSob_1_10.h" /* Array of doubles in hexadecimal form */ float tab [] = { 0x1p-1, 0x1.8p-1, 0x1p-2, 0x1.8p-2, 0x1.cp-1, 0x1.4p-1, 0x1p-3, 0x1.8p-3, 0x1.6p-1, 0x1.ep-1, }; /* Initialization of the UQRNG*/ void inituqrng() { cur = ( float * )( tab ); } Fig. A.2 : A source file generated by the QRNGUnroll program Blaise Pascal University, France 362 OICMS 2005