CSCE GPU PROJECT PRESENTATION SCALABLE PARALLEL RANDOM NUMBER GENERATION

Transcription

1 CSCE GPU PROJECT PRESENTATION SCALABLE PARALLEL RANDOM NUMBER GENERATION Farhad Parsan 10/25/2010

2 OUTLINE Introduction Design Performance Live Demonstration Conclusion 2

3 INTRODUCTION Scalable Parallel Random Number Generation (SPRNG) application Monte Carlo Simulation studying physical systems with a large degrees of freedom modeling phenomena with significant uncertainty in inputs 3

4 INTRODUCTION SPRNG library methods[1] 48 bit Linear Congruential 64 bit Linear Congruential Additive Lagged Fibonacci Multiplicative Lagged Fibonacci Combined Multiple Recursive Prime Modulus Linear Congruential 4

5 INTRODUCTION 64 bit Linear Congruential Generator (LCG) X(n) = A * X(n - 1) + C (Mod 2^64) Additive Lagged Fibonacci Generator (LFG) X(n) = X(n - k) + X(n - l) (Mod 2^64) 5

6 INTRODUCTION 64 bit Linear Congruential Generator (LCG) X(n) = A * X(n - 1) + C (Mod 2^64) A = C = Additive Lagged Fibonacci Generator (LFG) X(n) = X(n - k) + X(n - l) (Mod 2^64) k = 1279 l = 861 6

7 INTRODUCTION How to parallelize? There are 4 main techniques[2]: Leapfrog Sequence Splitting Independent Sequences Parameterization of the Generator 7

8 INTRODUCTION Leapfrog X P, X P+N, X P+2N, Sequence Splitting X PL, X PL+1, X PL+2, Independent Sequences using different seeds Parameterization of the Generator using different parameters 8

9 DESIGN LFG Seed initialization[3] 9

10 DESIGN LFG Seed initialization m = 64, k = 1279, l = 861 b 0,233 = 1 10

11 DESIGN LFG Seed initialization 11

12 DESIGN LCG Leapfrog method X 1 = ax 0 + c X 2 = a(ax 0 + c) + c = a 2 X 0 + c(a+1) X 3 = a(a 2 X 0 + c(a+1)) + c = a 3 X 0 + c(a 2 +a+1).. X N = a N X 0 + c(a N-1 + +a 2 +a+1) 12

13 DESIGN LCG Leapfrog method X 1 = ax 0 + c X 2 = a(ax 0 + c) + c = a 2 X 0 + c(a+1) X 3 = a(a 2 X 0 + c(a+1)) + c = a 3 X 0 + c(a 2 +a+1).. X N = a N X 0 + c(a N-1 + +a 2 +a+1) X(n) = A * X(n - 1) + C (Mod 2^64) A = a N, C = c(a N-1 + +a 2 +a+1) 13

14 DESIGN LCG Leapfrog method X 0 X 1 X 2 X N-2 X N-1 14

15 DESIGN LCG Leapfrog method X 0 X 1 X 2 X N-2 X N-1 X N X N+1 X N+2 X 2N-2 X 2N-1 15

16 DESIGN LCG Leapfrog method X 0 X 1 X 2 X N-2 X N-1 X N X N+1 X N+2 X 2N-2 X 2N-1 X 3N X 3N+1 X 3N+2 X 4N-2 X 4N-1 16

17 DESIGN LCG initialization Generating initial seeds Calculating new A, C X 0 X 1 X 2 X N-2 X N-1 X N X N+1 X N+2 X 2N-2 X 2N-1 X 3N X 3N+1 X 3N+2 X 4N-2 X 4N-1 17

18 DESIGN LCG initialization Generating initial seeds Calculating new A, C A = a N, C = c(a N-1 + +a 2 +a+1) pow() cannot be used because a is unsigned long a and N both are large numbers thus a N exeeds unsigned long range! 18

19 DESIGN LCG initialization Generating initial seeds Calculating new A, C A = a N, C = c(a N-1 + +a 2 +a+1) X(n) = A * X(n - 1) + C (Mod 2^64) If a 1 = b 1 (mod m) and a 2 = b 2 (mod m) and a 1, b 1, a 2, b 2 and m are all integers then: a 1 *a 2 = b 1 *b 2 (mod m) 19

20 DESIGN LCG initialization Creating initial seeds Calculating new A, C 20

21 DESIGN Design specifications The generated random numbers are consumed immediately In LCG method all random numbers are stored in shared memory to eliminate global memory access In LFG method there is not enough space to store random numbers in shared memory so they are stored in global memory 21

22 DESIGN LFG Kernel 22

23 DESIGN LCG Kernel 23

24 PERFORMANCE Reported reference performance Platform Number of Blocks Number of Generators Number of RNs Rate (MPS) LFG LCG e Fermi e CPU - 1 1e e Fermi e CPU - 1 1e9 331 LFG speedup : 28X LCG speedup : 98X 24

25 PERFORMANCE Achieved performance Platform Number of Blocks Number of Generators Number of RNs Rate (MPS) LFG LCG e Fermi e CPU - 1 1e e Fermi e CPU - 1 1e9 325 LFG speedup : 15X? 32X LCG speedup : 125X 25

26 PERFORMANCE CONSIDERATIONS LFG seed memory location pattern X 0,0 X 0,1 X 0,2 X 1,0 X 1,1 X 1,2 X 0,0 X 1,0 X 2,0 X 0,1 X 1,1 X 2,1 Proper register usage to decrease global memory access and redundant operations 26

27 PERFORMANCE CONSIDERATIONS LFG seed generation loop change (1.7X speedup) LCG shared memory usage (2.2X speedup) 27

28 LIVE DEMONSTRATION 28

29 CONCLUSION Two methods of parallel random number generation were discussed: Lagged Fibonacci Generator (LFG) Linear Congruential Generator (LCG) To parallelize the two methods using GPU some parallelization techniques were introduced: Leapfrog to parallelize LCG Independent Sequences to parallelize LFG By correct usage of GPU memory and computation resources it is possible to get 15x speedup over CPU in LFG method and 125x speedup in LCG method The achieved performance surpasses the performance reported in [4] Sufficient amount of shared memory could increase performance in LFG method remarkably by eliminating global memory access. 29

30 REFERENCES [1] Scalable Parallel Pseudo Random Number Generators Library, [2] P.D. Coddington, and A.J. Newell, JAPARA - A Java Parallel Random Number Generator Library for High-Performance Computing, Proc. of Java in Parallel and Distributed Processing Symposium (IPDPS'04), Santa Fe, April 2004 [3] Pryor, D. V., Cuccaro, S. A., Mascagni, M., and Robinson, M. L., Implementation of a portable and reproducible parallel pseudorandom number generator, In Proceedings of the 1994 Conference on Supercomputing,Washington, D.C., 1994 [4] S. Gao and G. D. Peterson, GPU accelerated scalable parallel random number generators, in Proc Symposium on Application Accelerators in High Performance Computing (SAAHPC 10), Jul