Section 6.5: Random Sampling and Shuffling

Transcription

1 Section 6.5: Random Sampling and Shuffling Discrete-Event Simulation: A First Course c 2006 Pearson Ed., Inc Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 1/ 29

2 Section 6.5: Random Sampling and Shuffling Exactly m! different permutations (a 0,a 1,...,a m 1 ) can be formed from a finite set A with m = A distinct elements A random shuffle generator is an algorithm that will produce any one of these m! permutations in such a way that all are equally likely Example If A = {0, 1, 2, the 3! = 6 different possible permutations of A are (0,1,2) (0,2,1) (1,0,2) (1,2,0) (2,0,1) (2,1,0) A random shuffle generator can produce any of six possible permutations with equal probability 1/6 Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 2/ 29

3 Algorithm The intuitive way to generate a random shuffle of A is: draw the first element a 0 at random from A draw the second element a 1 at random from A {a 0 draw the third a 2 at random from A {a 0, a 1, etc. An in place algorithm: Algorithm for(i = 0; i < m - 1; i++){ j = Equilikely(i, m - 1); hold = a[j]; a[j] = a[i]; /* swap a[i] and a[j]*/ a[i] = hold; The algorithm assumes A is stored in array a Algorithm is an excellent example of the elegance of simplicity. Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 3/ 29

4 Algorithm ctd prior to a[0], a[3] swap i j prior to a[1], a[6] swap i j prior to a[2], a[4] swap i j The above figure(corresponding to m = 9) illustrates i, j, and the state of a[ ] for the first three loop iterations The algorithm is ideal for shuffling a deck of cards a useful simulation skill Later in the section, a minor modification to this algorithm makes it suitable for random sampling without replacement Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 4/ 29

5 Random Sampling A population P of m = P elements a 0,a 1,...,a m 1 If m is not large, the population could be stored in primary memory As an array a[0], a[1],..., a[m 1] As a linked list If m is large, the population may be stored in secondary memory In any case, the population is (conceptually) a list Want a random sample S of n = S elements x 0,x 1,...,x n 1 from list P If n is small, the sample can be stored in primary memory as x[0], x[1],..., x[n 1] Sometimes a linked list could be used Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 5/ 29

6 Random Sampling: Notation and Terminology (1) Generic algorithms use functions Get(&z, L,j) and Put(z, L,j) L is either P or S Get(&z, L, j) returns the value of the j th element in the list L as z Put(z, L, j) assigns the value of z to the j th element in the list L Order preservation: sample order is consistent with population order Important for some applications Sequential sampling algorithms: Traverse P once, in order Access to S may or may not be sequential Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 6/ 29

7 Random Sampling: Notation and Terminology (2) Sampling with replacement: S may contain multiple instances of one element of P n can be larger than m Sampling without replacement: S contains at most one instance of each element of P n m The phrase at random can be interpreted in two ways: 1 Each element of P is equally likely to be an element of S 2 Each possible sample of size n is equally likely to be selected Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 7/ 29

8 Algorithm Algorithm for(i = 0; i < n; i++){ j = Equilikely(0, m - 1); Get(&z, P, j); /* random access */ Put(z, S, i); Complexity is O(n) Algorithm is non-sequential and m must be known Sampling is with replacement: n can be larger than m Order is not preserved The number of possible samples is m n (if elements of P are distinct) All samples are equally likely to be generated Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 8/ 29

9 Example Suppose The population is stored as array a[0], a[1],..., a[m 1] The sample is stored as array x[0],x[1],...,x[n 1] Then, Algorithm is equivalent to Example for(i = 0; i < n; i++){ j = Equilikely(0, m - 1); x[i] = a[j]; Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 9/ 29

10 Algorithm Algorithm for(i = 0; i < n; i++){ j = Equilikely(i, m - 1); /* i not 1*/ Get(&z, P, j); /* random access */ Put(z, S, i); Get(&x, P, i); /* sequential access */ Put(z, P, i); /* sequential access */ Put(x, P, j); /* random access */ O(n) complexity; non-sequential; m must be known Sampling is without replacement: n m Order is not preserved in S; order is destroyed in P by shuffling Number of possible samples: m(m 1) (m n + 1) = m!/(m n)! All samples are equally likely to be generated Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 10/ 29

11 Example Suppose the population and sample are stored as arrays a[0],a[1],...,a[m 1] and x[0],x[1],...,x[n 1] respectively Algorithm is equivalent to Example for(i = 0; i < n; i++){ j = Equilikely(i, m - 1); x[i] = a[j]; a[j] = a[i]; a[i] = x[i]; Algorithm is a simple extension of Algorithm Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 11/ 29

12 Sequential Sampling (without replacement) Basic idea: traverse P once, in order, and select elements to put in S Algorithms are O(m) The selection or non-selection of elements is random with probability p as illustrated Sequential Sampling Get(&z, P, j); if (Bernoulli(p)) Put(z, S, i) For algorithm 6.5.4, p is independent of i and j For algorithms and 6.5.6, the probability of selection changes based on the values of i, j and the number of elements selected Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 12/ 29

13 Algorithm Algorithm i = 0; j = 0; while( more data in P){ Get(&z, P, j); /* sequential access */ j++; if (Bernoulli(p)){ Put(z, S, i); i++; Order is preserved Each element of P is selected, independently, with probability p Algorithm does not make use of either m or n explicitly m may be unknown Sample size n is a Binomial(m, p) random variate Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 13/ 29

14 Example The population is stored in secondary memory as a sequential file and the sample is stored as array x[0],x[1],...,x[n 1] Algorithm is equivalent to Example i = 0; while( more data in P){ z = GetData(); if (Bernoulli(p)){ x[i] = z; i++; Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 14/ 29

15 Applications Discrete event simulation Real time data acquisition Objective is to sample, at random, some percentage (say, 1%) of the population Algorithm with p = 0.01 would provide the ability but cannot specify the sample size n exactly Algorithm provides the ability, and can specify n exactly, provided m is known Algorithm provides the ability, can specify n, even if m is unknown Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 15/ 29

16 Algorithm Algorithm i = 0; j = 0; while(i < n){ Get(&z, P, j); /* sequential access */ p = (n - i) / (m - j); j++; if (Bernoulli(p)){ Put(z, S, i); i++; m = P is known. Order is preserved. The key is that a j is selected with a probability (n i)/(m j) Number of possible samples is m!/(m n)!n! All samples are equally likely to be generated Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 16/ 29

17 Example The population is stored in secondary memory as a sequential file and the sample is stored as array x[0],x[1],...,x[n 1] Algorithm is equivalent to Example i = 0; j = 0; while( i < n){ z = GetData(); p = (n - i) / (m - j); j++; if (Bernoulli(p)){ x[i] = z; i++; Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 17/ 29

18 Algorithm Sequential random sampling without replacement, unknown m Algorithm for (i = 0; i < n; i++){ Get(&z, P, i); /* sequential access */ Put(z, S, i); j = n; while (more data in P){ Get(&z, P, j); /* sequential access */ j++; p = n / j; if (Bernoulli(p)){ i = Equilikely(0, n -1); Put(z, S, i); Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 18/ 29

19 Explanation of Algorithm The sample is initialized with the first n elements of the population For j n, population element a j overwrites an existing sample element with probability n/(j + 1) Access to P is sequential Access to S is not sequential Thus, the algorithm is not order preserving The number of possible samples is m!/(m n)!n! All samples are equally likely to be generated (with a caveat...) Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 19/ 29

20 Example The population is stored in secondary memory as a sequential file and the sample is stored as array x[0], x[1],..., x[n 1] Algorithm is equivalent to Example for (i = 0; i < n; i++){ z = GetData(); x[i] = z; j = n; while (more data in P){ z = GetData(); j++; p = n / j; if (Bernoulli(p)){ i = Equilikely(0, n -1); x[i] = z; Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 20/ 29

21 Algorithm Differences Algorithm Requires knowledge of m = P Preserves order Algorithm Does not require knowledge of m = P Does not preserve order Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 21/ 29

22 Example P = (0,1,2,3,4); m = 5 Use algorithm to select samples of size n = 3 Sampling is sequential and order is preserved There are 5!/2!3! = 10 possible (ordered) samples Algorithm generates possible samples (0,1, 2)(0, 1,3)(0, 1,4)(0, 2, 3)(0,2, 4)(0,3, 4)(1, 2,3)(1, 2,4)(1, 3,4)(2, 3, 4) each with equal probability 0.10 Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 22/ 29

23 Example If Algorithm is used, 13 samples are possible (0,3, 4) 1,4) (3,4, 2) (0,1, 2)(0, 1,3)(0, 1,4)(0, 3, 2)(0,4, 2) (3, 1,2)(4, 1,2)(3, (0,4, 3) (4, 1,3) (4,3, 2) Note order is not preserved. Each of these samples are not equally likely Algorithm produces the correct number of equally likely samples only if all alike-but-for-permutation samples are combined Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 23/ 29

24 URN Models Four examples with models: Binomial(n, p) Hypergeometric(n, a, b) Geometric(p) Pascal(n, p) All of these models can be motivated by drawing, at random, from a conceptual urn initially filled with a amber balls and b black balls Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 24/ 29

25 Binomial n balls are drawn from the urn, with replacement Let x be the number of amber balls drawn A straightforward Monte Carlo simulation of this random experiment: Use algorithm Population is the urn; sample is the drawn balls x is number of amber balls in sample More elegant: Let p = a/(a + b) and use the O(n) algorithm Binomial Urn x = 0; for (i = 0; i < n; i++) x +=Bernoulli(p); return x; Random variate x is Binomial(n,p) Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 25/ 29

26 Hypergeometric n balls are drawn from the urn, without replacement Use a modified version of previous algorithm Hypergeometric Urn m = a + b; x = 0; for (i = 0; i < n; i++){ p = (a - x) / (m - i); x += Bernoulli(p); return x; Random variate x is Hypergeometric(n,a,b) Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 26/ 29

27 More Hypergeometric The Hypergeometric(n, a, b) pdf is ( )( ) a b f (x) = x n x ( ) x = max(0,n b),...,min(a,n) a + b n Lower limit of x: if n > b, at least n b amber balls will be drawn Upper limit of x: if n > a, then at most a amber balls will be drawn In applications where n is smaller than both a and b, the range of possible values is x = 0,1,2,...,n Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 27/ 29

28 Geometric Draw with replacement until the first black ball is obtained Let x be the number of amber balls drawn With p = a/(a + b), the following algorithm simulates this random experiment Geometric Urn x = 0; while(bernoulli(p)){ x++; return x; Random variate x is Geometric(p) This stochastic model is commonly used in reliability studies: p is usually close to 1.0 and x counts the number of successes before the first failure Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 28/ 29

29 Pascal Draw with replacement until the n th black ball is obtained Let x be the number of amber balls drawn With p = a/(a + b), the following algorithm simulates this random experiment Pascal Urn x = 0; for(i = 0; i < n; i++){ while(bernoulli(p)) x++; return x; Random variate x is Pascal(n,p) This stochastic model is commonly used in reliability studies: p is usually close to 1.0 and x counts the number of successes before the n th failure Discrete-Event Simulation: A First Course Section 6.5: Random Sampling and Shuffling 29/ 29