Monte Carlo Simulations

Transcription

1 Monte Carlo Simulations DESCRIPTION AND APPLICATION

2 Outline Introduction Description of Method Cost Estimating Example Other Considerations

3 Introduction Most interesting things are probabilistic (opinion) Not really probabilistic once you get really down in the weeds but we generally can t control everything so effectively probabilistic When analyzing problems we want to understand the outcome based on the inputs Range Mean Mode Full distribution, if possible

4 Gambling Example Rolling a (fair) 6 sided die The probability of landing on each side is the same What is the average value of a roll? The analytical average is EE xx = 6 xx=1 xx pp(xx) 6 = xx 1 xx=1 6 = 3.5

5 Gambling Example (cont d) We could easily test this by rolling die over and over again Set 1 6, 3, 2, 2, 5, 6, 2, 4, 3, 3 Avg = 3.6 Set 2 1, 1, 2, 1, 1, 2, 1, 4, 4, 1 Avg = 1.8 Set 3 5, 3, 5, 1, 6, 5, 5, 2, 5, 2 Avg = 3.9 For 1000 samples the Avg is 3.53 based on that sample If we know the how to find the answer, why are we doing this? What if there are two dice?

6 Sums of Random Variables The distribution of a sum of independent random variables can be found using a convolution Ex: XX with ff XX xx, YY with ff YY yy Then ZZ = X + Y has ff ZZ zz = ff gg zz = ffxx zz yy gg yy dddd Convolutions for general distributions can be very hard to compute May not have a closed form for many different distribution types Real world is NOT independent! Not guaranteed to be Normal or approximately normal Usually resort to numerical methods to approximate Simulation is the only way to get an answer There may not exist a closed form total distribution

7 Monte Carlo Methods Monte Carlo Methods definitions A class of computational algorithms that rely on repeated random sampling to compute their results (Wikipedia) Any method which solves a problem by generating suitable random numbers and observing that fraction of the numbers obeying some property or properties (Wolfram MathWorld) Specifically NOT a computer but a computing algorithm You could do it all be hand (and Fermi did some of this) Rolling a die is an example of a Monte Carlo simulation Typically use computers because of their speed, memory and reliability Computers can do simple problems really fast without mistakes

8 Monte Carlo History Statistical Sampling methods have been known for many years but are generally infeasible by hand (unless you have a lot of time and patience) Estimating ππ through Buffon s Experiment (18th Century) Predicting experiment results using statistical sampling (Fermi) Calculating the probability of winning solitaire (Ulam) The Modern Age Near the End of WWII, ENIAC provided an opportunity use computers to support the nuclear bomb effort Next several computers were built to increase capability to do Monte Carlo! More Samples is Better! The name Monte Carlo was derived from Stanislaw Ulam s uncle who would borrow money from the family by saying that I just have to go to Monte Carlo

9 Monte Carlo Algorithm Given a specified model with inputs and outputs 1. Define the distribution of possible inputs 2. Generate a set of random inputs 3. Compute (deterministic) outputs based on randomly selected inputs 4. Record data 5. Repeat steps 2-4 as necessary The stopping conditions could be Predetermined number of iterations Precision of outputs (the answer stops changing)

10 Generating Input Samples For univariate distributions, use the inverse of the CDF Inverse CDFs only require numbers (probability values) in [0, 1]

11 Generating Input Samples With the Inverse input distribution(s) selected we only need to generate random numbers in [0,1] Two Options: Observe some physical process Atmospheric noise, Lottery machines, etc. (TRNG) Use an algorithm to generate sufficiently random numbers Pseudorandom number generator (PRNG)

12 Pseudorandom Presented at the 2017 ICEAA Professional Development & Training Workshop Number Generators Algorithms that use a seed value or a previous value to generate a new number in a sequence There are lots of pseudo random number generators Only some of them are actually good

13 PRNG Quality Quality PRNGs for Monte Carlo Simulations have the following characteristics Good Distribution Coverage Seemingly unpredictable Long Period Repeatability Efficiency Any PRNG that doesn t have these qualities may be insufficient for Monte Carlo sampling Crystal Ball uses an Multiplicative Congruential Generator (older but has 8 different options (Mersenne Twister as default)

14 Other Considerations AKA - ADVANCED TOPICS

15 Correlation It is essential to account for relationships among inputs! Correlation captures the essence of Drucker s Law*: If one thing goes wrong, everything else will, and at the same time. Correlation is a pairwise measure of how two inputs vary at the same time Values range from [-1, 1] Positive as one goes up, so does the other Negative as one goes up, the other goes down *Drucker, Peter F. Management, Tasks, Responsibilities, and Practices, p. 681

16 Latin Hypercube Sampling When the Monte Carlo iterations are expensive, using a different sampling approach can reduce the number of overall iterations required to achieve a tolerance. Standard MC sampling picks numbers from anywhere within [0,1] Latin Hypercube Sampling breaks [0,1] into subintervals and then samples Assures samples fill out the distribution

17 Latin Hypercube Sampling Results Standard MC sampling may be clustered Latin Hypercube Sampling (LHS) can provide for faster convergence of statistical properties The LHS advantage over MC sampling diminishes when As the number of samples increases As the number of sampled parameters increases

18 Common Distributions for Cost Estimating Presented at the 2017 ICEAA Professional Development & Training Workshop *From 2016 ICEAA CEA06 Monte Carlo : Looking Under the Hood

19 Deterministic Application

20 Deterministic Application How can you calculate pi? Throw Darts Pick random points in [-r,r]x[-r,r] If point (xx, yy) has xx 2 + yy 2 rr 2, then the point is in the circle Blue square has area = 4 rr 2 Gray Circle has area = ππrr 2 The percentage of points inside the circle should be ππ 4

21 Monte Carlo Sampling Set up Assume uniform distribution for X and Y points Use Excel Rand and scale to [-r,r] Pi Estimate is 4* CUM % in

22 Results As number of sample point increases: The percentage of points inside and outside the circle better approximates the respective areas The ππ estimate error decreases

23 Excel Only Approach It is possible to use an Excel Only approach but there are some major limitations Storage of numbers (inputs and outputs) Generating inputs from various distributions Handling Correlation It is possible to use VBA code, but most of the same limitations exist, now just hidden in code that may not be easily traceable Excel Add-ins can provide a fairly easy way to accomplish the uncertainty quantification and risk analysis

24 Calculating Pi with Crystal Ball Taking advantage of the Excel Add-in Crystal Ball we can make a simple model and let it handle the bookkeeping

25 Results Monte Carlo software tools can Manage the data Present results through descriptive statistics and charts Actual sample data can be extracted for additional analysis if desired

26 Applications to Cost Estimating

27 Cost Estimating Models Cost Estimates are models Outputs may be nonlinearly related to many inputs Inputs may be of a large variety of distributions Continuous or Discrete Inputs may be related The distribution of any output is not guaranteed to have a nice solution CLT may not apply! We may also require additional information Sensitivity Information Best Case / Worst Case Etc.

28 Example Demo File Missile System Estimate TBE already established Need to quantify the uncertainty in the model Our SMEs already told us what the distributions are We need to input them into our Monte Carlo Tool (Crystal Ball)

29 Select Distribution Wide Variety available or define your own!

30 Set Parameters The Assumption cells will contain the random sample at each iteration

31 Set Outputs The Forecast cells require an equation to calculate the output

32 Simulation Preferences Always double check your run preferences!

33 Results We got an answer, but it doesn t look quite right

34 The Problem of N Recall the ππ calculation How many Monte Carlo iterations does it take to get the average? Looks pretty good after 1000 Depends on your tolerance Generally, the more variables you have the more samples you need

35 Results Increase the number of iterations to improve result

36 Correlation Matrix Most Monte Carlo Tools use a Rank correlation to handle the relationship of many different types of distributions

37 Results (Positive) Correlation increases the over distribution coverage

38 Questions?