Monte Carlo Integration COS 323

Transcription

1 Monte Carlo Integration COS 323

2 Last time Interpolatory Quadrature Review formulation; error analysis Newton-Cotes Quadrature Midpoint, Trapezoid, Simpson s Rule Error analysis for trapezoid, midpoint rules Richardson Extrapolation / Romberg Interpolation Gaussian Quadrature Class WILL be held on Tuesday 11/22

3 Today Monte Carlo integration Random number generation Cool examples from graphics

4 Integration in d Dimensions? One option: nested 1-D integration Evaluate the latter numerically, but each sample of g(y) is itself a 1-D integral, done numerically

5 Integration in d Dimensions? Midpoint rule in d dimensions? In 1D: (b-a)/h points In 2D: (b-a)/h 2 points In general: O(1/h d ) points Exponential growth in # of points for a fixed order of method Curse of dimensionality Other problems, e.g. non-rectangular domains

6 Rethinking Integration in 1D

7 We Can Approximate

8 Or We Can Average

9 Estimating the Average 1 0 f (x)dx 1 N N i=1 f (x i )

10 Other Domains b a f (x)dx b a N N i=1 f (x i )

11 Monte Carlo Integration No exponential explosion in required number of samples with increase in dimension (Some) resistance to badly-behaved functions

12 Variance Sample variance of f : Var[ f (x)] = 1 N [ f (x N 1 i ) E( f (x))] 2 i=1 E(f(x))

13 Variance E(f(x)) Sample variance of f : [ f (x N 1 i ) E( f (x))] 2 i=1 N N Var Y i = Var[ Y ] i Var[ f (x)] = 1 i=1 N i=1 Var[ E( f (x))] = 1 N Var[ f (x)] Variance decreases as 1/N Error of E decreases as 1/sqrt(N)

14 Variance Problem: variance decreases with 1/N Increasing # samples removes noise slowly E(f(x))

15 Variance Reduction Techniques Problem: variance decreases with 1/N Increasing # samples removes noise slowly Variance reduction: Stratified sampling Importance sampling

16 Stratified Sampling Estimate subdomains separately

17 Stratified Sampling This is still unbiased E k (f(x)) E = k vol(m ) j f (x ) jn j =1 N j N j n =1

18 Stratified Sampling Less overall variance if less variance in subdomains Var[ E] = k j =1 vol(m j ) 2 N j Var[ f (x)] M j E k (f(x)) Total variance minimized when number of points in each subvolume M j proportional to error in M j.

19 Importance Sampling Put more samples where f(x) is bigger E(f(x)) f (x)dx = 1 N Y i Ω N i=1 where Y i = f (x i ) p(x i ) and x i drawn from P(x)

20 Importance Sampling This is still unbiased E(f(x)) for all N

21 Importance Sampling Variance depends on choice of p(x): E(f(x)) Var(E) = 1 N N n =1 f (x n ) p(x n ) 2 E 2

22 Importance Sampling Zero variance if p(x) ~ f(x) E(f(x)) Less variance with better importance sampling

23 Random number generation

24 True random numbers

25 Generating Random Points Uniform distribution: Use pseudorandom number generator 1 Probability 0 Ω

26 Pseudorandom Numbers Deterministic, but have statistical properties resembling true random numbers Common approach: each successive pseudorandom number is function of previous

27 Desirable properties Random pattern: Passes statistical tests (e.g., can use chi-squared) Long period: Go as long as possible without repeating Efficiency Repeatability: Produce same seuqence if started with same initial conditions Portability

28 Linear Congruential Methods Choose constants carefully, e.g. a = b = c = Results in integer in [0, c) Not suitable for MC: e.g. exhibit serial correlations

29 Problem with LCGs

30 Fibonacci Generators Takes form x n = x n-j x n-k Standard choices of j, k: e.g., (7, 10), (31, 63), (168, 521) Proper initialization is important and hard Built-in correlation! Not totally understood in theory (need statistical tests to evaluate)

31 Seeds Why? _Method_for_seeding_a_pseudo_rand.html?id=ou0gAAAAEBAJ

32 Pseudorandom Numbers To get floating-point numbers in [0..1), divide integer numbers by c + 1 To get integers in range [u..v], divide by (c+1)/(v u+1), truncate, and add u Better statistics than using modulo (v u+1) Only works if u and v small compared to c

33 Generating Random Points Uniform distribution: Use pseudorandom number generator 1 Probability 0 Ω

34 Sampling from a non-uniform distribution Specific probability distribution: Function inversion Rejection

35 Sampling from a non-uniform distribution Inversion method Integrate f(x): Cumulative Distribution Function

36 Sampling from a non-uniform distribution Inversion method Integrate f(x): Cumulative Distribution Function Invert CDF, apply to uniform random variable

37 Sampling from a non-uniform distribution Specific probability distribution: Function inversion Rejection

38 Sampling from a non-uniform distribution Rejection method Generate random (x,y) pairs, y between 0 and max(f(x))

39 Sampling from a non-uniform distribution Rejection method Generate random (x,y) pairs, y between 0 and max(f(x)) Keep only samples where y < f(x) Doesn t require cdf: Can use directly for importance sampling.

40 Quasi-Random Sampling

41 Example: Computing pi

42 With Stratified Sampling

43 Monte Carlo in Computer Graphics

44 or, Solving Integral Equations for Fun and Profit

45 or, Ugly Equations, Pretty Pictures

46 Computer Graphics Pipeline

47 Rendering Equation

48 Rendering Equation This is an integral equation Hard to solve! Can t solve this in closed form Simulate complex phenomena

49 Rendering Equation This is an integral equation Hard to solve! Can t solve this in closed form Simulate complex phenomena

50 Monte Carlo Integration

51 Estimate integral for each pixel by random sampling Monte Carlo Path Tracing

52 Global Illumination

53 Monte Carlo Global Illumination Rendering = integration Antialiasing Soft shadows Indirect illumination Caustics

54 Monte Carlo Global Illumination Rendering = integration Antialiasing Soft shadows Indirect illumination Caustics Eye Pixel x Surface

55 Monte Carlo Global Illumination Rendering = integration Antialiasing Soft shadows Indirect illumination Caustics Surface

56 Monte Carlo Global Illumination Rendering = integration Antialiasing Soft shadows Indirect illumination Caustics

57 Monte Carlo Global Illumination Rendering = integration Antialiasing Soft shadows Indirect illumination Caustics

58 Monte Carlo Global Illumination Rendering = integration Antialiasing Soft shadows Indirect illumination Caustics

59 Monte Carlo Global Illumination Rendering = integration Antialiasing Soft shadows Indirect illumination Caustics

60 Monte Carlo Global Illumination Rendering = integration Antialiasing Soft shadows Indirect illumination Caustics

61 Challenge Rendering integrals are difficult to evaluate Multiple dimensions Discontinuities Partial occluders Highlights Caustics

62 Challenge Rendering integrals are difficult to evaluate Multiple dimensions Discontinuities Partial occluders Highlights Caustics

63 Monte Carlo Path Tracing Big diffuse light source, 20 minutes

64 Monte Carlo Path Tracing 1000 paths/pixel

65 Monte Carlo Path Tracing Drawback: can be noisy unless lots of paths simulated 40 paths per pixel:

66 Monte Carlo Path Tracing Drawback: can be noisy unless lots of paths simulated 1200 paths per pixel:

67 Reducing Variance Observation: some paths more important (carry more energy) than others For example, shiny surfaces reflect more light in the ideal mirror direction Idea: put more samples where f(x) is bigger

68 Importance Sampling Idea: put more samples where f(x) is bigger

69 Effect of Importance Sampling Less noise at a given number of samples Equivalently, need to simulate fewer paths for some desired limit of noise

70 Other examples

71 More information on Monte Carlo: A review of Monte Carlo Ray Tracing Methods: CESCG97/mcrt.html

72 Matlab functions rand, randi, randn (normal) rng: Configure your random number generator! Quasi-random: haltonset, sobolset