Physics 736. Experimental Methods in Nuclear-, Particle-, and Astrophysics. - Statistics and Error Analysis -

Transcription

1 Physics 736 Experimental Methods in Nuclear-, Particle-, and Astrophysics - Statistics and Error Analysis - Karsten Heeger heeger@wisc.edu

2 Feldman&Cousin what are the issues they deal with? what processes is it applicable to? how is it done?

3 Feldman&Cousin what are the issues they deal with? measurements & limits coverage of confidence intervals how to construct confidence intervals what processes is it applicable to? Poisson process with background Gaussian errors with bounded physical region how is it done?

4 Goodness of Fit Tests Example of Run Test with X 2 Test A simulated data sample is shown with distribution function that was not used to generate the data. There are 20 bins. Distribution function was normalized to match number of events in data sample. Does the model fit the data?

5 Goodness of Fit Tests Example of Run Test with X 2 Test The value of χ 2 for this distribution is 25.2 for 19 d.o.f. resulting in P χ 2 =0.16 There are 7 bins with negative (n i np 0 i ) and 13 positive bins, with only 5 runs in the sign so that p(r =5)= This is a reason to reject the hypothesis. The combination u = 2(lnP χ 2 + lnp(r)) = 13.5 corresponds to a probability of 0.009

6 Confidence Intervals: Measurements and Limits rate or flux or # of events x confidence interval (CL=68.3%) confidence interval (CL=99%) What if some measurements are in a non-physical region?

7 Frequentist vs Bayesian Approach Two philosophies Bayesian approach probability = degree of belief that something will happen or that a parameter will have a given value Frequentist approach probability = relative frequency of something happening one can define frequentist probability for observing data (which are random) but not for the true value of a parameter independent of observer

8 Frequentist vs Bayesian Approach Two philosophies Bayesian approach require as input the prior beliefs of the physicist doing the analysis. necessarily subjective, and not allowed in frequentist method. Frequentist approach require as input probabilities of observing all data, including both the data actually observed and that which could have been observed (the Monte Carlo). not allowed in Bayesian method.

9 (Frequentist) Definition of the confidence interval for the measurement of a quantity x: If the experiment were repeated and in each attempt a confidence interval is calculated, then a fraction α of the confidence intervals will contain the true value of x (called µ). A fraction 1-α of the confidence intervals will not contain µ. Note: Experiments must not be identical

10 Frequentist vs Bayesian Approach classical approach Bayesian approach A precise experiment and an imprecise one with a statistical fluctuation can give the same limit! Q= prior belief function But it is not possible to combine the results of experiments that just quote a mass interval and confidence level.

11 Random Numbers Monte Carlo Techniques Karsten Heeger, Univ. of Wisconsin NUSS, July 13, 2009

12 Random Numbers & Monte Carlo Techniques Who has used a Monte Carlo before? Who has written a Monte Carlo before? what are elements of Monte Carlo?

13 Random Numbers & Monte Carlo Techniques Monte Carlo (MC) refers to any procedure that makes use of random numbers MC methods are used in simulation of natural phenomena simulation of experimental apparatus numerical analysis (e.g. integration of many variables)

14 Random Numbers & Monte Carlo Techniques Simulating Data or Experiment

15 Random Numbers & Monte Carlo Techniques Simulating Radioactive Decay

16 Random Numbers & Monte Carlo Techniques Estimating the Area of a Circle ** * ** ++? 't-.r o o F o z -o x 2.s Area of circle hits from 100 pairs of random numbers uniformly distributed between -1 and +1 # of hits inside circle give area estimate circle area estimates obtained from 100 MC runs, each with 100 pairs of random numbers. Gaussian curve based on mean and standard deviation of 100 estimated areas.

17 Random Numbers & Monte Carlo Techniques Random Numbers What is a random number? Is 3 a random number?

18 Random Numbers & Monte Carlo Techniques Random Numbers What is a random number? Is 3 a random number? No such thing as a single random number. A sequence of random numbers or a set of numbers that have nothing to do with the other numbers in the sequence. In a uniform distribution of random numbers in the range of [0,1] every number has the sam chance of turning up is as likely as 0.5.

19 Random Numbers & Monte Carlo Techniques How to Generate Random Numbers chaotic system e.g. lottery random process radioactive decay thermal noise cosmic ray arrival random number tables computer code

20 Random Numbers & Monte Carlo Techniques Random Number Tables

21 Random Numbers & Monte Carlo Techniques How to Generate Random Numbers all algorithms produce a periodic sequence of numbers sequence of numbers in a uniform distribution in the range [0,1] algorithms generate integers between 0 and M and return a real value x n = I n /M to obtain effectively random values, use small subset of a single period e.g. Mersenne twister algorithm = long period

22 Random Numbers & Monte Carlo Techniques How to Generate Random Numbers Middle Square, Von Neumann, 1946 generate a sequence of 10 digit integers, start with one, square it, and then take the middle 10 digits from answer as next number in sequence sequence is not random since each number is completely determined from previous one. but it appears random. a more complex algorithm does not lead to a better random sequence. it is better to use an algorithm that is well understood.

23 Random Numbers & Monte Carlo Techniques RANDU from IBM in 1960s RANDU 2D I n+1 =(65539 I n )mod2 31 RANDU 3D

24 Random Numbers & Monte Carlo Techniques How to Generate Random Numbers not all random number generators are good! For example, in ROOT TRandom3 recommended by ROOT TRandom too short of a period For example, in Numerical Recipes authors have admitted that RAN1 and RAN2 in first edition are mediocre generators ran0, ran1, ran2 are much better in second edition

25 Random Numbers & Monte Carlo Techniques How to Improve Generators improve behavior and increase period by modifying algorithms I n =(a I n 1 + b I n 2 )mod m this has 2 initial seeds and can have a period greater than m RABMAR generator in CERNLIB requires 103 seeds. the ultimate random number generator.

26 Random Numbers & Monte Carlo Techniques Simulating Distributions so far we have only considered random number in [0,1] more complicated problems generally require random numbers generated according to specific distributions we can generate random numbers according to certain distributions (e.g Poisson for radioactive decay) Goal: obtain a random deviate x from any probability density distribution function f(x) can use special purpose algorithms. use numerical libraries and routines. we will discuss 2 techniques here...

27 Acceptance/Rejection Method (von Neumann) Problem: generate a series of random numbers, xi, which follow a distribution f(x) Method: choose trial value, xtrial. accept with probability f(xtrial) choose trial x with random number λ1 x trial = x min +(x max x min )λ 1 random points are chosen inside the box and rejected if the ordinate exceeds f(x)

28 Acceptance/Rejection Method (von Neumann) random points are chosen inside the box and rejected if the ordinate exceeds f(x) bounding region is method to increase efficiency efficiency of method = ratio of areas keep Ch(x) as close as possible to f(x) Method applicable if - f(x) is too complex for other techniques - f(x) can be computed beware of normalization

29 Acceptance/Rejection Method (von Neumann) rejection algorithm is not efficient if the distribution has one or more larger peaks (or poles). in this case trial events are seldomly accepted. algorithm does not work when the range of x is [-, + ]

30 Inverse Transform Method applicable for simple distribution functions Method probability density function is f(x) in [-, + ] integrated probability up to point a is F(a) for x a F(a) is itself a random variable which will occur with uniform probability density on [0,1] we can find a unique x for a given u if u = F (x) provided we can find inverse x = F (u) 1

31 Inverse Transform Method Use of a random number u chosen from a uniform distribution [0,1] to find a random number x from a distribution with cumulative distribution function F(x) PDG

32 Inverse Transform Method Practical Method 1. normalize distribution function so that it becomes a probability distribution function (PDF) 2. integrate PDF from xmin to arbitrary x. this is probability of choosing a value less than x. 3. equate this to a uniform random number and solve for x. the resulting x will be distributed according to PDF. in other words, solve following equation for x given a uniform random number λ x x f(x)dx min xmax f(x)dx = λ x min

33 Inverse Transform Method convenient when you can calculate the inverse function e.g. exp(x), (1-x) n, 1/(1+x 2 ) there are som packages that do this for you. e.g. UNU.RAN in ROOT Examples generate x between 0 and 4 according to f(x) =x 0.5 generate x between 0 and according to f(x) =e x

34 Random Numbers & Monte Carlo Techniques What if rejection technique is impractical and you cannot invert the integral of the distribution function? Replace the distribution function f(x) by an approximate form f*(x)for which the inversion technique can be applied Generate trial values for x with inversion technique according to f*(x), and accept trial value with probability proportional to weight w = f(x)/f (x) f (x) rejection technique = special case where f*(x) is constant

35 Random Numbers & Monte Carlo Techniques Multidimensional Simulation (simulating a distribution in more than one dimension) if distribution is separable then variables are uncorrelated, each can be generated as before f(x, y) =g(x)h(y) generate x according to g(x) and y according to h(y) otherwise, distribution along each dimension needs to be calculated ymax D x (x) = f(x, y)dy y min f (x, y)dx find approximate distribution so that f (x, y)dy are invertible weights for trial events are given by w = f(x, y) f (x, y)

36 Monte Carlo Numbering Scheme To facilitate interfacing between event generators, detector simulators, and analysis packages used in particle physics