REAL 03-T-7 March, 2003

Size: px

Start display at page:

Download "REAL 03-T-7 March, 2003"

Zoe Lindsey
5 years ago
Views:

1 The Regonal Economcs Applcatons Laboratory (REAL) s a cooperatve venture between the Unversty of Illnos and the Federal Reserve Bank of Chcago focusng on the development and use of analytcal models for urban and regonal economc development. The purpose of the Dscusson Papers s to crculate ntermedate and fnal results of ths research among readers wthn and outsde REAL. The opnons and conclusons expressed n the papers are those of the authors and do not necessarly represent those of the Federal Reserve Bank of Chcago, Federal Reserve Board of Governors or the Unversty of Illnos. All requests and comments should be drected to Geoffrey J. D. Hewngs, Drector, Regonal Economcs Applcatons Laboratory, 607 South Matthews, Urbana, IL, , phone (217) , FAX (217) Web page: A SHORT NOTE ON THE NUMERICAL APPROXIMATION OF THE STANDARD NORMAL CUMULATIVE DISTRIBUTION AND ITS INVERSE by Chokr Drd REAL 03-T-7 March, 2003

2 A SHORT NOTE ON THE NUMERICAL APPROXIMATION OF THE STANDARD NORMAL CUMULATIVE DISTRIBUTION AND ITS INVERSE (March, 2003) Chokr Drd Department of Agrcultural and Consumer Economcs, Regonal Economcs Applcatons Laboratory, Unversty of Illnos at Urbana-Champagn. ABSTARCT: We ntroduce and evaluate the results of two numercal ntegraton methods: the ratonal fracton approxmaton, and the composte Gauss- Legendre quadrature. We provde computer codes n ANSI-C and Python for a fast and accurate computaton of the cumulatve dstrbuton functon (cdf) of the standard normal dstrbuton and the nverse cdf of the same functon. For the cdf we use the 5 th order Gauss-Legendre quadrature whch gves more accurate results compared to Excel and Matlab. The Inverse cdf computed usng ratonal fracton approxmatons gves a result that s seven-decmal place accurate.. Keywords: C/C++, Gauss-Legendre Quadarture, Normal dstrbuton, Numercal Integraton, Ratonal fracton approxmatons, Software. JEL Classfcaton: C63, C88, C89 All copyrghted materal and trademarks cted are the property of ther owners. Computer code ncluded herewth s provded as s wthout any warrantes. Computer codes n Python and ANSI-C are avalable from: 1

3 1. INTRODUCTION Varous research projects requre buldng software components from the ground up. If the ntegraton and calls to functons n commercal software are mpossble then unless large fnancal resources are commtted for the development of solutons smlar to the ones used n commercal software, code components for the project have to be bult n-house wth the explct purpose of havng a reasonable accuracy, speed, and low development costs. Numercal ntegraton s certanly one of the most used technques for runnng smulatons and statstcal analyss especally that more and more researchers n economcs and other areas of socal scences started tacklng problems that cannot be solved n closed form, Geweke (1995) gves examples n macroeconomcs nvolvng numercal ntegraton. Popular ntegraton methods range from very smple and naccurate methods lke rectangular and trapezodal rules (Kreyszg, 1993) to more complex and relatvely more accurate approaches lke the Newton-Cotes formulae and Gaussan quadratures, surveyed n Press et al. (1992), and Judd (1998). In the next secton, we ntroduce and evaluate the results of two numercal ntegraton methods: the ratonal fracton approxmatons, and the composte Gauss- Legendre quadrature. In secton 3, the computaton of the nverse cdf usng ratonal fracton approxmatons s evaluated. Secton 4, concludes ths short note. Computer code n Python and ANSI-C s avalable from the provded web URLs. We compare results provded by computer programs wrtten n Python and ANSI-C aganst results provded by bult-n functons n Mcrosoft's Excel, software usually used by students for quck calculatons, and Mathworks' Matlab, a more computaton orented software used by researchers. 2. NUMERICAL APPROXIMATION OF THE NORMAL STANDARD CUMULATIVE FUNCTION Most commercal statstcal and mathematcal applcatons nclude the error functon and ts complement n ther set of bult-n functons, however many powerful programmng envronment such as C/C++ and Python do not recognze such functons. The error functon s useful for most statstcal and econometrc purpose as t allows dervng the cumulatve normal dstrbuton (cdf) easly and accurately. The error functon and ts complement are defned as (Kennedy & Gentle, 1980): erf erfc x 2 x t dt ; x 0 (1) π 2 ( ) = exp( ) 0 2 x t dt π x ; x < 0 (2) = 1 erf 2 ( ) = exp( ) ( x) 2

4 Form (1) and (2), the exact standard normal cumulatve dstrbuton functon s gven by: ( ) F x ( x ) 1+ erf / 2 ; x 0 2 = 1 erf ( x / 2) ; x < 0 2 (3) Obvously, the accuracy of the results n (3) depends on the ntegraton method used to evaluate the ntegral functon n the error functon and on machne round-off errors. Most ntegraton method perform rather well on regons far from the tals of the dstrbuton however, as noted by Greene (2000, p.177), the tal areas of the normal dstrbuton are of mportance for econometrcans. In what follows, we examne two dfferent approxmaton technques, ratonal fracton approxmatons, and Gauss- Legendre quadrature, to llustrate the trade-off between speed and accuracy Ratonal Fracton Approxmatons Cody (1969) provdes ratonal fracton approxmatons that are relatvely accurate for the tme the artcle was publshed, the error functon s defned by: ( ) ( ) erf x = xr x ; 0< x 0.5 (4) erfc erfc 1 2 ( x) exp( x ) R2 ( x) 2 ( x ) ( ) = ; x 4.0 (5) exp x = + x R3 ( x ) x π ; x 4.0 (6) In (4)-(6), the ratonal fractons are defned by (7) where the coeffcents are provded by Kennedy & Gentle (1980, p ): p and q R ( x) = 3 = = 0 p x qx 2 2 ; R ( x) = 7 = = 0 p x qx ; R ( x) = 4 = = 0 p x qx 2 2 (7) 3

5 Gauss-Legendre(C) cdf Cody(Python) cdf x FIGURE 1: Gauss-Legendre quadrature vs. Ratonal fracton approxmatons before adjustment Usng the algorthm descrbed above wth Python 1 gves accurate enough results only for x [ 0.5,0.75]. Fgure 1 compares the cdf gven by Cody's algorthm and the Gauss-Legendre quadrature that we examne below, t s clear that Cody's algorthm cannot be of use f we seek accuracy. If we alter Cody's algorthm as follows we obtan a much better precson wthout affectng ts speed: ( ) ( ) erf x = xr x ; 0< x 1.5 (8) 1 ( ) ( x) ( x ) R1( x ) R2( x ) R3( x ) erfc = exp ; x 4.0 (9) The results of the adjusted algorthm match the overall shape of the cumulatve dstrbuton functon however, lke wth Matlab, t reaches zero (resp. 1) at x (resp. x ), whch creates a problem f we need to dvde by (resp. log) the probablty. We wll see next that the adjusted Cody algorthm whle fast s less precse than the Composte Gauss-Legendre quadrature (fgures 2 and 3). 1 The Python code avalable requres Numerc-22.0, the Numercal Extenson to Python, wrtten by Paul F. Dubos. 4

6 2.2. Composte Gauss-Legendre Quadrature b The Gauss quadrature, conssts n approxmatng the ntegral ( ) ( ) by the summaton wf( x) K where: f : k = 1, ( ) I = w x f x dx w x a weght functon, the x [ ab, ] are roots of Legendre polynomals of order K that are orthogonal to ( ) a w x, and the are the roots' weghts. Ths approach has been used wth varous weght functons, the smplest and yet very accurate one s attrbuted to Legendre, see Press et al. (1992) for more detals regardng the methodology and other versons of the Gauss quadrature. The ntegraton method known as Gauss-Legendre quadrature s Gauss's quadrature wth w( x) = 1; x [ 1,1 ], unlke Smpson's method and Newton-Cotes formulae, that use arbtrary and equally spaced ponts, the Gaussan quadrature determnes precse ponts n [ ab, ] symmetrcally around zero, but not necessarly equally spaced, therefore t s not approprate for tabulated data. Press et al. (1992), offer addtonal explanatons and a computer routne that helps fndng Legendre polynomals roots and ther respectve weghts, the polynomals are defned by: P0 ( x ) = 1; P1 ( x) = x; ( k + 1) P = ( 2k + 1) xp kp ; k 1 (10) k+ 1 k k 1 The followng theorem announces a property of Gauss-Legendre quadrature, relatng the order of the quadrature and the shape of the ntegrand to the precson of the approxmaton. THEOREM: The Gauss-Legendre quadrature s exact f f ( x ) s a polynomal of degree less than or equal to 2n 1, where n s Legendre polynomal number of roots. Recall that the Gauss-Legendre quadrature apples only over the nterval [ 1,1], n Harrs & Stoker (1998, p. 571), for any chosen nterval [ ab, ], the ntegral f ( ) rewrtten wth the substtuton b a 1 b a b a b+ a f ( x) dx = f z+ dz b a b + x = z+ a to gve: 2 2 b a w 's x dx s (11) 5

7 Wth k beng the order of Lagrange polynomal (.e. number of ponts), the error expected from the Gauss-Legendre quadrature after the change of varable s (Chapra & Canale, 1998): E k = 2k ( k! ) ( k ) ( k) ! 3 f ( 2k ) ( ξ ) ; 1 ξ 1 (12) 1.00E-04 Excel v. GL Matlab v. GL 8.00E E E E E Absolute Error FIGURE 2: Absolute error of the Gauss-Legendre quadarture compared to Excel and Matlab Lmtng ourselves to the ffth order of Legendre polynomal, we approxmate the normal cdf, usng the code n Python; the error n (12) becomes very small and s (10) f ( ξ ) approxmately equal to ; ξ [ 1,1]. Whle the algorthm gves accurate 1.23E+09 results t s conspcuously slow 2, ths requres usng a faster programmng language lke ANSI-C. The advantage of the composte Gauss-Legendre quadrature s that t s precse and does not converge to zero or one over a large nterval; n our test, we use the [- 15.0,15.0] nterval wth 0.25 step sze. The analyss of the error on the Gauss-Legendre quadarature shows that the maxmum error compared to results from Matlab and Excel 2 On a Pentum III 450 Mhz. 6

8 s 1.0E-4 (fgure 2), and that t s always less than the absolute error between Cody's adjusted algorthm and Matlab (fgure 3). 2.50E-03 Gauss-Legendre v. Adj. Cody Matlab v. Cody 2.00E E E E E x FIGURE 3: Absolute error between Gauss-Legendre quadrature, Matlab, and Ratonal fractons approxmaton after adjustment 3. INVERSE OF STANDARD NORMAL CUMULATIVE DISTRIBUTION Wth F (). beng the normal cumulatve dstrbuton functon (cdf), gven a probablty p, we seek to approxmate the pont x such that: 1 ( ) = = ( ) ; ( px, ) [ 0,1] F x p x F p (13) However, the nverse functon of F (.) s not possble to obtan n a closed form, varous approxmaton methods were suggested. We test the algorthm developed by Odeh & Evans (1974) and descrbed n Kennedy & Gentle (1980, p ), the algorthm s based on the approxmaton of ratonal fractons derved from Taylor seres. Compared to results gven by Matlab and Mcrosoft Excel n table 1, the algorthm wrtten n Python s seven-decmal-place accurate, whch s an approprate approxmaton consderng that x. 7

9 Table 1: Evaluaton of the results of the nverse cdf approxmaton p MS-Excel Matlab Approx. 0 #NUM! -Inf -1.00E #NUM! +Inf 1.00E CONCLUSION Varous research problems arse when a model cannot be solved n a closed form, n addton to smulatng the model nstead of solvng t, researchers can analyze the problem numercally. In ths short note, we provde explanatons and computer code to compute the cumulatve standard normal dstrbuton functon and ts nverse; we retan results form the 5 th order composte Gauss-Legendre quadrature n ANSI-C and the ratonal fracton approxmatons n Python as precse and fast numercal approxmatons. The composte Gauss-Legendre quadrature may be mproved upon by usng more ponts for better precson, and fewer subntervals for more speed. Addtonal ponts and ther weght can be computed (Press et al., 1992 and Vetterlng et al., 1992) and found n varous references (Judd, 1998). REFERENCES Chapra, S. C. & R. P. Canale (1998): Numercal Methods for Engneers: Wth Programmng and Software Applcatons, 3 rd ed., McGraw-Hll, Boston. Cody, W. J. (1969): "Ratonal Chebyshev Approxmatons for the Error functon, Mathematcal Computaton 23, Geweke, J. (1995): "Monte Carlo Smulaton and Numercal Integraton", Federal Reserve Bank of Mnneapols, Research Department Staff Report 192. Greene, W. H. (2000): Econometrc Analyss, 4 th ed., Prentce Hall, New Jersey. Harrs, J. W. & H. Stocker (1998): Handbook of Mathematcs and Computatonal Scence, Sprnger-Verlag, New York. Judd, K. L. (1998): Numercal Methods n Economcs, MIT Press, Cambrdge. 8

10 Kennedy, W. J. Jr. & J. E. Gentle (1980): Statstcal Computng, Marcel Dekker, New York. Kreyszg, E. (1993): Advanced Engneerng Mathematcs, 7 th ed., John Wley & Sons, New York. Odeh, R. E. & J. O. Evans (1974): "Algorthm AS 70: Percentage Ponts of the Normal Dstrbuton", Appled Statstcs 23, Press, W. H., S. A. Teukolsky, W. T. Vetterlg, & B. P. Flannery (1992): Numercal Recpes n C: The Art of Scentfc Computng, 2 nd ed., Cambrdge Unversty Press, Cambrdge. Vetterlng, W. T., S. A. Teukolsky, W. H. Press, & B. P. Flannery (1992): Numercal Recpes: Examples Book (C), 2 nd ed., Cambrdge Unversty Press, Cambrdge. 9

11 APPENDIX A1. ANSI-C CODE FOR THE CDF (CDF.C) /*********************************************************************/ /* Purpose: Computes cdf of standard normal dst. usng a composte */ /* ffth-order Gauss-Legendre quadrature */ /* Code by: Chokr Drd (December, 2002) */ /*********************************************************************/ #nclude <math.h> #nclude <stdo.h> long double GL(long double, long double); /* ntegraton over closed nterval */ long double cdf (long double); /* cdf functon */ long double f(long double); /* functon to ntegrate */ double p=0.; nt man () { /* the man functon to get the cdf s cdf() */ /* the man() block s used just to generate values for testng */ long double x=-15; whle (x<=15.){ p=cdf(x); prntf("%.17e\n",p); x=x+.25; return 0; /* cdf functon */ long double cdf(long double x){ f(x>=0.){ /* Integraton on a closed nterval */ long double GL(long double a, long double b) { long double y1=0, y2=0, y3=0, y4=0, y5=0; return (1.+GL(0,x/sqrt(2.)))/2.; else { return (1.-GL(0,-x/sqrt(2.)))/2.; long double x1=-sqrt( *sqrt(70.))/21., x2=-sqrt( *sqrt(70.))/21.; long double x3=0, x4=-x2, x5=-x1; long double w1=( *sqrt(70.))/900., w2=( *sqrt(70.))/900.; long double w3=128./225.,w4=w2,w5=w1; nt n=4800; long double =0, s=0, h=(b-a)/n; 10

12 for (=0;<=n;++){ y1=h*x1/2.+(h+2.*(a+*h))/2.; y2=h*x2/2.+(h+2.*(a+*h))/2.; y3=h*x3/2.+(h+2.*(a+*h))/2.; y4=h*x4/2.+(h+2.*(a+*h))/2.; y5=h*x5/2.+(h+2.*(a+*h))/2.; s=s+h*(w1*f(y1)+w2*f(y2)+w3*f(y3)+w4*f(y4)+w5*f(y5))/2.; return s; /* Functon f, to ntegrate */ long double f(long double x){ return (2./sqrt(22./7.))*exp(-pow(x,2.)); A2. PYTHON CODE FOR THE CDF USING GAUSS-LEGENDRE QUADRATURE (CDF-GL.PY) """ Purpose: Computes cdf of standard normal dst. usng a composte ffth-order Gauss-Legendre quadrature Code by: Chokr Drd (December, 2002) """ from Numerc mport * " cdf functon " def cdf(x): f x>=0.: return (1.+GL(0,x/sqrt(2.)))/2. else: return (1.-GL(0,-x/sqrt(2.)))/2. " Integraton on a closed nterval " def GL(a,b): y1=0. y2=0. y3=0. y4=0. y5=0. x1=-sqrt( *sqrt(70.))/21. x2=-sqrt( *sqrt(70.))/21. x3=0. x4=-x2 x5=-x1 w1=( *sqrt(70.))/900. w2=( *sqrt(70.))/900. w3=128./225. w4=w2 w5=w1 n=4800 s=0. 11

13 h=(b-a)/n for n range(0,n,1): y1=h*x1/2.+(h+2.*(a+*h))/2. y2=h*x2/2.+(h+2.*(a+*h))/2. y3=h*x3/2.+(h+2.*(a+*h))/2. y4=h*x4/2.+(h+2.*(a+*h))/2. y5=h*x5/2.+(h+2.*(a+*h))/2. s=s+h*(w1*f(y1)+w2*f(y2)+w3*f(y3)+w4*f(y4)+w5*f(y5))/2.; return s; " Functon f, to ntegrate " def f(x): return (2./sqrt(22./7.))*exp(-x**2.); A3. PYTHON CODE FOR THE CDF USING RATIONAL FRACTIONS APPROXIMATION (CDF.PY) """ Purpose: Algorthm to compute cdf of a Gaussan dstrbuton The cdf s 0 for all x < and s 1 for all x > Before adjustment, the accuracy of ths algorthm s acceptable only for ponts -0.5<=x<=0.75 Coded by: Chokr Drd (December, 2002) Based on: Kennedy & Gentle(1980): Statstcal Computng, Marcel Dekker, p """ from Numerc mport * def cdf(x): f x > 0.: y=x else: y=-x f y >= 0. and y <= 1.5: p=(1.+erf(y/sqrt(2.)))/2. f y > 1.5: p=1.-erfc(y/sqrt(2.))/2. f x > 0.: return p else: return 1.-p def erf(x): " for 0<x<=0.5 " return x*r1(x) def erfc(x): " for <=x<=4. " f x > and x < 4.: return exp(-x**2.)*(0.5*r1(x**2.)+0.2*r2(x**2.)+0.3*r3(x**2.)) f x >= 4.: " for x>=4. " 12

14 return (exp(-x**2.)/x)*(1./sqrt(22./7.)+r3(x**-2.)/(x**2.)) def R1(x): N=0. D=0. p=[ e2, e1, , e-2] q=[ e2, e1, e1,1.] for n range(0,3,1): N=N+p[]*x**(2.*) D=D+q[]*x**(2.*) return N/D def R2(x): N=0. D=0. p=[ e2, e2, e2, e2, e1, , e-1, e-7] q=[ e2, e2, e2, e2, e2, e1, e1,1.] for n range(0,7,1): N=N+p[]*x**(-2.*) D=D+q[]*x**(-2.*) return N/D def R3(x): N=0. D=0. p=[ e-3, e-2, e-1, e-1, e- 2] q=[ e-2, e- 1, , ,1.] for n range(0,4,1): N=N+p[]*x**(-2.*) D=D+q[]*x**(-2.*) return N/D A4. PYTHON CODE FOR THE INVERSE CDF (INVCDF.PY) """ Purpose: Algorthm to compute nverse cdf of a Gaussan dstrbuton for values of p; 1.0E-20<p<1 Coded by: Chokr Drd (November, 2002) Based on: Kennedy & Gentle(1980): Statstcal Computng, Marcel Dekker, p """ from Numerc mport * def nvcdf(p): 13

15 f p>0.5: return -nv(p) else: return nv(p) def nv(p): xp=0. lm = 1.e-20 p0 = p1 = -1.0 p2 = p3 = p4 = e-4 q0 = q1 = q2 = q3 = q4 = e-2 f p < lm or p == 1.: return -1./lm f p == 0.5: return 0. f p > 0.5: p=1.-p y = sqrt(log(1./p**2.)) xp= y+((((y*p4+p3)*y+p2)*y+p1)*y+p0)/((((y*q4+q3)*y+q2)*y+q1)*y+q0) f p < 0.5: xp = -xp return xp 14

An Accurate Evaluation of Integrals in Convex and Non convex Polygonal Domain by Twelve Node Quadrilateral Finite Element Method

An Accurate Evaluation of Integrals in Convex and Non convex Polygonal Domain by Twelve Node Quadrilateral Finite Element Method Internatonal Journal of Computatonal and Appled Mathematcs. ISSN 89-4966 Volume, Number (07), pp. 33-4 Research Inda Publcatons http://www.rpublcaton.com An Accurate Evaluaton of Integrals n Convex and