Numerical integration methods

Transcription

1 Chpter 1 Numericl integrtion methods The bility to clculte integrls is quite importnt. The uthor ws told tht, in the old dys, the gun ports were cut into ship only fter it ws flot, loded with equivlent crgo, nd rigged. This is becuse it ws impossible to clculte the wter displced volume, i.e. the integrl of the hull-describing function, with the rudimentry mth known t tht time. Consequently, there ws no wy to properly estimte the buoynt force. Thus, the loction of the wterline ws unknown until the ship ws fully flot. Additionlly, not every integrl cn be computed nlyticlly, even for reltively simple functions. Exmple The Guss error function defined s erf(y) = 2 π y 0 e x2 dx cnnot be clculted using only elementry functions. 1.1 Integrtion problem sttement At first, we consider the evlution of one dimensionl integrl, lso clled qudrture, since this opertion is equivlent to finding of the re under the curve of given function. b f(x)dx Surprisingly, one does not need to know ny high level mth to do so. All you need is precision scle nd scissors. The recipe goes like this: plot the function on some preferbly 1

2 dense nd hevy mteril, cut the re of interest with scissors, mesure the mss of the cutout, divide the obtined mss by the density nd thickness of the mteril nd you re done. Well, this is, of course, not very precise nd sounds so low tech. We will employ more modern numericl methods. Words of wisdom Once you become proficient with computers, there is tendency to do every problem numericlly. Resist it! If you cn find n nlyticl solution for problem, do so. It is usully fster to clculte, nd more importntly, will provide you some physicl intuition for the problem overll. Numericl solutions usully do not possess predictive power. 1.2 The Rectngle method The definition of the integrl vi Riemnn s sum: b N 1 f(x)dx = lim f(x i )(x i+1 x i ) (1.1) N i=1 where x i b nd N is the number of points, f(x) f 1 f 2 f 3 f 4 f 5 f 6 f i f N-1 f N X 1 X 2 X 3 X 4 X 5 X 6 X i X N-1 X N h b x Figure 1.1: The rectngle method illustrtion. Shded boxes show the pproximtions of the re under the curve. Here, f 1 = f(x 1 = ), f 2 = f(x 2 ), f 3 = f(x 3 ),..., f N = f(x N = b). Riemnn s definition gives us directions for the rectngle method: pproximte the re under the curve by set of boxes or rectngles (see fig. 1.1). For simplicity, we evlute our function t N points eqully seprted by the distnce h on the intervl (, b). 2

3 Rectngle method integrl estimte where b f(x)dx N 1 i=1 f(x i )h, (1.2) h = b N 1, x i = + (i 1)h, x 1 = nd x N = b (1.3) The MATLAB implementtion of this method is quite simple: Listing 1.1: integrte_in_1d.m (vilble t with_matlab_book/ch_integrtion/code/integrte_in_1d.m) function integrl1d = integrte_in_1d(f,, b) % integrtion with simple rectngle/box method % int_^b f(x) dx N=100; % number of points in the sum x=linspce(,b,n); s=0; for xi=x(1:end 1) % we need to exclude x(end)=b s = s + f(xi); end %% now we clculte the integrl integrl1d = s*(b )/(N 1); To demonstrte how to use it, let s check 1 0 x2 dx = 1/3 f x.^2; integrte_in_1d(f,0,1) ns = Well, is quite close to the expected vlue of 1/3. The smll devition from the exct result is due to the reltively smll number of points; we used N = 100. If you hve previous experience in low level lnguges (from the rry functions implementtion point of view) like C or Jv, the bove implementtion is the first thing tht comes to your mind. While it is correct, it does not use MATLAB s bility to use mtrices s rguments of function. A better wy, which voids using loop, is shown below. Listing 1.2: integrte_in_1d_mtlb_wy.m (vilble t progrmming_with_matlab_book/ch_integrtion/code/integrte_in_1d_mtlb_wy. 3

4 m) function integrl1d = integrte_in_1d_mtlb_wy(f,, b) % integrtion with simple rectngle/box method % int_^b f(x) dx N=100; % number of points in the sum x=linspce(,b,n); % if function f cn work with vector rgument then we cn do integrl1d = (b )/(N 1)*sum( f( x(1:end 1) ) ); % we exclude x(end)=b Words of wisdom In MATLAB, loops re generlly slower compred to the equivlent evlution of function with vector or mtrix rgument. Try to void loops which iterte evlution over mtrix elements Rectngle method lgorithmic error While the rectngle method is very simple to understnd nd implement, it hs wful lgorithmic error nd slow convergence. Let s see why this is so. A closer look t fig. 1.1 revels tht box often underestimtes (see, for exmple, the box between x 1 nd x 2 ) or overestimtes (for exmple, the box between x 6 nd x 7 ) the re. This is the lgorithmic error which is proportionl to f h 2 /2 to the first order, i.e. the re of tringle between box nd the curve. Since we hve N 1 intervls, this error ccumultes in the worst cse scenrio. So we cn sy tht the lgorithmic error (E) is Rectngle method lgorithmic error estimte ) ( ) E = O ((N 1) h2 (b ) 2 f 2 = O 2N f (1.4) In the lst term, we replced N 1 with N under the ssumption tht N is lrge. The O symbol represents the big O nottion, i.e. there is n unknown proportionlity coefficient. 1.3 Trpezoidl method The pproximtion of ech intervl with trpezoid (see fig. 1.2) is n ttempt to circumvent the wekness of the rectngle method. In other words, we do liner pproximtion of the 4

5 f(x) f 1 f 2 f 3 f 4 f 5 f 6 f i f N-1 f N X 1 X 2 X 3 X 4 X 5 X 6 X i X N-1 X N h b x Figure 1.2: The illustrtion of the trpezoidl method. Shded trpezoids show the pproximtions of the re under the curve. integrted function. Reclling the formul for the re of trpezoid, we pproximte the re of ech intervl s h(f i+1 + f i )/2, nd then we sum them ll. Trpezoidl method integrl estimte b f(x)dx h ( 1 2 f 1 + f 2 + f f N 2 + f N f N) (1.5) The 1/2 coefficient disppers for inner points, since they re counted twice: once for the left nd once for the right trpezoid re. Trpezoidl method lgorithmic error To evlute the lgorithmic error, one should note tht we re using liner pproximtion for the function nd ignoring the second order term. Reclling the Tylor expnsion, this mens tht to the first order, we re ignoring contribution of the second derivtive (f ) terms. With bit of ptience, it cn be shown tht Trpezoidl method lgorithmic error estimte ( ) (b ) 3 E = O 12N f 2 (1.6) Let s compre the integrl estimte with rectngle (eq. (1.2)) nd trpezoidl (eq. (1.5)) 5

6 methods. b b f(x)dx h (f 2 + f f N 2 + f N 1 ) + h (f 1 ), (1.7) f(x)dx h (f 2 + f f N 2 + f N 1 ) + h 1 2 (f 1 + f N ) (1.8) It is esy to see tht they re lmost identicl, with the only difference in the second term of either h (f 1 ) for the rectngle method or h 1 2 (f 1 + f N ) for the trpezoidl one. While this might seem like minor chnge in the underlying formul, it results in the lgorithmic error decresing s 1/N 2 for the trpezoidl method, which is wy better thn the 1/N dependence for the rectngle method. 1.4 Simpson s method f(x) f 1 f 2 f 3 f 4 f 5 f 6 f N-2 f i f N-1 f N X 1 X 2 X 3 X 4 X 5 X 6 X i X N-2 X N-1 h b X N x Figure 1.3: Simpson s method illustrtion The next logicl step is to pproximte the function with second order curves, i.e. prbols (see fig. 1.3). This leds us to the Simpson method. Let s consider triplet of consequent points (x i 1, f i 1 ), (x i, f i ), nd (x i+1, f i+1 ). The re under the prbol pssing through ll of these points is h/3 (f i 1 + 4f i + f i+1 ). Then we sum the res of ll triplets nd obtin Simpson s method integrl estimte b f(x)dx h 1 3 (f 1 + 4f 2 + 2f 3 + 4f f N 2 + 4f N 1 + f N ) (1.9) N must be in specil form N = 2k + 1, i.e. odd, nd 3 Yet gin, the first (f 1 ) nd lst (f N ) points re specil, since they re counted only once, while the edge points f 3, f 5,... re counted twice s members of the left nd right triplets. 6

7 Simpson s method lgorithmic error Since we re using more terms of the function s Tylor expnsion, the convergence of the method is improving nd the error of the method decreses with N even fster. Simpson method lgorithmic error estimte ( ) (b ) 5 E = O 180N f (4) 4 (1.10) Perhps one would be surprised to see f (4) nd N 4. This is becuse when we integrte triplet re, we go by the h to the left nd to the right of the centrl point, so the terms proportionl to the x 3 f (3) re cnceled out. 1.5 Generlized formul for integrtion A creful reder my hve lredy noticed tht the integrtion formuls for ll bove methods cn be written in the sme generl form. Generlized formul for numericl integrtion b where w i is the weight coefficient. f(x)dx h i=1 N f(x i )w i, (1.11) So, one hs no excuse to use the rectngle nd trpezoid method over Simpson s, since the clcultionl burden is exctly the sme, but the clcultion error for Simpson s method drops drsticlly s the number of points increses. Strictly speking, even Simpson s method is not so superior in comprison to others which use even higher order polynomil pproximtion for the function. These higher order methods would hve exctly the sme form s eq. (1.11). The difference will be in the weight coefficients. One cn see more detiled discussion of this issue in [1]. 7

8 y b y y x x b x Figure 1.4: The pond re estimte vi the Monte Crlo method. 1.6 Monte Crlo Integrtion Toy exmple: finding the re of pond Suppose we wnt to estimte the re of pond. Nively, we might rush to grb mesuring tpe. However, we could just wlk round nd throw stones in every possible direction, nd drw n imginry box round the pond to see how mny stones lnded inside (N inside ) the pond out of their totl number (N totl ). As illustrted in fig. 1.4, the rtio of bove numbers should be proportionl to the rtio of the pond re to the box re. So, we conclude tht the estimted re of the pond (A pond ) A pond = N inside N totl A box (1.12) where A box = (b x x )(b y y ) is the box re. The bove estimte requires tht the thrown stones re distributed rndomly nd uniformly 1. It is lso good ide to keep the re of the surrounding box s tight s possible to increse the probbility of hitting the pond. Imgine wht would hppen if the box ws huge compred to the pond size: we would need lot of stones (while we hve only finite mount of them) to hit the pond even once, nd until then the pond re estimte would be zero in ccordnce with eq. (1.12). 1 This is not trivil tsk. We will tlk bout this in chpter 11. For now, we will use rnd function provided by MATLAB. 8

9 1.6.1 Nive Monte Crlo integrtion y H f(x) b x Figure 1.5: Estimte of the integrl by counting points under the curve nd in the surrounding box. Well, the clcultion of the re under the curve is not much different from the mesurement of the pond re s shown in fig So bx x f(x)dx = N inside N totl A box Monte Crlo integrtion derived The bove method is not optiml. Let s focus our ttention on nrrow strip round the point x b (see fig. 1.6). For this strip N inside N totl H f(x b ) (1.13) So, there is no need to wste ll of these resources if we cn get the f(x b ) right wy. Thus, the improved estimte of the integrl by the Monte Crlo method looks like 9

10 H y f(x ) b x b b x Figure 1.6: Explntion for the improved Monte Crlo method The Monte Crlo method integrl estimte Choose N uniformly nd rndomly distributed points x i inside [, b] b f(x)dx b N N f(x i ) (1.14) i=1 The Monte Crlo method lgorithmic error A creful look t eq. (1.14) shows tht 1/N N i=1 f(x i) =< f > is ctully sttisticl estimte of the function men. Sttisticlly, we re only sure in this estimte within the stndrd devition of the men estimte. This brings us to the following expression. 10

11 Monte Crlo method lgorithmic error estimte ( b ) E = O < f 2 > < f > 2 N (1.15) where < f > = 1 N < f 2 > = 1 N N f(x i ) i=1 N f 2 (x i ) i=1 The right most term under the squre root is the estimte of the function stndrd devition σ = < f 2 > < f > 2. Looking t the bove expression, you might conclude tht you hve wsted precious minutes of your life reding bout very mediocre method, which reduces its error proportionlly to 1/ N. This is worse even compred to the otherwise wful rectngle method. Hold your horses. In the following section, we will see how the Monte Crlo method outshines ll others for the cse of multidimensionl integrtion. 1.7 Multidimensionl integrtion If someone sks us to clculte multidimensionl integrl, we just need to pply our knowledge of how to del with single dimension integrls. For exmple, in the two dimensionl cse, we simply rerrnge terms: bx by x y f(x, y) dx dy = bx The lst single dimension integrl is the function of only x: x by dx dy f(x, y) (1.16) y by y dy f(x, y) = F (x) (1.17) So, the two-dimensionl integrl boils down to the chin of two one-dimension ones, which we re lredy fit to process: bx by x y f(x, y) dx dy = bx x dx F (x) (1.18) 11

12 Miniml exmple for integrtion in 2D The listing below shows how to do 2D integrls by chining the one-dimension ones; note tht it piggybcks on the single dimensionl integrl in listing 1.1 (but feel free to use ny other method). Listing 1.3: integrte_in_2d.m (vilble t with_matlab_book/ch_integrtion/code/integrte_in_2d.m) function integrl2d=integrte_in_2d(f, xrnge, yrnge) % Integrtes function f in 2D spce % f is hndle to function of x, y i.e. f(x,y) should be vlid % xrnge is vector contining lower nd upper limits of integrtion % long the first dimension. % xrnge = [x_lower x_upper] % yrnge is similr but for the second dimension % We will define (Integrl f(x,y) dy) s Fx(x) Fx f(x,y), yrnge(1), yrnge(2) ); % ^^^^^ we fix 'x', ^^^^^^^^^^^here we reuse this lredy fixed x % so it reds s Fy(y) % This is quite cumbersome. % It is probbly impossible to do generl D dimensionl cse. % Notice tht mtlb folks implemented integrl, integrl2, integrl3 % but they did not do ny for higher thn 3 dimensions. integrl2d = integrte_in_1d(fx, xrnge(1), xrnge(2) ); end Let s clculte 2 dx (2x 2 + y 2 ) dy (1.19) f 2*x.^2 + y.^2; integrte_in_2d( f, [0,2], [0,1] ) ns = It is esy to see tht the exct nswer is 6. The observed devition from the nlyticl result is due to the smll number of points used in the clcultion. 12

13 1.8 Multidimensionl integrtion with Monte Crlo The bove "chin" method cn be expnded to ny number of dimensions. Cn we rest now? Not so fst. Note tht if we would like to split the integrtion region by N points in every of the D dimensions, then the number of evlutions, nd thus the clcultion time, grows N D. This renders the rectngle, trpezoidl, Simpson s, nd like methods useless for high-dimension integrls. The Monte Crlo method is notble exception; it looks very simple even for multidimensionl cse, mintins the sme N evlution time, nd its error is still 1/ N. A 3D cse, for exmple, would look like this bx x by bz dx dy dz f(x, y, z) (b x x )(b y y )(b z z ) y z N nd the generl form is shown below Monte Crlo method in D-spce N f(x i, y i, z i ) (1.20) i=1 dv D f( x) = dx 1 dx 2 dx 3...dx D f( x) V D V D V D N N f( x i ) (1.21) i=1 where V D is the D-dimensionl volume, x i rndomly nd uniformly distributed points in the volume V D Monte Crlo method demonstrtion To see how elegnt nd simple the implementtion of the Monte Crlo method cn be, we will evlute the integrl in eq. (1.19). f 2*x.^2 + y.^2; bx=2; x=0; by=1; y=0; % first we prepre x nd y components of rndom points % rnd provides uniformly distributed points in the (0,1) intervl N=1000; % Number of rndom points x=x+(bx x)*rnd(1,n); % 1 row, N columns y=y+(by y)*rnd(1,n); % 1 row, N columns % finlly integrl evlution integrl2d = (bx x)*(by y)/n * sum( f(x,y) ) 13

14 integrl2d = We used only 1000 points nd the result is quite close to the nlyticl vlue of Numericl integrtion gotchs Using very lrge number of points Since the lgorithmic error of numericl integrtion methods drops with n increse of N, it is very tempting to increse it. But we must remember bout round-off errors nd resist this tempttion. So, h should not be too smll or equivlently N should not be too big. N definitely should not be infinite s the Riemnn s sum in eq. (1.1) prescribes, since our lifetime is finite. Using too few points sin(x) π x, rdins Figure 1.7: The exmple of bdly chosen points ( ) for integrtion of the sin(x) function ( ). The smples give the impression tht the function is horizontl line. 14

15 There is dnger of under smpling if the integrted function chnges very quickly nd we put points very sprsely (see for exmple fig. 1.7). We should first plot the function (if it is not too computtionlly txing), then choose pproprite mount of points: there should be t lest 2 nd idelly more points for every vlley nd hill of the function 2. To see if you hit sweet spot, try to double the mount of points nd see if the estimte of the integrl stops chnging drsticlly. This is the foundtion for the so clled dptive integrtion method, where n lgorithm utomticlly decides on the required number of points MATLAB functions for integrtion Below is short summry for MATLAB s built-in functions for integrtion: 1D integrtion integrl trpz employs modified trpezoidl method qud employs modified Simpson s method Here is how to use qud to clculte 2 0 3x3 dx >> f 3*x.^2; >> qud(f, 0, 2) ns = As expected, the nswer is 8. Note tht qud expects your function to be vector friendly. Multidimensionl integrtion integrl2 2D cse integrl3 3D cse Let s clculte the integrl 1 0 >> f x.^2 + y.^2; >> integrl2( f, 0, 1, 0, 1) ns = (x2 + y 2 )dxdy which equls to 2/3. There re mny others built-ins. See MATLAB s numericl integrtion documenttion to lern more. 2 We will discuss it in detil in the section devoted to the discrete Fourier trnsform (chpter 15). 15

16 MATLAB s implementtions re more powerful thn those which we discussed, but deep inside, they use similr methods Self-Study Generl comments: Do not forget to run some test cses. MATLAB hs built-in numericl integrtion methods, such s qud. You might check the vlidity of your implementtions with nswers produced by this MATLAB built-in function. qud requires your function to be ble to work with n rry of x points, otherwise it will fil. Of course, it is lwys better to compre to the exct nlyticlly clculted vlue. Problem 1: Implement the trpezoidl numericl integrtion method. Cll your function trpezint(f,,b,n), where nd b re left nd right limits of integrtion, N is the number of points, nd f is hndle to the function. Problem 2: Implement the Simpson numericl integrtion method. simpsonint(f,,b,n). Remember bout specil form of N=2k+1. Cll your function Problem 3: Implement the Monte-Crlo numericl integrtion method. Cll your function montecrloint(f,,b,n). Problem 4: For your tests clculte 10 0 [exp( x) + (x/1000) 3 ]dx Plot the integrl bsolute error from its true vlue for the bove methods (include rectngulr method s well) vs. different number of points N. Try to do it from smll N=3 to N=10 6. Use loglog plotting function for better representtion (mke sure tht you hve enough points in ll res of the plot). Cn you relte the trends to eqs. (1.4), (1.6), (1.9) nd (1.15). Why does the error strt to grow with lrger N? Does it grow for ll methods? Why? Problem 5: Clculte π/2 0 sin(401x)dx Compre your result with the exct nswer 1/401. number of points to clculte this integrl. Provide discussion bout required 16

17 Problem 6: Clculte 1 1 dx 1 0 dy(x 4 + y 2 + x y) Compre results of the Monte-Crlo numericl integrtion method nd MATLAB s built-in integrl2. Problem 7: Implement method to find the volume of the N-dimensionl sphere for the rbitrry dimension N nd the sphere rdius R V (N, R) = dx 1 dx 2 dx 3 dx N x 2 1 +x2 2 +x x2 N R2 Clculte the volume of the sphere with R = 1 nd N =

18 Bibliogrphy [1] W. H. Press, S. A. Teukolsky, W. T. Vetterling, nd B. P. Flnnery. Numericl Recipes 3rd Edition: The Art of Scientific Computing. Cmbridge University Press, New York, NY, USA, 3 edition,