Computational Economics and Finance

Size: px

Start display at page:

Download "Computational Economics and Finance"

Hugo Lee
5 years ago
Views:

1 Computational Economics and Finance Part I: Elementary Concepts of Numerical Analysis Spring 2015

2 Outline Computer arithmetic Error analysis: Sources of error Error propagation Controlling the error Rates of convergence Compute and verify 2

3 Computer Arithmetic Unlike pure mathematics, computer arithmetic has finite precision and is limited by time and space. Real numbers are represented as floating-point numbers of the form ±d 0.d 1 d 2... d p 1 β e. 0.d 1 d 2... d p 1 is called the significand (old: mantissa) with d j {0, 1,..., β 1} and has p digits. β is called the base. e {e min, e min + 1,..., e max } is the exponent. 3

4 Example Consider the decimal number 0.1. If β = 10 and p = 3, then is exact. If β = 2 and p = 24, then is not exact. In fact, with β = 2 the number 0.1 lies strictly between two floating-point numbers and is not exactly representable by either of them. 4

5 Double Precision Most widely-used standard for floating-point computation: IEEE Standard for Floating-Point Arithmetic (IEEE 754) IEEE: Institute of Electrical and Electronics Engineers Followed by many hardware (central processing unit, CPU and floating-point unit, FPU) and software implementations Current version is IEEE , published in August 2008 Perhaps most widely used: IEEE 754 double-precision binary floating-point format: binary64 5

6 binary64 Base β = 2, exponent and significand written in binary form total of 64 bits 1 bit for +/- sign, 11 bits for exponent, 52 bits for significand Normalized such that most significant bit d 0 = 1 for all numbers Exponent is biased by 1023 The number ( 1) sign (1.d 1 d 2... d 52 ) 2 e 1023 has the value ( 1) sign i=1 d i 2 i 2 e

7 Machine Epsilon Smallest quantity ɛ such that 1 ɛ and 1 + ɛ are both different from one; smallest possible difference in the significand between two numbers Double precision has at most 16 decimal digits of accuracy Matlab: eps = e 016 Mathematica: $MachineEpsilon =

8 Mathematica: $MaxMachineNumber = Machine Infinity Largest quantity that can be represented; overflow error occurs if an operation produces a larger quantity. Double precision has maximal exponent 2 10 = 1024 = and Bias in representation: 1023 Largest number (2 eps) Matlab: realmax = e + 308

9 Mathematica: $MinMachineNumber = Machine Zero Any quantity that cannot be distinguished from zero. Underflow error occurs if an operation on nonzero quantities produces a smaller quantity. Double precision has smallest exponent By convention this number represents 0 since normalization requires d 0 = 1. Smallest positive number Matlab: realmin = e 308

10 Extended Precision Often desirable and occasionally necessary to increase precision Some Software packages can produce arbitrary precision arithmetic Mathematica: $MinNumber = $MaxNumber =

11 Computer Arithmetic A computer can only execute the basic arithmetic operations of addition, subtraction, multiplication, and division. Everything else is approximated. Relative speeds, old values (Exercise 2.7): speed relative operation to addition subtraction 1.03 multiplication 1.03 division 1.06 exponentiation 5.09 sine function

12 Computer Arithmetic Efficient evaluation of polynomials (Horner s method) a 0 + a 1 x + a 2 x 2 + a 3 x 3 = a 0 + x(a 1 + x(a 2 + xa 3 )) Efficient computation of derivatives (automatic differentiation) Consider f(x, y, z) = (x α + y α + z α ) β Then f x = (xα + y α + z α ) β 1 βαx α 1 = f(x, y, z) (x α + y α + z α ) βαxα x 12

13 Error Analysis: Sources of Error Model error: an economic model is only an approximation of a real phenomenon Data error: parameters of the model have to be estimated, forecasted, simulated or approximated; data may be missing; available data may not well reflect the true but unknown process Numerical errors: solving a model on a computer typically results in an approximation of the solution; such approximations are the essence of numerical analysis and involve two types of numerical errors, round-off errors and truncation errors reality model numerical solution 13

14 Numerical Analysis: Sources of Error Numbers are represented by finite number of bits. Real numbers with significand longer than the number of bits available have to be shortened. Examples: irrational numbers, finite numbers that are too long, finite numbers in decimal form that have no finite exact representation in binary form Round-off error: chopping off extra digits or rounding 2 3 stored as or as

15 Round-off Errors If β = 2 and p = 24, then the binary floating point representation of the decimal number 0.1, , (in single precision) is not exact Round-off errors are likely to occur when the numbers involved in calculations differ significantly in their magnitude, or when two numbers that are nearly identical are subtracted from each other. 15

16 Example in Matlab Solve the quadratic equation using the quadratic formula x x = 0 x = b ± b 2 4ac 2a Exact solutions: x 1 = 100 and x 2 =

17 Round-off Errors in Matlab f ormat long; a = 1; b = ; c = 0.01; RootDis = sqrt(b 2 4 a c) RootDis = x1 = ( b + RootDis)/(2 a) x1 = 100 x2 = ( b RootDis)/(2 a) x2 = e

18 Truncation Errors Truncation errors occur when numerical methods used for solving a mathematical problem use an approximate mathematical procedure. Example: The infinite sum e x = x n n=0 n! becomes N x n n=0 n! for some finite N. Truncation error is independent of round-off error and occurs even when the mathematical operations are exact. 18

19 Error Analysis: Controlling Rounding Error Rules of thumb: Avoid unnecessary subtractions of numbers of similar magnitude. First add the smaller numbers and then add the result to the larger numbers. 19

20 Mathematica: Example: Rounding Error Exercise 2.3: Consider the system of linear equations x y = x y = 0 Exact solution: x = and y = However, double-precision arithmetic yields x = e and y = e due to catastrophic cancelation: x = = ( ) ( ) Matlab:

21 Solving the Equations in Matlab A = [ ; ]; b = [ 1 ; 0 ]; A \ b ans = 1.0e+008 *

22 Solving the Equations in Mathematica Clear[x, y]; Solve[{ x y == 1, x y == 0}, {x, y}] {} According to Mathematica the system has no solution. Clear[x, y]; Solve[{ x y == 1, /2 x y == 0}, {x, y}] {{x > , y > }} Now Mathematica finds the exact solution correctly. 22

23 Error Analysis: Controlling Rounding Error Exercise 2.5: Compute 83521y x 2 y 4 2x 4 + 2x 6 x 8 for x = and y = Exact answer: However, double-precision arithmetic yields e (depending on ordering) Individual terms: 83521y 8 = e x 2 y 4 = e x 4 = e x 6 = e x 8 = e y 8 x 8 = e

24 Exercise 2.5 in Matlab x= ; y= ; y x 2 y 4 2 x x 6 x 8 ans = e

25 Exercise 2.5 in Mathematica x = ; y = ; y x 2 y 4 2 x x 6 x Mathematica finds the correct solution. x = ; y = ; y x 2 y 4 2 x x 6 x 8 0 Mathematica states that the solution is zero! 25

26 Error Analysis: Controlling Truncation Error Truncation error occurs in the application of many numerical methods. Example: iterative method x (k+1) = g (k+1) (x (k), x (k 1),...) Need stopping rules to stop sequence {x (k) } when we are close to unknown solution x. Unless sequence x (k) converges for small k stopping rule leads to truncation error. 26

27 Stopping Rules Stop when the sequence is not changing much anymore. Stop when x (k+1) x (k) is small relative to x (k), for small ε. x (k+1) x (k) x (k) ε This rule may never stop the sequence if x (k+1) converges to zero. General stopping rule: stop and accept x (k+1) if x (k+1) x (k) 1+ x (k) ε 27

28 Failure of General Stopping Rule Consider the sequence x k = k j=1 1 j This sequence diverges, but x k tends to infinity very slowly, e.g. x = For ε = the general stopping rule would stop the sequence at k = 1159 with x 1159 = General stopping rule is not reliable. 28

29 Rates of Convergence Key measure for the performance of an algorithm Suppose sequence {x (k) } with x (k) R n converges to x. {x (k) } converges at rate q > 1 to x if for all k sufficiently large. x (k+1) x x (k) x q M < Quadratic convergence: q = 2 Example: (2k) converges at rate q = 2 to 1 29

30 Linear Convergence {x (k) } converges linearly to x at rate β if for all k sufficiently large. x (k+1) x x (k) x β < 1 Example: k converges linearly to 1 at rate β = 0.5. Superlinear convergence: lim k x (k+1) x x (k) x = 0 Example: 1 + k k converges superlinearly to 1. 30

31 Error Analysis: Controlling Truncation Error Adaptive stopping rule: Suppose the sequence {x (k) } converges linearly at rate β to x and x (k+1) x β x (k) x. Then x (k+1) x x(k+1) x (k) 1 β Stop and accept x (k+1) if x (k+1) x (k) ε(1 β). Estimate β as the maximum over x (k l) x (k+1) x (k l 1) x (k+1) for l = 0, 1, 2,

32 Error Analysis: Controlling Truncation Error Exercise 2.11a: Consider the sequence x k = k 3 n n=1 n!. Note that lim k x k = e 3 1 = General stopping rule Adaptive stopping rule: ˆβ = max l=0,1,2,... x (k l) x (k+1) x (k l 1) x (k+1) β = x(k) x (k+1) x (k 1) x (k+1) 32

33 Compute and Verify First, compute an approximate solution to your problem. Second, verify that it is an acceptable approximation according to economically meaningful criteria. Example: Consider the problem of solving f(x) = 0. The exact solution is x, our approximate solution is ˆx. Forward error analysis: How far is ˆx from x? Backward error analysis: Construct a similar problem ˆf such that ˆf(ˆx) = 0. How far is ˆf from f? Compute and verify: How far is f(ˆx) from its target value of 0? 33

34 Compute and Verify Example: f(x) = x 2 2 = 0 Approximate solution ˆx = 1.41 f(1.41) = < 0, f(1.42) = > 0 Bound on forward error ˆx x < 0.01 ˆf(x) = x satisfies ˆf(ˆx) = 0 Backward error ˆf(x) f(x) = f(ˆx) = How large or important is this error? 34

35 Compute and Verify Quantify the importance of the error in economically meaningful terms. Example: Excess demand function E(p) = D(p) S(p) What does E(ˆp) = 0.01 mean? Not much. What does E(ˆp) D(ˆp) = 0.01 mean? A lot. Interpretation of relative error in this example Leakage between demand and supply due to market frictions. Optimization error of boundedly rational agents. 35

36 Compute and Verify Relative errors in economically meaningful terms Advantage: Generally applicable (unlike forward error analysis). Disadvantage: More than one solution may be deemed acceptable (like backward error analysis). 36

37 Summary Computer arithmetic Error analysis: Sources of error Error propagation Controlling the error Rates of convergence Compute and verify 37

Computational Economics and Finance

Computational Economics and Finance Part I: Elementary Concepts of Numerical Analysis Spring 2016 Outline Computer arithmetic Error analysis: Sources of error Error propagation Controlling the error Rates