Lecture 6: Interpolation

Lecture 6: Interpolation Fatih Guvenen January 10, 2016 Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 1 / 25

General Idea Suppose you are given a grid (x 1, x 2,..., x n ) and the function values (y 1, y 2,...,y n ) at corresponding points generated by function f (x) Q: How to find values off the grid points that provides a good approximation to f (x)? A good approximation is often taken to mean to minimize f (x) b f (x) according to some norm (L p, sup-, etc). In economics another very important concern is to preserve the shape ie, concavity or convexity of the interpolated (e.g, utility or value) function Popular methods are linear-, Chebyshev-, and Splineinterpolations. I will spend most of my time on Splines. Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 2 / 25

Polynomial Approximation Weierstrass Approximation Theorem. Suppose f is a continuous real-valued function defined on the real interval [a, b]. For a given ">0, there exists a polynomial of order n, P n (x), such that for all x in [a, b], we have kf (x) P n (x)k 1 <". In the limit, lim kf (x) P n(x)k n!1 1 = 0. Runge. Let f (x) =1/(1 + x 2 ) on [-5,5], and let L m f be the unique polynomial of order m that interpolates f at m equally-spaced points. Then: lim sup f (x) L m f = m!1 ( 0 if x < 3.633..., 1 if x > 3.633... The two theorems are not contradictory. The first theorem does not provide a way of finding the right P n (x) and the second one shows a naive approach can fail spectacularly. Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 3 / 25

Runge Example

Splines: Three Objectives 1 Match the function values at grid points (y 1, y 2,...,y n ) exactly. 2 Generate first derivatives that are continuous and differentiable for all x 2 [x 1, x n ]. 3 Generate second derivatives that are continuous for all x 2 [x 1, x n ] Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 5 / 25

Splines: Building from Ground Up Begin with the interval between two generic knots, x i and x i+1. If we were to construct a linear interpolant: y = Ay j + By j+1 A(x) x j+1 x B(x) 1 A = x x j (1) x j+1 x j x j+1 x j Although linear interpolation is sometimes useful, a shortcoming is that I I First derivative changes abruptly at the knot points, i.e., the interpolants have as many kinks as the knot points. Second derivative does not exist at knot points. This can create many problems, such as difficulties with derivative based minimization algorithms. Question: How can we modify (1) to allow differentiable first derivatives and continuous second derivatives at all points? Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 6 / 25

Splines: Building from Ground Up Generalize (1) using second derivative values at knot points: y = A(x)y j + B(x)y j+1 + C(x)y 00 j + D(x)y 00 j+1 (2) where C(x) = 1 6 (A3 (x) A(x))(x j+1 x j ) 2 and D(x) = 1 6 (B3 (x) B(x))(x j+1 x j ) 2 Note that you only need to know A and B to calculate everything. Verify that d 2 y dx 2 = A(x)y 00 j + B(x)y 00 j+1 Since A(x j )=1and B(x j+1 )=1 A(x j+1 )=1, the second derivative agrees with y 00 at end points. But how to find y 00 j and y 00 j+1? Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 7 / 25

Splines: Building from Ground Up Differentiate (2) to obtain an expression for dy dx and y 00 j+1. that will involve y 00 j Then impose the condition that dy dx calculated using (x j, x j+1 ) and using (x j+1, x j+2 ) equal each other at x j+1. We get: x j x j 1 y 00 j 1 6 + x j+1 x j 3 y 00 j + x j+1 x j 6 y 00 j+1 = y j+1 y j x j+1 x j y j y j 1 x j x j 1 For interior knot points, j = 2,...,N 1 we have an equation like this. But we have N unknowns (y j for j = 1,...,N). Impose two boundary conditions to pin down y 00 00 1 and yn. (Set them to zero or some reasonable value given the economics of the problem). Notice: no condition is imposed for y 0 or y 00 to agree with f 0 and f 00, since these are unknown. This can create problems as we will see. Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 8 / 25

Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 9 / 25

Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 10 / 25

Comparing Interpolation Methods for U(C)

Comparing Boundary Conditions Figure: N = 500 pts -10 6-10 7 Log Axis -10 8-10 9-10 10 y 1 = U (0.05) y 1 =0 Natural Spline True Function -10 11 0.04 0.06 0.08 0.1 0.12 0.14 0.16

Comparing Boundary Conditions 10 10 5 Figure: N = 100 pts Linear Axis 0-5 -10 y 1 = U (0.05) y 1 =0 Natural spline True Function 0.05 0.1 0.15 0.2 0.25

Interpolation: What Can Go Wrong 2 x 108 1.5 1 0.5 0 0.5 CRRA function with γ =10 8th-order polynomial Cubic spline Shumaker quadratic spline 1 0 0.5 1 1.5 2 Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 14 / 25

Interpolation: What Can Go Wrong 10 0 10 20 30 40 50 60 0 0.5 1 1.5 2 Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 15 / 25

Standard Spline vs Shape-Preserving 1.5 1 0.5 0 0.5 1 y=tanh(x) at x= 5, 4, 3,... Cubic spline Shumaker s spline y=tanh(x) continuous 1.5 5 4 3 2 1 0 1 2 3 4 5 Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 16 / 25

Spacing of Grid Points: Crucial!! One heuristic: put more grid points where f has more curvature. Another one: put more points near the parts of the function that are more relevant. I e.g., in an RBC style model, put more points near K, even though the functions of interest may have more curvature at low K values. In incomp. mkts models, V 00 (!) is largest for very low!. So should put more points there. Is this true if there are not many individuals near the constraint? (Answer: Typically, Yes. But why?) In some DP problems, with max operator on the RHS, the value function may have a kink or significant curvature somewhere in the middle of the state space. I Linear interpolation maybe your best choice. Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 17 / 25

Figure: Grid Point Locations: 51-Point Expanding Grid From 0 to 250 θ = 1 θ = 2 θ = 3 θ = 4 θ = 1 θ = 2 θ = 3 θ = 4 0 1 2 3 4 5 Wealth 220 225 230 235 240 245 250 Wealth (a) Low End of Grid (b) High End of Grid Note: The number of grid points between 0 and 4.99 is 1, 8, 14, and 19 when is equal to 1, 2, 3, and 4, respectively.

Algorithmus 1 : CREATING APOLYNOMIALLY-EXPANDING GRID Step 1. First, create an equally-spaced [0,1] grid: { x j : x j = j 1 N 1, j = 1,...,N}. Step 2. Shift and expand the grid: x = {x j : x j = a +(b a) x j }, where >1 is the expansion factor. Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 18 / 25

Spline w/ Expanding Grid(1000 pts) Figure: N = 1000 pts 100 50 Equally-spaced, θ =1.0 θ = 1.05 θ = 1.10 0-50 -100 0 0.2 0.4 0.6 0.8 1 Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 19 / 25

Spline w/ Expanding Grid (100 pts) Figure: N = 100 pts 10 5 U(C) θ = 1.0 θ = 2.0 θ = 3.0 0-5 -10 0 1 2 3 4 5 Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 20 / 25

AWordaboutB-Splines A fundamental theorem about B-splines (see de Boor (1978)): Constructing a cubic spline interpolation with (m, n) points in (x, y) direction requires: I I O((mn) 2 ) operations using regular splines. O(m 3 + m 2 n + mn 2 + n 3 ) operations with B-splines. Suppose m = n = 10: I I Regular spline requires 1, 000, 000 operations B-splines require: 4, 000 operations! ) Learn about B-splines! Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 21 / 25

ATricktoReducetheCurvatureofV(w) Samuelson (1969) showed that in a standard portfolio choice problem with CRRA preferences and a linear budget set, the value function inherits the curvature of U: U(c 0, c 1,...)= 1X t=1 t c1 t 1 ) V (!, A) = (A)! 1 The same result holds approximately true in a variety of different problems. With incomplete markets V (w) will typically have even more curvature than U(c) especially at low wealth levels. As we have seen so far, this high curvature creates a lot of headache when you try to interpolate the value function. Fortunately, there is a way out! Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 22 / 25

ATricktoReducetheCurvatureofV(w) There is an alternative formulation of CRRA preferences: U(c 0, c 1,...)= 1X t=1 t c (1 ) t! 1/(1 ) This is a special case of Epstein-Zin (1989, E trica) utility and represents the same preferences as CRRA utility with RRA =. Now the value function is linear: V (!, A) = (A)! Although incomplete markets introduces some curvature, this value function is much easier to interpolate than the one above. In fact, I once solved a GE model with asset pricing and a risk aversion of 4 using only 30 points in the wealth grid and linear interpolation! Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 23 / 25

-10-10 0.21-10 -5 Standard CRRA Utility (left axis) -10 0 0.20-10 5 Epstein-Zin Value Func. (right axis) -10 10 0.19-10 15 0 1 2 3 4 5 Figure: Which Function Would You Rather Interpolate? Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 24 / 25

Final Thoughts For any model that you solve, you MUST eventually re-solve it on a much finer grid and confirm that your main results are not changing (much). This is the only realistic way to check if approximation errors coming from interpolations are important. You will be surprised to find that some bad-looking interpolations actually yield the same results as much more accurate (and more costly to compute) interpolations. And vice versa.. Some problems are especially sensitive to any kind of approximation errors. We will see examples. Fatih Guvenen ( ) Lecture 6: Interpolation January 10, 2016 25 / 25