Examples of Linear Approximation

Transcription

1 M28Spring07 J. Simon Examples of Linear Approximation Here are several examples of using derivatives to find the "best" linear [I'd rather say "affine", but the text says "linear"] approximation to a given function at a particular point. This all is a generalization of the idea that a line tangent to a curve is a close approximation to that curve around the point of tangency. > with(plots):with(linalg):with(plottools): Warning, the name changecoords has been redefined Warning, the protected names norm and trace have been redefined and unprotected Warning, the assigned name arrow now has a global binding EXAMPLE Let's start by thinking of Calc I situations. The function f(x) = x^2 and the function g(x) = x+3 both have value 9 when x=3. > f:=x->x^2; g:=x->x+6; > plot({f(x), g(x)}, x=2..4); f := x x 2 g := x x + 6

2 x Since both functions are continuous, it is true that the limit of f(x) as x--> 3 is f(3) = 9, and the limit of g(x) as x approaches 3 is g(3) = also 9. So we could say > Limit(abs(`f(x)-g(x)`),x=3)=0; lim x 3 f(x)-g(x) = 0 So the function g(x)=x+6 is *some kind* of linear [again, I'd prefer the word "affine"] approximation of f(x) for points x near 3. For example, at the point x=3.0 > f(3.0); > g(3.0); > abs(f(3.0)-g(3.0)); When we move x by amount 0.0 from value x=3, the difference between f(x) and g(x) is around 0.05, or 5

3 times the change in x. Of all the linear functions g(x) that have g(3)=9, there is one that is the "best" approximation to f(x). We get this by making the slope of g(x) the same as the slope of f(x) at x=3. We find that slope by taking the derivative of f, that is f ' (3). > dfdx:=diff(f(x),x); dfdx := 2 x > dfdx_at3:=subs(x=3,dfdx); dfdx_at3 := 6 We want to define a linear function h(x) that has two properties: slope = 6, and value h(3)=9. You know a nice way to do this - the "point-slope" form of equation for a line. Instead of writing "y-y = m(x-x)", we write "y = y + m(x-x)". That says the same thing. Note that 9+6(x-3) simplifies to 6x-9. > h:=x->9+6*(x-3); > plot({f(x), h(x)}, x=2..4); h := x x x

4 Notice that when x is near 3, the graphs of f and g are almost identical. Let's see what the difference between them is for some point near x=3. > f(3.0); > h(3.0); > abs(f(3.0)-h(3.0)); Notice that when we move the domain point by 0.0, we get a change of in the output. Another way to capture this improvement is to observe that not only does f(x)-h(x) go to 0 as x-->3, but in fact the ratio f(x)-h(x) / x-3 goes to 0 as x-->3. > Error:=abs(`f(x)-h(x)`); Error := f(x)-h(x) > Error:=x->abs(f(x)-h(x)); Error := x f( x) - h( x) > EvenThisIsSmall:=x->Error(x)/abs(x-3); EvenThisIsSmall := x Error( x) x - 3 > EvenThisIsSmall(3.); > EvenThisIsSmall(3.0); > EvenThisIsSmall(3.00); Note that as x-->3, the value of EvenThisIsSmall goes to 0 at the same rate as the difference between x and 3. > Limit(Error(x)/abs(x-3),x=3)=0; æ ç lim ç x 3 è x x x - 3 Remark. You learned in Calc I some algebra tricks for checking such a limit. That is not my emphasis here, but just to remind you... > factor(x^2+9-6*x); ( x - 3) 2 ö ø = 0

5 So the numerator factors as (x-3)^2; we divide top and bottom by (x-3) to get that the quotient really is (up to + or -) just x-3, which certainly goes to 0 as x--> 3. Let's do one more example from Calc I. EXAMPLE 2 > f:=x->sin(x); f := x sin( x) Let's find a linear approximation to f near the domain point x=. > evalf(f()); > dfdx:=diff(f(x),x); > evalf(subs(x=,dfdx)); dfdx := cos( x) So we want a linear function h(x) that has h()= , and that has slope = > h:=x-> *(x-); h := x ( x - ) > simplify(h(x)); x > plot({f(x),h(x)}, x=0..2);

6 x Note that when x is close to, the functions f(x) and h(x) are very close - they are "tangent" at x=. Let's check the error...look at (how much to f and h differ) divided by (how far have we moved x from the point where h=f). > E:=(f(.)-h(.))/(0.); > E2:=(f(.0)-h(.0))/(0.0); E := E2 := > E3:=(f(.00)-h(.00))/(0.00); E3 := Notice that as x-->, not only do h(x) and f(x) get close, but they get so close that even when we divide the difference by the tiny numbers (x-), the quotient still goes towards 0. ################################

7 EXAMPLE 3 Now let's look at an example where f is a function of 2 variables. > f:=(x,y)->x^2/y+sin(y); f := ( x, y) x 2 y + sin( y) > plot3d([x,y,f(x,y)], x=0..3, y=..4, axes=boxed, labels=[x,y,z], orientation=[-58,64]); Z X Y Let's construct a linear approximation of f(x,y) near the domain point (x,y)= (2,2). > f22:=f(2,2);f22:=evalf(%); f22 := 2 + sin( 2) f22 :=

8 > Df:=[diff(f(x,y),x),diff(f(x,y),y)]; Df := 2 x y, - x 2 + cos( y) ù y 2 > Df22:=subs(x=2,y=2,Df);Df22:=evalf(%); Df22 := [ 2, - + cos( 2) ] Df22 := [ 2., ] > h:=(x,y)->f22+df22[]*(x-2)+df22[2]*(y-2); h := ( x, y) f22 + Df22 ( x - 2) + Df22 2 ( y - 2) > h(x,y); x y Let's check that we did this correctly, so that h(2,2) = f(2,2), and df/dx = dh/dx, df/dy = dh/dy at point (2,2). > h(2,2); > f(2,2);evalf(%); > diff(h(x,y),x); > diff(h(x,y),y); 2 + sin( 2) Since h is a linear function, dh/dx and dh/dy are constant. And they are constant with values equal to the partial derivatives of f at point (2,2). Now let's see the graphs: > plotsurface:=plot3d([x,y,f(x,y)], x=0..3, y=..4, axes=boxed, labels=[x,y,z], orientation=[-58,64], grid=[0,0]): > plotplane:=plot3d([x,y,h(x,y)], x=0..3.5, y=..4.5, axes=boxed, labels=[x,y,z], orientation=[-58,64], style=patchnogrid, color=blue): > plotpoint:=sphere([2,2, ],., color=black): > display({plotsurface, plotplane, plotpoint});

9 Z X Y 4.5 Now let's see how f(x,y) and h(x,y) differ for points (x,y) close to (2,2). > for i from to 5 do delta_x:=/(2*i); delta_y:=/(3*i); DiffBetFandH:=evalf(f(2+delta_x, 2+delta_y)-h(2+delta_x, 2+delta_y)); SizeOfPerturbation:=evalf(sqrt(delta_x^2+delta_y^2)); HopeQuotientGoesToZero:=evalf(DiffBetFandH/SizeOfPerturbation); od; delta_x := 2 3 DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := 4

10 6 DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := 6 9 DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := 8 2 DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := 0 5 DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := > for i from 00 by 00 to 500 do delta_x:=/(2*i); delta_y:=/(3*i); DiffBetFandH:=evalf(f(2+delta_x, 2+delta_y)-h(2+delta_x, 2+delta_y)); SizeOfPerturbation:=evalf(sqrt(delta_x^2+delta_y^2)); HopeQuotientGoesToZero:=evalf(DiffBetFandH/SizeOfPerturbation); od; delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero :=

11 delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := The convergence is not quick, but it does still seem that as (x,y) --> (2,2), the functions f(x,y) and h(x,y) are so close that the quiotient (f(x,y)-h(x,y)) / (x-2, y-2) approaches 0. > for i from 000 by 000 to 5000 do delta_x:=/(2*i); delta_y:=/(3*i); DiffBetFandH:=evalf(f(2+delta_x, 2+delta_y)-h(2+delta_x, 2+delta_y)); SizeOfPerturbation:=evalf(sqrt(delta_x^2+delta_y^2)); HopeQuotientGoesToZero:=evalf(DiffBetFandH/SizeOfPerturbation); od;

12 CONCLUSION: delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := delta_x := DiffBetFandH := SizeOfPerturbation := HopeQuotientGoesToZero := THE ABOVE EXAMPLES ILLUSTRATE THE FOLLOWING PRINCIPLE:

13 For a function f(x,y), around a point (a,b), the affine function h(x,y) = f(a,b) + (df/dx)(x-a) + (df/dy)(y-b) [here the partial derivatives are evaluated at (x,y) = (a,b)] is the best possible affine approximation of f(x,y). In particular, the limit as (x,y) approaches (a,b) is... > Limit(abs(`f(x,y)-h(x,y)`)/abs(`(x,y)-(a,b)`)) = 0; æ f(x,y)-h(x,y) ö Limitç = 0 ç (x,y)-(a,b) è ø This is actually the formal definition of "differentiable function". We say that f(x,y) is differentiable at the point (a,b) IF THERE EXISTS an affine function h(x,y) with the above property. ################ EXAMPLE 4 Example of non-differentiable function: > f:=(x,y)->+sqrt(x^2+y^2); f := ( x, y) + x 2 + y 2 > PlotCone:=plot3d([x,y,f(x,y)], x=-2..2, y=-2..2, axes=boxed, labels=[x,y,z], orientation=[-58,64], grid=[,]): > PlotPlane:=plot3d( [x,y,.*x+.2*y+],x=-2..2, y=-2..2, axes=boxed, labels=[x,y,z], orientation=[-58,64], grid=[5,5], color=tan): > display({plotcone,plotplane});

14 Z X Y There are many planes that pass through the point (0,0,); equivalently, there are many affine functions h(x,y) that agree with f(x,y) at the point (x,y)=(0,0). But there is no plane tangent to the cone at that point; equivalently, there is no affine function h(x,y) that is so close to f(x,y) near (0,0) that the quotients f(x,y)-h(x,y) / (x,y) - (0,0) would be near 0. ########################### EXAMPLE 5 Here is one more example of a differentiable function and its "best affine approximation". Suppose f is a function with 2-dimensional input and 2-dimensional output, that is f: R^2 --> R^2. For example, > f:=(x,y)->[x^2*y, x^2-y^2]; f := ( x, y) [ x 2 y, x 2 - y 2 ] Let's consider the domain point (x,y)=(2,3). Just as in the simpler examples above, we want to define an affine

15 function of the form h(x,y) = f(2,3) + Df(2,3)*[x-2,y-3]. Here f(2,3) is a 2-dimensional vector, and the "derivative" Df(2,3) is the 2x2 matrix of partial derivatives. > DfMatrix:=matrix([[Diff(f(x,y)[],x), Diff(f(x,y)[],y)],[Diff(f(x,y)[2],x), Diff(f(x,y)[2],y)]]); Evaluate the partial derivatives to get DfMatrix := ( x x 2 y) ( x x 2 - y 2 ) y x 2 ù ( y) ( y x 2 - y 2 ) > Df:=matrix([[diff(f(x,y)[],x), diff(f(x,y)[],y)],[diff(f(x,y)[2],x), diff(f(x,y)[2],y)]]); Df := 2 x y x 2 ù 2 x -2 y Plug in x=2, y=3 to get the derivative matrix Df at the point (2,3), that is Df(2,3). > Df23:=matrix([[2*2*3, 2^2],[2*2, -2*3]]); Df23 := 2 4ù 4-6 To define h(x,y), we also need to know the value (here a vector) of f(2,3). > f(2,3); [ 2, -5] So now we can write out h(x,y). [There is a problem with notation. When we write the derivative matrix of f acting on the vector [x-2, y-3], the right way to do that is to write [x-2, y-3] as a column vector; then the "action" of the matrix Df(2,3) is just matrix multiplication. Then we need also to write the constant vector f(2,3)=[2,-5] as a column vector too. Maple is a little awkward with this; don't worry about the red Maple commands that I am using to do this, just look at the

16 resulting blue results.] > print(`h`=matrix([[2],[-5]])+ evalm(df23)* matrix([[x-2],[y-3]])); h = 2ù ù 4-6 x - 2ù y - 3 > h:=(x,y)->matrix([[2+2*(x-2)+4*(y-3)],[-5+4*(x-2)-6*(y-3)]]); x + 4 yù h := ( x, y) x - 6 y To compare h(x,y) with f(x,y), we should also write the output of f(x,y) as a column vector. > F:=(x,y)->matrix([[f(x,y)[]], [f(x,y)[2]]]);f(x,y); f( x, y) ù F := ( x, y) x 2 y ù x 2 - y 2 f( x, y) We claim that ff is very close to h(x,y) when (x,y) is near (2,3); so close, that if we divide the difference by the size (x,y)-(2,3), we still get something very small. Here are some examples to see if this is right. Case. (x,y) = (2.0, 3.02) > F(2.0,3.02); ù > h(2.0, 3.02); 2.20ù > ErrorVector:=F(2.0,3.02)-h(2.0,3.02);ErrorVector:=evalm(%); ù 2.20ù ErrorVector := ErrorVector := ù

17 > SizeOfErrorVector:=sqrt(ErrorVector[,]^2 + ErrorVector[2,]^2); SizeOfErrorVector := > Perturbation:=evalm([[.0], [.02]]); Perturbation := 0.0ù 0.02 > SizeOfPerturbation:=sqrt(Perturbation[,]^2+Perturbation[2,]^2); SizeOfPerturbation := > Quotient:=SizeOfErrorVector/SizeOfPerturbation; Quotient := So not only is h(x,y) close to f(x,y), but the difference is so small that we can divide it by.02 and still get a small number. Let's try a smaller perturbation, and then declare "Victory" for this topic. Case 2. (x,y) = (2.0003, 3.000) > F(2.0003,3.000); > h(2.0003, 3.000); ù ù > ErrorVector:=F(2.0003,3.000)-h(2.0003,3.000);ErrorVector:=evalm(%); ErrorVector := ErrorVector := ù ù ù > SizeOfErrorVector:=sqrt(ErrorVector[,]^2 + ErrorVector[2,]^2); SizeOfErrorVector := > Perturbation:=evalm([[.0003], [.000]]); Perturbation := ù 0.000

18 > SizeOfPerturbation:=sqrt(Perturbation[,]^2+Perturbation[2,]^2); SizeOfPerturbation := > Quotient:=SizeOfErrorVector/SizeOfPerturbation; Quotient := The Quotient is not as small as the perturbation, but still it is smaller than the previous Quotient. As the perturbation gets closer to 0, the Quotient gets closer to 0. This says that our affine function h(x,y) is tangent to the given function f(x,y) at the point (2,3). ########## End of Handout #############