Numerical Methods 5633

Transcription

1 Numerical Methods 5633 Lecture 3 Marina Krstic Marinkovic mmarina@maths.tcd.ie School of Mathematics Trinity College Dublin Marina Krstic Marinkovic 1 / Numerical Methods

2 Organisational Assignment 0 - summary of the comments To appear Submission DL Assignment Assignment Assignment Last 2 weeks - Tue. or Thurs. (+Fri.) lectures? 15.11/22.11 (14-16h) or 17.11/24.11 (14-16h) Module MA3463 (Computation theory and logic)? -> Tue. Marina Krstic Marinkovic 2 / Numerical Methods

3 Curve Fitting From Wikipedia: Curve fitting is the process of constructing a curve, or mathematical function, that has the best fit to a series of data points, possibly subject to constraints. References for this lecture: Svante Wold Spline Functions in Data Analysis, 1974, Technometrics, Vol. 16, No. 1. Link to the paper D.G.Rossiter, Technical note: Curve fitting with the R Environment for Statistical Computing (link to the note) Marina Krstic Marinkovic 3 / Numerical Methods

4 Linear regression Least squares (linear): given a set of observations, (xi, yi), we seek a best fitting line (pair of numbers (a,b)) such that the sum of the squares (yi - bxi - a)^2 attains a minimum value y(x) =a + bx R 2 X y i y(x) 2 R 2 (a, 2 = 2 = 2 Least squares (polynomial): line > curve nx yi (a + bx i ) 2 R programming: The command to perform the least square regression is the lm command i=1 nx yi (a + bx i ) =0 i=1 nx yi (a + bx i ) x i =0 i=1 Marina Krstic Marinkovic 4 / Numerical Methods

5 Linear regression Least squares (linear): given a set of observations, (xi, yi), we seek a best fitting line (pair of numbers (a,b)) such that the sum of the squares (yi - bxi - a)^2 attains a minimum value y(x) =a + bx R 2 X y i y(x) 2 ss xx = nx nx x i x 2 ss yy = nx i=1 ss xy = x i x (y i ȳ i=1 i=1 y i ȳ 2 2 x = ss xx n 2 y = ss yy n cov(x, y) = ss xy n Least squares (polynomial): line > curve R programming: The command to perform the least square regression is the lm command Marina Krstic Marinkovic 5 / Numerical Methods

6 Linear regression Least squares (linear): given a set of observations, (xi, yi), we seek a best fitting line (pair of numbers (a,b)) such that the sum of the squares (yi - bxi - a)^2 attains a minimum value y(x) =a + bx R 2 X y i y(x) 2 2 x = ss xx n 2 y = ss yy n cov(x, y) = ss xy n b = cov(x, y) 2 x = ss xy ss xx a =ȳ b x Least squares (polynomial): line > curve R programming: The command to perform the least square regression is the lm command Marina Krstic Marinkovic 6 / Numerical Methods

7 Linear regression Least squares (linear): given a set of observations, (xi, yi), we seek a best fitting line (pair of numbers (a,b)) such that the sum of the squares (yi - axi - b)^2 attains a minimum value R 2 X y i y(x, a 1,a 2,...,a n ) 2 minimising the sum of the squares of the vertical deviations of a (n-1) th order polynomial Least squares (polynomial): line > curve R programming: The command to perform the least square regression is the lm command Marina Krstic Marinkovic 7 / Numerical Methods

8 Example in R: linear regression x <- c(32,64,96,118,126,144,152.5,158) y <- c(99.5,104.8,108.5,100,86,64,35.3,15) #we will make y the response variable and x the predictor #the response variable is usually on the y-axis plot(x,y,pch=19) #fit first degree polynomial equation: fit <- lm(y~x) #second degree fit2 <- lm(y~poly(x,2,raw=true)) #third degree fit3 <- lm(y~poly(x,3,raw=true)) #fourth degree fit4 <- lm(y~poly(x,4,raw=true)) #generate range of 50 numbers starting from 30 and ending at 160 xx <- seq(30,160, length=50) plot(x,y,pch=19,ylim=c(0,150)) lines(xx, predict(fit, data.frame(x=xx)), col="red") lines(xx, predict(fit2, data.frame(x=xx)), col="green") lines(xx, predict(fit3, data.frame(x=xx)), col="blue") lines(xx, predict(fit4, data.frame(x=xx)), col="purple") Marina Krstic Marinkovic 8 / Numerical Methods

9 Example in R: linear regression y x Marina Krstic Marinkovic 9 / Numerical Methods

10 Runge s function f(x) = x 2 f <- function(x) { 1/(1+25*x*x) } source( Runge_function.R ) x<-seq(-1,1,by=0.01) y<-f(x) #assigning values of Runge function to y plot(x,y,col='blue',xlim=c(-1.1,1.1),ylim=c(-0.5,1.2)) #second degree fit2<-lm(y ~ poly(x, degree=2)) points(x,fitted(fit2),col= green') #fifth degree fit5<-lm(y ~ poly(x, degree=5)) points(x,fitted(fit5),col= red') #tenth degree fit10<-lm(y ~ poly(x, degree=10)) points(x,fitted(fit10),col='black') par(mar=c(5, 4, 4, 8) + 0.1) mtext("d=2", side=3, line=3,col='green') mtext("d=5", side=3, line=2,col='red') mtext("d=10", side=3, line=1,col='black') Marina Krstic Marinkovic 10 / Numerical Methods

11 Runge s function f(x) = x 2 f <- function(x) { 1/(1+25*x*x) } d=2 d=5 d=10 y x Marina Krstic Marinkovic 11 / Numerical Methods

12 Runge s function - polynomial fit error plot(x,y-fitted(fit10),col='black',xlim=c(-1.1,1.1),ylim=c(-0.5,1.2),ylab="fit error") points(x,y-fitted(fit5),col='red') points(x,y-fitted(fit2),col='green') d=2 d=5 d=10 fit error Marina Krstic Marinkovic 12 / Numerical Methods x

13 Piecewise polynomial interpolation vs. Splines To obtain interpolants that are better behaved, we look at other forms of interpolating functions y If we are given a set of data points (x 1,y 1 ) (x n,y n ) x 1< x 2 < <x n the simplest way to connect the points (x j,y j ) is by straight line segments, Piecewise linear interpolation x 2.or by using piecewise quadratic interpolation y {(x 1,y 1 ),(x 2,y 2 ),(x 3,y 3 )} {(x 3,y 3 ),(x 4,y 4 ),(x 5,y 5 )} Or by constructing k-th order polynomial {(x 1,y 1 ),(x 2,y 2 ),,(x k+1,y k+1 )} Polynomial Interpolation x Marina Krstic Marinkovic 13 / Numerical Methods

14 Piecewise polynomial interpolation vs. Splines Smooth non-oscillatory interpolation: splines If we are given a set of data points (x 1,y 1 ) (x n,y n ) x 1< x 2 < <x n Consider functions s(x) fulfilling properties: 1.s(x i )=y i i=1 n, 2.s(x i ),s (x i ),s (x i ) are continuous on [x 1, x n ] Then among such functions s(x) satisfying 1. and 2., find the one which minimises the integral Z xn x 1 s 00 (x) 2 dx The idea of minimising the integral is to obtain an interpolating function for which the first derivative does not change rapidly [Illustrations from Wikipedia] Marina Krstic Marinkovic 14 / Numerical Methods

15 Natural and complete spline approximation Natural spline boundary conditions: s (x 1 ) = s (x n ) = 0; Complete spline boundary conditions: s (x 1 ) = f (x 1 ); s (x n ) = f (x n ); This leads to better approximation properties, and does not require that we know the second derivative at the end points. See pdf of lecture notes, for derivations and more examples. Marina Krstic Marinkovic 15 / Numerical Methods