Cubic smoothing spline

Transcription

1 Cubic smooting spline Menu: QCExpert Regression Cubic spline e module Cubic Spline is used to fit any functional regression curve troug data wit one independent variable x and one dependent random variable y. Number of points (x i, y i ) is n. e regression model y = f(x) + ε is composed of p cubic curves defined on p adjacent segments, xu,1 xu,1, xu,2... xu, p 1, of te x-axis separated by p 1 knots. e values x u,i are knots and can be defined by te user. Analytical properties of te curve (like smootness, curvature, continuity) and statistical properties (residual variance, prediction variance) are subject to modeling. e polynomial regression spline can be written generally as y x S x a, 2 S x 1, x, x,..., x ; 0 were is te polynomial order, in case of =3 it is a cubic spline. S (x) is te vector or predictors and a k is vector of regression coefficients in te k-t segment. If te measured value at x i is y i, and x i is in te k-t interval (k-t segment) wic is te interval (x u,k-1 ; x u,k ), ten te predicted value f(x i ) is given by f(x i ) = a k,1 + a k,2 x i + a k,3 x i 2 + a k,4 x i 3 Parameters a k,i are computed by linear least squares regression. If no continuity for te function at knots was required, te model would be p independent regression polynomials as sown on Fig. 1. e sape of te regression curve depends eavily on te cosen number of segments as sown on Fig. 2a, b. k Fig. 1 Example four independent cubic regression polynomials (discontinuous spline)

2 Fig. 2 a, b Discontinuous regression splines for p = 3 and p = 6 e above models are discontinuous, tey sow flexibility of regression splines, tey are of little use owever. Usually, we want te model to be continuous in f(x), so we impose te condition S x k 1 S x a a k. Furter, we can make use of easy differentiability of polynomials and introduce conditions of continuous first and second derivatives at knots. For our cubic splines we ave = 3. S x k 1 S x a a, S x a S x k k 1 a were te first and second derivative of te polynomial are defined as follows: x 2 1 S S x 0,1,2 x,3 x,..., x x 2 S x 2 S x 0,0,2,6 x,..., 1 x 2 x Imposing tese conditions will result in a curve wit second order differentiability. Suc a spline curve will be suitable for a wide class of applications. is smoot curve will. k

3 ave continuous second derivative and smoot first derivative in te sense tat S 3 '' (x) S 3 '' (x+δ) < ε for any ε > 0 and sufficiently small nonzero δ. Compared to kernel smooters or moving average (see Basic Statistics module), cubic splines eliminate bias in estimating mean, especially near maxima or minima. anks to using a linear regression model it is possible to compute statistical caracteristics suc as confidence intervals, standard deviation, covariance matrices, error estimates. Curves on Fig. 3 troug Fig. 5 sow te same data fitted wit splines wit continuous f(x), continuous f(x) and f '(x), and continuous f(x), f '(x) and f ' '(x). e data are te same as in Fig. 1. No boundary conditions are posed at te beginning or te end of te curve in te Cubic Spline module. Fig. 3 Fit te data wit a continuous spline function f(x) Fig. 4 Fit te data wit continuous f(x) and f '(x)

4 Fig. 5 Fit te data wit continuous f(x) and f '(x) and f ''(x) (te smootest curve) Knots can be defined by te user (by default, te knots are spaced on x so tat te number of points in eac segment be te same if possible). e influence of number and position of knots on te spline curve is illustrated on Fig. 6. e smallest number of knots is 1, dividing te spline into two segments (number of segments p = 2). Maximum number of knots is n 1 (not generally recommended), were n is number of data points. In tis case, te spline can provide no statistical information (suc as variances or confidence intervals) as it passes troug all points. Since te spines can sometimes be overdetermined a generalized Moore-Penrose pseudoinverse is used to compute te regression coefficients to ensure correct solution. Fig. 6 Influence of knots selection (same data in all 4 plots)

5 Polynomial form of te spline curve allows analytical evaluation of derivatives and integrals of te curve wic can be used for analysis of pysical or cemical processes as suggested on Fig. 7 and Fig. 8. Fig. 7 Using spline to find area under curve and derivatives Fig. 8 Using Spline module to determine inflex points on a titration curve Data and parameters Data X and Y are expected in two columns in te Data seet. Furter columns may contain positions of user-defined knots, new x-data for prediction and weigts for individual measurements. An example of data structure is sown on Fig. 9 Fig. 10. It is good to keep in mind tat eac segment may contain at most one maximum, one minimum and one inflex

6 point. If te data for prediction is defined, te predicted y-values, teir 1 α confidence interval and analytical derivatives and integrals are computed in te protocol. Note: e number m of columns of te caracteristic matrix is m = 4p. ere may be an issue of numerical instability in case of too many segments. It is recommended to use rater small number of segments, say 2 to 10, but not more tan 50. A possible consequence of too many segments is illustrated on Fig. 11. Fig. 9 Basic data structure for spline Fig. 10 Full possible structure of spline data A: p=5 B: p=30 C: p=50 Fig. 11 Suitable (A, B) and too big (B, C) number of segments p for n=150

7 Fig. 12 Dialog window for Cubic spline module Open te Cubic Spline dialog window (Menu: QC.Expert Regression Cubic Spline). Coose te columns wit independent and dependent variable. Optionally, coose te column wit description of individual cases, ceck te Weigts and Prediction ceck boxes and select appropriate columns. In te group Knots select No of segments (knots will be cosen automatically), or Knots from table (knots must be given in te selected column of te data table). It is recommended to coose number of segments p so tat number of points n is divisible by p. Generally, number of points in te first (p 1) segments is equal to IN(n/p), te last p t segment contains te rest of te points. For example, if n=12 and p=7 ten first six segments will contain one point wile te last segment will contain 6 points. Continuity requirements are set in te group Continuity in. e user may select any combination of continuity in function values, first and second derivatives. By default, all boxes are cecked (recommended). Cecking te box Data and residuals in te group Protocol will produce a table of data and residuals. is table as as many rows as te original data and may be avoided if not necessary. Clicking OK will run te computation. Protocol ask name ask name from te dialog window Data Selected data subset (All, Marked, Unmarked, or user-defined filter) Independent variable Selected independent variable column Dependent variable Selected dependent variable column No of points Number of data rows n No of knots Number of knots (p 1) No of sections Number of segments p Continuous function Required continuity in function values (Yes / No) Continuous first derivatives Required continuity in first derivatives (Yes / No)

8 Continuous second derivatives Required continuity in second derivatives (Yes / No) Knots positions Number and x-position of knots Model parameters Number of data points in individual segments and estimated regression coefficients of te polynomial y = a 0 + a 1 x + a 2 x 2 + a 3 x 3 ; A(0)=absolute term, A(1)=linear term, A(2)=quadratic term, A(3)=cubic term able of predicted values If Prediction was selected in te Cubic Spline dialog window tis table gives predicted value (Y-pred) for all a-values given in te Prediction column (X-value) togeter wit confidence interval of prediction (Lower bound, Upper bound), and values of first and second derivatives. Residual squares sum Sum of te squared residuals (estimated errors) Residual std. deviation Standard deviation of te residuals Residual variance Variance of te residuals Mean std. deviation Residual standard deviation Eff. degrees of freedom Effective degrees of freedom, ν eff = n + c*(p 1) 4p, were p is number of segments and c is number of continuity requirements. If te degrees of freedom is less tan 1 no statistics is computed. ables of extremes and inflexes Consists of two tables a table of extremes and a table of inflexes in te interval of x-data (x min, x max ). If no extremes and inflexes are found tis table is not created. No of extremes is table provides analytically computed extremes (minima and maxima) on te regression curve. Columns in te table include: ype of extreme (MIN or MAX), X- and Y-coordinate and second derivative (first derivative is always zero at te extreme). able of inflexes is table provides analytically computed inflexes on te regression curve. Columns in tis table include: X- and Y- coordinate and first derivative (second derivative is always zero at te inflex point). able of data and residuals It te ceckbox Data and residuals was cecked in te Cubic Spline dialog window tis table provides original data ( measured X and Y values), predicted Y-values and residuals (difference between measured and predicted Y values). Graps Spline function Plot of regression spline function. If degrees of freedom ν eff > 0 confidence intervals of te mean are plotted as red lines around te spline curve (plot A). If ν eff = 0 te curve is called interpolation spline as it passes troug data (plots B-D). Plots (C) and (D) are splines wit loosened continuity conditions. However, in most real

9 cases, ν eff > 0 and p<<n wit all continuity conditions imposed will be advisable. (A) (B) (C) (D) Spline function wit knots Same as te previous plot wit visible knot values.

10 First derivative f (1) (x). of te spline function. Zero-points correspond to local minima and maxima of te spline function. Numerical values of te first derivative are also available in Protocol in able of predicted values and ables of extremes and inflexes. Second derivative f (2) (x). of te spline function. Zero-points correspond to inflex points of te spline function. Numerical values of te second derivative are also available in Protocol in able of predicted values and ables of extremes and inflexes. Integral of te spline function. x xmin f z dz Plot of residuals y i f(x i ). It tere is a trend in te residuals te knots may ave been cosen improperly and canges in te model sould be considered. Q-Q plot of te residuals. If te points lie close to te line te residuals (not necessarily te actual errors) ave approximately normal distribution. Plot of Y-prediction plots te measured data against predicted y- values. e closer lie te points to a line te closer fit troug te data. It is not generally te goal to ave te closest fit (like interpolation spline). Rater we try to find reasonably complex model to describe te data realistically.