Curve Fitting
Content Concept of Curve Fitting Difference with Interpolation Estimation of Linear Parameters by Least Squares Curve Fitting by Polynomial Least Squares Estimation of Non-linear Parameters by Least Squares Curve Fitting Using Excel
Concept of Curve Fitting Difference with Interpolation One of the most extensively used techniques in numerical methods is the estimation of parameters by the principle of least squares. This technique is employed to drive information about the functional relation between xx and yy, assuming such a relation exists, from a set of data pairs xx ii, yy ii (ii = 00, nn). The estimation of parameters by least squares causes an smoothing to a given set of data and eliminates, to some degree, errors in observation, measurement, recording, transmission and conversion, as well as other types of random errors which may have been introduced in the data. This is one of the most important functions of the principle of least squares, and one which distinguishes it from the interpolation. (Recall that an interpolating polynomial exactly fills all data points used, such that any error in the data will be held in the interpolation).
Concept of Curve Fitting Difference with Interpolation There are two distinct but related categories of techniques based on the principle of least squares: 1. The estimation of linear parameter by least squares 2. The estimation of non-linear parameter by least squares
Estimation of Linear Parameters by Least Squares Given a set of data pairs xx ii, yy ii (ii = 00, nn), which can be interpreted as the measured coordinates of the coordinates of the points on the graph of yy = ff(xx) values, let us assume that the unknown function ff(xx) can be approximated by a linear combination of suitably chosen functions ff 00 xx, ff 11 xx,, ff mm (xx) of the form FF xx = aa 00 ff 00 xx + aa 11 ff 11 xx + + aa mm ff mm xx where the unknown coefficients aa 00, aa 11, aa 22,, aa mm are independent parameters to be determined, and mm < nn. The difference between the approximating function value FF(xx ii ) and the corresponding data values yy ii is called a residual rr ii and is defined by the relation rr ii = FF xx ii yy ii (ii = 00, nn) We have then a residual rr ii for each data pair xx ii, yy ii (ii = 00, nn).
Estimation of Linear Parameters by Least Squares The function FF(xx) that best approximates the given set of data in a least squares sense is the linear combination aa 00 ff 00 xx + aa 11 ff 11 xx + + aa mm ff mm (xx) of functions ff kk (xx) that produces the minimum value of the sum QQ of the squared residuals QQ = rr 22 ii ii ii FF xx ii yy ii 22 Rewriting the above equation in its expanded form, we get QQ = ii aa 00 ff 00 xx + aa 11 ff 11 xx + + aa mm ff mm xx yy ii 22 Considering the parameters aa 00, aa 11, aa 22,, aa mm independent variables of the function QQ, minimizing the sum, that is, differentiating and equating to zero, we obtain QQ aa kk 22 ii FF xx ii yy ii xx ii aa kk = 00 (kk = 00, mm)
Estimation of Linear Parameters by Least Squares The constrains imposed by this equation form a system of mm + 11 independent algebraic equations (called normal equations) which are linear in mm + 11 parameters aa kk (kk = 00, mm). The solution (aa 00, aa 11, aa 22,, aa mm ) of this system of normal equations is that set of parameters aa kk which produces the minimum sum of squared residuals. The normal equations can be reduced to a form suitable for computation by the following steps: First, substitute the relation FF(xx ii ) = ff aa kk (xx ii ) kk and express FF(xx ii ) equation in its expanded form QQ 22 aa aa 00 ff 00 xx + aa 11 ff 11 xx + + aa mm ff mm xx yy ii ff kk (xx ii ) = 00 (kk = 00, mm) kk ii
Estimation of Linear Parameters by Least Squares We can rewrite the normal equations in the form
Estimation of Linear Parameters by Least Squares The mm + 11 normal equations obtained from the above equation, when evaluated for kk = 00, kk = 11, kk = 22,, kk = mm, can be written as a single matrix equation of the form where all summations are over ii (ii = 00, nn). The solution (aa 00, aa 11, aa 22,, aa mm ) of the matrix of the normal equations is the set of parameters aa kk (kk = 00, mm) that minimizes the sum QQ of the squared residuals.
Curve Fitting by Polynomial Least Squares Let us now consider the special case of the least-squares estimation of linear parameters in which the functions ff kk xx = xx kk (kk = 00, mm) so the FF(xx) equation becomes an mth-degree polynomial, mm < nn, denoted PP mm (xx) of the form PP mm xx = aa 00 xx 00 + aa 11 xx 11 + aa 22 xx 22 + + aa mm xx mm That is, we will approximate function yy = ff(xx) by an mth-degree polynomial PP mm (xx) over the range of data pairs xx ii, yy ii (ii = 00, nn). The parameters aa 00, aa 11,, aa mm are then determined such that QQ = rr 22 ii ff ii PP mm xx ii yy ii 22 is a minimum. That is, we will fit an mth-degree polynomial curve to the data in a leastsquares sense, as defined earlier. This special case of the estimation of linear parameters is commonly referred to as polynomial curve-smoothing by least squares.
Curve Fitting by Polynomial Least Squares The normal equations that determine aa 00, aa 11,, aa mm for this special case can be obtained directly by substituting xx ii kk (i.e., xx ii to the kth power) for ff kk (xx ii ) obtained in the above matrix equation. This substitution gives us These normal equations for the least-squares polynomial can then be written as
Estimation of Non-linear Parameters by Least Squares You can use the method previously seen with nonlinear functions, linearizing the function. For example if we have the exponential function We can linearize the function as follows:
Estimation of Non-linear Parameters by Least Squares For this case, the matrix representation of nominal equations is as follows:
Curve Fitting Using Excel
Curve Fitting Using Excel
Homework 8 (Individual) 1. Given the following data set, fit a quadratic least-squares polynomial (degree 2): x y 0.0 1.000000 0.1 1.105171 0.2 1.221403 0.3 1.349859 0.4 1.491825 0.5 1.648721 2. Given the following data set, fit to an exponential function of the form yy = aaee bb by least squares: x y 0.0 0.500000 0.1 0.674929 0.2 0.911059 0.3 1.229802 0.4 1.660058 0.5 2.240845 Consider for both problems 6 digits of precision.
Computer Program 7 (by team) Submit a computer program that compute the curve fitting of a set of data by the following methods: a) Polynomial Least Squares b) Non-linear Parameters by Least Squares, exponential fit Hand over: Computational algorithm (printed) Source Code (printed and file) Executable (file)
Curve Fitting