Post-Processing Radial Basis Function Approximations: A Hybrid Method

Size: px

Start display at page:

Download "Post-Processing Radial Basis Function Approximations: A Hybrid Method"

Camilla Sullivan
5 years ago
Views:

1 Post-Processing Radial Basis Function Approximations: A Hybrid Method Muhammad Shams Dept. of Mathematics UMass Dartmouth Dartmouth MA mshams@umassd.edu August 4th 2011 Abstract With the use of Fourier Series we are able to approximate other functions. As we take more terms in the series the approximation becomes increasingly accurate. However, when approximating a piecewise continuous function there remains an error noticeable at the jump discontinuity of the function. This error, known as the Gibbs Phenomenon, does not disappear with more terms being taken but instead slides closer to the point of discontinuity. The Gibbs Phenomenon is not restricted only to Fourier approximations but can be seen in all Global approximations. By using Gegenbauer polynomials and re-projecting the approximation in Gegenbauer space we can post-process these series, extracting accurate data, and removing the Gibbs Phenomenon. Like Fourier series, Radial Basis Functions can be used as a Global method of approximation. RBFs become increasingly accurate as more points are taken. When using RBFs to approximate a piecewise continuous function the Gibbs Phenomenon is, once again, observed. Using Gegenbauer polynomials, like in the Fourier case, we can post-process RBF approximations, removing the Gibbs Phenomenon and recovering accuracy. 1 Introduction This project was undertaken as a continuation of past work with Fourier approximations and Gegenbauer Post- Processing. From the beginning the project has been overseen and provided with guidance by Sigal Gottlieb. Her work with Saeja Oh Kim, Jae-Hun Jung, Daniel Higgs, and Chris L. Bresten is directly related to, and has benefitted, this project. The use of Radial Basis Functions has been rising in recent history and as such the need to develop techniques of reducing their error has become more important. In this paper, the techniques involved will be explained, numerical results will be shown, and some conclusions and speculation will be presented. 2 Approximation Techniques 2.1 Radial Basis Functions Radial Basis Functions have already proven useful in a number of areas. They can be used for scattered data problems in higher dimensions and solving partial differential equations. RBFs as approximation techniques have recently received more attention. RBFs are a family of functions depent on the distance from point to point or the distance from center points. Being that they are equidistantly spaced, they are mesh free and very adaptable. RBFs employ the use of a shape parameter designated as ɛ that play a pivotal role in approximations. Within the family of RBFs there are many different basis. The selection of shape parameter is unique to each RBF basis. 1

2 In the context of this project, there are four basis that were used. Multiquadric Gaussian Inverse Multiquadric Inverse Quadratic φ j = φ j = (x i x j ) 2 + ɛ 2 (1) φ j = e (ɛ(xi xj))2 (2) φ j = 1 (xi x j ) 2 + ɛ 2 (3) 1 (x i x j ) 2 + ɛ 2 (4) The approximation of a function using a RBF is done by, first, building an Interpolation matrix. The Interpolation matrix is a linear combination of basis vectors. The matrix is created using one of the RBF basis and a set number of points. The matrix multiplied by unknown coefficients, λ, equals the real values of the approximated function. φ 1, x 1 φ 2, x 1... φ n, x 1 φ 1, x 2 φ 2, x 2... φ n, x φ 1, x n φ 2, x n... φ n, x n λ 1 λ 2... λ n = f 1 f 2... f n. (5) Next, the inverse of the interpolation matrix is multiplied by the function values in order to obtain the value of the coefficients. After which, the interpolation matrix and the coefficients are multiplied in order to build the approximated value of the function. 2.2 Gegenbauer Post-Processing λ = M 1 f (6) Gegenbauer Post-Processing is an emerging technique that removes Gibbs error in function approximations. The technique uses the Gegenbauer family of polynomials. Gegenbauer polynomials are a family of functions that are generated by a particular power series, aptly named a generating function. n=0 C (α) n (x)t n (7) The Gegenbauer family of polynomials is a completely orthogonal basis over the interval [ 1, 1] with respect to the weight function: (1 x 2 ) α 1/2 (8) In order to apply the Post-Processing technique, the original function approximation also needs to be orthogonal over [ 1, 1]. While this is true of the Fourier case, it is not true of the RBF case. However, we notice that the technique still seems to work with RBFs. The Gegenbauer process begins by generating the polynomials up to the number of terms used in the approximation, or in the case of RBFs, the number of points taken. The generation process deps largely on the number of points taken and the ratio of two variables, m and λ. 2

3 Both m and λ are depent on the number of points taken in the RBF and are related to each other. After the polynomials have been generated, their inverses are multiplied by the bad function approximation, much like in the RBF case of solving for coefficients. This finds the new Gegenbauer coefficients. From here, a new approximation is built by multiplying the Gegenbauer polynomials by the new coefficients. 3 Methods 3.1 Matlab Code The first half of the code is the RBF approximation method. First, the parameters are set up and the function to be approximated is designated. Next, the basis to be used is selected. Finally, the coefficients are calculated and the approximation is built. Included in the code is the capability to plot the function, it s approximation, and errors of the approximation. N=96; x = -cos(pi*[0:n]/n); xc=x; f=sign(x); f=f(:); ep=10/n; for i=1:length(x) for j=1:length(xc) %phi(i,j)=sqrt((x(i)-xc(j))ˆ2+(ep)ˆ2); %Multiquadric %phi(i,j)=exp(-1*(ep)ˆ2*(x(i)-xc(j))ˆ2); %Gaussian %phi(i,j)=1/(sqrt((x(i)-xc(j))ˆ2+(ep)ˆ2)); %Inverse Multiquadric phi(i,j)=1/((x(i)-xc(j))ˆ2+(ep)ˆ2); %Inverse Quadratic lambda=phi\f; lambda=lambda(:); xx = -0.5*cos(pi*[0:N]/N) -0.5; %xx = -0.5*cos(pi*[0:N]/N) +0.5; for i=1:length(xx) for j=1:length(xc) %phi(i,j)=sqrt((xx(i)-xc(j))ˆ2+(ep)ˆ2); %phi(i,j)=exp(-1*(ep)ˆ2*(xx(i)-xc(j))ˆ2); %phi(i,j)=1/(sqrt((xx(i)-xc(j))ˆ2+(ep)ˆ2)); phi(i,j)=1/((xx(i)-xc(j))ˆ2+(ep)ˆ2); fn=phi*lambda; figure(1) %plot(x,lambda, c, LineWidth,2) plot(x,f, k, LineWidth,2),hold on plot(xx,fn, b, LineWidth,2),hold on figure(2) 3

4 rbferror=abs(fn+1); %rbferror=abs(fn-1); plot(xx,log10(rbferror), b, LineWidth,2),hold on The second half of the code is the Post-Processing method. The parameters are set up and the Gauss-Lobatto quadrature is used. Also, the appropriate quad weights for the quadrature are included. Next, the polynomials are computed and finally, the new approximation is built. Included in the code is the capability to plot the Post-Processed approximation as well as the errors of the new approximation. Nq = N; T = N; xxi = cos(pi*([0:nq]-0.5)/nq) ; xi = -0.5*cos(pi*[0:Nq]/Nq) -0.5; %xi = -0.5*cos(pi*[0:Nq]/Nq) +0.5; w = xi*0 + pi/nq; x=xi; m=t/16; lambda=t/16; GPa = zeros(nq+1,m+1); GPa(:,1) = 1+0*xxi; hl(1)=pi*2ˆ(1-2*lambda)*gamma(2*lambda)/lambda/(gamma(lambda)ˆ2); hl(1)=sqrt(hl(1)); GPa(:,2) = 2*lambda*xxi; hl(2)=pi*2ˆ(1-2*lambda)*gamma(1+2*lambda)/(1+lambda)/(gamma(lambda)ˆ2); hl(2)=sqrt(hl(2)); for im=3:m+1 imr = im - 1; GPa(:,im) = (2*(imr+lambda-1)*xxi(:).*GPa(:,im-1)-(imr+2*lambda-2)*GPa(:,im-2))/(imr); hl(im)=pi*2ˆ(1-2*lambda)*gamma(imr+2*lambda)/(factorial(imr)*(imr+lambda)* (gamma(lambda))ˆ2 ); hl(im)=sqrt(hl(im)); for im=1:m+1 GPa(:,im)=GPa(:,im)/hl(im); for im=1:m+1 for ix=1:nq+1 int2(ix)=(1-xxi(ix)ˆ2)ˆ(lambda)*fn(ix)*gpa(ix,im); quad2(im) =int2*w; for ix=1:nq+1 gfn(ix)= GPa(ix,:)*quad2 ; figure(1) plot(x,gfn, r, LineWidth,2),hold on figure(2) 4

5 gegerror=abs(gfn+1); %gegerror=abs(gfn-1); plot(x,log10(gegerror), r, LineWidth,2),hold on The code must run twice first over the interval [ 1, 0] and then over [0, 1]. The reasoning for this being that the point of discontinuity in the Square Wave function is at x = 0. We have to break the the interval down into two intervals of smoothness. 3.2 Parameters In the first portion of the code, generating the RBF approximation, there are a few parameters that need to be tinkered with to suit a specific need. The obvious parameter, N, exists to allow different amounts of points. The value of N is bounded by time. What is meant by this is that, there doesn t seem to be an upper limit for the value of N other than how much time the code will take to run. The highest tested value of N was 10,000 points when approximating the Square Wave function. It clocked in around 90 minutes for the code to run the entire interval [ 1, 1]. The next parameter was the shape parameter, ɛ. It was found that having a shape parameter depent on N worked well, but had the possibility of confusing the the reconstruction process later in the Gegenbauer step. In the Multiquadric case 1/N was used. N/5 was used for the Gaussian basis and 10/N was used for both the Inverse Multiquadric and Inverse Quadratic basis. None of the shape parameter selections would be considered definitive. More work is necessary to be done with them but the presented selections work well enough. In the second portion of the code, generating the Gegenbauer approximation, there are a few dummy variables worthy of note. Nq and T are parameters within the code, left over from when there was a difference between the number of points taken and the number of terms in the series taken in the Fourier case. In the RBF case they are both equal to each other and to N. The next two parameters were m and λ. As mentioned before, m and λ are related to each other and to N or T, the dummy variable as it is written in the code. Of all the parameters presented thus far, these two require the most attention in order to receive accurate results. For values of N below 100, m = λ = T/16 was used. Part of the reason m and λ require more thought in selecting is that they are supposed to be integer values. Later in the paper, results taken with different values of m and λ as well as results were they do not vary with N will be touched upon. 4 Results 4.1 Approximations The Multiquadric RBF approximation (blue) as well as the Post-Processed approximation (red) for the Square Wave function (black) over the interval [ 1, 1], { 1 for 1 x < 0, F (x) = (9) 1 for 0 x 1, is shown in Figure 1. The approximation was taken with N = 96, ɛ = 1/N, and m = λ = T/16. The other three basis would build similar approximations. To avoid redundancy, only the Multiquadric approximation is shown. 5

FIGURE 1: MULTIQUADRIC APPROXIMATION AND RECONSTRUCTION 4.2 Errors As we increase N, the errors of the RBF approximation decay but near the point of discontinuity, x = 0, the error remains large.

6 FIGURE 1: MULTIQUADRIC APPROXIMATION AND RECONSTRUCTION 4.2 Errors As we increase N, the errors of the RBF approximation decay but near the point of discontinuity, x = 0, the error remains large. In Figure 2, the pointwise errors (in logscale) of a Multiquadric RBF, with ɛ = 1/N, over the interval [ 1, 0], are shown for N = 16 (blue), 32 (red), 48 (green), 64 (magenta), 80 (cyan), and 96 (black). On the right, in Figure 3, we have the pointwise errors (in logscale) of the Post-Processed Multiquadric RBF with m = λ = N/16 with the same values of N denoted by the same colors as in Figure 2. FIGURE 2: BEFORE RECONSTRUCTION FIGURE 3: AFTER RECONSTRUCTION In Figure 4, the pointwise errors (in logscale) of a Gausian RBF, with ɛ = N/5, over the interval [ 1, 0], are shown for N = 16 (blue), 32 (red), 48 (green), 64 (magenta), 80 (cyan), and 96 (black). On the right, in Figure 5, we have the pointwise errors (in logscale) of the Post-Processed Gaussian RBF with m = λ = N/16 with the same values of N denoted by the same colors as in Figure 4. 6

FIGURE 4: BEFORE RECONSTRUCTION FIGURE 5: AFTER RECONSTRUCTION In Figure 6, the pointwise errors (in logscale) of an Inverse Multiquadric RBF, with ɛ = 10/N, over the interval [ 1, 0], are shown for

On the right, in Figure 7, we have the pointwise errors (in logscale) of the Post-Processed Inverse Multiquadric RBF with m = λ = N/16 with the same values of N denoted by the same colors as in

7 FIGURE 4: BEFORE RECONSTRUCTION FIGURE 5: AFTER RECONSTRUCTION In Figure 6, the pointwise errors (in logscale) of an Inverse Multiquadric RBF, with ɛ = 10/N, over the interval [ 1, 0], are shown for N = 16 (blue), 32 (red), 48 (green), 64 (magenta), 80 (cyan), and 96 (black). On the right, in Figure 7, we have the pointwise errors (in logscale) of the Post-Processed Inverse Multiquadric RBF with m = λ = N/16 with the same values of N denoted by the same colors as in Figure 6. FIGURE 6: BEFORE RECONSTRUCTION FIGURE 7: AFTER RECONSTRUCTION In Figure 8, the pointwise errors (in logscale) of an Inverse Quadratic RBF, with ɛ = 10/N, over the interval [ 1, 0], are shown for N = 16 (blue), 32 (red), 48 (green), 64 (magenta), 80 (cyan), and 96 (black). On the right, in Figure 9, we have the pointwise errors (in logscale) of the Post-Processed Inverse Quadratic RBF with m = λ = N/16 with the same values of N denoted by the same colors as in Figure 8. 7

FIGURE 8: BEFORE RECONSTRUCTION FIGURE 9: AFTER RECONSTRUCTION In all four basis, the selection of m and λ needs to be refined before the results can be as accurate as would be hoped for.

8 FIGURE 8: BEFORE RECONSTRUCTION FIGURE 9: AFTER RECONSTRUCTION In all four basis, the selection of m and λ needs to be refined before the results can be as accurate as would be hoped for. From the plots we can see the slow convergence near the point of discontinuity before reconstruction. In Table 1, the L errors are shown for the Multiquadric basis, before and after reconstruction, with the same parameters as in Figures 2 and 3, respectively. ( n) 10 n. N Before Reconstruction After Reconstruction (-1) 5.19 (-2) (-1) 2.05 (-2) (-1) 3.20 (-3) (-1) 1.50 (-3) (-1) 1.40 (-3) (-1) 4.64 (-4) TABLE 1: L ERRORS MULTIQUADRIC In Table 2, the L errors are shown for the Gaussian basis, before and after reconstruction, with the same parameters as in Figures 4 and 5, respectively. ( n) 10 n. N Before Reconstruction After Reconstruction (-1) 8.84 (-2) (-1) 1.92 (-2) (-1) 8.50 (-3) (-1) 6.90 (-3) (-1) 5.19 (-4) (-1) 1.50 (-3) TABLE 2: L ERRORS GAUSSIAN In Table 3, the L errors are shown for the Inverse Multiquadric basis, before and after reconstruction, with the same parameters as in Figures 6 and 7, respectively. ( n) 10 n. 8

9 N Before Reconstruction After Reconstruction (-1) 8.69 (-2) (-1) 1.91 (-2) (-1) 8.30 (-3) (-1) 7.00 (-3) (-1) 6.36 (-4) (-1) 1.50 (-3) TABLE 3: L ERRORS INVERSE MULTIQUADRIC In Table 4, the L errors are shown for the Inverse Quadratic basis, before and after reconstruction, with the same parameters as in Figures 8 and 9, respectively. ( n) 10 n. N Before Reconstruction After Reconstruction (-1) 8.65 (-2) (-1) 1.92 (-2) (-1) 8.20 (-3) (-1) 7.00 (-3) (-1) 6.87 (-4) (-1) 1.50 (-3) TABLE 4: L ERRORS INVERSE QUADRATIC In Tables 1 through 4 it is easy to see that the errors before reconstruction do not decay with N. After reconstruction, the errors decay rapidly. In all cases except the Multiquadric case, as N reaches higher values the reconstruction ceases to decay and instead becomes worse. This is due to the ill conditioning of the Interpolation Matrix, possibly as a result of the shape parameter confusing the Post-process. 5 Conclusions 5.1 Speculation The question of whether m and λ need to be depent on N came up early on in this project. While it makes sense that they should be depent on N, earlier results showed that with sufficient manipulation of the ratio of m to λ just as accurate if not more accurate reconstructions could be built. Initial problems with this method were making sure m and λ remained integers. Whether the indepent m and λ were actually extracting data from the bad approximation or were just appearing to do so are yet to be determined. As it stands now, m and λ should vary with N. Early on in the writing of the code, there appeared to be an upper limit to N. The code could not run when values of N larger than approximately 250 points were taken. I continued working on writing the code and fixing problems as they came up. Somewhere along the way, the code became able to handle far higher values of N. The reasoning behind this is still unknown to me but suffice to say, it worked out for the best. I tried to write into the code a way for it to run a Poly Harmonic Spline Basis and later a Thin Plate Spline Basis. However, this failed as I did not fully understand how those basis worked. Unlike the other four basis used, these two do not involve a shape parameter. 5.2 Remarks This project has been a wonderful ride from start to finish. Working on the project has taught me basic to intermediate Matlab skills as well as some more advanced Latex skills. I have reached a comfortable level with Matlab programming and a competent level of Latex writing. 9

10 This project has taught me to better voice myself in a more concise manner and has made me more comfortable with giving talks. Other than learning the tools necessary to work on this project, I have an increased understanding of the subject matter. It was worthwhile to work with real numerical analysis and Approximation theory. My background in RBFs is a little stronger now, which I find exciting, considering the growing RBF community. 6 References [1] Sigal Gottlieb, Jae-Hun Jung, and Saeja Kim. A Review of David Gottliebs Work on the Resolution of the Gibbs Phenomenon, volume x of Communications in Computational Physics. Global-Science Press, [2] Jae-Hun Jung, Sigal Gottlieb, Saeja Oh Kim, Chris L. Bresten, and Daniel Higgs. Recovery of High Order Accuracy in Radial Basis Function Approximations of Discontinuous Problems, volume 45 of Journal of Scientific Computing. Springer Science+Business Media,

Adaptive Node Selection in Periodic Radial Basis Function Interpolations

Adaptive Node Selection in Periodic Radial Basis Function Interpolations Muhammad Shams Dept. of Mathematics UMass Dartmouth Dartmouth MA 02747 Email: mshams@umassd.edu December 19, 2011 Abstract In RBFs,