16th European Signal Processing Conference (EUSIPCO 2008), Lausanne, Switzerland, August 25-29, 2008, copyright by EURASIP

Transcription

1 16t European Signal Processing Conference (EUSIPCO 008), Lausanne, Switzerland, August 5-9, 008, copyrigt by EURASIP ADAPTIVE WINDOW FOR LOCAL POLYNOMIAL REGRESSION FROM NOISY NONUNIFORM SAMPLES A. Sreenivasa Murty 1 and T. V. Sreenivas Department of Electrical Communication Engineering Indian Institute of Science, Bangalore, India s: { 1 asmuvce, tvsree}@ece.iisc.ernet.in ABSTRACT We consider te problem of local polynomial regression of noisy nonuniform samples of a time-varying signal in te presence of observation noise. We formulate te problem in te time domain and use te pointwise minimum mean square error (MMSE) as te cost function. Te coice of te window lengt for local regression introduces a bias-variance tradeoff wic we solve by using te intersection-of-confidence-intervals (ICI) tecnique. Tis results in an adaptive pointwise MMSE-optimal window lengt. Te performance of te adaptive window tecnique is superior to te conventional fixed window approaces. Simulation results sow tat te improvement in reconstruction accuracy can be as muc as 9 db for 3 db input signal-to-noise ratio (SNR). 1. INTRODUCTION Sampling is a fundamental operation in digital signal processing. Nearly six decades of signal processing advances ave been made possible tanks to te Sannon-Wittaker-Kotelnikov uniform sampling teorem 1]. However, uniform sampling comes wit its own limitations. In many cases, nonuniform sampling is unavoidable or natural in te acquisition process. Consider for example signals sampled at instants were tey cross a certain tresold (level-crossing sampling). Suc event-based sampling scemes ave certain advantages wic may be useful in some applications, 3]. Sometimes, nonuniform sampling is deliberately incorporated to overcome te limitations of uniform sampling. For example, in parameter estimation of pase signals, nonuniform sampling overcomes te parameter aliasing problem tat is often encountered in uniform sampling 3, 4]. Yet anoter scenario in wic nonuniform sampling is natural is kernel density estimation 7]. Unlike uniform sampling, tere are no generalized results for signal reconstruction from nonuniform samples. Reconstruction in te nonuniform case is mostly done by using suitable basis functions and iterative/noniterative algoritms are used wit a certain regularization constraint. Te basis functions can be localized or nonlocalized depending on te properties of te underlying signal. In 5], te autors present a multilevel algoritm for regularized reconstruction of a signal from noisy nonuniform samples using trigonometric polynomials as basis functions. In 9], te autors propose a polynomial filtering tecnique but te solution is not associated wit a mean square error (MSE) criterion. Te objective of tis paper is to overcome tis drawback by formulating a MSE cost function. In many practical applications, te nonuniform samples are also corrupted by measurement noise. In suc a scenario, one is interested in estimating te underlying signal by te optimization of a suitable cost function. In tis paper, we propose te use of a pointwise minimum mean square error (PMMSE) cost function wic is also suitable for time-varying signals. We coose te simplest of basis functions, namely, te polynomial, and address te reconstruction problem in te framework of local polynomial regression. Local polynomial regression is a well establised area of researc and as found many applications 7, 8]. One can also coose a different set of basis functions, but we prefer to use polynomials because of teir simplicity. We sow tat te optimization of te pointwise mean square error cost function leads to a bias-variance tradeoff, a problem tat is well known in te statistical signal processing literature 10, 11]. We sow tat te tradeoff can be solved in a near-optimal fasion by using te intersection-of-confidence-intervals(ici) tecnique. In doing so, we sow tat local polynomial regression togeter wit te PMMSE criterion adapts to te signal local beavior.. PROBLEM FORMULATION Consider te practical sampling scenario sown in Fig. 1, in wic te output of an analog source s(t) is corrupted by additive noise w(t). Te samples of te process y(t) = s(t) + w(t) (1) are taken at ordered nonuniform instants {t n, n Z}. Te noise may be inerent in te recording system or it may be due to te cannel. We assume tat w(t) is wite Gaussian in nature, zero-mean and variance σw. Tis scenario occurs in many practical cases suc as observation of astronomical data, data measured in a moving veicle for oceanograpic applications, magnetic/gravitational field measurements in geopysics, jitter in sampling, data loss in images or audio signals due to cannel erasures etc. Te observations are given by y(t n ) = s(t n ) + w(t n ). () Te sampling instants are distinct and strictly ordered in time i.e., t n < t n+1. Sampling can be random or deterministic; in tis paper, we assume te latter. Te objective is to compute an estimate of s(t) from te noisy measurements, {y(t n ), n Z} suc tat te mean square error is least. We do not assume tat s(t)

2 16t European Signal Processing Conference (EUSIPCO 008), Lausanne, Switzerland, August 5-9, 008, copyrigt by EURASIP Analog signal source Figure 1: Scematic of a nonuniform sampling system. is bandlimited; infact, many natural signals are not bandlimited. We assume tat te signal is continuous and smoot so as to permit a local polynomial representation to a desired accuracy (Weierstrass s teorem 6]). 3. POINTWISE MINIMUM MEAN SQUARE ERROR Let t be a point were te estimate of s(t) is computed by a local p t order polynomial regression to te signal s(t). Te least squares approximation error is given by C(t, a) = n y(t n ) p a k (t)tn] k (t t n ) (3) k=0 were a = a 0 (t) a 1 (t) a (t)... a p (t)] are te time-varying polynomial coefficients. Te polynomial approximation is localized by a window function (t) wic is cosen to be positive-valued, symmetric and satisfies te properties: + (t)dt = 1 and + (t)dt <. (4) Te coefficients, {a k (t), 0 k p} are functions of time and are estimated as â l (t) = arg min a l C(t, a). (5) By using te polynomial coefficient estimates, we can compute an estimate ŝ(t) of te signal s(t). Usually, one is interested in te beaviour of te estimator as te number of samples witin te observation window tends to infinity. Tis limiting case leads to asymptotic bias and variance expressions, wic in te present case are given by Bias (ŝ(t)) = ( 1)p+1 s p+1 (t) β p+1 (β)dβ (6) (p + 1)! (β)dβ and Var (ŝ(t)) = σw (β)dβ ]. (7) (β)dβ Te derivations for te above expressions can be found in 7]. Altoug te context in 7] is one of kernel density estimation, te expressions are valid for our problem too since te underlying teme is te same namely, local polynomial regression. From (6) and (7), we note tat te bias and variance depend on (t). We now study te effect of (t), its sape and size. In tis paper, we confine ourselves to two commonly-encountered window functions: 1. Rectangular window: Te rectangular window of compact time support σ is defined as { 1 (t) = σ for t < σ (8) 0 oterwise Te corresponding bias is given by ( 1) p+1 s p+1 (t) Bias (ŝ(t)) = p+1 σ p+1 p odd (p + )! 0 p even, (9) and te variance is given by Var (ŝ(t)) = σ w σ. (10). Gaussian window: Te infinite-support centered- Gaussian window of variance σ given by yields Bias (ŝ(t)) = (p + 1)! and ( 1) p+1 s p+1 (t) 1 (t) = e t σ. (11) πσ p σ p+1 p odd 0 p even Var (ŝ(t)) = (1) σ w πσ. (13) Wit respect to te window parameter σ, te bias and variance of te estimator sow complementary caracteristics. As σ 0, Bias (ŝ(0)) 0 and Var (ŝ(0)). On te oter and, as σ, Bias (ŝ(0)) and Var (ŝ(0)) 0. For te same value of σ, te Gaussian window function yields an estimate wit a variance tat is π times less tan tat obtained by te rectangular window. Te bias is appreciably smaller wit te rectangular window tan te Gaussian window. For example, wit p = 3, te bias wit te rectangular window is 80/3 times smaller tan tat obtained wit te Gaussian window. Tis reduction in bias is more tan te reduction in variance tat te Gaussian window offers. Also, in a practical application, due to te finite data availability, te rectangular window is a better coice. Since te bias and variance ave complementary caracteristics wit respect to σ, it is not possible to minimize bot. Instead, one can minimize te mean square error MSE(ŝ(t)) = E { s(t) ŝ(t)] }. Te MSE is equal to te sum of te squared bias and te variance and is given by: MSE(ŝ(t)) = Bias(ŝ(t))] + Var(ŝ(t)) (14) ( s p+1 ) (0) = p+1 σ p+ + σ w. (15) (p + )! σ

3 16t European Signal Processing Conference (EUSIPCO 008), Lausanne, Switzerland, August 5-9, 008, copyrigt by EURASIP Te optimum MMSE coice of σ is given by σ opt = arg min MSE(ŝ(t)) σ σ w p+ ((p + )!) = (p + )(s p+1 (t)) ] 1 p+3 (16) Since te MSE is a function of time (pointwise MSE), we refer to te above optimization as te pointwise MMSE criterion. Te optimal value of σ requires apriori opt knowledge of te iger-order derivatives of s(t) wic may not be available in practice. However, by exploiting te complementary bias-variance caracteristics, one can obtain a near-optimal solution by using te ICI tecnique wic we present next. 4. NEAR-OPTIMAL PMMSE SOLUTION BY THE ICI TECHNIQUE Te ICI tecnique, in its present form, proposed by Stankovic and Katkovnik 1] as been successfully applied to a variety of problems to solve te bias-variance tradeoff. Some applications of te ICI tecnique include instantaneous frequency estimation by using timefrequency distributions 1, 13] and zero-crossings 14]. In tis paper, we promote te application of te ICI tecnique for te problem of reconstruction from noisy nonuniform samples. Te summary of te ICI tecnique is given below. Define H = { σ (i) } σ (i) = i σ (0), i = 0, 1,,..., i max. i max is cosen suc tat σ (imax) is te largest number tat is less tan te number of samples observed. 1. Initialization: i = 0; σ (0) is cosen as te window lengt about t encompassing (p + 1) data samples. Te signal estimate, ŝ(t) is obtained by local polynomial regression to te centered-data, wit σ (0) as te window parameter. Denote te estimate by ŝ (0) (t). We compute te κ confidence interval J 0 as ] J 0 = ŝ (0) (t) κσ, ŝ (0) (t) + κσ. (17). Iteration: Wit te window parameter σ (i+1), we compute te estimate, ŝ (i+1) (t) and te associated confidence interval: ] J i+1 = ŝ (i+1) (t) κσ, ŝ (i+1) (t) + κσ. (18) 3. Stopping Condition: If J i J i+1 = (empty set), ten σ opt = σ i, else i i+1 and go to step above. Te confidence interval computation requires te knowledge of σ w but it can be estimated. Apriori knowledge of te bias is not needed werein lies te advantage of te ICI tecnique. Te only parameter tat needs to be determined is te confidence interval parameter κ. κ sould be as small as possible and still keep te probability of coosing a wrong window lengt below a certain tresold. A reliability analysis conducted in 1] sowed tat κ =.5 is a reasonable coice. Te corresponding confidence interval coverage probability is approximately Earlier results on local polynomial CMSE(dB)!10!15!0!5 ADAPTIVE FIXED! SNR(dB) Figure : Cumulative mean square error performance for fixed vs. adaptive window tecniques as a function of te SNR. regression 7] sowed tat tird-order polynomials are optimal for yielding smoot estimates; terefore, we use p = EXPERIMENTAL RESULTS 5.1 Experiment-1 First, we conduct an experiment using s(t) = e αt sin(ωt), α = 0.005, ω = 0.05rad/s. Te noise is wite Gaussian wit a variance σw. An arbitrary sampling sequence is generated at te beginning of te experiment by using a pseudorandom generator associated wit a uniform distribution. We fix te sampling sequence for all te statistical Monte-Carlo realizations of te noisy signal since we ave assumed a deterministic sampling sequence. In te experiment, te noisy signal is sampled at 56 locations in 0, 55] as determined by te sampling sequence. Te signal s(t) is estimated at 56 uniform locations 0, 1,,..., 55] using te adaptive window tecnique. Te initial window lengt is cosen as te interval encompassing (p+1) samples. At eac iteration of te window parameter estimation tecnique, te increment is cosen as te window lengt encompassing (p + 1) additional samples. We repeat te experiment 100 times wit a different random realization of te noise sequence at eac repetition. Since te signal s(t) is known to us in tis simulated experiment, we can compute te cumulative mean square error (CMSE) to quantify te estimation accuracy. Te CMSE is given by CMSE = 1 RN R N r=1 n=1 ŝ (r) (n) s(n)], (19) were s (r) (n) is te estimate of s(n) in te r t realization (R: number of realizations), and were N is te number of points at wic te estimates are obtained. Te CMSE accumulates te MSE at eac point in te observation interval and gives an objective measure for

4 16t European Signal Processing Conference (EUSIPCO 008), Lausanne, Switzerland, August 5-9, 008, copyrigt by EURASIP Figure 3: (a) Te estimated signal wit te σ-confidence interval and te actual signal; (b) Mean square error performance as a function of te SNR. performance comparison. To sow tat te adaptive window tecnique is superior to a fixed window tecnique, we repeat te experiment wit a fixed window of k samples; in our experiments, we cose k = 7. Te CMSE corresponding to te fixed and adaptive window lengts are sown in Fig.. Note tat te adaptive tecnique is consistently superior to te fixed window tecnique. Te CMSE of te adaptive window regression is lower tan tat of te fixed window by about 9 db for 3 db input SNR. As SNR increases, te CMSE of bot tecniques decreases but te accuracy advantage of te adaptive tecnique still exists. 5. Experiment- We conduct anoter experiment wit a linear frequencymodulated cirp wit an exponentially decaying envelope. Tis can also be seen as a syntesized isolated formant in a speec signal. We cose te envelope to decay by a factor of 1 e witin te observation window. We cose te frequency to increase linearly from 0.05 to 0.1 (tese are normalized frequencies wit 0.5 corresponding to te Nyquist sampling frequency). Te estimated signal and te mean square error performance at te 111 t position (arbitrarily cosen point corresponding to ig local SNR) and 00 t position (corresponding to low local SNR) in te window is sown in Fig. 3. Note tat te new tecnique offers accurate estimation for moderate to ig SNRs even for time-varying signals. Te signal estimation accuracy is iger in regions of ig local SNR tan in regions of poor local SNR, wic is intuitive and acceptable. 6. CONCLUSIONS We addressed te problem of signal reconstruction from noisy nonuniform samples witin te framework of local polynomial regression, using a pointwise minimum-mean square error criterion. Tis involves a bias-variance tradeoff, wic we solved by using te intersection-of-confidence-intervals tecnique. Te tecnique enables one to tradeoff adaptability to signal variations (bias term) and robustness to noise (variance term). Te performance of te tecnique is superior to te fixed window-based approac. Software MATLAB software for te tecniques reported in tis paper can be requested from te autors by . REFERENCES 1] M. Unser, Sampling50 Years After Sannon, Proc. IEEE, Vol. 88, No. 4, pp , April 000. ] F. Marvasti, Nonuniform sampling: Teory and practice, Kluwer Academic Publisers, New York, ] S. C. Sekar and T. V. Sreenivas, Auditory motivated level-crossing approac to instantaneous frequency estimation IEEE Trans. on Sig. Proc., Vol. 53, Issue 4, pp , Apr ] J. A. Legg and D. A. Gray, Performance bounds for polynomial pase parameter estimation wit nonuniform and random sampling scemes IEEE Trans. on Sig. Proc., Vol. 48, Issue, pp , Feb ] M. Raut and T. Stromer, Smoot approximation of potential fields from noisy scattered data, Geopysics, Vol. 63, No. 1, pp , ] S. K. Sen and E. V. Krisnamurty, Numerical Algoritms: Computations in Science and Engineering, Affiliated East-West Press, ] J. Fan and I. Gijbels, Local polynomial modeling and its application, Capman and Hall, London, ] W. Hardle, Applied nonparametric regression, Cambridge University Press, Cambridge, 1990.

5 16t European Signal Processing Conference (EUSIPCO 008), Lausanne, Switzerland, August 5-9, 008, copyrigt by EURASIP 9] T.I. Laakso, A. Tarczynksi, N.P. Murpy and V. Valimaki, Polynomial filtering approac to reconstruction and noise reduction of nonuniformly sampled signals, Signal Processing, pp , ] S. M. Kay, Modern spectral estimation: Teory and applications, Prentice Hall, ] S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation Teory, Upper Saddle River, New Jersey, USA: Prentice-Hall, ] L. Stankovic and V. Katkovnik, Algoritm for te instantaneous frequency estimation using time frequency distributions wit adaptive window lengt, IEEE Sig. Proc. Lett.,, vol. 5, No.9, pp. 4-7, Sept ] S. C. Sekar and T. V. Sreenivas, Adaptive spectrogram vs. adaptive pseudo-wigner-ville distribution for instantaneous frequency estimation, Signal Proc., vol. 83 no.7, pp , July ] S. C. Sekar and T. V. Sreenivas, Adaptive Window Zero-Crossing-Based Instantaneous Frequency Estimation, EURASIP J. on Applied Sig. Proc., vol. 004, no. 1, pp , 004.