/06/$ IEEE 364

Transcription

1 006 IEEE International ympoium on ignal Proceing and Information Technology oie Variance Etimation In ignal Proceing David Makovoz IPAC, California Intitute of Technology, MC-0, Paadena, CA, Abtract We preent a new method of etimating noie variance. The method i applicable for D and D ignal proceing. The eence of thi method i etimation of the catter of normally ditributed data with high level of outlier. The method i applicable to data with the majority of the data point having no ignal preent. The method i baed on the hortet half ample method. The mean of the hortet half ample (horth and the location of the leat median of quare are among the mot robut meaure of the location of the mode. The length of the hortet half ample ha been ued a the meaurement of the data catter of uncontaminated data. We how that computing the length of everal ub ample of varying ize provide the neceary information to etimate both the catter and the number of uncontaminated data point in a ample. We derive the ytem of equation to olve for the data catter and the number of uncontaminated data point for the Gauian ditribution. The data catter i the meaure of the noie variance. The method can be extended to other ditribution. the ueful noie data. If the number of pure noie data point i greater than the number of noiy ignal data point, then one can apply the method developed below to etimate the noie power without doing any explicit eparation of the noie from the noiy ignal. (a Index Term oie variance etimation, nonlinear filter, robut etimation, catter etimation I. ITRODUCTIO oie etimation i a major tak in all area of ignal proceing, be it peech or image proceing. ignal proceing algorithm for egmentation, clutering, retoration, noie reduction, tatitical inference etc, depend on the knowledge of the noie variance. The literature on the noie variance etimation in peech and image abound []- [7]. WE preent a new algorithm that ue very few aumption about the data, namely that the noie ha Gauian ditribution and that the majority of the data point in the data et have no ignal preent. In ignal proceing in general one deal with noiy data z, where each data point i it i a combination of the clean ignal i i the clean ignal and v i the noie: z i = i + v i. In many application the data contain a number of data point for which the ignal i either not preent or much maller than the noie. In (a (b Figure we how two illutration of uch data: a noiy peech waveform and an atronomical image. The ditribution of the data point conit of the noie ditribution and the noiy ignal ditribution. The noie ditribution in many cae i or can be approximated by the Gauian ditribution. If one to conider thi ditribution from the point of view of etimating the noie power, i.e. the width of the Gauian ditribution, then the noie data point become the ueful data and the noiy peech data point become outlier preent in (b Figure. Example of (ad data and (b D data, in which the number of ignal data point i maller than the number of background noie only data point. In ection II we preent the background on data catter etimation in outlier contaminated data. In ection III we introduce the new method and derive the algorithm for computing the catter of normally ditributed outlier contaminated data. In ection IV we preent reult of the imulation. II. BACKGROUD Etimation of the peak and the catter of data in a ample i a common problem encountered in many divere area of tatitical data proceing. If the data are known to have Gauian ditribution, the mot common etimator of the peak and the catter are the mean and the tandard deviation of the data around the mean. If there are outlier preent in the data ample, mean-baed etimator break down almot immediately; even one outlier can reult in a completely miguided mean. The ame i true about the tandard deviation from the mean a an etimator of the data catter. A more robut etimator of the peak i the median. But even the median erode a the number of outlier i increaing and approache 50% of the ample ize. There exit two general approache in dealing with outlier contaminated data. The firt approach, and our method /06/$ IEEE 364

2 belong to thi group, i to deal with the whole ample and devie robut etimator which are to a great extent inenitive to the preence of outlier. The econd approach i to devie robut method of identifying and excluding outlier and then to treat the uncontaminated ample with the conventional tatitical method. Out method i built on one of the exiting method of mode etimation. The mode a an etimator of the peak of a ditribution i very robut; it i motly inenitive to up to 50% outlier in the data ample. However, wherea computing mean and median i traightforward and both etimator have a unique value for any given data ample, mode etimation i notoriouly difficult and moreover for multimodal ditribution there i no unique mode. There exit a whole cla of mode etimator baed on the notion [8,9] of the hortet half ample. The horth the mean of the hortet half ample - wa propoed in [8]. In [9] it wa hown that the location of the one-dimenional leat median of quare, which i the mean of the minimum and maximum data point of the hortet half ample can be ued a a robut etimator of the mode of a data ample. Thi etimator ha a higher bia than the horth. A low biaed variant of mode etimator wa reported in [0]. It i computed by repeatedly taking the hortet half ample within hortet half ample. In [9] it wa propoed to ue the length of the hortet half ample a a robut etimator of the data catter. However, wherea the middle point of the hortet half ample i not enitive to the preence of outlier in the data ample, the length of the hortet half ample depend on the number of outlier. In the preence of outlier only data point out of the total of actually belong to the parent ditribution. The half ample i only uch with repect to all point in the ample, but it i more than half-ample with repect to the point from the parent ditribution with the outlier excluded. ince the catter etimate depend critically on the fact that the hortet half ample actually encompae half of the uncontaminated data point, it become meaningle in preence of outlier. The novelty of our approach i that we derive a way of imultaneouly etimating both the data catter and the ective ize of the ample. Thi allow u to etimate the data catter for outlier contaminated data. III. METHOD A. Multiple hortet ubample We tart with the noiy data point z i and ort them in the acending order. We will notation x i for the orted data point. Let X = (x x be an ordered ample of ize. In order to find the hortet ub ample coniting of n data point one find i=m that minimize ( xn+ i xi, where i =,, -n. We introduce the fractional ub ample ize r = n/. We etimate the mode by the median of the hortet ub ample Mode(r = x m+n/. The mode etimator Mode(r i applicable for any ditribution. The catter etimator, that we derive here, i applicable only for the data that ha the Gauian ditribution with mean μ and varianceσ : = ( x μ f ( x exp / ( πσ σ ( For the Gauian ditribution the value of Mode(r i an unbiaed etimator of μ. If the hortet ub ample i identified a decribed above, then the length of the hortet ub ample i: r = x m+n -x n The fractional ub ample ize r approximate the integral of the ditribution function from point x n to point x m+n (ee Fig.. Therefore, to the extent that Mode(r give the correct etimate of the peak of the Gauian, the following relation involving the error function erf hold between the ub ample fractional ize r and it length L: r / σ r r = exp( t dt = erf ( π σ 0 The critical fact here i that the normalization factor depend on the ize of the uncontaminated ample, i.e. the outlier hould be excluded from thi count. If one know, for example if there are no outlier, then in order to compute σ one imply invert the equation. f(x r x m x m+n/ x m+n Figure The haded area, defined a the integral of f(x from x m =Mode(r r/ to x m+n =Mode(r r/, i equal to r. If i not known, which i the cae under conideration, one can find r for everal value of r and obtain a ytem of equation to olve for both and σ. We introduce notation Mr for the etimator of the catterσ of the normally ditributed data. (ML tand for multiple hortet length. We will call the et of fractional ize r the upport of the etimator ML (r. B. Derivation of the Equation for catter Etimation ince Mr depend on everal parameter, there i a certain degree of freedom in electing the mot ective way of computing it. The more traightforward way i to find r for value of r and olve the ytem of two equation for two unknown: 365

3 r = u erf r = u erf u ( r ( r ; v σ Factoring out u lead to the following equation for v: r erf ( r = r erf ( r. (4 Another approach to finding and σ i by to perform the leat-quare fit to the data. Quantitie r and r are meaured for ub ample. The following quantity i minimized: ( ru erf ( L r = Minimization with repect to u and v ru erf r v u v = ( ( ( ru erf ( r = 0, = lead to the following et of equation = = r ( r u erf ( r r exp = 0; (3 ( (5 = 0; ( ( r r u erf ( r ( = 0. Again, factoring out u lead to the following equation for v: = r erf r = ( r r r exp( ( r = r exp ( ( r erf ( r = 0. The olution of either ytem of equation 3 or 7 i ubject to the obviou contraint: u. (9 The way we apply thi contraint i to olve the ytem firt, and if u >, then et u = and compute v a erf ( r v = (0 = r Baed on the imulation we concluded that the only gain achieved by uing the econd approach i in the execution time. Below we preent only reult obtained by olving the equation 4. Here i the ummary of the firt approach. One find the length r of the hortet ubet of r data point and r of the hortet ubet of r data point. The value of r, r, r, and r are ued to olve equation (4 for v, which i directly related by equation (3 to the varianceσ. (6 (7 (8 C. Further Refinement-Automated ML Once Mr and are found a further refinement i poible. We how in ection III baed on the reult of imulation, that in general the greater r i, the more accurate are the ML (r and etimator. We do iterative refinement uing the following the trategy. We tart with reaonably mall r, meaure r, and compute Mr and. The value r are incremented and Mr and are recomputed until at leat one of the three topping criteria i met. Two empirical parameter R max and dr min are ued to define topping criteria of the algorithm. The iteration terminate if / exceed R max, or if the change in / between two iteration drop below dr min. Another topping criterion i baed on the aumption that the change in the Mode(r between two conecutive iteration cannot exceed the value of the catter Mr. The mode i found for the bigget r in the et. For example, if the upport i r = (0.4; 0.5, then the mode i found for r = 0.5. We ue notation AMr (automated Mr for the catter etimator obtained uing thi iterative proce. upport r in thi cae refer to the initial et of value of the fractional ub ample ize. The reult of the imulation preented below were obtained uing the following empirical value of the parameter R max = 0.95, dr min = 0.. IV. IMULATIO A. Detail of the imulation In order to tet the etimator derived in the previou ection we generated everal et of clean and contaminated data. The uncontaminated data are zero-mean Gauian with the tandard deviation of. We run our imulation for three ample ize of 30, 00, and 000 data point. In order to compute the average and tandard deviation of the etimator ML and AML we repeated the imulation time for the ample ize 30, for the ample ize of 00, and 0000 time for the ample ize of 000. We generated two kind of outlier that were added to replace the data point from the main ditribution. The firt kind conited of data point uniformly ditributed from 3 to 8. The econd kind conited of data point normally ditributed with the ame tandard deviation of a the main data and the mean of 4 ( percentile. The outlier fraction F=- / ued for the uniformly ditributed data wa 0., 0.3, 0.4, and 0.5. The outlier fraction F ued for the normally ditributed outlier wa 0., 0.3, and 0.4. The ratio of F = 0.5 obviouly could not be ued for the normally ditributed outlier, ince in thi cae they are no longer outlier. In Fig. we how two hitogram for two extreme cae of contaminated data: one ample of data with the uniform outlier with the fraction F = 0.5 and one ample of data the Gauian outlier with the fraction F = 0.4; for the ample ize in both cae i = 000. For comparion we alo computed the median abolute deviation (MAD []. The value of MAD i a meaure of the cale or diperion of a ditribution about the median. It i 366

4 often calculated a the median of the abolute-value ditance of the point about the median: MAD = median{ x_i - median(x } and multiplied by a factor of.486 to achieve conitency with the tandard deviation for aymptotically normal ditribution. f(x f(x x Figure 3 The top figure how a hitogram of a contaminated ample with the uniformly ditributed outlier with the fraction F = 0.5; the bottom figure how a hitogram of a contaminated ample with the normally ditributed outlier with the fraction of F = 0.4. The ample ize in both cae i 000. B. imulation of Contaminated Data ow we conider outlier contaminated data. Firt, in Table we how the reult for the ML a a function of the contamination fraction F. We alo preent the reult for the relative uncontaminated ize T : T = true true = ( F To compute thee quantitie we ue the upport r = (0.4;0.5, which i the bigget upport one can ue with no a priori knowledge of the outlier level, only auming that it doe not exceed 50% of the ample ize. The reult for lower level of contamination diplay a higher bia and diperion. The cloer i the ub ample ize to the ize of the uncontaminated ample, the better are the reult for the ML and T :. The refined etimator AML i devied with the idea of improving the accuracy of etimation for lower level of contamination. A part of the algorithm the upport i increaed to come a cloe a poible to the ize of the uncontaminated data without overtepping that boundary and without including the outlier in the ub ample defined by the upport r. In Table 3 we give the reult of the imulation for the initial upport r = (0.5;0.35. The goal i achieved, a AML i a better etimator for the lower outlier level and i very cloe to the ML for the highet outlier level. We illutrate the reult for the ML and AML etimator in Fig. 4. We alo diplay the value obtained for the MAD etimator for comparion. Only the etimator computed for the uniform outlier ditribution are plotted. The reult for both Gauian and uniform outlier are very imilar, the only difference i that for the uniform outlier the ML and AML etimator can be computed in the extreme cae of 50% outlier. LIMITATIO The method developed here ha certain limitation. One of them i that the equation (4 doe not alway have a olution for a mall ample ize. The failure rate become negligible for the ample ize > 00 data point. The econd limitation i the cae of highly contaminated data with the outlier having a more dene ditribution than the main data. In thi cae the mode of the outlier could be picked over that of the main data. Further invetigation i planned to etablih a well defined limitation of thi method and alo to extend it to the other than Gauian ditribution. COCLUIO We derived a new method of etimating the catter of normally ditributed data with high level of contamination. Our method i very table and perform well for the fraction of outlier of up to 50%. Thi method can be applied to etimating noie variance in noiy data, where the number of data point containing only noie i greater than the number of data point containing both ignal and noie. 367

5 5 Mr T ample Fraction F of Uniform Outlier Fraction F of Gauian Outlier ize ( ( ( ( ( ( ( (0.30 (0.4 (0.5.0 (0..00 ( ( (0.30 ( ( ( ( (0.34 (0.5.0 ( (0.0.0 ( ( ( ( ( ( ( ( ( ( (0.4 (0.5.0 (0..00 ( (0.4 ( ( ( (0.9.0 (0.05 Table. The average (tandard deviation of the catter Mr and relative uncontaminated ize T etimator for outlier contaminated ample. The upport r = (0.4; 0.5. ample Fraction F of Uniform Outlier Fraction F of Gauian Outlier ize AMr (0.33 (0.39 (0.49 (0.65 (0.33 (0.39 ( ( ( ( (0.50 ( (0.34. ( ( (0..0 ( ( ( (0..0 ( T (0.5 (0.8 (0.3 (0.37 (0.5 (0.8 ( (0.. (0.5.5 ( ( (0.. (0.5.7 ( ( (0.7.0 ( ( ( ( (0.4 Table. The average (tandard deviation for the catter AMr and relative uncontaminated ize T etimator computed uing the refined trategy. The initial upport wa r = (0.5;

6 6 catter Etimate catter Etimate ample ize 30 erie AML erie3 ML erie MAD Outlier Fraction F erie AML ample ize 00 erie3 ML erie MAD Outlier Fraction F [4] J.L. tarck and F. Murtagh, Automatic oie Etimation from the Multireolutional upport, Publication of the Atronomical ociety of the Pacific, 0, 93, 998 [5] M. Janen, oie Reduction by Wavelet Threholding, pringer Verlag ew York Inc., 00 [6] A.B. Hamza and H. Krim, Image Denoiing: A onlinear Robut tatitical Approach, IEEE Tranaction on ignal Proceing, 49, 3045, 00 [7] L. Alparone, G. Corini, M. Diani, oie modeling and etimation in image equence from thermal infrared camera, Optic in Atmopheric Propagation and Adaptive ytem VII, Proceeding of the PIE, 5573, 38,004 [8] D.F. Andrew, P.J. Bickel, F.R. Hampel, P.J. Huber, W.H. Roger, and J.W. Tukey, Robut Etimate of Location, Princeton Univerity Pre, Princeton,.J., 97 [9] P.J. Roueeuw and A.M. Leroy, Robut Regreion and Outlier Detection, John Wiley & on, ew York, Y, 987 [0] D. R. Bickel, Robut etimator of the mode and kewne of continuou data, Computational tatitic and Data Analyi 39, (00 0 erie AML erie3 ML erie MAD ample ize 000 catter Etimate Outlier Fraction F 0. Figure 4. Three etimator of the data catter: AML, ML, and MAD a function of the outlier fraction F. ML i computed with the upport r = (0.4; 0.5, and AML i computed with the initial upport r = (0.5; REFERECE [] K.K. Paliwal, Etimation of noie variance from the noiy ar ignal and it application in peech enhancement, IEEE Tranaction on Acoutic, peech and ignal Proceing, 36, 9, 988 [] H. Attia, L. Deng, A. Acero, and J. C. Platt, A new method for peech denoiing and robut peech recognition uing probabilitic model for clean peech and for noie, In Proc. Europeech, 3, 903, 00 [3] K. Kontantinide, B. atarajan, and G.. Yovanof, oie Etimation and Filtering Uing Block-Baed ingular Value Decompoition, IEEE Tranaction of Image Proceing, 6, 479,