A NEW LEAST SQUARES ADAPTATION SCHEME FOR THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS

Transcription

1 A NEW LEAST SQUARES ADAPTATION SCHEME FOR THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS Luis A. Azpicueta-Ruiz, Aíbal R. Figueiras-Vidal, ad Jeróimo Areas-García Departmet of Sigal Theory ad Commuicatios Uiversidad Carlos III de Madrid Legaés-Madrid, Spai ABSTRACT Adaptive combiatios of adaptive filters are a efficiet approach to alleviate the differet tradeoffs to which adaptive filters are subject. The basic idea is to mix the outputs of two adaptive filters with complemetary capabilities, so that the combiatio is able to retai the best properties of each compoet. I previous works, we proposed to use a covex combiatio, applyig weights λ() ad 1 λ(), with λ() (, 1), to the filter compoets, where the mixig parameter λ() was updated to miimize the overall square error usig stochastic gradiet descet rules. I this paper, we preset a ew adaptatio scheme for λ() based o the solutio to a least-squares (LS) problem, where the mixig parameter is allowed to lie outside rage [, 1]. Such affie combiatios have recetly bee show to provide additioal gais. Ulike some previous proposals, the ew LS combiatio scheme does ot require ay explicit kowledge about the compoet filters. The ability of the LS scheme to achieve optimal values of the mixig parameter is illustrated with several experimets i both statioary ad trackig situatios. Idex Terms Adaptive filters, least squares (LS), combiatio of filters. 1. INTRODUCTION Adaptive filterig schemes have become crucial compoets i may sigal processig applicatios [1], [2]. Whatever kid of adaptive filter is used, a compromise ivolvig speed of covergece, trackig capabilities, ad steady-state error is always preset. I order to alleviate these performace compromises, schemes that maage filter parameters (see, amog may others, [3], [4]) have traditioally bee used. A recet alterative cosists i usig adaptive combiatios of adaptive filters with complemetary capabilities [5], [6]. This approach is gaiig popularity due to its simplicity ad ability to deal with almost ay kid of performace tradeoff i may applicatios [7]-[11]. I the basic combiatio scheme, two adaptive filters combie their outputs by meas of a mixig parameter to obtai a overall output of improved performace: y() = λ()y 1 + [1 λ()]y 2(), (1) This work was partly supported by by the Spaish Miistry of Educatio ad Sciece uder grat CICYT TEC ad by Madrid Commuity grat S-55/TIC/223. where y i(), i = 1, 2 are the outputs of the compoet filters, y() is the overall output, ad is the time idex. It has bee show that, whe mixig parameter λ() is appropriately selected, the overall filter performs at least as the best idividual compoet, or eve better tha ay of them [6], [12]. Therefore, the desig ad study of good adaptatio rules for λ() is of primary importace to get the best out of the combiatio. I [6] we studied the performace of oe such adaptatio rules, i which the mixig parameter was updated to miimize the overall square error usig a gradiet descet algorithm. Additioally, the mixig parameter was costraied to iterval (, 1) by defiig it as the output of a sigmoid activatio fuctio, λ() = [1+e a() ] 1, so that (1) is a covex combiatio. The, parameter a() was adapted usig the least-mea-square (LMS) algorithm: a( + 1) = a() µ a e 2 () a(), (2) where e() = d() y(), ad d() is the desired output of the filter at each iteratio. The value of λ() ca be recovered from a() at each iteratio. Update (2) provides satisfactory performace whe the step-size parameter µ a is appropriately chose. However, its correct adjustmet depeds o certai characteristics of the filterig sceario, such as the sigal-to-oise (SNR) ratio or the speed of chages i trackig situatios. I order to simplify the selectio of the step size for the mixig parameter, a ormalized LMS (NLMS) adaptatio scheme for the covex combiatio of two filters was preseted i [13]. Both the LMS ad NLMS based adaptatio schemes for λ() share the advatage of beig completely geeral, i the sese that they ca be used with ay kid of adaptive filters ad their applicatio does ot require ay kowledge about the operatio mechaisms of the compoet filters. Recetly, it has bee theoretically foud that the optimal values of the mixig parameter ca take values outside rage [, 1], providig additioal (though geerally ot very sigificat) performace gais [12]. For istace, for the particular case of a combiatio of two LMS filters with step sizes ad ( > ) for the first ad secod compoets, respectively, the optimal λ() takes egative steady-state values i statioary situatios. I order to beefit from the potetial advatages of egative mixig parameters, Bershad et. al [12] proposed to use

2 the followig rule for selectig the mixig parameter. ê2 «λ() = 1 κerf 1 () ê 2 2 (), (3) where ê 2 i(), i = 1, 2 are estimatios of the istataeous mea square errors of both compoet filters, which ca be obtaied as time averages over a rectagular widow of legth K, i.e., ê 2 i() = 1 X e 2 i(m), (4) K m= K+1 where e i() = d() y i(). This scheme has the advatage of ot requirig to adjust ay step size for the mixig parameter, while allowig egative values for λ() whe κ > 1. Selectio of a appropriate value for κ, however, requires to kow which is the optimal steady-state value for the mixig parameter, ad this result is i geeral depedet o the particular filters beig combied, as well as the characteristics of the filterig sceario. Based o a statistical aalysis of the LMS filter, ad usig several assumptios, a appropriate value of κ was foud to be κ = 1 + (5) 2( ) for the particular case of a combiatio of two LMS filters. However, selectio of κ is ot evidet whe a exact aalysis of the mea-square performace of the compoet filters (ad of the cross-correlatio betwee their errors) is ot plausible, or whe it is subject to approximatio errors. Furthermore, the fact that this adaptatio scheme is ot explicitly searchig for the miimizatio of the square error ca result i suboptimal operatio i certai circumstaces, as we will see later i the experimets sectio. I this paper, we preset a ew adaptatio scheme for λ() based o the solutio of a very simple least-squares (LS) problem. As i [12], we allow the mixig parameter to lie outside iterval [, 1], ad rely also o time averages through a widow of the most recet samples to obtai good estimatios of appropriate values for the parameter. Our scheme, however, does ot require ay specific kowledge about the particular kid of filters i the combiatio, but it proceeds directly to the miimizatio of the square error. I the ext sectio, we itroduce the ew LS adaptatio rule for λ(), ad provide some aalytical isight about its expected performace. I Sectio 3 we will carry out several experimets, showig the efectiveess of the ew scheme both i statioary ad trackig coditios, ad comparig its behavior to that of some previous proposals. 2. LEAST SQUARES ADAPTATION OF THE MIXING PARAMETER 2.1. Algorithm Descriptio Cosider the followig LS problem: J[λ()] = X β(, i)e 2 (, i) (6) where β(, i) are the weightig coefficiets associated to each time istat, e(, i) = d(i) y(, i), ad y(, i) would be the predictio of the overall filter for d(i) if the outputs of the compoet filters were combied with parameter λ(), i.e, y(, i) = λ()y 1(i) + [1 λ()]y 2(i) = y 2(i) + λ()[y 1(i) y 2(i)]. I order to obtai the value of the mixig parameter that miimizes (6), we ca take derivatives with respect to λ(), J[λ()] λ() = 2 (7) X β(, i)e(, i)[y 1(i) y 2(i)]. (8) Now, makig this expressio equal to, ad solvig out for λ(), we arrive at the followig equatio for the optimal value of the mixig parameter at iteratio : P β(, i)[d(i) y2(i)][y1(i) y2(i)] λ LS() = P β(, i)[y1(i). (9) y2(i)]2 This result ca be iterpreted i a ituitive way i the light of the secod lie of (7). That expressio for the overall filter output suggests that the combiatio of filters ca be see as a secod-level oe tap filter with weight λ(), whose iput sigal ad desired output are y 1() y 2() ad d() y 2(), respectively. The, (9) is coheret with the solutio of a LS problem, sice the deomiator ca the be iterpreted as the autocorrelatio of the iput sigal (with zero lag), while the umerator would be a estimatio of the crosscorrelatio betwee the iput ad desired sigals (also with zero lag). Selectio of the widow β(, i) plays a importat role i the proposed algorithm. The choice of a expoetiallyweighted widow, β(, i) = β i, with ( < β 1), would make it easy to derive a RLS-like algorithm for updatig λ(). For this oe-tap filterig problem, however, computatioal coveiece is ot a issue, sice the direct applicatio of (9) is very easy ad computatioally efficiet. I the practice, we have foud that a rectagular widow j 1 β(, i) =, i < K, i > K where K is the widow legth, offers improved performace with respect to the expoetially-weighted widow, as we shall discuss later i the experimets sectio Aalysis of the LS adaptatio rule for λ() I this sectio we give some theoretical isight about the ability of (9) to provide appropriate values for the mixig parameter. We assume a statioary data model similar to that i [6]: d() = w T o u()+e (), where w o is a ukwow system, u() is the iput to the compoet filters, satisfyig E{u()} = ad E{u()u T ()} = R, ad e () is zeromea i.i.d. oise with variace σ. 2 Uder this model, we ca defie the a priori errors of the compoet filters as e a,i() = e i() e (), i = 1, 2. Usig similar argumets to those i [6] ad [12], it ca be show that the optimal value for the mixig parameter is give by J 2() λ o() = (1) J 1() + J 2() where we have defied J i() = E{e 2 a,i() e a,1()e a,2()}, i = 1, 2.

3 O the other had, if we take the expectatio of (9), it is easy to see that P β(, i)e{[d(i) y2(i)][y1(i) y2(i)]} E{λ LS()} P β(, i)e{[y1(i) y2(i)]2 } (11) where we have approximated the expectatio of the quotiet by the quotiet of the expectatios of the umerator ad deomiator. It ca be show that such a approximatio itroduces a egligible error for ot too small K, sice the umerator ad deomiator i (9) are already reasoably good estimates of the correlatio ad cross-correlatio of the ivolved sigals usig time averages over several iteratios. Now, otig that y 1(i) y 2(i) = e a,2(i) e a,1(i), ad that d(i) y 2(i) = e 2(i) = e a,2(i)+e (), ad after some algebraic maipulatios, we arrive at P β(, i) J2(i) E{λ LS()} P β(, i)[ J1(i) + J2(i)] (12) Obviously, i steady state, this value approximates the optimal value for the mixig parameter give by (1), with the approximatio beig more accurate as the legth of the widow icreases. O the other had, i trasiet situatios, or i other time-varyig situatios, J i() chages over time. Therefore, i these cases, it would be desirable to use shorter widows to reduce the bias i the estimatio of the mixig parameter (obviously, at the cost of a icreased variace). 3. EXPERIMENTS I this sectio we study the performace of the ew LS scheme for adaptig the mixig parameter of a affie combiatio i a plat idetificatio setup. Two sets of experimets have bee carried out. First, we focus o the ew LS rule, ad study how differet kids of widows ca ifluece its performace, both i statioary ad trackig coditios. I the secod block of experimets, the performace of the ew rule is compared to that of previous schemes for adaptig the mixig parameter Performace of the LS adaptatio rule For these simulatios we have cosidered a real plat with M = 16 coefficiets, whose iitial values are take at radom from iterval [ 1, 1]. Whe studyig covergece ad statioary behavior, plat coefficiets are chaged abruptly durig the experimet, to study the ability of the algorithm to recoverge after a fast trasitio i the plat. The iput sigal, u() is i.i.d. Gaussia oise with power σ 2 u = 1 16, so that Tr(R) = 1. The output aditive oise, e (), is the same kid of sigal, ad its variace has bee selected to get a SNR of 2 db. We have used the excess mea-square-error (EMSE), EMSE() = E{e 2 () e 2 ()} as a figure of merit of the behavior of the combiatio schemes, averagig the results over 1 idepedet realizatios. I Fig. 1 we have displayed the behavior of a combiatio of two LMS filters whe usig LS adaptatio for the mixig parameter. The step sizes for the LMS compoets are = LS exp LS Rect (K=5) LS Limit (K=5) (c) LS Limit (K=5) Fig. 1. EMSEs of the compoet LMS filters ( =.1 ad =.2) ad of the proposed LS algorithm (9). Usig a expoetially-weighted widow. Usig a uiform widow (K = 5). (c) Usig a uiform widow (K = 5) ad imposig λ() 1. (d) Usig a uiform widow (K = 5) ad imposig λ() 1..1 ad =.1, so that the first filter adapts faster. Two differet widows were selected for the adaptatio of λ() as follows: (d) LS-exp [Subfig. 1]: expoetially-weighted widow, β(, i) = β i. LS-rect [Subfig. 1]: uiform widow of legth K. The memory of both schemes, cotrolled by parameters β ad K, was fixed log eough to guaratee that a earlyoptimal steady-state value of λ() was achieved, leadig to β =.9997 ad K = 5. I the light of the results, it seems evidet that a more adequate performace is obtaied whe usig the rectagular widow (LS-rect). The expoetiallyweighted widow itroduces a very log delay i the trasfer betwee filter compoets. This is due to the fact that quadratic errors durig covergece (i.e., just after = ad = 4) are much larger tha the errors icurred after the two compoet filters have coverged, thus havig a sigificat ifluece i the cost fuctio (6), eve whe multiplied by a small factor.

4 As we discussed at the ed of Sectio 2, a log memory allows for a very accurate estimatio of the steady-state value of λ o(), but ca actually damage the combiatio performace i trasiet situatios. This is i fact the reaso for observig that the overall EMSE situates well above that of the fast compoet followig the trasitios i the plat. I order to avoid this egative effect, it is eough to impose a upper limit o the mixig parameter: λ LS() 1, as we ca check i Subfig. 1(c). We have fially explored how the legth of the widow (parameter K) affects the EMSE of the combiatio. I Subfig. 1(d) we have represeted EMSE evolutio for K = 5. As we ca see, the trasfer betwee filter compoets occurs ow with almost o delay, but the steady-state EMSE suffers a slight icremet. I order to assess the performace of the proposed LS adaptatio rule for λ(), it is ecessary to study its trackig abilities. Trackig situatios have bee aalyzed assumig a radom-walk model for the plat coefficiets w o( + 1) = w o() + q(), where q() are i.i.d. zero-mea Gaussia radom vectors with covariace matrix Q = E{q()q T ()} = σqi. 2 We will study filter performace for varyig, which ca be see as a measure of the degree of o-statioarity i the plat. I these experimets, the ormalized square deviatio (NSD) of a filter, defied as the ratio of its EMSE to that of the LMS filter with optimal step size, is used as the figure of merit. Note that i the trackig situatio we are cosiderig, there exists a LMS filter with optimal performace, whose step size depeds o the speed of chages ad is give by [6, Eq. (48)]: s µ opt = σ 2 Tr(R) + []2 σ 4 2 2σ. (13) Therefore, the steady-state ( ) NSD of ay filter (either a compoet or the combiatio) is defied as NSD( ) = EMSE( ) EMSE opt( ), (14) where EMSE opt( ) is the steady-state EMSE of the LMS filter with step size give by (13). Fig. 2 represets, as a fuctio of, the steady-state NSD achieved by the two LMS compoets, as well as that icurred by their combiatio usig the proposed LS adaptatio rule for the mixig parameter. We have cosidered both the cases where o limits are imposed o λ LS() [Subfig. 2], ad the case i which a upper limit of 1 has bee imposed [Subfig. 2]. All results have bee averaged over 25 iteratios oce the algorithms reached steady-state, ad over 2 idepedet rus. As it ca be see, the combiatio scheme offers a appropriate behavior for a wide rage of degrees of o-statioarity, achievig, at least, the smallest of the NSDs of the compoet filters. Whe comparig Subfigs. 2 ad 2, it is clear that imposig a upper limit of 1 o the mixig parameter has a very mior ifluece o filter performace. Therefore, i the ext subsectio, we will focus o this particular implemetatio of the LS combiatio scheme, ad compare its NSD( )[db] NSD( )[db] LS Rect (K=5) LS Limit (K=5 ) Fig. 2. Steady-state NSD of two LMS filters, ad of their adaptive combiatio usig LS adaptatio for λ() with a uiform widow (K=5). No costraits are imposed o λ LS(). λ LS() 1. performace to that of some other schemes for learig the mixig parameter Compariso to other adaptatio rules For compariso purposes, i this subsectio we have cosidered two other methods for the adaptatio of the mixig parameter: 1) a ormalized gradiet descet algorithm from [13], ad 2) update rule (3), take from [12]. Fig. 3 represets the covergece ad steady-state performace of all three combiatio schemes for the same sceario used i the previous subsectio (otice the differet scalig of the time axis), ad compare the to the optimal (but urealizable) EMSE of ay affie combiatio of the ad -LMS filters, that would be obtaied if usig λ o(). All three combiatio schemes are able to follow the fast covergece of the -LMS ad achieve the lower steady-state

5 LS Optimum 2 4 Normalized Optimum 2 4 Scheme from [12] Optimum 2 4 Fig. 3. EMSEs of differet schemes for adaptively combiig adaptive filters. The optimal (but urealizable) EMSE of ay affie combiatio of filters ad -LMS filters is also icluded for compariso purposes. LS adaptatio of the mixig parameter (this paper). Normalized adaptatio rule from [13]. (c) Scheme from [12]. error of the secod compoet. Differet deviatios are observed, however, with respect to the optimal affie combier. Regardig LS [Subfig. 3], we ca observe certai delay i the trasitio betwee filters. As for the ormalized adaptatio rule [Subfig. 3], i this case it seems to preset a better behavior tha ay of the two other schemes, but it has the icoveiece of ot allowig egative values for the mixig parameter, what ca result i suboptimal operatio i other cases. Fially, a bump is observed for the scheme from [12] [Subfig. 3(c)], which is probably a cosequece of the fact that this scheme is ot explictly miimizig the overall square error. I some other filterig scearios this bump may ot appear, ad this scheme could provide a more satisfactory performace. I Fig. 4 we have represeted the steady-state NSD of the three combiatio schemes, agai as a fuctio of. The performace of the LS ad the ormalized adaptatio rules is very similar, with LS achievig a very mior advatage for slowly chagig scearios, as a cosequece of usig egative values for the mixig parameter. The scheme from [12] ca also exploit the advatages of egative values for λ(). O the other had, it shows suboptimal performace for a wide rage of. (c) 4. CONCLUSIONS Adaptive combiatios are a flexible ad versatile approach to improve the performace of adaptive filters. I order to λ() NSD( )[db] LMS LMS Scheme from [12] Normalized LS Scheme from [12] Normalized LS Fig. 4. Trackig performace of differet schemes for adaptively combiig adaptive filters. NSD steady-state performace. Steady-state value of the mixig parameter for each method. get the best out of them, however, the combiatio eeds to be appropriately adjusted at each iteratio. I this paper, we have preseted a ovel rule for adaptig the parameter that cotrols the combiatio, which is based o the solutio to a very simple LS problem. The behavior of our proposal has bee illustrated by meas of several experimets i both statioary ad trackig situatios, ad show to be competitive with respect to existig schemes. 5. REFERENCES [1] A. H. Sayed, Fudametals of Adaptive Filterig, New York: Wiley, 23. [2] S. Hayki, Adaptive Filter Theory, Upper Saddle River, NY: Pretice Hall, 22. [3] H.-C. Shi, A. H. Sayed, ad W.-J. Sog, Variable step-

6 size NLMS ad affie projectio algorithms, IEEE Sigal Process. Lett., vol. 11, pp , Feb. 24. [4] W. Zhuag, RLS algorithm with variable forgettig factor for decisio feedback equalizer over time-variat fadig chaels, Wireless Persoal Commu., vol. 8, pp , Aug [5] M. Martíez-Ramó, J. Areas-García, A. Navia- Vázquez, ad A. R. Figueiras-Vidal, A adaptive combiatio of adaptive filters for plat idetificatio, i Proc. 14th Itl. Cof. Digital Sigal Processig, Satorii, Greece, 22, pp [6] J. Areas-García, A. R. Figueiras-Vidal, ad A. H. Sayed, Mea-square performace of a covex combiatio of two adaptive filters, i IEEE Tras. Sigal Process., vol. 54, pp , Mar. 26. [7] Y. G. Zhag ad J. A. Chambers, Covex Combiatio of Adaptive Filters for a variable tap-legth LMS algorithm, IEEE Sigal Proc. Letters, vol. 13, pp , 26. [8] A. H. Sayed ad C. Lopes, Adaptive Processig over Distributed Networks, IEICE Tras. Fudametals of Electroics, Commuicatios ad Computer Scieces, vol. E9A, pp , 27. [9] L. A. Azpicueta-Ruiz, A. R. Figueiras-Vidal, ad J. Areas-García, Acoustic echo cacellatio i frequecy domai usig combiatios of filters, i 19th Itl. Cogress o Acoustics (ICA), Madrid, Spai, 27. [1] D. P. Madic, et al. Olie trackig of the degree of oliearity withi complex sigals, i Proc. ICASSP 8, Las Vegas, NV, 28, pp [11] M. T. M. Silva ad V. H. Nascimeto, Improvig the trackig capabilities of adaptive filters via adaptive combiatio, IEEE Tras. Sigal Process., to appear, 28. [12] N. J. Bershad, J. C. M. Bermudez, ad J.-Y. Toureret, A affie combiatio of two LMS adaptive filters trasiet mea-square aalysis, IEEE Tras. Sigal Process., vol. 56, pp , 28. [13] L. A. Azpicueta-Ruiz, A. R. Figueiras-Vidal, ad J. Areas-García, A ormalized adaptatio scheme for the covex combiatio of two adaptive filters, i Proc. ICASSP 8, Las Vegas, NV, 28, pp