Low-complexity widely linear RLS filter using DCD iterations

Transcription

1 XXX SMPÓSO BASLEO DE TELECOMUNCAÇÕES - SBrT 12, DE SETEMBO DE 212, BASÍLA, DF Low-complexity widely linear LS filter using DCD iterations Ferno G. Almeida Neto, Vítor H. Nascimento Yuriy V. Zakharov esumo ecentemente, filtros adaptativos amplamente lineares estão sendo usados para acessar completamente estatísticas de segunda ordem de sinais impróprios, com o objetivo de melhorar a estimação. Essa característica torna esses filtros vantajosos em relação aos seus equivalentes estritamente lineares, apesar de apresentarem maior complexidade computacional. Nesse sentido, com o intuito de reduzir o custo computacional do LS amplamente linear (WL-LS), em um artigo anterior foi proposta uma versão de complexidade reduzida que foi chamada de C-WL-LS e que apresentou um quarto da complexidade computacional do WL-LS. Contudo, o algoritmo obtido ainda manteve complexidade O(N 2 ) (em que N representa o comprimento do vetor regressor). No presente trabalho, o C-WL-LS é modificado e é proposta uma modificação do algoritmo dichotomous coordinate descent (DCD) para iterativamente resolver as equações normais. Com essa abordagem, o número de multiplicações por iteração é reduzido e um algoritmo numericamente estável e de complexidade linear com N é obtido. Simulações são realizadas para comprovar o funcionamento do algoritmo proposto. Palavras-Chave Estimação amplamente linear, Estimação com complexidade reduzida, Algoritmo LS amplamente linear de complexidade reduzida, Algoritmo dichotomous coordinate descent (DCD) Abstract Widely linear (WL) adaptive filters have been receiving much attention recently, since they take advantage of the full second-order statistics of improper signals. While this approach provides estimation gains if compared to their strictly linear (SL) counterparts, WL algorithms generally present higher computational complexity. n order to reduce the computational cost, we presented in a previous paper a reduced-complexity (C) version of the WL-LS algorithm the C-WL-LS algorithm which was shown to be 4 times less expensive than the WL- LS, but still keeps the O(N 2 ) complexity ( N is the length of the regressor vector). n this paper, we modify the C- WL-LS, applying the dichotomous coordinate descent (DCD) algorithm to iteratively solve the normal equations. With this approach, we reduce the number of multiplications per iteration obtain a numerically stable widely-linear LS algorithm with computational complexity linear on N. Simulations are presented to support our approach. Keywords Widely linear estimation, educed-complexity estimation, educed-complexity widely linear LS, Dichotomous coordinate descent (DCD) algorithm. NTODUCTON t is well-known that the ecursive Least-Squares (LS) algorithm 1] converges faster than the Least Mean-Squares (LMS) algorithm 1], that the LS is also less sensitive Ferno G. Almeida Neto Vítor H. Nascimento are with the Dept. of Eletronic Systems, Escola Politécnica of the Univeristy of São Paulo, São Paulo, SP, Brasil. {fganeto, vitor}@lps.usp.br. Yuriy V. Zakharov is with the Dept. of Electronics of the University of York, York, U.K.. yury.zakharov@york.ac.uk to the spread of the eigenvalues in the covariance matrix. However, while the LMS presents linear computational cost, the LS complexity is quadratic with the regressor vector length (O(N 2 )). Moreover, the LS algorithm also suffers with numerical instability, requiring more care during the implementation in order to avoid divergence. The computational cost is also aggravated if we use a widely linear (WL) LS algorithm 2]. n this case, the length of the regressor vector is duplicated to use both the complex data their conjugate, which provides smaller mean-square error (MSE) for improper (or non-circular) input 3]. This approach is used to account for full second-order statistics of improper signals achieve better estimation. n order to reduce the complexity of the WL-LS, a reduced-complexity (C) version of the WL-LS was proposed in 4]. n that paper, a linear transformation was used to obtain an algorithm with exactly the same performance, but with a complexity of one-fourth the complexity of the WL-LS. Although it was shown that the C-WL-LS has complexity similar to the strictly linear (SL) LS algorithm, the cost was still O(N 2 ) the numerical stability still remained unsolved. (See Table to compare the number of real operations per iteration of the algorithms, we call strictly linear (SL) LS the traditional complex LS algorithm.) Note that the quadratic cost of the algorithms comes from the update of the inverse autocorrelation matrix. n 5] 6], the dichotomous coordinate descent (DCD) was proposed as an alternative to iteratively solve the normal equations. t was shown in 7] that the DCD is an efficient stable alternative to implement the LS algorithm in FPGAs, since this algorithm uses less multiplications to solve the normal equations, less multipliers lead to the reduction of the consumed area. The DCD is particularly attractive for the reason that one can obtain an LS algorithm with complexity proportional to N. Moreover, since the DCD-LS does not propagate the inverse autocorrelation matrix, it does not suffer from numerical instability. n this paper we modify the DCD algorithm to use with the C-WL-LS. As a result, we obtain an algorithm that presents low-complexity that is, linear on N which is numerically stable. We compare the obtained algorithm (the DCD-WL-LS) with the C-WL-LS show that the new approach provides a low-complexity algorithm with almost the same performance of the C-WL-LS, which is supported by our simulations. This paper is organized as follows: Section presents the C-WL-LS algorithm. n Section we modify the C-

2 XXX SMPÓSO BASLEO DE TELECOMUNCAÇÕES - SBrT 12, DE SETEMBO DE 212, BASÍLA, DF WL-LS to obtain the DCD-WL-LS. n Section V, we present the version of DCD algorithm used here. Simulations comparing the algorithms are performed in Section V, we conclude the paper in Section V.. THE C-WL-LS ALGOTHM The C-WL-LS was first presented in 4] as a reducedcomplexity version of the WL-LS algorithm. The main difference between those algorithms is that using the linear transformation U = 1 j 1 j ] N, j = 1, N is the N-order identity matrix represents the Kronecker product 8], we can modify the complex WL-LS regressor vector ] s (k)+js s WL (k) = (k) s (k) js (k) to the real C-WL-LS regressor s C (k) = s T (k) st (k) ] T, s (n) s (n) are N 1 vectors containing the real the imaginary parts of the data, respectively. n this case, using a real regressor vector, we replace many complexcomplex multiplications (which cost 4 real multiplications 2 real sums each) by real-complex multiplications (which cost only 2 real multiplications each), reducing the complexity. n 4], it is shown that both the algorithms achieve exactly the same performance, since UU H /4 = 2N. Thus, the C-WL- LS is able to use full second-order statistics, but with reduced complexity. The algorithm is summarized in Table. nitialization w C () = 2N 1 P C () = δ 2N TABLE C-WL-LS ALGOTHM for k =,1,... γ C (k) = (λ C +s H C (k)p C(k)s C (k)) 1 k C (k) = P C (k)s C (k)γ C (k) e C (k) = d(k) w H C (k)s C(k) w C (k +1) = w C (k)+k C (k)e C (k) P C (k +1) = 1 λ C PC (k)+k C (k)k H C (n)γ C(k) ] Note that d(k) is the desired signal e C (k) is the estimation error. w C (k) is a vector containing the coefficient estimates, while P C (k) is the estimate of the inverse covariance matrix. We call k C (k) the Kalman gain λ C γ C (k) are the forgetting factor the conversion factor, respectively. We use an identity matrix multiplied by a small constantδ to initialize P C (), w C () is initialized with a zero-vector. From Table, we present the computational complexity of this algorithm, one could think that, besides the complexity reduction of the C-WL-LS, one could do better, since the algorithm is still quadratic in the complexity. n Section we present the DCD-WL-LS algorithm, which presents still lower computational complexity.. THE DCD-WL-LS ALGOTHM n this section, we modify the transversal DCD-LS proposed in 7] to develop our DCD-WL-LS with linear complexity cost. We assume in this paper that both s (k) s (k) are tap-delay lines, i.e., s (k) = s (k) s (k 1) s (k N +1) ] T s (k) = s (k) s (k 1) s (k N +1) ] T, respectively. Sinces WL (k) is composed of two tap-delay lines, the fast algorithm in 7] cannot be directly used. We show below how the DCD-LS can be modified to work with WL data in the present situation. When we deal with the C-WL-LS algorithm, for each k iteration we want to find the solution for the normal equations (k) = (k) = w(k)β(k), (1) λ k i s C (i)s T C(i) = λ(k 1)+s C (k)s T C(k) (2) is the 2N 2N correlation matrix λ is the forgetting factor. (k) is real symmetric. β(n) is a vector given by β(k) = λ k i d(i)s C (i) = λβ(k 1)+d(k)s C (k). (3) We can write eq. (2) as ] (k) (k) = (k) T (k), (k) (k) = λ k i s (i)s T (i) = λ (k 1)+s (k)s T (k), (k) = (k) = (4) λ k i s (i)s T (i) = λ (k 1)+s (k)s T (k), (5) λ k i s (i)s T (i) = λ (k 1)+s (k)s T (k) (6) are N N matrices. Note that (k) (k) are symmetric matrices, but (k) is not symmetric in general. (k) (k) can be evaluated using the low-cost update proposed in 7]: (k) = r (k) ρ T (k) ρ (k) (1:N 1,1:N 1) (k 1) ], (7) the notation (1:N 1,1:N 1) (k 1) sts for a matrix with the elements of (k 1) from row 1 to N 1 from column 1 to N 1. r (k) is a real number ρ (k) is an (N 1) 1 vector. From eq. (7), we note that we only need to update the first column of (k), since we already have (1:N 1,1:N 1) (k 1) from the last iteration (k) is symmetric. n order to obtain the first row of (k) we

3 XXX SMPÓSO BASLEO DE TELECOMUNCAÇÕES - SBrT 12, DE SETEMBO DE 212, BASÍLA, DF only need to copy the values of the first column. Thus, we calculate only the first column of (k) that is (1:N,1) (k) to update eq. (7), i.e., (1:N,1) (k) = λ (1:N,1) (k 1)+s (k)s (k). Similarly, we update the first column of eq. (5) using (1:N,1) (k) = λ (1:N,1) (k 1)+s (k)s (k). n order to update (k), we write (6) as the matrix r (k) ρ T (k) = (k) ] ρ (k) (1:N 1,1:N 1), (8) (k 1) (1:N 1,1:N 1) r (k) = ρ (k) = ρ (k) = λ k i s (i)s (i), λ k i s (i)s (i 1), (9) λ k i s (i)s (i 1) (1) (k 1) = = λ k i s (1:N 1) k 1 = λ k i s (2:N) = (2:N,2:N) (k). (i 1)s (1:N 1) (i)s (2:N) (i) (i i) (11) Note from eq. (11) that (1:N 1,1:N 1) (k 1) is a block matrix obtained from iteration k 1, which does not need to be updated. Equations (9) (1) show that, in general ρ ρ, which means that they need to be calculated separately. To obtain the first column of (k), we define the column vector (1:N,1) (k) = r (k) ρ T (k) ] T = k λk i s (i)s (i) = k 1 λk i s (i)s (i)+s (k)s (k) = λ (1:N,1) (k 1)+s (k)s (k), (12) which is recursively updated. The first row of (k) is updated in a similar manner, i.e., (1,2:N) (k) = λ (1,2:N) (k 1)+s (k)s (2:N)T (k). (13) Note that in eq. (13) we do not calculate the first element of the row, since it is already computed in eq. (12). Using this approach, we can calculate matrix (k) with the update of three N N block-matrices. ecall that for each blockmatrix, we only need to update the first row in the case of (k) (k) or the first row the first column (in the case of (k)). We use this low-cost form to update matrix (k) in the DCD-WL-LS algorithm (see Table ). Assume that we update (k) with the procedure shown before. n order to obtain the DCD-WL-LS, recall eq. (1), but in the instant k 1, i.e., (k 1) = w(k 1)β(k 1). (14) We define the residue of the normal equations, after obtaining an approximate solution ŵ(k 1), as the vector Moreover, define r(k 1) = β(k 1) (k 1)ŵ(k 1). (15) (k) = (k) (k 1), (16) β(k) = β(k) β(k 1), (17) w(k) = w(k) ŵ(k 1). (18) Substituting eqs. (16), (17) (18) in (14), after some algebra manipulation, we obtain (k) w(k) = β (k), (19) β (k) = r(k 1)+ β(k) (k)ŵ(k 1). (2) Solving equation (19) instead of (1) is a way of taking advantage of the previous solution ŵ(k 1) to reduce the complexity of the algorithm. This is due to the iterative structure of DCD solving (19) is equivalent to start DCD with ŵ(k 1) as initial condition, reducing the number of DCD iterations ( the complexity) necessary at each time instant. We apply the DCD algorithm of Table V to iteratively solve this system of equations in the DCD-WL-LS algorithm obtain the approximate solution ŵ(k) = ŵ(k 1)+ ŵ(k). (21) We can also write eq. (15) in terms of β (k) ŵ(k), using eqs. (2) (21), i.e., r(k) = β (k) (k) ŵ(k). ecall (16) (17). Using the second identity of (2) (3) in eqs. (16) (17), respectively, we can express (k) = (1 λ)(k 1)+s C (k)s T C (k) (22) β(k) = (1 λ)β (k)+d(k)s C (k). (23) Define the filter estimative y(k) = s T C (k)ŵ(k 1). Using (22) (23) in eq. (15), (k)ŵ(k 1) = = (λ 1)β(k 1) r(k 1)]+s C (k)y(k), (24) which can be used in (2) to obtain β (k) = λr(k 1)+e(k)s C (k), e(k) = d(k) y(k) corresponds to the a priori error. The DCD-WL-LS algorithm is summarized in Table,

4 XXX SMPÓSO BASLEO DE TELECOMUNCAÇÕES - SBrT 12, DE SETEMBO DE 212, BASÍLA, DF Step TABLE DCD-WL-LS ALGOTHM Equation nitialization: ŵ() = 2N 1, r 2N 1 () =, = δ N, = δ N, = N for k = 1,2,... (k) = λ (1,1:N) (k 1)+s (k)s (k) 2 (1,1:N) (k) = λ (1,1:N) (k 1)+s (k)s (k) 3 (1,1:N) (k) = λ (1,1:N) (k 1)+s (k)s (k) 4 (2:N,1) (k) = λ (2:N,1) (k 1)+s (k)s (2:N)T (k) 5 (k), (k), (k) (k) 6 y(k) = s T C (k)ŵ(k) 7 e(k) = d(k) y(k) 8 β (k) = λr(k 1)+e(k)s C (k) 9 (k) w(k) = p (k) ŵ(k), r(k) 1 ŵ(k) = ŵ(k 1)+ ŵ(k) 1 (1,1:N) we also provide the initialization of the algorithm. Note that the DCD algorithm, which is presented in Section V, is used to solve the 9 th row of Table. Table compares the complexity of the developed algorithm with the SL, WL C-WL-LS. Note that there are two lines referring to the DCD-WL-LS. The first presents the complexity considering that λ can be any real number such that < λ < 1. The second assumes that λ = 1 2 m, m N, which allows the substitution of multiplications by bit-shifts sums in steps 1, 2, 3, 4 8. N u is the maximum number of successful iterations in DCD (see Section V). TABLE COMPUTATONAL COMPLEXTY OF THE SL-LS, WL-LS, C-WL-LS AND DCD-WL-LS N TEMS OF EAL OPEATONS PE TEATON Algorithm + SL-LS 6N 2 +14N 1 7N 2 +21N +1 1 WL-LS 24N 2 +28N 1 28N 2 +42N +1 1 C-WL-LS 6N 2 +11N 8N 2 +14N +1 1 DCD-WL-LS N(8 +4N u +2M b ) 14N 2 - +N u 1 DCD-WL-LS N(14+4N u +2M b ) 8N 2 - (λ = 1 2 m ) +N u 1 V. THE DCD ALGOTHM The dichotomous coordinate descent (DCD) algorithm 5], 6] was proposed as a multiplication-free alternative to solve the least-squares (LS) problem equations 1]. The algorithm is implemented using sums bit-shift operations to avoid multiplications divisions see Table V, we present the DCD algorithm version considered in this paper. The cyclic DCD algorithm updates the solution vector ŵ in the cyclic order n = 1,...,N, which is used by the DCD- WL-LS algorithm to solve the normal equations. We assume that the elements of ŵ have amplitude such that ŵ p H ( H is a positive number) that we use M b bits to represent them. The constantα is the step-size of the DCD.N u is the maximum number of successful iterations is usually chosen as N u << N. With this approach, the algorithm updates the most significant bits first the less significant bits are updated later. Note that during the DCD operation, we have two kinds of iterations: the successful the unsuccessful iterations. The first kind occurs when the condition in step 3 is true, the updates in steps 4 6 are performed. Otherwise, the update is called unsuccessful. The computational complexity of the TABLE V DCD ALGOTHM Step Equation nitialization: ŵ = 2N 1, r = β α = H, q = for m = 1,...,M b 1 α = α/2 2 flag = for n = 1,...,2N 3 if r p > (α)/2) p,p then 4 ŵ p = ŵ p + sign(r p)α 5 r = r sign(r p)α (1:2N,1) 6 q = q +1 7 if q > N u, the algorithm stops 8 if flag = 1, then repeat step 2 DCD algorithm, presented in Table V, is a rom variable, which motivates the calculation of the worst case complexity. For the worst case, the complexity corresponds to P m = multiplications P a N(2N u + M b 1) + N u sums. ecall that this complexity is a pessimistic bound, since in general this upper bound is not reached. V. SMULATONS n this section, we compare our new DCD-WL-LS with the C-WL-LS algorithm. For this purpose, we consider the same identification system model used in 4], a non-circular signal is applied to justify the WL approach. We assume that the input signal is given by s(k) = 1 ǫ 2 s (k)+jǫs (k), s (k) s (k) are two real-valued uncorrelated complex-gaussian processes, with zero-mean variance σ 2 = σ2 = 1. We can choose ǫ between 1, the condition for circularity occurs when ǫ = 1/ 2. However, we use ǫ =.1 which corresponds to a non-circular process to perform our simulations. n figures 1 2 we assume that the optimum coefficients of the system to be identified are given by w 1 opt,i = β 1(1+cos(2π(i 3)/5) j(1+cos(2π(i 3)/1))], (25) for i = 1,2,...,5 β 1 =.432. n figure 3, in order to present the tracking performance of the DCD-WL-LS, we start the simulation with the optimum coefficients of eq. 25 after 2 iterations replace them by w 2 opt,i = β 2(1+cos(π(i 3)/5) j(1+cos(5π(i 3)/1))], (26) β 2 = β 1 /4. We assume that there is Gaussian noise η(k) added to the desired signal that the signal-to-noise ratio (SN) is 4dB. We obtain the desired signal with d(k) = e{w H opts(k)}+η(k). We define the C-WL-LS forgetting factor λ C = we initialize P C () = 1 4 1, for N = 5.

5 XXX SMPÓSO BASLEO DE TELECOMUNCAÇÕES - SBrT 12, DE SETEMBO DE 212, BASÍLA, DF For the DCD-WL-LS algorithm, we choose λ DCD equal to λ C, we defineh = 1. n our simulations, we compare the mean-square error (MSE) of the C-WL-LS the DCD-WL-LS algorithms in three situations: 1) varying the number of bits M b as 2, 4, 8 or 16 bits setting N u = 4; 2) setting M b = 16 bits choosing N u to be 1, 2, 4 or 8; 3) changing the system coefficients to verify the tracking performance of the DCD-WL-LS algorithm. We run 1 simulations to obtain the MSE as MSE = E{e 2 (k)}. Figures 1, 2 3 show the results. Note from figure 1 that for a fixed 1 C-WL-LS DCD-WL-LS(M b =2) DCD-WL-LS(M b =4) DCD-WL-LS(M b =8) DCD-WL-LS(M b =16) terations Fig. 1. MSE comparison between the C-WL-LS the DCD-WL-LS, using N u = 4 M b = 2,4,8,16, for non-circular input. 1 simulations were performed. 1 C-WL-LS DCD-WL-LS(N u=1) DCD-WL-LS(N u=2) DCD-WL-LS(N u=4) DCD-WL-LS(N u=8) terations Fig. 2. MSE comparison between the C-WL-LS the DCD-WL-LS, using M b = 16 N u = 1,2,4,8, for non-circular input.1 simulations were performed. N u, the precision of the DCD-WL-LS algorithm increases as the number of bits M b increases, when compared to the C- WL-LS implemented with Matlab precision. With this information, we can choose M b to guarantee an specific MSE of course bounded by the SN use the algorithm with the smaller number of bits necessary for a given implementation. n figure 2, we can see that for a fixed number of bits, when we increasen u, we accelerate the convergence. However, even for N u = 1, we note that the algorithm achieves a steady-state MSE similar to the C-WL-LS MSE, though 4 times slower than the C-WL-LS algorithm. n fact, when we apply the DCD to solve the normal equations, we reduce the complexity to O(N), but we pay for this reduction with the need of some more iterations to achieve the steady-state MSE. Since we can control the additional number of iterations by choosing a convenient N u, this approach can be very attractive for lowcost implementations. 1 C-WL-LS DCD-WL-LS(N u=1) DCD-WL-LS(N u=2) DCD-WL-LS(N u=4) DCD-WL-LS(N u=8) terations Fig. 3. Tracking performance of the C-WL-LS the DCD-WL-LS algorithms, when the system coefficients change. Non-circular input is used for a fixed M b = 16 N u = 1,2,4,8. 1 simulations were performed. n figure 3, we show the DCD-WL-LS algorithm subject to a modification of the system coefficients, which is used to verify the tracking performance. Note that for the different values of N u proposed, the algorithm presents approximately the same performance when the system changes. As shown in 7], the tracking performance of the DCD-WL-LS does not depend as much on N u. V. CONCLUSONS n this paper, we propose a widely-linear LS algorithm that is numerically stable (since it avoids the propagation of the inverse of the autocovariance matrix) has a computational complexity that is linear on the filter length. Our simulations showed that the modification proposed slows down the initial convergence of the algorithm, but with a smart choice of the DCD parameters, the algorithm quickly recovers the performance achieves the same steady-state MSE achieved by the C-WL-LS. The tracking performance of the DCD-WL- LS does not depend as much on N u. An analytical study of the DCD-WL-LS is still under research, since traditional tools of the LS analysis can not be applied here. EFEENCES 1] A.H. Sayed, Adaptive filters, Wiley-EEE Press, 28. 2] Jae-Jin Jeon, LS adaptation of widely linear minimum output energy algorithm for DS-CDMA systems, in Proc. of the Advanced ndustrial Conference on Telecommunications/Service Assurance with Partial ntermittent esources Conference/E-Learning on Telecommunications Workshop, July 25, pp ] B. Picinbono P. Chevalier, Widely linear estimation with complex data, EEE Transaction on Signal Processing, vol. 43, no. 8, pp , Aug ] F.G.A. Neto, V.H. Nascimento, M.T.M. Silva, educed-complexity widely linear adaptive estimation, in 7th nternational Symposium on Wireless Communication Systems, Sep. 21, pp ] Y.V. Zakharov T.C. Tozer, Multiplication-free iterative algorithm for LS problem, Electronics Letters, vol. 4, no. 9, pp , April 24. 6] Y. Zakharov, G. White, Jie Liu, Fast LS algorithm using dichotomous coordinate descent iterations, in Signals, Systems Computers. ACSSC 27. Conference ecord of the Forty-First Asilomar Conference on, Nov. 27, pp ] Y.V. Zakharov, G.P. White, Jie Liu, Low-complexity LS algorithms using dichotomous coordinate descent iterations, EEE Transaction on Signal Processing, vol. 56, no. 7, pp , July 28. 8]. A. Horn C.. Johnson, Topics in Matrix Analysis, Cambridge University Press, Cambridge, MA, 1991.