Diffusion Spline Adaptive Filtering

Transcription

1 24 th European Signal Processing Conference (EUSIPCO 16) Diffusion Spline Adaptive Filtering Authors: Simone Scardapane, Michele Scarpiniti, Danilo Comminiello and Aurelio Uncini

2 Contents Introduction Overview State of the art Theoretical background Spline adaptive filter Proposed diffusion SAF Network setting Derivation of the algorithm Experimental validation Setup Results Conclusions Conclusive remarks

3 Content at a glance Setting: L agents in a network received a streaming flux of data, wishing to infer a common predictive function with local communication. Problem: Standard diffusion LMS (D-LMS) is simple and efficient, however it may perform poorly when the underlying model is nonlinear. Objective: To design a novel nonlinear diffusion filter having similar computational complexity as the standard D-LMS.

4 Current approaches and possible shortcomings 1. Fixed nonlinear expansion [1]: communication overhead might be too significant. 2. Distributed kernel filtering [2]: resulting model is formulated in terms of data from all agents. 3. Distributed machine learning/optimization [3]: might not be adequate to a streaming setting with low computational power available locally. [1] Scardapane, S., Wang, D., Panella, M. and Uncini, A., Distributed learning for random vector functional-link networks. Inform. Sciences, 301, pp [2] Predd, J.B., Kulkarni, S.B. and Poor, H.V., Distributed learning in wireless sensor networks. IEEE Signal Process. Mag., 23(4), pp [3] Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein, J., Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. and Trends in Machine Learning, 3(1), pp

5 Contents Introduction Overview State of the art Theoretical background Spline adaptive filter Proposed diffusion SAF Network setting Derivation of the algorithm Experimental validation Setup Results Conclusions Conclusive remarks

6 Data model Denote by x n a buffer of the last M input samples. Assume that an unknown Wiener model is generating the desired response as follows: d[n] = f 0 ( w T 0 x n ) + ν[n], (1) A spline adaptive filter (SAF) computes the output similarly [1, 2], i.e. first a linear filtering operation: s[n] = w T nx n. (2) Then, the final output is computed via spline interpolation over s[n]. [1] Scarpiniti, M., Comminiello, D., Parisi, R. and Uncini, A., Nonlinear spline adaptive filtering. Signal Process., 93(4), pp [2] Scarpiniti, M., Comminiello, D., Scarano, G., Parisi, R. and Uncini, A., Steady-State Performance of Spline Adaptive Filters. IEEE Trans. on Signal Process., 64(4), pp

7 Visualization of the SAF () s Control points i.c. i ( u) q x,0 q q q q x, i x, i 1 x, i 2 x, i 3 q xq, s i th tract

8 Spline definition The spline is defined by a set of Q control points (called knots), denoted as Q i = [ q x,i q y,i ]. We suppose that the knots are uniformly distributed, i.e. q x,i+1 = q x,i + x, for a fixed x R. We also constrain the knots to be symmetrically spaced around the origin. The output is obtained by an interpolating polynomial of order P, passing by the closest knot to s[n] and its P successive knots.

9 Spline equations Given the index i of the closest knot, we define the normalized abscissa value between q x,i and q x,i+1 as: u = s[n] x From u we can compute the reference values: s[n]. (3) x u = [ u P u P 1... u 1 ] T (4) q i,n = [ ] T q y,i q y,i+1... q y,i+p (5) The output of the filter then given by: y[n] = ϕ(s[n]) = u T Bq i,n, (6) where B R (P+1) (P+1) is called the spline basis matrix.

10 Contents Introduction Overview State of the art Theoretical background Spline adaptive filter Proposed diffusion SAF Network setting Derivation of the algorithm Experimental validation Setup Results Conclusions Conclusive remarks

11 Network model The connectivity of the network is represented as a real-valued matrix C R L L, where entry C kl > 0 if nodes k and l are connected, 0 otherwise. The symbol N k will denote the inclusive neighborhood of node k. We require the mixing coefficients to define a convex combination for every node: C kl 0 and L C kl = 1 k, l = 1,..., L. (7) l=1 At a generic time ( instant n, each ) agent receives some input/output data denoted by, d (k) [n] x (k) n We assume that streaming data at the local level is generated according to: ( ) d (k) [n] = f 0 w T 0 x (k) + ν (k) [n]. (8) n

12 Visualization of the setting N k 1 k 2 3 ( k ) x n ( k ) w n s ( k ) [ n ] ( k y ) [ n ] ( k ) q in,

13 Algorithm (1) The network objective is to minimize: J glob (w, q) = L k=1 L J (k) loc (w, q) = k=1 { E e (k) [n] 2}, (9) In the diffusion SAF (D-SAF), we first diffuse the linear coefficients: ψ (k) n = C kl w (l) n. (10) l N k w-diffusion Then, nodes perform a second diffusion step over their nonlinear coefficients: ξ (k) i,n = C kl q (l) i,n. (11) l N k q-diffusion This step requires combination only of q (k) i,n, hence its complexity is defined only by the spline order P.

14 Algorithm (2) The spline output given the new span is obtained as: y (k) [n] = ϕ k (s (k) [n]) = u T Bξ (k) i,n. (12) From this, the local error is given as e (k) [n] = d (k) [n] y (k) [n]. The two gradient descent steps are given by: w (k) n+1 = ψ(k) n + µ (k) w e (k) [n]ϕ (s (k) [n])x (k) n, (13) w-adapt q (k) i,n+1 = ξ(k) i,n + µ(k) k e (k) [n]b T u. (14) q-adapt Note that D-LMS is a special case of the D-SAF, where each node initializes its nonlinearity as the identity, and µ (k) q = 0, k = 1,..., L.

15 Contents Introduction Overview State of the art Theoretical background Spline adaptive filter Proposed diffusion SAF Network setting Derivation of the algorithm Experimental validation Setup Results Conclusions Conclusive remarks

16 Experimental setup We set L = 10 agents with random connectivity. Weights w 0 are extracted from a normal distribution. Local input signal is generated according to: x k [n] = a k x k [n 1] + 1 a 2 kɛ[n], (15) Local correlation coefficients, noise variances and local step-sizes are chosen randomly in some given intervals. Experiments are repeated 15 times, by keeping fixed the topology of the network and the optimal parameters of the system. MATLAB code is available under open-source license

17 MSE evolution NC-LMS D-LMS NC-SAF D-SAF MSE Sample 10 4 Figure 1 : MSE evolution, averaged across the nodes.

18 MSD evolution D-SAF Node 1 Node 6 0 D-SAF Node 1 Node 6 MSD (linear part) Node 8 MSD (non-linear part) Node Sample 10 4 (a) Linear MSD Sample 10 4 (b) nonlinear MSD Figure 2 : MSD evolution for D-SAF and 3 representative nodes running NC-SAF.

19 Final nonlinearity 2 2 Real NC-SAF (3 nodes) Real D-SAF 1 1 AF output y[n ] 0 AF output y[n ] Linear combiner output s [n ] Linear combiner output s [n ] Figure 3 : Final estimation of the nonlinear model. (a) Three representative nodes running NC-SAF. (b) Final spline of the nodes running D-SAF.

20 Contents Introduction Overview State of the art Theoretical background Spline adaptive filter Proposed diffusion SAF Network setting Derivation of the algorithm Experimental validation Setup Results Conclusions Conclusive remarks

21 Conclusive remarks We have introduced a distributed algorithm for adapting SAFs. The algorithm allows for a flexible nonlinear estimation, with a small increase in computational complexity. In particular, the algorithm requires in average only twice as much computations as the standard D-LMS. Apart from a theoretical analysis, we plan to extend the algorithm to the case of second-order adaptation with Hessian information, ATC combiners, and asynchronous networks. Additionally, we plan on investigating diffusion protocols for more general architectures, including Hammerstein spline filters.

22 Questions?