Adaptive System Identification and Signal Processing Algorithms

Transcription

1 Adaptive System Identification and Signal Processing Algorithms edited by N. Kalouptsidis University of Athens S. Theodoridis University of Patras Prentice Hall New York London Toronto Sydney Tokyo Singapore

2 List of contributors Preface xiii xv 1 Introduction 1 N. Kalouptsidis and S. Theodoridis 2 Basic Concepts and Algorithmic Schemes 7 N. Kalouptsidis and S. Theodoridis 2.1 Introduction System identification Signal processing Efficiency issues in adaptive algorithms Linear regression Steady-state performance of the LR estimate and the Wiener filter State-space forms of the LR estimate and the RLS algorithm Approximate realizations of the LR estimate and the LMS algorithm The instrumental variable method General prediction models The general LS optimization problem and the prediction error method Predictors in state-space form and the Kaiman filter Fast algorithms for Wiener filtering The Levinson algorithm The split Levinson algorithm The lattice family The Schur algorithm Fast algorithms for linear regression Fast RLS algorithms Square root QR and lattice algorithms The multichannel case 53 References 55

3 vi Contents General Structure of Adaptive Algorithms: Adaptation and Tracking 58 Lennart Ljung 3.1 Introduction Optimal algorithms for tracking drifting parameters A basic signal model A signal model with global and local trends Some ad hoc algorithms for tracking drifting parameters The RLS algorithm The LMS algorithm Estimating the unknown covariances Algorithms for tracking abruptly changing parameters Formulation Detection algorithms ML-type algorithms Algorithms for general non-linear regressions Asymptotic properties of the decreasing gain case Tracking ability of the algorithms A useful lemma The tracking error for M-dependent regressor sequences The tracking error for mixing regressor sequences 3.11 Evaluation of the error in the frequency domain 3.12 Conclusions References The Least Mean Square Family 84 William A. Sethares 4.1 Introduction LMS and its children Normalized LMS Leakage Dead zone Signed-error LMS Signed-regressor LMS Sign-sign LMS Quantized state LMS Least mean fourth Median LMS Expected behaviour approach The deterministic approach Analytical background Persistence of excitation The stochastic approximation approach Theoretical development 106

4 vii 4.6 Examples, comparisons and discussion LMS Normalized LMS Leakage Dead zone Signed error Signed regressor Sign-sign Quantized state Least mean fourth Median LMS Convergence and tracking of LMS and variants Conclusion 119 References 120 Fast Transversal RLS Algorithms 123 Dirk Т. М. Slock and Thomas Kailath 5.1 Introduction Adaptive filtering algorithms The LMS algorithm The RLS algorithm Comparison of the tracking performance of LMS and RLS Comparison of various RLS algorithms Windowing issues in fast LS algorithms Recursive least squares (RLS) Prewindowed FTRLS algorithm derivation, definition and interpretation of algorithmic quantities Vector space formulation and projection updating Algorithm derivation (steady state) Initialization and restarting Computational redundancies and numerical stabilization Hyperbolic rotations and numerical instability An approach for the analysis of roundoff errors Introducing redundancies in the FTRLS algorithm Compact representation of the FTRLS algorithm Averaging analysis of the error propagation The limited range for Я and a possible way out Wordlength considerations Divisions Growing-and sliding-window covariance FTRLS algorithms The growing-window covariance (GWC) FTRLS algorithm The sliding-window covariance (SWC) FTRLS algorithm Modular (circular) multichannel and/or muitiexperiment FTRLS algorithms 174

5 viii Contents The multichannel RLS problem The multiexperiment RLS problem The multichannel multiexperiment RLS problem The modular multichannel multiexperiment FTRLS algorithm Triangular factorization 182 Notes 184 References Lattice algorithms 191 Fuyun Ling 6.1 The statistical view of order recursive estimation and the lattice structure MMSE estimation and its geometric interpretation Order recursive MMSE estimation Lattice structure for MMSE estimation Properties of the MMSE lattice estimator Time-and-order recursive LS estimation and the basic LS lattice algorithm Basic relations of ORLS estimation Basic LS lattice algorithm with prewindowed data Variations and extensions of the basic LS lattice algorithm Variations of basic LS lattice algorithm with prewindowed data Gradient lattice algorithms Multichannel LS lattice algorithm Other ORLS estimation algorithms Convergence and numerical properties of lattice algorithms Initial convergence of LS lattice algorithms Tracking behaviour of lattice algorithms Numerical stability and accuracy of lattice algorithms Summary and conclusions 256 References The QR family 260 /. C. McWhirter and I. K. Proudler 7.1 Introduction Narrowband beamforming QR decomposition Givens rotations Parallel implementation Square-root-free version Direct residual extraction Weight freezing and flushing 274

6 ix Parallel weight extraction Comparison with recursive modified Gram-Schmidt algorithms Comparison with Kaiman filter algorithms Adaptive FIR filtering The QRD approach Forward linear prediction Backward linear prediction The QRD least-squares lattice algorithm The 'fast QRD' algorithm Physical interpretation of fast algorithm parameters Weight extraction from fast algorithms Computer simulations Wideband beamforming Multichannel adaptive filters Multichannel lattice Multichannel fast QRD algorithm Algorithm listings 318 Notes 319 References 320 Spectral analysis Theodoridis and N. Kalouptsidis 8.1 Introduction Basic guidelines from the theory of spectral analysis of stochastic processes Parametric models and autoregressive spectral estimation AR spectral estimation based on observation data Schemes based on the Toeplitz structure of the autocorrelation matrix Least-squares method The forward backward LS method Adaptive techniques ARMA spectral analysis: basic directions Modified Prony's technique for sinusoidal modelling 374 Appendix A 376 Appendix В 380 References 385 Channel equalization 388 C.F.N. Cowan 9.1 Introduction 9.2 Linear FIR equalizers

7 9.3 Adaptive algorithm performance Subsymbol-spaced equalizers Decision feedback equalizers Complex equalizers Non-linear equalizers Conclusions 405 References 405 Echo cancellation 407 Fuyun Ling 10.1 Echoes in telephone networks and their cancellation Echoes in telephone networks and their impact on voice and data transmission Voice echo control and cancellation Modem data echo cancellation Digital subscriber-loop echo cancellation Data-driven Nyquist echo cancellers and their converging and tracking characteristics Structures of modem data echo cancellers Convergence characteristics of LMS Nyquist echo cancellers Steady-state excess MSE and selection of step sizes Comparison of analytic and real passband echo cancellers Finite wordlength effects in echo cancellation How finite wordlength affects echo canceller operation Gradient averaging algorithm for echo canceller implementation Related topics and references Far echo frequency offset compensation Analog and digital sampling rate conversion Fast echo canceller training Topics related to ISDN echo cancellation Topics related to voice telephone echo cancellation Concluding remarks 462 Notes 463 References 463 Interference rejection and channel estimation for spread-spectrum communications 466 Ronald A. litis 11.1 Definition of spread-spectrum communications Introduction Spread-spectrum signal models 467

8 xi 11.2 Interference rejection using the Wiener filter and the LMS algorithm Interference rejection in a direct-sequence receiver Performance and analysis of the optimum interference rejection filter An adaptive DS receiver using the LMS algorithm Statistics of the misadjustment filter Interference rejection in frequency-hopped spread-spectrum systems FFH receiver BER analysis Joint channel estimation and interference rejection using the RLS algorithm Multipath channel and interferer model Joint estimation of channel and interferer parameters Optimum prewhitening filter and composite channel estimates BER analysis of the RLS-based receiver Joint estimation of PN code delay, multipath and interference using the extended Kaiman filter Parameterization of the interferer, channel and code delay Extended Kaiman filter for real parameters and complex measurements EKF interference, multipath and delay estimator Digital RAKE receiver and BER analysis Summary 509 References 510 Neural networks for adaptive signal processing 512 Simon Haykin and Andrew Ukrainec 12.1 Introduction Simplified model of a neuron Why neural networks for adaptive signal processing? Classification of neural networks Back-propagation networks Complex back-error propagation network The complex back-error propagation algorithm Special case of back-error propagation algorithm: real parameters Non-linear prediction example Issues in learning Radial basis function networks Some preliminaries 540

9 xii Contents Unsupervised learning of hidden layer parameters Extended metric clustering Chaotic time series prediction problem Concluding remarks 547 Notes 549 References 549 Index 554