On Sparse Bayesian Learning (SBL) and Iterative Adaptive Approach (IAA)

Transcription

1 On Sparse Bayesian Learning (SBL) and Iterative Adaptive Approach (IAA) Jian Li and Xing Tan Dept. of Electrical and Computer Eng. University of Florida Gainesville, Florida

2 Outline Sparse Signal Recovery Algorithms Benchmark Sparse Bayesian Learning (SBL-0) Iterative Adaptive Approach (IAA) The SBL-α algorithm Probing Further 2

3 Sparsity Based Approaches l 1 -Norm Based Optimization Methods: LASSO, BP, LARS, Dantzig Selector, DASSO, L1-SVD User/tuning parameter, hard to choose Performance sensitive to user parameter Iterative Weiner Filtering Based Approaches: FOCUSS, M-FOCUSS, DPC, AP-DPC Two user parameters, hard to choose (varying with iterations) Performance sensitive to user parameters Sparse Bayesian Learning (SBL) SBL, M-SBL EM based SBL takes a long time to converge Type-II ML base SBL trades performance for faster convergence Easy to use, as found by practitioners SAL, Dept. of ECE 3

4 Benchmark: SBL-0 4

5 Sparse Bayesian Learning Sparse Bayesian Learning (Tipping 01): Originally proposed in machine learning literature Use Bayesian model with inverse gamma priors for and Recommend Estimate by a type-ii ML and an EM algorithm 5

6 Widely Used SBL SBL for Basis selection (Wipf et al. 04): Uses the same Bayesian model but with flat priors for and Estimate by an EM approach Equivalent to Tipping s EM with his recommended priors ( ) Bayesian Compressive Sensing (Ji et al. 08): Applies Type-II ML based SBL to compressive sensing Show that it is more accurate than l 1 -Norm Based Optimization Methods The widely used SBL is easy to use, not sensitive to initial conditions. 6 6

7 A SPARCO Problem SPARCO Problem 902 (Scientific Computing Laboratory, UBC) is an irregular sampling matrix and is a DCT transform matrix (A = MB). K = 200, M = 40, K = 3, SNR = 10 db. 0 7

8 SPARCO Example Type-II ML based SBL is 0 faster (10 times) than -5 SBL-0 type-ii ML EM based SBL (called SBL-0 hereafter). But this comes at the Re econstruction n error (db) cost of performance -25 (reconstruction error) degradation. SBL-0 used as benchmark hereafter SNR (db) 8

9 IAA 9

10 Passive Array Processing Data Model measurements from M sensors steering matrix signal waveform noise no. of snapshots Estimate signal waveforms locations 10

11 Iterative Adaptive Approach (IAA) IAA minimizes the weighted least squares cost function where IAA signal waveform estimate is diagonal Q is obtained from the previous estimates IAA is iterative 11

12 IAA IAA waveform estimate can be rewritten as Requires only one matrix inversion per iteration Amenable to parallel implementation IAA Merits Nonparametric User parameter-free Quadratic convergence Works with Uncorrelated/Coherent sources Few (even single) snapshots Arrays with arbitrary geometries 12

13 IAA illustration 13

14 IAA illustration 14

15 SPARCO Example IAA can be viewed as an approximation to the widely used SBL (SBL-0). Reco onstruction error (db) Their performances (reconstruction ti errors) are -15 similar. IAA converges faster (10 times) and hence computationally more SBL-0 IAA efficient SNR (db) 15

16 Achieving Sparsity with IAA Sparse IAA estimates are obtained with the BIC (Bayesian information criterion) Minimize no. of peaks kept IAA with BIC: IAA waveform estimates η of them Penalty term Peaks of IAA that minimize BIC are picked as sources Provides sparsity by estimating the number of sources 16

17 SPARCO Example IAA with BIC gives lower reconstruction error than the widely used SBL (SBL-0). The BIC part of the computation is negligible. construction n error (db) Re SBL-0 IAA+BIC SNR (db) 17

18 IAA with RELAX RELAX A parametric cyclic approach Requires number of sources to be known Iteratively re-determine the parameters by subtracting the estimated ones from the data IAA with BIC estimates can be used to initialize RELAX Helps improve IAA with BIC results even further Works with off-grid sources 18

19 Array Processing Example average delayand-sum Two uncorrelated sources at 82 and 90 degrees SNR = 40 db, 50 independent trials no. of snapshots, N = 2 19

20 Active Sensing Applications Active Sensing Radar/sonar range-doppler (intra-pulse Doppler) imaging 20

21 21

22 22

23 Range-Doppler Imaging Example min. SNR = 5 db 23

24 MIMO STAP Example (a) Conventional SIMO; (b) MIMO-S, with switching (c) MIMO-RS, with random switching; (d) MIMO SAL, Dept. of ECE 24

25 IAA for MIMO STAP MIMO-RS SAL, Dept. of ECE 25

26 SIMO Angle-Doppler Imaging in MIMO STAP MIMO-RS MIMO-S MIMO SAL, Dept. of ECE No steering vector error; no jamming. 26

27 SBL-αα 27

28 SBL-α algorithm The same hierarchical Bayesian model But with the priors: the a prior ri pdf f(p ) n α=0.5 α=1 α=1.5 α=2 Equivalent to widely used 50 SBL if we set Increasing will increase sparsity in the estimate of p n EM based 28

29 When the true signal is sparse, SBL-1 can perform much better than existing methods. SBL-1 performs SPARCO Example Re econstructio on Error (db B) better and faster IAA IAA+BIC type-ii ML SBL-0 SBL-1 (5 times) than SBL SNR (db) SBL-1 is sensitive to initial condition, so IAA is used to initialize SBL-1. 29

30 A SAR Imaging Example Backhoe DAS IAA SBL-1 30

31 Probing Further Based on Tipping s Bayesian model: A Monte-Carlo Expectation Maximization (MCEM) algorithm can devised to automatically estimate the hyper parameters a and b. MCEM can give sparser and more accurate results than SBL-0. A belief propagation based sparse Bayesian learning approach can be developed to solve large-scale compressive sensing problems efficiently, in O(N log N), when A is sparse. 31

32 THANK YOU! SAL, Dept. of ECE 32