Signal Processing with Side Information A Geometric Approach via Sparsity João F. C. Mota Heriot-Watt University, Edinburgh, UK
Side Information Signal processing tasks Denoising Reconstruction Demixing (source separation) Compression Inpainting, super-resolution, prior information multi-modal heterogeneous Recommender systems Medical imaging Consumer electronics Robotics MRI PET How to represent multi-modal or heterogeneous data? How to process it? 2/23
Outline Compressed Sensing with Prior Information Application: Video Background Subtraction N Deligiannis VUB-Belgium M Rodrigues UCL X-ray Image Separation Conclusions 3/23
Compressed Sensing Sucess rate (50 trials) Compressed Sensing (CS) sparse Our bound CS bound iid Gaussian CS + PI CS performance Basis pursuit number of measurements What if we know? prior information How do we integrate in the problem? Reconstruction guarantees? medical images, video, 4/23
Intuition measurements random orientation solutions of Tangent cone of at Our approach prior information (PI) model for PI small 5/23
Good components Bad components 6/23
L1-L1 minimization sparse i.i.d. Theorem (BP) [Chandrasekaran, Recht, Parrilo, Willsky, 2012] Theorem (L1-L1 minimization) [M, Deligiannis, Rodrigues, 2017] parameter-free support overestimation 7/23
Experimental Results Gaussian Sucess rate (50 trials) L1-L1 Mod-CS L1-L2 Mod-CS BP BP bound [Vaswani and Lu, 2010] number of measurements 8/23
Summarizing Prior Information can help, but can also hinder L1-L1 works better than L1-L2 (theory and practice) (Computable) bounds are tight for L1-L1, but not for L1-L2 Theory predicts optimal ; indicates how to improve Limitations: Gaussian matrices; bounds depend on unknown parameters 9/23
Outline Compressed Sensing with Prior Information Application: Video Background Subtraction N Deligiannis VUB-Belgium M Rodrigues UCL X-ray Image Separation Conclusions A Sankaranarayanan CMU-USA V Cevher EPFL-CH 10/23
Compressive Background Subtraction linear operation CS camera observed How to recover from online? How many measurements from frame? 11/23
Compressive sensing for background subtraction [Cevher, Sankaranarayanan, Duarte, et al, 2008] foreground sparse background Basis Pursuit Assumption: background is static & known fg measurements Problems Prior frames are ignored fixed; depends on foreground area Our Approach Estimate from past frames: Integrate into BP via minimization 12/23
Problem Statement Model sparse arbitrary function sparse time measurements Problem Compute a minimal # of measurements Reconstruct perfectly online algorithm w/ adaptive rate 13/23
Algorithm : computed at iteration Acquire Set with Estimate L1-L1 minimization Gaussian parameters of # measurements of oversampling factor and repeat... 14/23
Estimating a Frame estimation extrapolation linear motion overlap: take average; gaps: fill w/ average of neighbors state-of-the-art in video coding 15/23
Experimental Results 280 frames Number of measurements Ours CS oracle modified CS (nonadaptive) prior state-of-the-art [Warnell et al, 2014] reduction of 67% L1-L1 oracle Frame 16/23
Experimental Results 280 frames Relative error estimation modified CS (nonadaptive) reconstruction determined by solver Frame 17/23
Outline Compressed Sensing with Prior Information Application: Video Background Subtraction N Deligiannis VUB-Belgium M Rodrigues UCL X-ray Image Separation Conclusions B Cornelis VUB-Belgium I Daubechies Duke-USA 18/23
Motivation: X-Ray of Ghent Altarpiece Mixed X-Ray Can we use the visual images to separate the x-rays? 19/23
Approach: Coupled Dictionary Learning Training step coupling X-Ray Visible w/ sparse columns learn dictionaries by alternating minimization Demixing step mixed x-ray visual front visual back 20/23
Results reconstructed x-rays MCA [Bobin et al, 07 ] mixed x-ray multiscale MCA w/ksvd Ours visuals in grayscale 21/23
Summary / Conclusions data dictionaries sparse columns X-ray separation sparse sparse prior information Reconstruction w/ PI measurements observations Low-rank model multi-modal features Better models? Guarantees? Scalable algorithms? Applications medical imaging (MRI + PET + ECG) super-resolution (depth + visual) SAR + microwave imaging robotics (laser + sonar) 22/23
References J. F. C. Mota, N. Deligiannis, M. R. D. Rodrigues Compressed Sensing with Prior Information: Optimal Strategies, Geometries, and Bounds IEEE Transactions on Information Theory, Vol 63, No 7, 2017 J. F. C. Mota, N. Deligiannis, A. C. Sankaranarayanan, V. Cevher, M. R. D. Rodrigues Adaptive-Rate Reconstruction of Time-Varying Signals with Application in Compressive Foreground Extraction IEEE Transactions on Signal Processing, Vol 64, No 14, 2016 N. Deligiannis, J. F. C. Mota, B. Cornelis, M. R. D. Rodrigues, I. Daubechies Multi-Modal Dictionary Learning For Image Separation With Application in Art Investigation IEEE Transactions on Image Processing, Vol 26, No 2, 2017 23/23