Compressed-Sensing Recovery of Images and Video Using Multihypothesis Predictions

Transcription

1 Using Compressed-Sensing Recovery of Video Using Multihypothesis Chen Eric W. Tramel, and James E. Department of Electrical & Computer Engineering Geosystems Research Institute Mississippi State University, MS USA November 2011

2 CS Overview Using Compressed Sensing (CS) For x R N and a set of linear projections, Φ R M N where M N; i.e., y = Φx x can be reconstructed from y by solving solving min x x 1 s.t. Φx = y Complications For N large and dense Φ: Explicit storage of Φ becomes impractical High computational complexity for reconstruction

3 Block Compressed Sensing Using Block Compressed Sensing (BCS) Image partitioned into small blocks (B B) y j = Φ B x j Φ B : M B B 2, x j : block j of image Smooth Projected Landweber (SPL) BCS-SPL (Gan DSP2007, Mun & ICIP2009) Iterative-thresholding reconstruction Wiener filter to smooth blocky artifacts Simple, fast

4 Multiscale BCS-SPL Using Multiscale BCS-SPL MS-BCS-SPL ( et al. EUSIPCO2011) Assumes wavelet-domain sampling with blocks; i.e., BCS is deployed within each subband of DWT Different subrate at each decomposition level Reconstruction uses similar SPL procedure

5 Residual Reconstruction Using Residual Reconstruction (RR) Prediction: x Residual: r = x x Residual projection: q = Φr = Φ(x x) = y Φ x Residual r tends to be more compressible than x Reconstruction: ˆx = x +Reconstruct(q, Φ) where Reconstruct( ) is some suitable CS recovery

6 Residual Reconstruction Using The Prediction Problem We want to find x so that x x; i.e., x = arg min p P(x ref ) x p 2 2 but:x is unknown at reconstruction Use some initial reconstruction ˆx instead: x = arg min p P(x ref ) ˆx p 2 2 Alternate approach cast problem into measurement domain: x = arg min p P(x ref ) y Φp 2 2

7 Prediction for Video Using for Video for Images Video-Frame Reconstruction Form prediction of each frame in a video sequence Block-based sampling in BCS permits ME/MC on blocks in measurement domain Multi-pass recovery: use recovered key frames as references for multi-hypothesis () prediction of subsequent frames

8 Prediction for Video Using for Video for Images Linear Combination of Hypotheses Prediction for block x t,i is linear combination of hypothesis blocks, H t,i : x t,i = H t,i w Estimating w : an ill-posed problem Distance-Weighted l 2 (Tikhonov) Regularization ŵ = argmin w y t,i ΦH t,i w λ Γw 2 2 l 1 -Regularization (Do et al. ICIP2009, Prades-Nebot et al. PCS2009) ŵ = argmin w y t,i ΦH t,i w λ w 1

9 Prediction for Video Using for Video for Images Tikhonov Regularization Distance Weighting Bias regularization toward solutions with projections close to measurements: y t,i Φh Γ =... 0 y t,i Φh K 2 Experimental results show distance-weighted l 2 regularization yields better predictions than l 1 regularization

10 -BCS-SPL for Images Using for Video for Images Subblock-Based Prediction (a) current subblock hypothesis subblock search window S1 S3 B S2 S4 (b) b S1 S2 S3 S4 0 0 zero-padding S1 0 0 S2 0 0 B B blocks divided into b b subblocks Minimum subblock size, b = 1 2 B Maximum subblock size, b = B Multiple hypotheses drawn from spatial surrounding area of a subblock 0 0 S3 0 0 found for subblocks zero padded to block size 0 0 S4 (c)

11 -BCS-SPL for Images Using for Video for Images Algorithm 1: INITIALIZATION: B = 32, b = 16, w = 32 2: x = BCS-SPL(y, Φ,Ψ,B) 3: repeat 4: x i = Prediction( x,y, Φ,b,w,B) 5: r = y Φ x i 6: ˆr = BCS-SPL(r, Φ, Ψ,B) 7: ˆx i = x i + ˆr 8: if update criterion satisfied then 9: b b 2, w w 2 10: end if 11: Update x ˆx i 12: i = i +1 13: until stopping criterion satisfied x: initial reconstruction Φ: measurement operator, Ψ: sparsity transform

12 -MS-BCS-SPL Using for Video for Images -MS-BCS-SPL Combines MS-BCS-SPL with predictions carried out in wavelet domain Block size B l = [16,32,64] Initial subblock size b l = 1 8 B l

13 -MS-BCS-SPL Using for Video for Images Algorithm 1: repeat 2: ẋ i = Ω x 3: for 1 l L do 4: for each subband θ {H,V,D} do 5: ẋ i(θ) = Prediction(ẋ i(θ),y(θ), Φ,B l,b l,w) 6: end for 7: end for 8: x i = Ω 1 ẋ i 9: ˆx i = x i +BCS-SPL(y Φ x i, Φ,Ψ, {B l}) 10: if update criterion satisfied then 11: for 1 l L do 12: b l b l 2 13: end for 14: w w 2 15: end if 16: Update x ˆx i 17: until stopping criterion satisfied

14 Video Frame Recovery Performance Bidirectional Recovery of Foreman Frame Using for Video for Images Recovery PSNR(dB) Subrate(M/N) RR w/-tik RR w/-gpsr BCS-SPL

15 Video Frame Recovery Performance Bidirectional Recovery of Football Frame Using for Video for Images Recovery PSNR(dB) Subrate(M/N) RR w/-tik RR w/-gpsr BCS-SPL

16 Image Recovery Performance, Lenna Using for Video for Images Recovery PSNR(dB) Subrate(M/N) -MS MS BCS-SPL TV

17 Image Recovery Performance, Barbara Using for Video for Images Recovery PSNR(dB) Subrate(M/N) -MS MS BCS-SPL TV

18 Visual Comparison Barbara Using for Video for Images

19 Visual Comparison Barbara Recovery Inset, S = 0.1 Using for Video for Images BCS-SPL TV MS-BCS-SPL

20 Visual Comparison Barbara Inset, S = 0.1 Using for Video for Images MS-GPSR -MS-BCS-SPL -BCS-SPL

21 for Images Using Runtime Comparison for Video for Images

22 Using predictions use a distance-weighted Tikhonov regularization to find the best linear combination of hypotheses Multiple predictions were used to create a measurement domain residual of the signal to be recovered, which is generally more compressible than the original signal predictions work both for video and still images and the performance improvement is significant MATLAB code fowler/bcsspl/