Compressive Single Pixel Imaging Andrew Thompson University of Edinburgh. 2 nd IMA Conference on Mathematics in Defence

Transcription

1 Compressive Single Piel Imaging Andrew Thompson University of Edinburgh 2 nd IMA Conference on Mathematics in Defence

2 About the project Collaboration between the University of Edinburgh and SELEX Galileo Ltd. Jared Tanner and Coralia Cartis (UoE/E CoS) David Humphreys and Robert Lamb (SELEX)

3 About the project Collaboration between the University of Edinburgh and SELEX Galileo Ltd. Jared Tanner and Coralia Cartis (UoE/E CoS) David Humphreys and Robert Lamb (SELEX) Partly funded by the Underpinning Defence Mathematics (UDM) programme of the MoD Facilitated by the Knowledge Transfer Network

4 Outline of presentation Background The Rice single piel camera design A mathemacal model Compressed Sensing

5 Outline of presentation Background The Rice single piel camera design A mathemacal model Compressed Sensing Research undertaken Reconstruction algorithms investigated Results of numerical tests Etension to 3D dynamic images

6 Outline of presentation Background The Rice single piel camera design A mathemacal model Compressed Sensing Research undertaken Reconstruction algorithms investigated Results of numerical tests Etension to 3D dynamic images Design issues and key findings

7 The Compressed Sensing Single Piel Camera (Rice University, )

8 A sampling model Model light incident on DMD as an N piel image Represent as a vector of dimension N

9 A sampling model , , y i i Model light incident on DMD as an N piel image Represent as a vector of dimension N Model each measurement as an inner product with a random basis vector

10 A sampling model , , y i i Model light incident on DMD as an N piel image Represent as a vector of dimension N Model each measurement as an inner product with a random basis vector Model photon counting noise and quantization error as Gaussian noise Take n < N samples e y T n T T 2 1 e

11 The Compressed Sensing paradigm CS: We can recover signals from undersampling if we sample incoherently (e.g. randomly) the signal has low information content

12 The Compressed Sensing paradigm CS: We can recover signals from undersampling if we sample incoherently (e.g. randomly) the signal has low information content Images are often compressible in some transform domain: information can be captured in relatively few coefficients

13 The Compressed Sensing paradigm CS: We can recover signals from undersampling if we sample incoherently (e.g. randomly) the signal has low information content Images are often compressible in some transform domain: information can be captured in relatively few coefficients Reconstruction by solving an optimization problem min y where z is k 2 -sparse

14 Algorithm options Adapted three algorithms for the problem: l 1 projection (based on SPGL1) Normalized Iterative Hard Thresholding (NIHT) Iterative tree thresholding (etension of NIHT)

15 Algorithm options Adapted three algorithms for the problem: l 1 projection (based on SPGL1) Normalized Iterative Hard Thresholding (NIHT) Iterative tree thresholding (etension of NIHT) A choice of three 2D sparsifying transforms: Discrete Cosine transform (JPEG) Haar wavelet Daubechies 9 7 wavelet (JPEG 2000)

16 Accuracy of reconstruction Metric: RMSE 6464 lena original 1 N ˆ 2 2 ±1 sampling l 1 projection algorithm Daubechies 9 7 wavelets

17 Some eample reconstructions

18 Running time (s)

19 Effect of sampling noise on RMSE (σ=0)

20 Effect of sampling noise on RMSE (σ=2.5)

21 Effect of sampling noise on RMSE (σ=25)

22 Comparison of different wavebands original visible SW MW LW range reconstruction data RMSE : t : 29.71s RMSE : 9.13 t : 72.57s RMSE : t : 54.24s RMSE : 8.60 t : 52.60s RMSE : 3.87 t : 22.65s tuning : 0.55 tuning : 0.8 tuning : 0.6 tuning : 0.7 tuning : 0.85 ±1 sampling; undersampling ratio 15%; l 1 projection algorithm; Haar wavelet

23 Effect of foreground/background clutter without trees with trees data original recovered recovered original data SW RMSE : 9.13 t : 72.57s tuning : 0.8 RMSE : t : 26.86s tuning : 0.4 MW RMSE : t : 54.24s tuning : 0.6 RMSE : t : 55.14s tuning : 0.5 LW RMSE : 8.60 t : 52.60s tuning : 0.7 RMSE : t : 34.33s tuning : 0.45

24 Peak signal to noise reduction (LW) 1.35km original l 1 l 1 debias NIHT 89% 90% 95% 5km 79% 92% 94% 10km 79% 88% 97%

25 Peak signal to noise reduction (LW) 1.35km original l 1 l 1 debias NIHT 89% 90% 95% 5km 79% 92% 94% 10km 79% 88% 97% CS techniques (using wavelets) can effectively preserve PSNR even at distance

26 Etension to 3D dynamic images Modelling assumption: the sequence can be viewed as a series of static frames slow moving scene. y i F i i i 1,, Perform separate 2D reconstructions as before. F

27 Etension to 3D dynamic images Modelling assumption: the sequence can be viewed as a series of static frames slow moving scene. y i F i i i 1,, Perform separate 2D reconstructions as before. Frame by frame reconstruction implemented F Reads in a 3D tensor video sequence Same 2D model options available Writes output to AVI file.

28 Joint 3D reconstruction Alternative: eploit temporal dependence between frames model enre datacube as sparse in the 3D wavelet domain. F 2 1 F F y

29 Joint 3D reconstruction Alternative: eploit temporal dependence between frames model enre datacube as sparse in the 3D wavelet domain. 3D transforms implemented: 3D Haar wavelets 3D Daubechies D8 orthogonal wavelets. F 2 1 F F y

30 3D reconstruction eample CAMEOSIM video sequence of moving vehicle datacube Reconstruction using l 1 projection and Haar wavelets ; δ = 0.4 Original:

31 3D reconstruction eample CAMEOSIM video sequence of moving vehicle datacube Reconstruction using l 1 projection and Haar wavelets ; δ = 0.4 Frame by frame:

32 3D reconstruction eample CAMEOSIM video sequence of moving vehicle datacube Reconstruction using l 1 projection and Haar wavelets ; δ = 0.4 Joint:

33 Design issues Quantifying constraints on sampling time controlling photon counting noise limits on the speed at which the micromirrors can be flipped

34 Design issues Quantifying constraints on sampling time controlling photon counting noise limits on the speed at which the micromirrors can be flipped DMD technology availability in different wavebands cost and bulkiness

35 Design issues Quantifying constraints on sampling time controlling photon counting noise limits on the speed at which the micromirrors can be flipped DMD technology availability in different wavebands cost and bulkiness Quantifying increase in dynamic range/quantization error

36 Design issues Quantifying constraints on sampling time controlling photon counting noise limits on the speed at which the micromirrors can be flipped DMD technology availability in different wavebands cost and bulkiness Quantifying increase in dynamic range/quantization error Etension to 3D: quantifying constraints on speed of motion of dynamic images

37 Summary of findings It is possible to undersample providing the image in question is suitably compressible

38 Summary of findings It is possible to undersample providing the image in question is suitably compressible The compressibility of the image will depend upon the waveband presence of foreground/background clutter

39 Summary of findings It is possible to undersample providing the image in question is suitably compressible The compressibility of the image will depend upon the waveband presence of foreground/background clutter The model shows robustness to sampling noise

40 Summary of findings It is possible to undersample providing the image in question is suitably compressible The compressibility of the image will depend upon the waveband presence of foreground/background clutter The model shows robustness to sampling noise Good signal to noise preservation even at distance

41 Summary of findings It is possible to undersample providing the image in question is suitably compressible The compressibility of the image will depend upon the waveband presence of foreground/background clutter The model shows robustness to sampling noise Good signal to noise preservation even at distance Possibility of etending to 3D dynamic imaging provided the scene is suitably slow moving

42 References Single Piel Imaging via Compressive Sampling (M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly & R. Baraniuk); IEEE Signal Processing Magazine, Vol. 25(2), pp (2008). Compressive Imaging for Video Representation and Coding (M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly & R. Baraniuk); Proceedings of the Picture Coding Symposium, Beijing, China (April 2006). Phase Transitions for Greedy Sparse Approimation Algorithms (J. Blanchard, C. Cartis, J. Tanner & AT); Applied Computational & Harmonic Analysis, Vol. 30(2), pp (2011); Compressive Single Piel Imaging (AT); Technical report (2011);

43 Thank you for your attention!