Compressive Single Piel Imaging Andrew Thompson University of Edinburgh 2 nd IMA Conference on Mathematics in Defence
About the project Collaboration between the University of Edinburgh and SELEX Galileo Ltd. Jared Tanner and Coralia Cartis (UoE/E CoS) David Humphreys and Robert Lamb (SELEX)
About the project Collaboration between the University of Edinburgh and SELEX Galileo Ltd. Jared Tanner and Coralia Cartis (UoE/E CoS) David Humphreys and Robert Lamb (SELEX) Partly funded by the Underpinning Defence Mathematics (UDM) programme of the MoD Facilitated by the Knowledge Transfer Network
Outline of presentation Background The Rice single piel camera design A mathemacal model Compressed Sensing
Outline of presentation Background The Rice single piel camera design A mathemacal model Compressed Sensing Research undertaken Reconstruction algorithms investigated Results of numerical tests Etension to 3D dynamic images
Outline of presentation Background The Rice single piel camera design A mathemacal model Compressed Sensing Research undertaken Reconstruction algorithms investigated Results of numerical tests Etension to 3D dynamic images Design issues and key findings
The Compressed Sensing Single Piel Camera (Rice University, 2006 08)
A sampling model Model light incident on DMD as an N piel image Represent as a vector of dimension N
A sampling model 5 4 3 2 1, 1 1 0 0 1, y i i Model light incident on DMD as an N piel image Represent as a vector of dimension N Model each measurement as an inner product with a random basis vector
A sampling model 5 4 3 2 1, 1 1 0 0 1, y i i Model light incident on DMD as an N piel image Represent as a vector of dimension N Model each measurement as an inner product with a random basis vector Model photon counting noise and quantization error as Gaussian noise Take n < N samples e y T n T T 2 1 e
The Compressed Sensing paradigm CS: We can recover signals from undersampling if we sample incoherently (e.g. randomly) the signal has low information content
The Compressed Sensing paradigm CS: We can recover signals from undersampling if we sample incoherently (e.g. randomly) the signal has low information content Images are often compressible in some transform domain: information can be captured in relatively few coefficients
The Compressed Sensing paradigm CS: We can recover signals from undersampling if we sample incoherently (e.g. randomly) the signal has low information content Images are often compressible in some transform domain: information can be captured in relatively few coefficients Reconstruction by solving an optimization problem min y where z is k 2 -sparse
Algorithm options Adapted three algorithms for the problem: l 1 projection (based on SPGL1) Normalized Iterative Hard Thresholding (NIHT) Iterative tree thresholding (etension of NIHT)
Algorithm options Adapted three algorithms for the problem: l 1 projection (based on SPGL1) Normalized Iterative Hard Thresholding (NIHT) Iterative tree thresholding (etension of NIHT) A choice of three 2D sparsifying transforms: Discrete Cosine transform (JPEG) Haar wavelet Daubechies 9 7 wavelet (JPEG 2000)
Accuracy of reconstruction Metric: RMSE 6464 lena original 1 N ˆ 2 2 ±1 sampling l 1 projection algorithm Daubechies 9 7 wavelets
Some eample reconstructions
Running time (s)
Effect of sampling noise on RMSE (σ=0)
Effect of sampling noise on RMSE (σ=2.5)
Effect of sampling noise on RMSE (σ=25)
Comparison of different wavebands original visible SW MW LW range reconstruction data RMSE : 12.94 t : 29.71s RMSE : 9.13 t : 72.57s RMSE : 11.37 t : 54.24s RMSE : 8.60 t : 52.60s RMSE : 3.87 t : 22.65s tuning : 0.55 tuning : 0.8 tuning : 0.6 tuning : 0.7 tuning : 0.85 ±1 sampling; undersampling ratio 15%; l 1 projection algorithm; Haar wavelet
Effect of foreground/background clutter without trees with trees data original recovered recovered original data SW RMSE : 9.13 t : 72.57s tuning : 0.8 RMSE : 26.87 t : 26.86s tuning : 0.4 MW RMSE : 11.37 t : 54.24s tuning : 0.6 RMSE : 18.78 t : 55.14s tuning : 0.5 LW RMSE : 8.60 t : 52.60s tuning : 0.7 RMSE : 15.97 t : 34.33s tuning : 0.45
Peak signal to noise reduction (LW) 1.35km original l 1 l 1 debias NIHT 89% 90% 95% 5km 79% 92% 94% 10km 79% 88% 97%
Peak signal to noise reduction (LW) 1.35km original l 1 l 1 debias NIHT 89% 90% 95% 5km 79% 92% 94% 10km 79% 88% 97% CS techniques (using wavelets) can effectively preserve PSNR even at distance
Etension to 3D dynamic images Modelling assumption: the sequence can be viewed as a series of static frames slow moving scene. y i F i i i 1,, Perform separate 2D reconstructions as before. F
Etension to 3D dynamic images Modelling assumption: the sequence can be viewed as a series of static frames slow moving scene. y i F i i i 1,, Perform separate 2D reconstructions as before. Frame by frame reconstruction implemented F Reads in a 3D tensor video sequence Same 2D model options available Writes output to AVI file.
Joint 3D reconstruction Alternative: eploit temporal dependence between frames model enre datacube as sparse in the 3D wavelet domain. F 2 1 F F y 2 1 2 1 0 0 0 0 0 0
Joint 3D reconstruction Alternative: eploit temporal dependence between frames model enre datacube as sparse in the 3D wavelet domain. 3D transforms implemented: 3D Haar wavelets 3D Daubechies D8 orthogonal wavelets. F 2 1 F F y 2 1 2 1 0 0 0 0 0 0
3D reconstruction eample CAMEOSIM video sequence of moving vehicle 646464 datacube Reconstruction using l 1 projection and Haar wavelets ; δ = 0.4 Original:
3D reconstruction eample CAMEOSIM video sequence of moving vehicle 646464 datacube Reconstruction using l 1 projection and Haar wavelets ; δ = 0.4 Frame by frame:
3D reconstruction eample CAMEOSIM video sequence of moving vehicle 646464 datacube Reconstruction using l 1 projection and Haar wavelets ; δ = 0.4 Joint:
Design issues Quantifying constraints on sampling time controlling photon counting noise limits on the speed at which the micromirrors can be flipped
Design issues Quantifying constraints on sampling time controlling photon counting noise limits on the speed at which the micromirrors can be flipped DMD technology availability in different wavebands cost and bulkiness
Design issues Quantifying constraints on sampling time controlling photon counting noise limits on the speed at which the micromirrors can be flipped DMD technology availability in different wavebands cost and bulkiness Quantifying increase in dynamic range/quantization error
Design issues Quantifying constraints on sampling time controlling photon counting noise limits on the speed at which the micromirrors can be flipped DMD technology availability in different wavebands cost and bulkiness Quantifying increase in dynamic range/quantization error Etension to 3D: quantifying constraints on speed of motion of dynamic images
Summary of findings It is possible to undersample providing the image in question is suitably compressible
Summary of findings It is possible to undersample providing the image in question is suitably compressible The compressibility of the image will depend upon the waveband presence of foreground/background clutter
Summary of findings It is possible to undersample providing the image in question is suitably compressible The compressibility of the image will depend upon the waveband presence of foreground/background clutter The model shows robustness to sampling noise
Summary of findings It is possible to undersample providing the image in question is suitably compressible The compressibility of the image will depend upon the waveband presence of foreground/background clutter The model shows robustness to sampling noise Good signal to noise preservation even at distance
Summary of findings It is possible to undersample providing the image in question is suitably compressible The compressibility of the image will depend upon the waveband presence of foreground/background clutter The model shows robustness to sampling noise Good signal to noise preservation even at distance Possibility of etending to 3D dynamic imaging provided the scene is suitably slow moving
References Single Piel Imaging via Compressive Sampling (M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly & R. Baraniuk); IEEE Signal Processing Magazine, Vol. 25(2), pp 83 91 (2008). Compressive Imaging for Video Representation and Coding (M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly & R. Baraniuk); Proceedings of the Picture Coding Symposium, Beijing, China (April 2006). Phase Transitions for Greedy Sparse Approimation Algorithms (J. Blanchard, C. Cartis, J. Tanner & AT); Applied Computational & Harmonic Analysis, Vol. 30(2), pp 188 203 (2011); http://ecos.maths.ed.ac.uk. Compressive Single Piel Imaging (AT); Technical report (2011); http://ecos.maths.ed.ac.uk.
Thank you for your attention!