Imager Design using Object-Space Prior Knowledge

Transcription

1 Imager Design using Object-Space Prior Knowledge M. A. Neifeld University of Arizona OUTLINE 1. The Last Slot. Introduction 3. PSF Engineering 4. Feature-Specific Imaging

2 Introduction: objects are not iid pixels. - Conventional cameras are designed to image iid pixels impulse-like point-spread-functions (identity transformation) generic metrics such as resolution, field of view, SNR, etc. - Real objects are not iid pixels so don t estimate pixels - This keeps the compression guys employed! - (10 6 pixels)(3 colors/pixel)(8 bits/color) =.4x10 7 bits - (10 11 people)(4x10 9 years)(10 9 images/year) = 4x10 9 images <100 bits - The set of interesting objects is small - Many ways to characterize interesting objects: power spectra, principal components, Markov fields, wavelet projections, templates, task-specific models, finite alphabets, etc. Information depends upon task: Option 1 - this is a random image I = 10 7 bits Option this is a battlefield image I =? bits how to quantify PDF! Option 3 this image either contains a tank or not I = 1bit task-specific source model

3 Introduction: post-processing exploits priors. - Linear Restoration: de-noising and de-blurring exploit noise statistics, object power spectra, principal components, wavelets, - Nonlinear Restoration: super-resolution uses finite support, positivity, finite alphabet, power spectra, wavelets, principal components, isolated points, - Recognition: features, templates, image libraries, syntax, invariance, - Finite Alphabet Post-Processing Examples LADAR Multi-Frame Super-Resolution Object largest return rmse = 7.3mm Object Measurement IBP 8% Wiener rmse = 5.8mm Viterbi rmse = 0.6mm Axial extent of target = Temporal pulse width = 30mm. Target feature size = Scan step size = 4.6mm IBPP 4% D4 - % Optical blur = 1.5 and pixel-blur =. Reconstruction from images, s = 1%

4 Introduction: plausibility of a single pixel imager. Measure only what you want to know Source volume r 1 Fluorescent markers Distant bright objects: aircraft, missile, stars Imager r y x r M M : Number of point sources z Strong Object Model: Equal-intensity monochromatic point sources Scene is completely specified by sources positions: Imager Goals: r 1 r r M Estimate point source position(s): { r 1 r r M } Conventional image may be formed as a post-processing processing step r 1 r r M Conventional image

5 Introduction: information-based design. Optimize imager based on information metric. Maximize measurement entropy. Select detector sizes and positions based on measurement pdf. Source Volume 1 40cm phase mask 1 3 source power = 0.5mW 1cm 3 Lens Measurement log-pdf Measurement log-pdf 3 Detector NEP=nW d 1,h 1 Measurement log-pdf cubic phase random phase

6 Introduction: single pixel imager results. Single Source in Volume Multiple Sources in Volume Detector(s) : Imager Type Conventional CPM RPM One detector in one aperture 1% 39% 65% Two detectors in one aperture 30% 54% 74% Two detectors in two apertures 36% 74% 89% Object-space prior knowledge should inform the optical design Let s utilize this viewpoint in a more useful problem domain

7 PSF ENGINEERING

8 PSF Engineering: Under-Sampled Imagers Imagers for which pixel size > optical spot size.. Large pixels result in under-sampling/aliasing. Sub-pixel shifted measurements to resolve ambiguity. spatial ambiguity shift camera.. Frame 1 Frame Frame K Optical degrees of freedom not exploited. We consider engineering optical point spread function.

9 Imaging Model Object: f Imaging operator: H Measurements: g.. N = 51x51 Optics details: Resolution = 0.mrad/1mm Field of view = 0.1 rad Thickness = 5mm Aperture =.75mm F/# = 1/1.8 Phase-mask Sub-pixel shifts Sensor details: Pixel = 7.5 mm Under-sampling = 15x Full well capacity = 49ke - Spectral bandwidth = 10nm Center wavelength = 550nm.. M = 34x34 Single frame signal to noise ratio: SNR = 10log[sqrt(N e )] = 3.3dB SNR can be improved via multi-frame averaging ~ sqrt(k) Total photon-count is kept constant over multiple-frames.

10 Linear Reconstruction u Linear imaging model: g = Hf + n (note: n is AWGN) u Block-wise shift-invariant imaging operator H is M x N u Problem: M << N (e.g., M=N/15) ^ u Linear minimum mean square error (LMMSE) reconstruction: f = Wg u LMMSE operator: W = RfHt(HR fht+rn)-1 u No Priors = flat PSD u Priors = power law PSD or triangle PSD Example training objects Power Lawà PSD(f) = 1/fη PSD model

11 Performance Measures Root Mean Squared Error: Object Composite Channel H c RMSE n = g ( f fˆ ) 55 LMMSE Reconstruction [%] RMSE=8.6% Angular resolution: Point Object f = δ(r) Composite Channel H c n + g Reconstruction to Diffraction-limited sinc ^ f θ arg min θ x sinc ˆ f θ D q =0.4mrad

12 Conventional/TOMBO Imager Results Conventional Imager TOMBO Imager Shift-sensor RMSE for TOMBO sub-pixel shift Resolution for TOMBO Sub-pixel shifted measurements

13 Alternate PSF Consider use of extended point spread function(psf) Design issue #1: retain full optical bandwidth Design issue #: tradeoff SNR for condition number Pseudo-Random Phase masks for extended PSF Realization of a spatial Gaussian random process. impulse-like PSF extended PSF ( ) R x φ = α exp x σ = γ σ, 4 ρ D - mask roughness r - mask correlation length Pseudo-Random Phase mask Enhanced Lens (PRPEL) Example PSF(D=0.5l,r=10 l) Modulation Transfer Function

14 Resolution Results Resolution for PRPEL and TOMBO All designs use optimal roughness. Note more rapid convergence of PRPEL compared to TOMBO. Higher resolution achieved by PRPEL at reduced number of frames. PRPEL achieves 0.3mrad resolution at K=5 compared to K=1 for TOMBO.

15 RMSE Results RMSE for PRPEL and TOMBO TOMBO K=1 PRPEL K=1 K= K= K=3 K=3 PRPEL makes effective use of prior knowledge at K=1 Note more rapid convergence of PRPEL. PRPEL consistently out-performs TOMBO.

16 PRPEL Summary 4% RMSE requirement Imager Typefi TOMBO Number of Framesfl PRPEL RMSE achieved at M=N/4 Imager Type fi TOMBO (K=4) PRPEL K 5 4 RMSE 4.% 3.9% 0.3mrad Resolution requirement Imager Typefi Number of Framesfl K TOMBO 1 PRPEL 5 Resolution achieved at M=N/4 Imager Type fi (K=4) Resolution TOMBO 0.60mrad PRPEL 0.35mrad PRPEL imager achieves 60% improvement in resolution. PRPEL imager obtains % improvement in RMSE.

17 PSF Engineering via SPEL Sine-Phase mask Enhanced Lens(SPEL) : N α ( x ) φ( x ) = isin i + i= 1 Amplitude Spatial-frequency Phase offset a w Phase-mask Pick N=3: yields 1 free parameters for optimization. ω ( f fˆ ) 100 Optimization criteria: RMSE = [%] 55 RMSE computed over object class using LMMSE operator. PSF is optimized for each value of K. θ i

18 Optimized PSF K=1 Observations Note smaller support of SPEL PSF compared to PRPEL PSF. SPEL PSF also contains subpixel structure. SPEL PSF has more efficient photon-distribution. K= Observations PSF support reduces with increasing K. SPEL PSF is array of delta pulses.

19 Optimized PSF: System Implications K=16 Observations SPEL PSF converges to delta pulses as K increases. In limit K 16 we observe that SPEL PSF to converge to TOMBO-like PSF.

20 RMSE : Power law PSD Results RMSE for SPEL, PRPEL, and TOMBO PRPEL K=1 SPEL K=1 K= K= K=3 K=3 SPEL provides best use of prior knowledge for K=1 SPEL outperforms TOMBO by 47% in terms of RMSE(K=8). SPEL improves RMSE by 35%compared to PRPEL (K=8).

21 Results Angular resolution Resolution for SPEL,PRPEL and TOMBO Note PSF optimization was performed over RMSE. SPEL out-performs TOMBO. SPEL performance compared to PRPEL improves with increasing K. PSF engineering can exploit weak object prior knowledge to improve performance Stronger object prior knowledge can enable non-traditional image measurement

22 FEATURE-SPECIFIC IMAGING

23 Passive Feature-Specific Imaging: Motivation Conventional imaging system PCA, ICA, Fisher, Wavelet, etc. Feature extraction Features Task noise noisy image Restoration, recognition, compression, etc. Feature-specific optics Features Task Feature-specific imaging system (FSI) noise Feature-Specific Imaging (FSI) is a way of directly measuring linear features (linear combinations of object pixels). Attractive solution for tasks that require linear projections of object space Let s consider a case for which task = pretty picture

24 FSI for Reconstruction PCA features provide optimal measurements in the absence of noise Noise-free reconstruction: y = Fx xˆ = My min ε = E{ x x } subject to F = max{ f } = 1 1 i ˆ m j= 1 photon count constraint ij PCA solution : M F M = F T pca pca pca General solution : F= AF A general pca is any invertable matrix T 1 = RFFR ( F ) x x Result using PCA features:

25 Optimal Features in Noise PCA features are not optimal in presence of noise Noise-free problem statement: y = Fx + n xˆ = My M ε opt = RF FRF + x T T -1 ( x σ I) Wiener - operator = Tr + + subject to T T -1 { FRxF ( FRxF σ I) } Tr{ Rx} m E{ x } SNR= 10log( ) σ F = max{ f } = 1 1 i j= 1 ij pca F = F Fpca 1 Object block size = 4x4 Noise = AWGN We use stochastic tunneling to optimize/search RMSE = 1.9 RMSE = 14 Note: PCA error is no longer monotonic in the number of features trade-off between truncation error and photon count constraint F opt RMSE = 1 RMSE = 11.8

26 Optimal Features in Noise Error increases as number of feature increases for PCA solution Reconstructed is improved significantly by using optimal solution Optical implementation requires non-negative projections

27 Passive FSI Result Summary Optimal FSI is always superior to conventional imaging Non-negative solution is a good experimental system candidate

28 Passive FSI for Face Recognition Face recognition from grayscale image feature measurements Class of 10 faces, 600 images per face Training = 3000 faces and testing = 3000 faces Features: wavelet, PCA, Fisher, Recognition algorithms: - k nearest neighbor based on Euclidean distance metric - -layer neural networks batch trained using back-propagation with momentum Sample images from face database [Each image is 18x96] First Wavelet feature of the above images [Each feature is 8x6] Recognition performance [%] Comparison of PCA recognition with AWGN 0 mux 0 conv 0_1 mux 0_1 conv FSI Conventional AWGN standard deviation

29 Passive FSI Optical Implementations

30 Active Feature-Specific Imaging: Motivation What is active illumination? Object Project known structure onto scene Additional degrees of freedom improve imager performance Projector Illumination pattern Conventional cameras Past work on active illumination focused on: Obtain depth-information for 3D objects Enhanced resolution for D objects Our goals: Improve object- and/or task-specific performance Simplify light collection hardware

31 FSAI System Flow Diagram Illumination patterns are eigenvectors (refer as PCA - FSAI) replication of eigenvector P 1 P P M Sequence of illumination patterns Object G Advantages [ diag(p i )]G Light Collection H (optics operator) Small number of detectors High measurement SNR detector Photodetector noise (AWGN) [ H][ diag ( Pi )] G (Estimate of feature weight) r = [ H ][ diag( P )] G + Task is to produce object estimate using these values i ~ αˆ i i n i r1 r R =.. r M Vector of Measurements ece

32 FSAI Post-Processing Measurement vector Post-processing operator W is obtained by minimizing J Wˆ = where and J = R R G ~ H G r1 r R =.. r M ˆ ˆ T E{ trace[( G WR)( G WR) ]} ( mean square error) ~ H = T ece ~ [ HR ~ = [ H i,j G ] object ~ H T M N + R n and ~ H correlatio n matrix, ] 1 Linear postprocessing, i,j W The MMSE operator is given by: = N R n= 1 Metric to evaluate reconstructions : RMSE = 1 number of objects n number of [ H][ diag ( P )]] = objects noise 1 N i n, j covariance N k = 1 i= 1 ( G k i G = W R? Gˆ matrix. k i ) αˆ i P (suboptimal in noise) N = number of pixels, M = number of patterns i

33 Illumination Using Optimal Patterns PCA vectors are not optimal in presence of noise J = ˆ ˆ T E{ trace[( G WR)( G WR) ]} ( R containsnoise) J PCA which is G G ˆ with Gˆ = Minimize the residual MMSE (J MMSE ) with respect to both P i s and T i s J = MMSE R G ( P,... PM, T1 where Wˆ ( P,... P 1 M, T 1,... T ~ T ~ ~ ( H ( P1... PM )) ( H ( P1... PM )) RG ( H ( P1... PM )),... T M ) ) Trace{ R T + σ diag T 1 K i= 1 ˆ ~ W H R 1 M G G Optimal features depend on M, SNR = σ, T a i } P i σ,... T M 1 PCA optimal SNR = 6 db PCA M = 4 M = 8 optimal ece

34 FSAI Results Average RMSE (LOG SCALE) SNR = 6 db U n i f o r m i l l u m i n a t i o n O p t i m a l f e a t u r e s (LOW NOISE) P C A - F S A I (u n i f o r m T) P C A - F S A I (n o n-u n i f o r m T) N u m b e r o f f e a t u r e s PCA-FSAI (uniform T) PCA-FSAI (optimal T) Original object Minimum from PCA-FSAI RMSE = Optimal FSAI Minimum from optimal FSAI RMSE = M = 4 M = 8 ece

35 FSAI Results Summary Algorithm Uniform illumination PCA FSAI (uniform T) PCA FSAI (nonuniform T) Optimal features Improvement of optimal FSAI compared to uniform illumination SNR = 6 db (M = 1) (M = 4) (M > 4) (M = 16) 31 % SNR = 16 db (M =1) (M = ) (M > ) 0.07 (M = 16) 54 % ece

36 Conclusions Objects are not iid pixels Pixel-fidelity should not be the goal of an imager Need new non-traditional design metrics Design should reflect prior knowledge of objects Object-specific imagers (e.g., SPEL) Joint design of optics and post-processing Design should reflect prior knowledge of application Task-specific imagers (e.g., FSI)