Recovering Camera Sensitivities using Target-based Reflectances Captured under multiple LED-Illuminations

Transcription

1 Recovering Camera Sensitivities using Target-based Reflectances Captured under multiple LED-Illuminations Philipp Urban, Michael Desch, Kathrin Happel and Dieter Spiehl

2 Motivation: Spectral Estimation Channel m Illumination Scene Camera Camera Response? Channel 1 Can we calculate spectral reflectances from the camera image? Spectral Image

3 Motivation: Spectral Estimation If yes, we could render/reproduce the image for any illuminant

4 Motivation: Spectral Estimation If yes, we could estimate pigments used by an artist Y. Zhao 2008

5 Spectral Estimation Approaches c 1 c m Signal Processing Training-based Methods Model-based Methods Spatio-Spectral Methods Multipoint spectral Measurement Methods Camera Sensitivities Required

6 Model-based Methods Use information to calculate spectral reflectance factor from camera responses Additional Information e.g. Distribution of noise,reflectances

7 Camera Model (Linear, Continuous) Illuminant X X d l r [s 1 s m ] T Reflectance Camera Sensitivities + c 1 = = c c m Camera Response (additive noise) Geometry factor* Spectral stimulus Required but usually unknown *Lambertian surface assumption

8 Camera Model (Linear, Discrete) Sensitivity Range Relative power\reflectance r = [ ] T Spectral representation by N-dimensional vector Geometry Factor Reflectance vector Diagonal matrix with illuminant as diagonal i-th sensitivity vector Noise i-th channel response

9 Monochromator-based Training Stimuli Monochromator Halon (r 1) Sensitivity Sampling the Sensitivities with monochromatic light + Very high effective dimension* (~N) Wavelength Very good sensitivity estimation without any additional assumptions *Hardeberg (2002)

10 Target-based Training Stimuli Low effective dimension of stimuli for typical targets (broadband) Measure m target reflectances Capture target for one illuminant Additional assumptions required (smoothness, non-negativity, ) Large influence of assumptions on estimated sensitivities

11 LED+Target-based Training Stimuli m x k x n equations High effective for recovering dimension of stimuli s1,,sn for typical targets (narrowband) Measure m target reflectances Capture target for LED illuminant 1 Capture target for LED illuminant k Additional assumptions required (smoothness, non-negativity, ) Small influence of assumptions on estimated sensitivities

12 Problem: Measurement Geometry 45 /0 Geometry factor differs Different measurement geometries: among patches and illuminants Distance between camera and patches Angle between camera and patches Difference Non-linear between equation spectrophotometer system and camera (sensitivities + geometry factors)

13 Solution: Use Chromaticities Divide each equation by the sum of channels (geometry factor can be reduced) Rearrange equations + consider noise where is a (k m n) (n N) dimensional matrix that depends on Noise, assumed to be uncorrelated zero-mean Gaussian with covariance matrix

14 What Assumptions are Reasonable? Sensitivities are non-negative 1 invalid 2 Sensitivities are likely to be smooth: Model wavelength-correlation using a Toeplitz-matrix Unlikely, but possible

15 What Assumptions are Reasonable? Covariance matrix for can be modeled as block-diagonal 3 (i.e. channel sensitivities are uncorrelated) Likelihood model: Prior distribution: where 2 3 Posterior distribution: where

16 Reconstruct Relative Sensitivities Solve Constrained Maximum-A-Posteriori (MAP) Problem: Minimize: subject to: The result are sensitivities that maximize the posterior distribution subject to constraints Last constraint is a normalization required to avoid zero sensitivities

17 Experimental Setup 1 Camera: Sinar 54H-based RGB camera (IR cutoff removed) Custom made two-stage filter wheel with yellow + blue filter Six channel camera (12 bit, 22MP) 2 Target: Color Checker measured using X-Rite s EyeOne

18 Experimental Setup 3 LED viewing Booth: JUST Normlicht LED Color Viewing Light (six different types of LED) 4 Spectroradiometer to measure LED spectral power distribution (SPD) Konica-Minolta CS UV/IR cutoff filter added to limit the sensitivity range of the camera to nm [EyeOne measurements do not cover the real sensitivity range]

19 Results Effective dimension: 15 Color Checker only: 9 Experimental Setup *CC = Color Checker patches Training Stimuli (24 CC* x 6 LED = 144)

20 Results Reconstructed Sensitivities Predicted vs. real channel responses for the training stimuli [Errors: mean ~ 0.3%, 95 th < 0.8%, max < 1.35%]* *Error percentage is relative to the max. channel response

21 Results (Test Stimuli) Capture the ESSER TE221 Target under 6 LED illuminant [1698 different test stimuli] Predict channel responses using the reconstructed sensitivities Prediction error histogram* Small mean errors < 0.5%* Small 95 th errors < 1.3%* But large max errors ~7.5%* Possible reason for large max error: Invalid Lambertian surface assumption Effective dimension of stimuli is still to small *Error percentage is relative to the max. channel response

22 Conclusion Camera sensitivities are required for many reflectance estimation methods If unknown they need to be measured/estimated Sensitivity reconstruction methods use assumptions (non-negativity, smoothness) and training stimuli: Different LED + Target Higher effective dimension of stimuli Smaller influence of assumptions Sensitivity reconstruction using Constrained Maximum-A-Posteriori (MAP) Test: 6-channel camera, 6-LED, Color Checker Results (Esser test target): small mean and 95 th -, large max prediction errors

23 Thanks for your attention Thanks to Maria Fedutina, Tanja Kaulitz, Henri Kröling, Karsten Rettig, Manfred Jakobi for constructing, manufacturing and assembling the filter wheel and to the Deutsche Forschungsgemeinschaft for the sponsorship of the project Philipp Urban, Michael Desch, Kathrin Happel and Dieter Spiehl