Efficient Regression for Computational Imaging: from Color Management to Omnidirectional Superresolution

Transcription

1 Efficient Regression for Computational Imaging: from Color Management to Omnidirectional Superresolution Maya R. Gupta Eric Garcia Raman Arora

2 Regression 2

3 Regression

4 Regression

5 Linear Regression: fast, not good enough

6 Problem: Device Dependent Colors Depend on Device

7 Color Management For each device, characterize the mapping between the native color space and a device independent color space. ICC Profile ICC Profile CIELab (Lab) ICC Profile ICC Profile 8/5/2009 7

8 Color Management For each device, characterize the mapping between the native color space and a device independent color space. ICC Profile ICC Profile CIELab (Lab) ICC Profile ICC Profile CIELab is a widely used deviceindependent color space that is perceptually uniform (i.e. Euclidean distance approximates human judgement of color dissimilarity) 8/5/2009 8

9 Color Management For each device, characterize the mapping between the native color space and a device independent color space. ICC Profile ICC Profile CIELab (Lab) ICC Profile ICC Profile Mapping from RGB -> CIELab and CIELab -> CMYK can be highly nonlinear 8/5/2009 9

10 Gamut mapping: linear transforms not adequate Original gamut Extended gamut Skin tones Skin tones Original Gamut Linear regression Nonlinear regression

11 Creating Custom Color Enhancements Ex: simulating illumination effects original transformed by artist to sunset 2 hrs. work in Photoshop

12 Example Convert an image to how it would look in Cinecolor based on 16 sample color pairs Original cinecolor

13 Color management: speed by LUT 8/5/

14 Color management: speed by LUT 8/5/

15 Color management: speed by LUT 15

16 Color management: speed by LUT

17 Color management: speed by LUT

18 Color management: speed by LUT

19 Color management: speed by LUT

20 Linear Interpolation is linear in the outputs

21 Linear Interpolation is linear in the outputs

22 Linear Interpolation is linear in the outputs

23 Lattice Regression Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data: 8/5/

24 Lattice Regression Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data: 8/5/

25 Lattice Regression Choose the lattice outputs to minimize the post-linear interpolation empirical risk on the data:

26 Effect of Different Lattice Regression Regularizers 8/5/

27 Effect of Different Lattice Regression Regularizers 8/5/

28 Lattice Regression Closed Form Solution Sparse: No more than 7 d m non-zero entries (of m 2 ) with cubic interpolation. 9/15/

29 Example Color Management Results

30 Example Color Management Results

31 Omnidirectional Super-resolution: 9/15/

32 Omnidirectional Superres Related Work State of the Art: Arican and Frossard (ICPR 2008 Best Paper Award) Interpolation with spherical harmonics Alignment with an iterative conjugate gradient approach.

33 Lattice Regression Approach Finding the correct registration of the low-resolution images is challenging non-convex optimization problem. Evaluate a candidate registration: use lattice regression on image subset -> high-res spherical grid sum interpolation error for all left-out low res image data

34 Lattice Regression Approach Finding the correct registration of the low-resolution images is challenging non-convex optimization problem. Evaluate a candidate registration: use lattice regression on image subset -> high-res spherical grid sum interpolation error for all left-out low res image data Finding the optimal joint registration is a 3(N-1)-d opt. problem We use FIPS to find the global optimum.

35

36 9/15/

37 Visual Homing START.. Lattice Regression Better For Visual Homing. HOME....

38 Some Conclusions

39 Some Conclusions

40 Some Conclusions

41 Some Conclusions

42 For details, see: Optimized Regression for Efficient Function Evaluation, Eric K. Garcia, Raman Arora, and Maya R. Gupta, (in review draft upon request). Lattice Regression, Eric K. Garcia, Maya R. Gupta, Neural Information Processing Systems (NIPS) Building Accurate and Smooth ICC Profiles by Lattice Regression, Eric K. Garcia, Maya R. Gupta, 17 th IS&T Color Imaging Conference "Adaptive Local Linear Regression with Application to Printer Color Management," Maya R. Gupta, Eric K. Garcia, and Erika Chin, IEEE Trans. on Image Processing, vol. 17, no. 6, , "Learning Custom Color Transformations with Adaptive Neighborhoods," Maya R. Gupta, Eric K. Garcia, and Andrey Stroilov, Journal of Electronic Imaging, vol. 17, no. 3, "Gamut Expansion for Video and Image Sets," Hyrum Anderson, Eric K. Garcia, and Maya R. Gupta, Computational Color Imaging Workshop, /5/

43 Color is an event light source reflection human cones respond: human perceives color L = long wave = red M = medium wave = green S = short wave = blue

44 What does it mean to see black? light source??? human perceives color human cones respond L = long wave = red M = medium wave = green S = short wave = blue

45 What does it mean to see white? light source??? human perceives color human cones respond L = long wave = red M = medium wave = green S = short wave = blue

46 What does it mean to see white? images from: You can see white given light made up of 2-spectra

47 Color Science Crash Course What we see can be represented by three primaries. match monochromatic light at some wavelength mixture of three primary colors Stiles-Burch 10 color matching functions averaged across 37 observers. Adapted from (Wyszecki & Stiles, 1982) by handprint.com. 8/5/

48 CIELab Based on spectral measurements integrated over CMF Color Distances of color, envelopes. Euclidean distance between two colors approximates the perceptual difference noticed by a human observer. Distance metrics created to correct for perceptual nonuniformities in the space: image source: 8/5/

49 2-D and 3-D Simulation d=2 d=3 8/5/

50 Color management for printers 8 bit RGB color patch Color printer printed color patch Human eye Measure CIEL*a*b* Goal: Print a given CIEL*a*b* value. Problem: What RGB value to input?

51 Inverse Device Characterization Step 1 Sample the device CIELab Output Measure Step 2 Build an inverse look-up-table Regression Look-up-table 8/5/

52 8/5/

53 Gaussian Process Regression Models data as being drawn from a Gaussian Process A leading method in geostatistics (2-d regression) also known as Kriging. Generally considered a state-of-the-art method by machine learning folks Parameters: Covariance Function (length scale L), Noise Power σ 2. (L large, σ 2 small) (L small, σ 2 small) (L large, σ 2 large) 8/5/

54 Gaussian Process Regression Models data as being drawn from a Gaussian Process A leading method in geostatistics (2-d regression) also known as Kriging. Generally considered a state-of-the-art method by machine learning folks Parameters: Covariance Function (length scale L), Noise Power σ 2. (L large, σ 2 small) (L small, σ 2 small) (L large, σ 2 large) Given Covariance form, parameters can be learned by maximizing marginal likelihood. (i.e. automatically from data). 8/5/

55 2-D Simulation 50 Training Samples 1000 Training Samples Gaussian Process Regression (Direct) Gaussian Process Regression (to nodes of lattice) Lattice Regression (GPR bias) Lattice Regression (Bilinear bias) 8/5/

56 3-D Simulation 50 Training Samples 1000 Training Samples Gaussian Process Regression (Direct) Gaussian Process Regression (to nodes of lattice) Lattice Regression (GPR bias) Lattice Regression (Bilinear bias) 8/5/