Scientific imaging of Cultural Heritage: Minimizing Visual Editing and Relighting

Transcription

1 Scientific imaging of Cultural Heritage: Minimizing Visual Editing and Relighting Roy S. Berns Supported by the Andrew W. Mellon Foundation

2 Colorimetry Numerical color and quantifying color quality b* a*

3 Color: Interaction of Light, Object, and Observer

4 1.25 Daylight 1 Relative power Wavelength, nm Relative sensitivity S M L Wavelength, nm 1 Reflectance factor, R Wavelength, nm

5 1.2 Relative power, S Wavelength, nm 1 SR Wavelength, nm 1 Reflectance factor, R Wavelength, nm Hundreds of wavelengths mapped to THREE signals Relative sensitivity Relative sensitivity Relative sensitivity Wavelength, nm Wavelength, nm Wavelength, nm Relative response Relative response Relative response Wavelength, nm Wavelength, nm Wavelength, nm L M S

6 Eye-Brain Codes Color: Red-Green, Yellow-Blue, Black-White Redness or Greenness Yellowness or Blueness Blackness or Whiteness Black-White Retinal Interconnections Red-Green L M S Yellow-Blue Rousseau, Centennial of Independence

7 Red-Green CIE Approximation of Color Vision CIELAB Full color L* a* b*

8 Monet, Sunrise (Marine) interpretation aid

9 Quality Metrics Color Difference Formulas Delta E 2000 CIELAB Delta E ΔL ' 2 + ΔC ' 2 ab S ΔE 00 = L S C ' ' ΔC +R ab ΔH ab T S C S H + ΔH ' ab S H 2 1/ b* b* a* a*

10 Vector Plots Average CIEDE2000 = 6.7

11 Scientific Imaging Minimizing visual editing

12 Color Reproduction Goals Preferred e.g. conventional photography and reprographics) Spectral e.g. research and scientific imaging Colorimetric e.g. scientific imaging recording the true colors

13 Cézanne, Portrait of Anthony Valabrègue Scanned photograph and visual editing Direct digital colorimetric

14 Five Rules of Colorimetric Imaging 1. Lighting correlated color temperature (CCT) near 5000 K (assuming D50 workflow) 2. Optimal exposure 3. Profile is based on minimizing E with outstanding lightness accuracy 4. Independent validation using target not used to profile camera 5. Encoding space does not clip scene colors

15 Evaluating_Solid_State_Lighting_TR_Jan_2014.pdf 1. Lighting CCT near 5000K

16 D Avg. 3.5 Max. CIEDE K 0.9 Avg. 3.8 Max.

17 D Avg. 3.4 Max. CIEDE K 1.6 Avg. 4.9 Max.

18 2. Optimal Exposure Renoir Albert Cahen d Anvers" Correct Under

19 3. Colorimetric Profile 2012 IS&T Archiving, Copenhagen

20 Test Targets

21 Results

22 RIT-Sinar Dual-RGB Optimized for color 1.5 Avg th 6.8 Max. CIEDE2000

23 Hasselblad H4D-50 Reprographics mode 4.6 Avg th 9.8 Max. CIEDE2000

24 PhaseOne IQ Avg th 11.6 Max. CIEDE2000

25 4. Independent Validation Artist Paint Target (APT) glossy black 50L* neutral cobalt blue titan buff (aged varnished white approximation)

26 CFA and Scanback Cameras ColorChecker Profiles CFA HVS Scanback Computational analysis (simulation)

27 Visualizing Results Measured Cobalt Phthalo CFA scanback

28 5. Encoding Space Does Not Clip Colors 2015 IS&T Archiving, Los Angeles

29 Out of Gamut Colors interpretation aid Adobe RGB Varnished artist palette Fluorescent paints Pointer colors ECI RGB

30 Imaging Surface Normal and Color Minimizing relighting and reshooting

31 Surface Normal A normal to a surface at a point is the same as a normal to the tangent plane to that surface at that point.

32 Normal Defined Using XYZ Coordinates n x Y n = n y n z Z X

33 False Color srgb for Directions X, Y, and Z Scene Normal map Rendered

34 Diffuse Materials Lambert s Law I diffuse = k d I light ( N L) I diffuse = k d = = diffuse reflected light diffuse albedo (color) I light = Light intensity N = L = surface normal in XYZ coordinates light direction in XYZ coordinates

35 R. Woodham, Photometric method for determining surface orientation from multiple images, Optical Engineering 19:1, (1980) Photometric because it uses radiance values Stereo because only two lighting geometries are required theoretically In practice, at least three geometries are required

36 Photometric Stereo I 3 = N L 3 Camera along z axis I 1 = N L 1 I 2 = N L 2

37 Combining Three Lights and Writing as Matrix I = Ln I = I 1 I 2 I 3 L = x 1 y 1 z 1 x 2 y 2 z 2 x 3 y 3 z 3 n = n = L 1 I Three unknowns: nx, ny, nz n x n y n z With images from three different directions, can solve for nx, ny, nz

38 Four Lights n = L + I + means pseudo-inverse, i.e., least squares n = n x n y n z L = x 1 y 1 z 1 x 2 y 2 z 2 x 3 y 3 z 3 x 4 y 4 z 4 I = I 1 I 2 I 3 I 4 More lights improves the precision and accuracy

39 Four-Light Imaging Simplified 2015 SPIE Electronic Imaging, San Jose

40 Art Institute of Chicago

41 Flat Fielded Images

42 Color Calibration cobalt blue CIEDE2000 = 0.18

43 Define Light Direction With Cue Ball

44 Diffuse and Normal Maps

45

46 Real Time Visualization

47 Computer Graphics Software Maya

48 Rendered Still Image More diffuse More directional ARTIC studio set up

49 Rendered as Metal with Directional Lighting

50 Movie

51 Summary Colorimetry for average observer and physically-non-realizable source CIELAB for numerical color specification a* is not redness opposed to greeness All Delta E s are not alike Evaluate E and vector plots

52 Summary Colorimetric rather than preferred color reproduction Five rules of colorimetric imaging: 5000 K lighting, optimal exposure, colorimetric profile and outstanding lightness accuracy, independent validation using target not used for profiling wide-gamut 16-bit (or more) encoding

53 Summary Measure surface normal and diffuse color for more flexibility Diffuse color to track long-term color changes Lighting a painting is done on the computer rather than in the studio Creating a virtual museum

54 Based on Monnier, P. and Shevell, S. K., Large shifts in color appearance from patterned chromatic backgrounds, Nature Neuroscience 6, , 2003