Computational color Lecture 1. Ville Heikkinen

Transcription

1 Computational color Lecture 1 Ville Heikkinen

2 1. Introduction - Course context - Application examples (UEF research) 2

3 Course Standard lecture course: - 2 lectures per week (see schedule from Weboodi) - exercises Course page (available in Moodle soon ): - lecture slides - exercises + data 3

4 Color in image processing 4

5 Computations Measurement devices produce data that are usually represented with vectors, matrices, and tensors. Efficient image analysis and processing usually requires understanding of color/spectral data and suitable computational methods. 5

6 Biometry, medical diagnosis, security Is illumination same in all images? Are images calibrated accurately? 6

7 Source of color data In this course we assume that measurement devices include standard color imaging devices (e.g. in mobile phone) and spectral imaging devices (multispectral- and hyperspectral cameras). 7

8 Color (trichromatic, RGB) imaging devices 8

9 Example: 3-dimensional vectors (pixelwise RGB vectors corresponding to a digital color image) 9

10 Is color enough? Color representation Surface reflectance (r) Wavelength [nm] 10

11 A simple spectral imaging system 11

12 A complex spectral imaging system 12

13 Spectral image as a data structure 13

14 Reflectance spectrum corresponding to a pixel 14

15 Sampling 15

16 Images corresponding to spectral bands 16

17 Applications for spectral data and calibrated color Biomedical imaging Biometry and security Color analysis, display, and printing Cultural heritage imaging Environmental monitoring Industrial machine vision 17

18 Course context: Color management and data analysis... 18

19 ...where the main phases are 1. Measurement 2. Processing 3. Accurate color representation 4. Data analysis (e.g. machine learning) 19

20 Course outline The main topics in this course are: 1. Vector space formulations for colorimetry. Relation between spectral spaces and color spaces. Fundamental colorimetry 2. Estimation of standard color values from device responses 3. Estimation of spectral data from device responses 4. Representation of spectral data using subspaces 5. Implementation of computational models using MATLAB or some other computational environment. 20

21 Practical/project work Data + small analysis task Students write a report of obtained results. The reports are written as a form of (approx. 6 page) scientific publication consisting of Abstract Introduction Methodology Experiments Discussion Conclusions 21

22 Example 1 Computational spectral imaging: Increasing the availability of spectral data for common people (as well as for experts) 22

23 A (real) hyperspectral imaging system (100k), UV-VIS-NIR A Wide Spectral Range Reflectance and Luminescence Imaging System, T. Hirvonen et al., Sensors, Vol. 13(11), ,

24 Properties of hyperspectral imaging + h Narrow spectral bands VIS, IR and UV Estimation of reflectance information can be done easily by using reference surfaces. - jcosts Measurement speed (slow) Light source properties (heat) Spatial properties of obtained images High-level of expertise Practicality of measurement setting Several systems are not mobile 24

25 Multispectral imaging system (1k) + mathematics + programming -> Estimated hyperspectral image in visual wavelength range Spectral imaging using consumer-level devices and kernel-based regression, V. Heikkinen et al., Journal of the Optical Society of America A, Vol. 33(6),

26 Properties of multispectral imaging + h Fast imaging (moving objects, video imaging) Inexpensive (if based on RGB technique) Large spatial resolution Low demands for light source Practical RGB based systems are in standard use in several applications Mobile - Estimation of reflectance information is not trivial Possibly broad spectral bands. Sensitivity of RGB-based system may be restricted to VIS region 26

27 Spectral images visualized as srgb Estimates visualized as srgb Spectral imaging using consumer-level devices and kernel-based regression, V. Heikkinen et al., Journal of the Optical Society of America A, Vol. 33(6),

28 Comparison of images using srgb color representation 3 line scans (Specim V10) RGB + Laptop system using Digital colorchecker in training Training data : Spectral imaging using consumer-level devices and kernel-based regression, V. Heikkinen et al., Journal of the Optical Society of America A, Vol. 33(6),

29 Comparison of images using PCA eigenimages 3 line scans (Specim V10) RGB + Laptop system using Digital colorchecker in training Training data : Supplements for Spectral imaging using consumer-level devices and kernelbased regression, V. Heikkinen et al., Journal of the Optical Society of America A, Vol. 33(6),

30 What kind of training data are needed for the imaging system? 3 line scans (Specim V10) RGB + Laptop system using Digital colorchecker in training Spectral imaging using consumer-level devices and kernel-based regression, V. Heikkinen et al., Journal of the Optical Society of America A, Vol. 33(6),

31 What kind of applications there are? 3 line scans (Specim V10) RGB + Laptop system using Digital colorchecker in training Spectral imaging using consumer-level devices and kernel-based regression, V. Heikkinen et al., Journal of the Optical Society of America A, Vol. 33(6),

32 Example 2: Spectral imaging as a tool for object analysis and sensor development 32

33 Hyperspectral imaging as tool for tree seeds screening Line scan imaging in nm using two cameras. Tree seeds in three classes Thermal and hyperspectral imaging for Norway spruce (Picea abies) seeds screening, J. Dumont, T. Hirvonen, V. Heikkinen et al., Computers and Electronics in Agriculture, Vol. 116, pp ,

34 Hyperspectral imaging and feature extraction Line scan imaging in nm using two cameras. Tree seeds in three classes Thermal and hyperspectral imaging for Norway spruce (Picea abies) seeds screening, J. Dumont, T. Hirvonen, V. Heikkinen et al., Computers and Electronics in Agriculture, Vol. 116, pp ,

35 analyzing feature selection Line scan imaging in nm using two cameras. Tree seeds in three classes Thermal and hyperspectral imaging for Norway spruce (Picea abies) seeds screening, J. Dumont, T. Hirvonen, V. Heikkinen et al., Computers and Electronics in Agriculture, Vol. 116, pp ,

36 ...and classifier construction with two indices (based on three, narrow spectral bands) Line scan imaging in nm using two cameras. Tree seeds in three classes Thermal and hyperspectral imaging for Norway spruce (Picea abies) seeds screening, J. Dumont, T. Hirvonen, V. Heikkinen et al., Computers and Electronics in Agriculture, Vol. 116, pp ,

37 Example 3. Remote sensing of forest areas A plot of pine trees in the ground (10 m x 10 m area) Evaluation of simulated bands in airborne optical sensors for tree species identification, P. Pant et al., Remote Sensing of Environment, Vol. 138, pp 27 37,

38 Supervised classification of tree species in the ground Evaluation of simulated bands in airborne optical sensors for tree species identification, P. Pant et al., Remote Sensing of Environment, Vol. 138, pp 27 37,

39 Example 4. Color calibration in remote sensing Color characterization for aerial cameras. Susanne Scholz. Applied Geoinformatics for Society and Environment (AGSE) proceedings

40 Example 5. Pigment mapping using spectral reflectance data (image segmentation) Sensor changes in 1000 nm 40

41 Example: Segmentation of painting image using correlation coefficient Let vector V be a spectral measurement (121-dimensional vector). Task: Find those pixels that have correlation coefficient > 0.99 with the vector V. 41

42 42

43 43

44 Task: Find those pixels that have correlation coefficient > with the vector V. 44

45 45

46 46