Colour calibration of an artificial vision system for industrial applications: comparison of different polynomial models

Transcription

1 Colour calibration of an artificial vision system for industrial applications: comparison of different polynomial models 1 Francesco Bianconi, 1 Stefano A. Saetta, 2 Giulia Sacchi, 1 Francesco Asdrubali, 1 Giorgio Baldinelli 1 Dip. Ingegneria Industriale, Università degli Studi di Perugia Via G. Duranti, Perugia (ITALY) bianco@ieee.org, stefano.saetta@unipg.it, fasdruba@unipg.it, baldinelli.unipg@ciriaf.it 2 Green Consulting S.r.l. Strada di Santa Filomena, Terni (ITALY) sacchi@greenconsulting.biz Abstract In this paper we investigate the performance of different polynomial models for colour calibration. We present the results of a set of experiments on a visual inspection system composed of a dome illuminator and an industrial, entry-level, camera. The system has been calibrated with six different polynomial models using the 24 colour patches of the X-Rite Color Checker as source colours. The colour measuring performance of the various models has been evaluated on a set of industrial materials by computing the difference ( E ab ) between the results obtained with the visual inspection system with those obtained with a Minolta Chroma Meter CR-200. The results show that the performance of the different calibration models does not vary significantly as the polynomial formulation changes, suggesting that a relation as simple as a linear one may produce as good results as higher degree polynomials. 1. Introduction Materials appearance represents an important feature of many industrial products, and governs, to a great extent, their price. It is therefore of primary importance that this feature be checked carefully to guarantee accurate uniformity and to comply with the requirements of customers. Such tasks have been traditionally demanded to trained human operators. Globalisation, however, has been calling for better ways to assess visual appearance of industrial products. Human-based control, nowadays, is no longer satisfactory, because the decisions made by human operators are subjective and non-repetitive. Besides, the entire process is difficult to document. As a consequence the industrial interest for the development of automated computer vision systems for assessing materials appearance is currently high. The main applications of such systems can be conveniently classified into three groups: classification, segmentation and content-based image retrieval. Classification is about grouping products into lots of similar appearance, a procedure that is usually referred to as surface grading. Segmentation is concerned with defect detection (i.e.: stains, veins, cracks, etc.), and it is usually indicated as surface inspection. Content-based image retrieval consists in searching for images in large databases based on the visual content. For general surveys on these applications the interested reader is referred to references [1, 2, 3]. In all cases the problem consists in measuring visual appearance in a quantitative way. It is widely accepted that this 1

2 is the result of two interacting features: colour and texture. In this paper we are concerned with the former. Colour can be easily measured through specific instruments, namely colorimeters and spectrophotometers. Unfortunately such instruments are not particularly suitable for on-line, continuous surface inspection; on the one hand because they are contactbased and hence intrinsically slow, on the other hand because they can only provide a punctual measurement of the object under inspection. On the contrary automatic visual inspection systems based on controlled illumination and industrial cameras are much more appropriate for this task, but produce device-dependent results which need to be converted into colorimetric data through colour calibration. This involves determining the relationship between input spectra and camera output. This problem, usually referred to as colour calibration, has received significant attention during the last two decades, due to the ample variety of possible applications in industry. The mathematical basis of the problem has been clearly described in the work of Vhrel and Russel [4]. Most related literature has been focused on defining a suitable functional form to model such a mapping. The various approaches that have been proposed can be conveniently classified into two families: polynomial models and non-polynomial models. Polynomial models [5] range from the simplest linear model to higher-degree polynomials, where high-order and cross-terms are used to approximate the nonlinear relationship. Polynomials models have some potential advantages: they are easy to implement and require the estimation of few parameters. As a consequence they also require relatively small training sets. Non-polynomial models employ different formulations to estimate the relationship between device-dependent and device-independent data. To this end various authors proposed different neural network-based methods [6, 7]. Theoretically they should provide good results, due to the excellent capability of neural networks of modelling non-linearities. Conversely, however, these methods need a far higher number of samples to train a model [8], which represents a potential problem in practical applications. In a comparative study conducted by Cheung et al. [9] the authors showed that the two classes of models give approximately the same performance, concluding that polynomial transforms are preferable for they are easier and less time-consuming to train than neural models. Based on these considerations we focused this manuscript on the first group of methods. In the remainder of the paper we briefly recall the basics of colour calibration (Section 2), describe the materials and instruments used in the experiments (Section 3), summarize the experimental activity (Section 4) and conclude with some final considerations and proposals for further investigations (Section 5). 2. Polynomial models for colour calibration As we mentioned in the preceding section, colour calibration is about determining a function F that maps device-dependent values into device-independent values: ( X Y, Z ) F ( R, B) 2, = (1)

3 Throughout this paper we assume, without loss of generality, that deviceindependent colour coordinates are given in the CIE XYZ colour space and devicedependent colour coordinates in the RGB space. Indeed a device-independent colour space can be defined as any space that has a one-to-one mapping to the CIE XYZ space [4]. The same applies to device-dependent colour spaces. For conversion formulas between colour spaces the interested reader is referred to references [10, 11]. In this paper we assume that F is a polynomial in R, G and and consider six polynomial models from degree one to three. The mathematical formulation of each model is reported in Equation 2. B] M3 3 RG1] M3 5 R G BR] M3 6 R G BR, RG1] R G BR, R, G, B,1] M R G BR, R, G, B, RG1] M 3 11 The calibration procedure consists in estimating the coefficients (parameters) of each model, which are expressed as matrices of order 3 3, 3 5, 3 6, 3 8, 3 9 and 3 11 respectively. In order to estimate the unknown parameters we need a set of colour patches of which both the device-dependent (R,B) and deviceindependent (X,Y,Z) colour coordinates are known. To this end we used, in the experiments, the 24 colour patches of the X-Rite Color Checker (from now on ColorChecker-24). From a mathematical standpoint the procedure is standard leastsquare estimation, which can be expressed as follows: arg min M = ( e) (3) R m ij where e, represents the residual (error) of the least-square procedure. In the case of the linear model, for instance, the residual can be expressed in the following way: M (2) e N dist i= 1 L2 = X,Y,Z,R,G,B M i i i i i i 3 3 (4) In the above equation (X i, Y i, Z i ) represent the device-independent coordinates of each of the N colour patches, (R i, G i, B i ) the corresponding device-independent coordinates and dist indicates the Euclidean distance. L 2 3

4 3. Materials and instruments The imaging device we wish to calibrate is composed of a support, a dome illuminator and an industrial camera, as shown in Fig. 1 The support serves as a basement for the dome. Inside the support there is a pocket where the surface to analyse is stowed. A circular series of threaded, self-tapping inserts enables rotation of both illuminator and camera making it possible taking rotated pictures of the specimen. The illuminator is a DL Monster Dome Light produced by Spectrum Illumination. It is composed of a moulded fibreglass hemisphere that reflects multiangle white LED lights onto the field of view. It is designed to provide uniform and diffuse lighting condition on the surface under inspection. A hole at the top of the dome permits the insertion of the optics. The camera is a DFK 31BU03 produced by The Imaging Source. The main characteristics of the camera are: single Sony 1/3 CCD, progressive scan, USB 2.0, C/CS mount, resolution In the experiments it has been equipped with a Computar T2314FICS-3 fixed focal length optics. It is worth mentioning that the camera response is linear, therefore inversion gamma correction is not required. Camera Dome illuminator Support Figure 1. The imaging device For system calibration we used the ColorChecker-24 (2), which provides a set of 24 scientifically prepared natural, chromatic, colored patches in a wide range of colors. Many of the patches represent natural objects, such as human skin, foliage and blue sky. Figure 2. The 24 colour patches of the ColorChecker 4

5 A Minolta Chroma Meter CR-200 has been used both to get the colour coordinates of the ColorChecker patches and to check the results obtained with the calibrated system. The performance of the artificial vision system has been evaluated over a set of 22 samples of industrial materials (Fig. 3) covering Alcantara, leather and plastics. Alcantara 1 0,246 0,231 0, Alcantara 2 0,247 0,316 0, Alcantara 3 0,223 0,239 0, Alcantara 4 0,591 0,602 0, Alcantara 5 0,057 0,054 0, Alcantara 6 0,058 0,053 0, Alcantara 7 0,041 0,047 0, Alcantara 8 0,037 0,044 0, Alcantara 9 0,040 0,046 0, Alcantara 10 0,988 0,552 0, Alcantara 11 0,448 0,198 0, Alcantara 12 0,333 0,061 0, Alcantara 13 0,076 0,057 0, Alcantara 14 0,112 0,081 0, Alcantara 15 0,085 0,065 0, Alcantara 16 0,731 0,595 0, Alcantara 17 0,287 0,206 0, Alcantara 18 0,278 0,209 0, Alcantara 19 0,729 0,684 0, Alcantara 20 0,503 0,455 0, Leather 1 0,352 0,159 0, Plastics 1 0,352 0,159 0, Figure 3. The 22 samples of industrial materials used to test the colour calibration models. Below the names are indicated the uncalibrated RGB coordinates returned by the artificial vision system (first row) and the XYZD65 colour coordinates returned by the Minolta Chroma Meter CR-200 (second row). 4. Experiments The experimental activity consisted in two stages: 1) colour calibration of the imaging device and 2) evaluation of the performance of the colour-calibrated device. 5

6 Following the pattern recognition jargon we refer to the two phases as training and validation, respectively [12]. In the training stage the imaging device has been calibrated using the six polynomial models described in Section 2. The unknown parameters have been determined through least square estimation using the 24 colour patches of the ColorChecker-24 (Fig. 2). This provides 24 3 = 72 equations. Since the number of unknowns ranges from nine for the linear model to 33 for the 3 11 model, the problem is always overconstrained. The device-dependent RGB values of the colour patches have been captured through the artificial vision system and averaged over an area of pixels for each patch. The corresponding device-independent values have been measured through the Minolta CR-200 under the D65 illuminant. In this case the measure is averaged over five points, one at the centre of the specimen and the others at the midpoint of each diagonal. For each polynomial model the unknown parameters have been estimated through a least-square error minimization procedure implemented on the MATLAB platform. The validation stage consisted in estimating, through the artificial vision system, the device-independent colour coordinates of the materials shown in Figure 3 and comparing them with the values obtained through the chroma meter. We refer to the first set of parameters as the estimated colour coordinates, and to the second set as the true colour coordinates. The generalization error of the system is given by the average colour difference, in the CIE L*a*b* space ( E ab ), between the true and the estimated colour coordinates. Calibration residual Average generalization error Colour difference 14,0 12,0 10,0 8,0 6,0 4,0 2,0 0,0 3 x 3 3 x 5 3 x 6 3 x 8 3 x 9 3 x 11 Type of polynomial model Figure 4. Results of the experiments. Figure 4 reports the results of the experiments. Light bars indicate the calibration residual obtained with each polynomial model. This is the average colour difference, in the L*a*b* space, between the true and the estimated colour coordinates of the 24 patches of the colour checker (Fig. 2). Dark bars indicate the average difference among the true and the estimated colour coordinates of the 22 material patches used in the experiment. The results put in evidence an interesting trend. Whereas the calibration error decreases significantly as the complexity of the model increases, 6

7 the same does not occur with the average generalization error. The absolute value of the calibration error shows the same trend obtained by Valous et al. [13]. 5. Conclusions and future work In this work we presented a comparison of different polynomial models for calibration of an artificial vision system composed of illuminator + camera. The results of the experiments suggest interesting conclusions. In terms of general trend it comes out that, whereas the calibration error decreases as the complexity of the model increases, the same does not occur with the generalization error. We believe that a reasonable explanation for this behaviour could be the well-known problem of overfitting [14]: a model typically adapts to training data, and therefore the calibration error is to be regarded to as an optimistic estimate of the generalization error. Besides, it is widely accepted that polynomial models are quite prone to this problem, particularly when their degree increases [15]. As for the absolute values of the generalization error, the results show an average E ab of around 12 units. It is commonly believed that a figure greater than four results in noticeable colour difference [16], therefore the performance of the system is not satisfactory. We believe that this depends both on the camera and calibration patches used in the experiments. Other authors [9] reported generalization errors as low as 2.5 units, but with a three-ccd camera and a higher number of train samples for calibration. This last parameter, in particular, plays a crucial role in the construction of the model, as remarkably pointed out by Jetsu et al. [17], who showed that as the size of the training set becomes larger, the performance of polynomial model improves significantly. This concept seems to be corroborated by the results presented by Heaghen et al. [18]: in an application of polynomial models for colour calibration of an imaging system for use in dermatology, they reported a generalization errors comparable with ours, even with a three-ccd camera, being the system calibrated with the ColorChecker-24. Future work will involve the use of a higher-level, possibly three-ccd camera and wider colour calibration charts, such as, for instance, the Digital ColorChecker S which provides up to 140 colour patches for calibration. An interesting point to clarify is to determine which of the two features (i.e: camera and colour patches) contributes most to the performance of the calibrated system. Aknowledgements This work was partially supported by a joint research programme among Mondial Marmi SpA (Perugia, Italy), Dipartimento Ingegneria Industriale (Università degli Studi di Perugia, Italy) and Ministero dell Istruzione dell Università e della Ricerca under the framework specified by the Ministerial Decree No. 593 of 8 August References [1] D. Lu and Q. Weng, A survey of image classification methods and techniques for improving classification performance, International Journal of Remote Sensing, 28(5), pp , [2] X. Xie, A Review of Recent Advances in Surface Defect Detection using Texture Analysis Techniques, Electronics Letters on Computer Vision and Image Analysis, 7(3), pp. 1-22, [3] E. González, F. Bianconi and A. Fernández, A comparative review of colour features for contentbased image retrieval, Anales de Ingeniería Gráfica, 21, pp. 7-14,

8 [4] M.J. Vhrel and H.J. Trussell, The Mathematics of Color Calibration. Proc. of the IEEE International Conference on Image Processing, Vol. 1, pp , Chicago (USA), October [5] G. Hong, M. R. Luo and P. A. Rhodes, A study of digital camera colorimetric characterization based on polynomial modeling. Color Research and Application, 26, pp [6] R. Schettini, B. Barolo and E. Boldrin, Colorimetric calibration of color scanners by backpropagation Pattern Recognition Letters, 16(10), pp , [7] Y Liu, H. Yu and J. Shi, Camera characterization using back-propagation artificial neural network based on Munsell system, Proc. of SPIE, Vol. 6621, 66210A-5, [8] M. Tong-sheng and S. Hui-liang Colorimetric characterization of imaging device by total color difference minimization, Journal of Zhejiang University SCIENCE A, 7(6), pp , [9] V. Cheung, S. Westland, D. Connah and C. Ripamonti A comparative study of the characterisation of colour cameras by means of neural networks and polynomial transforms, Coloration Technology, 120(1), pp , [10] G. Wyszecki and W.S. Styles, Color Science. Concepts and Methods, Quantitative Data and Formulae. Second Edition, Wiley-Interscience, [11] H.R. Kang, Computational Color Technology, Spie Press, [12] R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classification, John Wiley & Sons, [13] N.A. Valous, F. Mendoza, D. Sun and P. Allen. Colour calibration of a laboratory computer vision system for quality evaluation of pre-sliced hams, Meat Science, 81(1), pp , [14] T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning. Data Mining, Inference, and Prediction, Springer, [15] C.M. Bishop, Pattern Recognition and Machine Learning, Springer, [16] D. Ploumidis, Color Perception, Available online < site visited on Feb 7, [17] T. Jetsu, V. Heikkinen, J. Parkkinen, M. Hauta-Kasari, B. Martinkauppi, S. Deok Lee, H. Wook Ok and C. Yeong Kim, Color Calibration of Digital Camera Using Polynomial Transformation, Proc. of the Third European Conference on Color in Graphics, Imaging and Vision, Leeds (UK), pp , [18] Y.V. Haeghen, J.M.A.D. Naeyaert, I. Lemahieu and W. Philips An Imaging System with Calibrated Color Image Acquisition for Use in Dermatology, IEEE Transactions on Medical Imaging, 19(7), pp ,