Recovering spectral data from digital prints with an RGB camera using multi-exposure method

Transcription

1 Recoverng spectral data from dgtal prnts wth an RGB camera usng mult-exposure method Mkko Nuutnen, Prkko Ottnen; Department of Meda Technology, Aalto Unversty School of Scence and Technology; Espoo, Fnland Abstract The spectral data of a surface can be reconstructed from RGB camera response when reflectance spectrum of the surface and radance spectrum of the lght source are smooth enough. One factor that lowers the reconstructon performance of consumer level cameras s lmted dynamc range. Detals or ntensty levels n low and hgh lumnance regons can not be relably detected from a sngle mage. Wth multple exposures t s possble to capture hgh dynamc range mages. The am of ths study was to apply a mult-exposure method for spectral data reconstructon of prnts and fnd how accuracy t s. Our ntended applcaton area s prnted mage dgtzaton for mage qualty calculatons. We measured the ntensty levels and recovered spectral data of prnted samples by takng two mages usng dfferent exposure tmes. Based on the results mult-exposure method mproves the accuracy of spectral data reconstructon of prnt samples compared to tradtonal methods and s the preferred choce for prnted mage dgtzaton process. Introducton Spectral data provde the most useful nformaton for color reproducton measurements. The spectral data of a sample can be measured by spectroradometer. Spectroradometer measurements are precse but tme-consumng and pont-wse. Pont-wse measurements are not applcable for some applcatons. For example spatal varatons of spectral data n a natural scene cannot be measured usng a pont-wse devce. Respectvely spectral nformaton of a prnted photograph s dffcult to measure usng a pont-wse devce. Our ntended applcaton area s prnted mage dgtzaton. Ths dgtzaton process has been developed for our prnted mage qualty calculaton system [1]. The dea n our prnted mage qualty calculaton system s to dgtze the prnted mage and calculate the qualty of prnted mage by comparng the features of orgnal dgtal mage and prnted mages. When prnted mage qualty s calculated the accuracy and spatal samplng frequency of colour measurement should be hgh enough. Earler we have used (RGB to XYZ) charactersed dgtal camera for spatal samplng of prnted mages [1]. In some other studes [],[3] reflectve scanners have been used for spatal samplng. There are plenty of earler studes where a RGB camera (spatally samplng devce) s used as a spectral measurement devce. Ths s possble because the spectral data of natural scenes, objects and llumnaton are smooth. It s a known fact that the dynamc range of consumer dgtal cameras s the lmtng factor of ther magng performance. Often consumer dgtal cameras are not capable of detectng, n a sngle mage, the entre ntensty range httng the sensor. Relable detecton s possble only f more than one exposure s used. In ths we descrbe our experments n whch we nvestgated the performance of a mult-exposure method for spectral data reconstructon of prnted samples. In prevous studes the reconstructon accuracy has been mproved by ncreasng the number of response channels usng dfferent flter combnatons. Valero et. al. [4] measured that two or three flters mproved sgnfcantly the reconstructon performance. In some studes the number of channels was ncreased by producng mages under spectrally dfferent lght sources [5]. The prncpal component analyss (PCA) s the tradtonal lnear method for reconstructon of the spectral data from a camera response [5] or to analyse the dmensonalty of a sample spectra set [6]. Other base functon methods have also been studed. L and Berns [7] studed the performance of ICA analyss compared to the PCA. Mansour et. al. [8] studed the performance of Fourer and Wavelet bases. The second lnear method to estmate the spectral data from camera response s based on the camera s flter functons. Camera s a lnear system whch can be descrbed by the Equaton (1): C= RW (1) where vector C s response of the camera, vector R s spectral data of the sample and matrx W contans flter functons for the camera s channels [9]. The flter functons can be measured usng a monochromator or they can be estmated usng dfferent constrants [10] or parametrc functons [1]. Spectral data can be estmated also wthout a need for bass or flter functons [9]. For example the spectra, R, can be drectly estmated from the RGB values, C, usng a transform matrx M (Equaton ). Soll et. al. [9] solved transform matrx M usng Moore-Penrose pseudo-nverse of R. R = CM () The accuracy of the methods has been further mproved usng dfferent local weghts. Agahan et. al. [1] used a weghted prncpal component analyss for reconstructon of reflectance spectra. The method ncreased the nfluence of tranng samples that were close to the testng sample when base functons were formed. The closeness metrc whch was used was the colour dfference between tranng and testng samples. Shen and Xn [13] assumed that tranng samples u closer to a testng sample u are usually more relable and thus should contrbute more to the estmaton of the transformaton matrx W shen. They calculated weghts α for u as: 1 = ( ) 3 / 1/ T 1 α π UU exp ( u u) UU ( u u) (3) where UU s covarance matrx of u. By ncorporatng the weghtng, the mean square error between the measured and the predcted spectra can be formulated as:

2 1 L J = W L = 1 shen u α r α (4) In ths study we use the method of Shen and Xn [13] to test the performance of mult-exposure technology. We measure spectral data and camera response data for the prnted samples. Then we evaluate the colormetrc and spectral performance of mult-exposure method compared to the tradtonal sngleexposure method. Spectral data For performance evaluaton of mult-exposure method we used the 180 colour patches of Gretag Macbeth DC test target (teachng samples) and the 4 colour patches of Gretag Macbeth CC test target (testng samples). The dgtal test target mages were prnted on sx grades. Propertes of these s are lsted n Table 1. Papers P1 and P were copy s and s P3 - P6 nk-jet s. The gloss values for s P3 and P4 were hgh compared to the other s. Gloss values for s P3 and P4 were 75 and 77.1 and for s P1, P, P5 and P6 were We used Epson Stylus Pro 3800 nk-jet prnter. Paper-specfc ICC profles were determned n Proflemaker Pro software before prntng. Table 1. Paper types, grammage and gloss Paper Paper type Grammage (g/m ) Gloss (GU) P1 copy P copy P3 nk-jet P4 nk-jet P5 nk-jet P6 nk-jet The spectral data of the prnted samples were measured by the Photo Research PR-670 spectroradometer. The measurement envronment ncluded two halogen lamps. The lumnance level of a whte reflector on the measurement plane was 800 cd/m. The colour temperature of the lght sources was 3097 K. The structure of measurement envronment s shown n Fgure 1. The prnted test samples were photographed by Canon EOS 5D camera wth Canon EF 50 mm/.5 lens. The aperture of the lens was F9. The photometrc dstorton of the lens and possble uneven llumnaton on the plane was compensated by the correcton matrx. The correcton matrx ncluded pxelwse correcton factors for the camera mage. The factor values were derved from the low-pass fltered mage of the unform grey-surface that was photographed on the measurement plane. Fgure shows an ntensty mage of correcton factor matrx for green channel n our measurement envronment. Compensaton of uneven lghtng because of optc s a mportant step for dgtzaton process. Correcton factors can be even more than 1,5. Fgure 1. The measurement envronment ncludes two halogen lamps (3097 K), measurement plane for samples and measurement devce holder perpendcular to the measurement plane Fgure. Intensty mage of correcton factor matrx for green channel n our measurement envronment Mult-exposure method Our mult-exposure method produces an RGB ntensty mage by selectng ntensty values from two mages, I 1 and I, produced usng dfferent exposure tmes (1/40s and 1/80s). The purpose of the longer exposure (1/40s) mage I 1 was to detect the lower ntensty values of the samples and the purpose of the shorter exposure (1/80s) mage I was to detect the hgher ntensty values. The selected exposure tmes were based on the response of exposure 1/80 that produced the optmal sngleexposure mage n our lghtng envronment. Ths optmum mage was defned so that the response values dd not saturate. The threshold for selectng the response of exposure 1/40 or 1/80 was ntensty value of 0.5 (scale 0-1). If ntensty value of exposure 1/40 was more than 0.5, the ntensty of exposure 1/80 was selected for mult-exposure mage and multpled by factor of. If ntensty of exposure 1/40 was smaller than 0.5, the ntensty of exposure 1/40 was selected. Fgure 3 shows a flow chart of process for estmatng spectra pˆ for pxel poston (,j) where I ME s mult-exposure mage wth sze of N x M pxels, I C s pxel-wse correcton matrx, I MEc s corrected mult-exposure mage and W s transformaton matrx solved by the method proposed by Shen and Xn [13].

3 Fgure 4. Camera responses for exposures 1/80s and 1/40s n scale cd/m Fgure 3. Flow chart for spectra, pˆ, reconstructon n pxel poston, (,j), where I 1 s long-exposure mage (1/40s) and I s short-exposure mage (1/80s), I ME s mult-exposure mage, I C s pxel-wse correcton matrx, I MEc s corrected mult-exposure mage and W s transformaton matrx solved by the method proposed by Shen and Xn [13] Table shows the mnmum and maxmum lumnance values (cd/m ) for the samples n our lghtng envronment. Mnmum lumnance values were measured from full-tone black patches and maxmum lumnance values were measured from the plan s. Fgure 4 shows the response of the camera wth exposures of 1/40s and 1/80s. The response was measured from the transparency test target (Image Engneerng TE41) usng an ntegratng sphere (Image Engneerng LE6). The llumnaton level of the ntegratng sphere was set to encompass the llumnaton scale n our lghtng envronment. Fgure 5 shows the response of the camera n scale cd/m. The added dotted lnes show the mnmum lumnance levels for the samples. The added lnes show the lnearty level of the responses. Table. Mnmum and maxmum lumnance of the samples n test envronment Paper Mn lumnance (cd/m ) Max lumnance (cd/m ) P P 7 49 P P P P Fgure 5. Camera responses for exposures 1/80s and 1/40s n scale cd/m Based on Table the lght absorpton capablty of depends strongly on the grade. The mnmum lumnance levels of the s P1 and P were fvefold compared to the s P3 and P4. The mnmum lumnance levels of the s P5 and P6 were threefold compared to the s P3 and P4. Based on Fgure 5 the response of exposure 1/80 changes to be nonlnear after the lumnance of 100 cd/m. Respectvely, the response of exposure 1/40 starts to become nonlnear after the lumnance of 50 cd/m. Based on ths the usage of the exposure 1/40 expands the relable detecton area at low lumnance levels.

4 Performance metrcs for spectral reconstructon Two measures were used for the performance assessment of the mult-exposure method: the spectral goodness-of-ft coeffcent (GFC) [1] and the colormetrc CIEDE000 color dfference metrc [11]. GFC s defned n Equaton (5): GFC = ( λ= 400 pˆ( pˆ( p( λ= 400 λ= 400 ) ( 1/ 700 p( ) 1/ *100% where pˆ are the reconstructed spectra and p measured spectra. Values range from 0 to 100 %. Valero [4] used the defntons: when GFC (%) > 99.5 % reconstructon s acceptable and when GFC (%) reconstructon s almost-exact ft. The second metrc was CIEDE000 color dfference: L C H C H (6) CIEDE000= R T kls L kc SC kh SH kc SC kh S H where L s the lghtness dfference between samples, C s the chroma dfference between samples and H s hue dfference between two samples. Complete equatons and descrptons for varables L, C, H, S L, S C and S H can be found n [11]. CIEDE000 was calculated wth reference to the spectra of the whte patch n our lghtng envronment usng default parameter values of k L, k C and k H. (5) Table 5. Mean and maxmum CIEDE000 colour error values for dfferent grades usng sngle-exposure (SE) and mult-exposure (ME) methods (testng samples) Mean CIEDE000 Max CIEDE000 P P P P P P Table 6. Mean and mnmum GFC values for dfferent grades usng sngle-exposure (SE) and mult-exposure (ME) methods (testng samples) Mean GFC (%) Mn GFC (%) P P P P P P Spectral reconstructon results Table 3 shows the CIEDE000 color error and Table 4 shows the GFC values of the teachng samples for the sngleexposure and mult-exposure methods. Table 5 and Table 6 show the values of testng samples. Fgure 6 compares the mean and Fgure 7 compares the maxmum CIEDE000 color error values between sngle- and mult-exposure methods for the testng samples. Table 3. Mean and maxmum CIEDE000 colour error values for dfferent grades usng sngle-exposure (SE) and mult-exposure (ME) methods (teachng samples) Mean CIEDE000 Max CIEDE000 P P P P P P Table 4. Mean and mnmum GFC values for dfferent grades usng sngle-exposure (SE) and mult-exposure (ME) methods (teachng samples) Mean GFC (%) Mn GFC (%) P P P P P P Fgure 6. CIEDE000 mean colour error values of dfferent grades usng sngle-exposure and mult-exposure methods for testng samples Fgure 7. CIEDE000 maxmum colour error values of dfferent grades usng sngle-exposure and mult-exposure methods for testng samples

5 Based on the results the performance of the mult-exposure method s hgher than the performance of the sngle-exposure method. In the case of the testng samples all values of the mult-exposure method were better than the values of the sngle-exposure method, excludng mnmum GFC value of P4. Mean GFC values were between 99.83% 99.97% wth mult-exposure method and 99.83% % wth sngleexposure method for testng samples. Based on Valero [4] the average level of reconstructon s acceptable wth both multexposure and sngle-exposure methods. Mnmum GFC values were between 97.69% wth mult-exposure method and 97.39% % wth sngle-exposure method. Based on Valero [4] the mnmum GFC value wth both mult-exposure and sngle-exposure methods s under acceptable level for some type. Dfferences between mean CIEDE000 colour error values wth the sngle-exposure and mult-exposure methods were between unts. Dfferences between maxmum CIEDE000 colour error values were between Based on the mean CIEDE000 colour error for testng samples the mult-exposure method mproved most the performance of P3. Maxmum CIEDE000 colour error wth the sngleexposure method for P3 was 6.90 unts. The colour patch was the darkest neutral (patch number 4) n the Gretag Macbeth CC test target. CIEDE000 colour error for ths patch wth the mult-exposure method was.60 unts. Fgure 8 shows the measured spectra and the reconstructed spectra of ths patch. Based on Fgure 8 the mult-exposure method mproves the reconstructon accuracy mostly n the area of low- and mdwavelengths. Respectvely, the reconstructed spectra of the sngle- and mult-exposure method equal each other n the area of long-wavelengths. The second problematc patch wth the sngle-exposure method was the dark brown (patch number 1) n Gretag Macbeth CC test target. Fgure 9 shows the measured spectra and the reconstructed spectra of ths colour patch for P5. When method was the sngle-exposure the reconstructed spectra dffered from the measured spectra mostly n the area of long-wavelengths. The reconstructon accuracy of the multexposure method was hgh. Fgure 9. Spectral radances for dark brown patch (patch number 1) n Gretag Macbeth CC test target prnted on P5 Fgure 10 shows the CIEDE000 colour error values of P3 and Fgure 11 shows CIEDE000 colour error values of P5 for the testng samples (Gretag Macbeth CC target) ordered n ascendng order by measured lumnance values of the patches. Based on Fgure 10 and Fgure 11 the multexposure method mproved mostly the reconstructon performance of the dark patches. The one problematc patch for the mult-exposure method was patch number 3. Fgure 1 shows the measured spectra and reconstructed spectra for ths lght blue colour patch of P3. Fgure 10. CIEDE000 colour error values for p3 ordered n ascendng order by measured lumnance of patches Fgure 8. Spectral radances for dark neutral patch (patch number 4) n Gretag Macbeth CC test target prnted on P3 Fgure 11. CIEDE000 colour error values for p5 ordered n ascendng order by measured lumnance of patches

6 Fgure 1. Spectral radances for patch number 3 prnted on P3 Conclusons Based on the results the mult-exposure method mproves the reconstructon performance. In partcular the performance for reconstructon of dark colour patches wth the Multexposure method was hgh compared to the performance of the sngle-exposure method. Ths was an expected result. The mult-exposure method detects the lower lumnance levels more lnearly than the sngle-exposure method does. Mult-exposure method s preferred choce for the module of colour measurng n prnted mage dgtzaton system. Wth specal hgh dynamc range cameras the detecton s lnear over a wde lumnance range. In that case the multexposure method would be worthless. Anyway the camera used n our study s a hgh-end product and mult-exposure method ncreased ts performance even when the applcaton specfc requrements were pretty low compared to many other applcatons. For example the lumnance range n natural scenes s much wder than n reflectance prnts. References [1] M. Nuutnen, R. Halonen, T. Lest, P. Ottnen, Reducedreference qualty metrcs for measurng the mage qualty of dgtally prnted natural mages, Proc. SPIE 759, 7590I (010). [] T. Eerola, J.-K. Kamaranen, T. Lest, R. Halonen, L. Lensu, H. Kälvänen, P. Ottnen, G. Nyman, Fndng Best Measurable Quanttes for Predctng Human Vsual Qualty Experence, Proc. IEEE Internatonal Conference on Systems, Man, and Cybernetcs, p (008). [3] R. Halonen, T. Lest, P. Ottnen, The Influence of Image Content and Paper Grade on Qualty Attrbutes Computed from Prnted Natural Images, Proc. NIP 4, p (008). [4] E.M. Valero, J.L. Neves, S.M.C. Nascment, K. Amano, D.H. Foster, Recoverng Spectral Data from Natural Scenes wth an RGB Dgtal Camera and Colored Flters, Color Research and Applcaton, 3, 5 (007). [5] F.H. Ima, R.S. Berns, Spectral Estmaton Usng Trchromatc Dgtal Cameras, Proc. Int. Symp. on Multspectral Imagng and Color reproducton for Dgtal Archves, p (1999). [6] S.M.C. Nascmento, D.H. Foster, K. Amano, Psychophyscal estmates of the number of spectral-reflectance bass functons needed to reproduce natural scenes, J. Opt. Soc. AM. A,, 6 (005). [7] Z. L, R.S. Berns, Evaluaton of Lnear Models for Spectral Reflectance Dmensonalty Reducton, Proc. ICIS, p (006). [8] A. Mansour, T. Slwa, J.Y. Hardeberg, Y. Vosn, Representaton and Estmaton of Spectral Reflectances Usng Projecton on PCA and Wavelet Bases, Color research and applcaton, 33, 6 (008). [9] M. Soll, M. Andersson, R. Lenz, B. Kruse, Color Measurements wth a Consumer Dgtal Camera Usng Spectral Estmaton Technques, Proc. 14 th Scandnavan Conference on Image Analyss, vol of Lecturel notes n Computer Scence, p (005). [10] K. Barnard, B. Funt, Camera Characterzaton for Color Research, Color research and applcaton, 7, 3 (00). [11] M. Thomson, S. Westland, Colour-Imager Characterzaton by Parametrc Fttng of Sensor responses, Color research and applcaton, 6, 6 (001). [1] F. Agahan, S.A. Amrshah, S.H. Amrshah, Reconstructon of Reflectance Spectra Usng Weghted Prncpal Component analyss, Color research and applcaton, 33, 5 (008). [13] H.-L. Shen, J.H. Xn, Colormetrc and Spectral Characterzaton of a Color Scanner Usng Local Statstc, Journal of Imagng Scence and Technology, 48, 4 (004). [14] S. Westland, C. Rpamont, Computatonal Color Scence usng MATLAB, (John Wley & Sons, Ltd, England 004) 07 p. Author Bography Mkko Nuutnen receved the M. Sc. degree n automaton and system technology n 004 and Lc. Sc. (tech) n 007 from the Helsnk Unversty of Technology. He s currently a Ph.D. student at the depertment of Meda Technology at Aalto Unversty School of Scence and Technology. Hm research nterests nclude colour mage processng, mage devce measurements and mage qualty measurements..