A Noise Analysis for Recovering Reflectances of the Objects Being Imaged

Transcription

1 A Nose Analyss for Recoverng Reflectances of the Objects Beng Imaged September 0 Mkya HIRONAGA

2

3 A Nose Analyss for Recoverng Reflectances of the Objects Beng Imaged 画像入力による分光反射率復元に対するノイズの影響解析 September 0 Graduate School of Engneerng Osaka Cty Unversty Mkya HIRONAGA 広永美喜也

4 Preface Colors of objects vary under varous llumnants. Even under the exact same llumnants, pctures taken by dfferent cameras are not the same because pctures are always under the nfluence of the llumnants and the senstvtes of the sensors. Thus, recoverng the spectral reflectances vector r by use of the sensor response vector p s mportant for a reproducton of accurate colors or a stable recognton of colored objects, etc. The sensor response p s determned by p=slr+e, where S s the matrx representng the spectral senstvtes of the mage sensors, L s the matrx representng the spectral power dstrbutons of the llumnants and e s the addtve nose vector. The recoverng reflectances r from the sensor response p s an nverse problem and largely affected by the nose e. To recover accurate reflectances by applyng a reconstructon matrx W, the study of the nose s requred. Shmano proposed a model estmatng the nose varance n an mage acquston devce based on the Wener estmaton. He also derved an evaluaton model for the qualty of an mage acquston devce. In ths thess, Shmano s model s examned and extended to an comprehensve model and a robust frame work s proposed for estmatng the nose varance and analyzng the effect of the nose to the mage acquston. Ths thess s organzed as follows; In chapter, the background and the purpose of the study are brefly descrbed. In chapter, the spectral evaluaton model for the mage acquston devce based on the Wener model s examned and the spectral measure of the qualty (Qr) of the mage acquston devce for the recovery of the reflectances s studed and t s shown that the Qr can be appled to other reconstructon models wth expermental results and mathematcal proofs.

5 In chapter 3, t s shown that the above mentoned model stands n the subspace projected by the senstvtes of the human vson. The colormetrc measure of the qualty (Qc) s also appled to multple reconstructon models and ts nose robustness for the quantzaton error and samplng error s examned Also the concept of the NIF (Nose Influence Factor) s proposed and the reason of the nose senstvty of an mage acquston devce s analyzed. In chapter 4, the model estmatng the nose varance based on the Wener model s extended and modfed to a comprehensve model wth a reconstructon matrx W and t s appled to the reconstructon matrces of the Wener and the lnear models. The accuracy of the estmaton for the nose varance by the proposed model s confrmed by experments. The ncreases n the mean square errors (MSE) of the reconstructon for the reflectances are examned n the Wener and the lnear models and t s shown that the effect of the regularzaton for the ncrease n the MSE s well descrbed by the proposed model. In chapter 5, the overall dscussons and conclusons are summarzed. Keywords: spectral reflectances, recovery models, colormetry, evaluaton, spectral dscrmnaton, nose n magng systems

6 Contents Overall Introducton Reference s 6 Evaluatng a qualty of an mage acquston devce amed at the reconstructon of spectral reflectances by the use of the recovery models 9. Introducton 0. Models for the Reconstructon of Spectral Reflectances.. Wener Estmaton Usng Estmated Nose Varance.. Multple Regresson Analyss 5..3 Ima-Berns Model 6.3 Expermental Procedures 6.4 Results and Dscussons 9.5 Conclusons Reference s 8 3 Nose robustness of a colormetrc evaluaton model for mage acquston devces wth dfferent characterzaton models Introducton Models for the Colormetrc Evaluaton Wener Estmaton Usng Estmated Nose Varance Applcaton of the Multple Regresson Analyss to Fundamental Vector Evaluaton Applcaton of the Ima-Berns Model to Fundamental Vector Evaluaton Expermental Procedures Results and Dscussons MSE and Qc ( σ ) Effect of the Nose to the Relaton between MSE and Qc ( σ ) Effect of the Samplng Bts to MSE Effect of the Samplng Bts to NIF 5

7 3.4.5 Reason of the Nose Senstvty Error and Qualty Estmaton Model Conclusons 54 Reference s 55 4 Influence of the nose present n an mage acquston system on the accuracy of recovered spectral reflectances Introducton Models Prevous Models to Separate the MSE based on the Wener Estmaton Multple Regresson Analyss Ima-Berns Model Lnear Model Proposed Model Expermental Procedures Results and Dscussons Conclusons 78 Reference s 79 5 Overall Conclusons 83 v

8 Lst of Fgures. Concepts of spectral and colormetrc; vectors projected onto human vsual subspace (HVSS).. Spectral senstvtes of the sensors of the camera.. Spectral power dstrbuton of the llumnaton..3 MSEs of the recovered spectral reflectances by the Wener, Regresson, and Ima-Berns method for the GretagMacbeth ColorChecker (CC) and the Kodak Q60R (KK) are plotted as a functon of Qr ( σ )..4 MSEs of the recovered spectral reflectances by the Wener, Regresson, and Ima-Berns method for the GretagMacbeth ColorChecker (CC) and the Kodak Q60R (KK) are plotted as a functon of Qr ( 0), whch s the value of Qr ( σ ) when the estmated nose varance s zero. Ths s the case wthout consderaton for the nose..5 (a) Typcal example of the recovered spectral reflectance of the color red at a large Qr ( σ ) (Qr ( σ ) = ). (b) Color reproducton of the GretagMacbeth ColorChecker by the recovered spectral reflectance at Qr ( σ ) = (a) Typcal example of the recovered spectral reflectance of the color red at a mddle Qr ( σ ) (Qr ( σ ) = ). (b) Color reproducton of the GretagMacbeth ColorChecker by the recovered spectral reflectance at Qr ( σ ) = (a) Typcal example of the recovered spectral reflectance of the color red at a small Qr ( σ ) (Qr ( σ ) = ). (b) Color reproducton of the GretagMacbeth ColorChecker by the recovered spectral reflectance at Qr ( σ ) = Spectral senstvtes of the sensors of the camera sampled at nm. 3. Spectral power dstrbuton of the llumnaton sampled at nm. v

9 3.3 MSEs of the recovered fundamental vectors by the Wener, Regresson, and Ima-Berns model for the GretagMacbeth ColorChecker (CC) and the Kodak Q60R (KK) are plotted as a functon of Qc ( σ ). 3.4 MSEs of the recovered fundamental vectors by the Wener, Regresson, and Ima-Berns model for GretagMacbeth ColorChecker (CC) are plotted as a functon of Qc ( σ ) wth (a)8-bt mage and 0-nm samplng ntervals, (b)8-bt mage and 0-nm samplng ntervals, (c)6-bt mage and 0-nm samplng ntervals, and (d)6-bt mage and 0-nm samplng ntervals 3.5 MSEs of the recovered fundamental vectors by the Wener, Regresson, and Ima-Berns model for GretagMacbeth ColorChecker (CC) are plotted as a functon of Qc ( 0) wth 6-bt mage and 0-nm samplng ntervals (MSE ( 0) s are plotted for the Wener model). 3.6 MSEs of the recovered fundamental vectors by the Wener, Regresson, and Ima-Berns model for GretagMacbeth ColorChecker (CC) and the theoretcal MSE estmated wth Emax(- Qc ( σ ) ) wth 0-nm samplng ntervals are plotted as a functon of the samplng bts nm NIF(Nose Influence Factor) for three sensor sets are plotted as a functon of the samplng bts. 3.8 Nose varance and the square of sngular values for the sensor set 34 and 47 wth 8-bt and 6-bt mages, 0-nm samplng ntervals. 4. Spectral senstvtes of each sensor. 4. Spectral power dstrbuton of the recordng llumnaton. 4.3 Relaton between the ncrease n the MSE by the nose estmated by the prevous model ( MSE Wener ) and the MSE nose by the Wener, Regresson, Ima-Berns and lnear model for the GretagMacbeth ColorChecker. v

10 4.4 (a) Relaton between the ncrease n the MSE ( MSE ) by the lnear model and the MSE NOISE R ( σˆ = κ ) wth the nose varance estmated wth the prevous model for the GretagMacbeth ColorChecker. 4.4 (b) Relaton between the ncrease n the MSE ( MSE ) by the lnear model and the MSE NOISE R ( σˆ = κ W ) wth the nose varance estmated wth the reconstructon matrx W for the GretagMacbeth ColorChecker. 4.5 Relaton between the ncrease n the MSE ( MSE ) and MSE NOISE by the Wener and the lnear model. 4.6 Relaton between the ncrease n the MSE ( MSE R reconstructon matrx W ( = κ ) or the estmated nose varance. ) and the Nose Senstvty for the 4.7 Relaton between the ncrease n the MSE ( MSEp ) by the Wener, regresson, Ima-Berns and lnear model and the MSE NOISE ( σˆ R = κ reconstructon matrx W for the GretagMacbeth ColorChecker. W ) wth the nose varance estmated wth the v

11 Lst of Tables. Maxmum and mnmum Qr ( σ ) for each number of sensors of the Macbeth ColorChecker. 4. Estmated system nose varance and the MSE by the Wener model wth each estmated system nose varance. v

12 Chapter Overall Introducton Ths thess presents a framework for recoverng the spectral reflectances of objects beng maged by an mage acquston system. The proposed work estmates and analyzes the nose n the mage acquston devce and recovers the reflectances of objects accurately. Ths chapter addresses the overall ntroducton of ths thess. The purpose, the background and the sketch of ths thess are also descrbed n ths chapter.

13 The vsual nformaton plays an mportant role for a human beng to recognze envronments. The vsual nformaton manly conssts of shapes and colors. Colors of objects are dependent on the llumnant, the surface of the object, and the senstvty of the sensors. The llumnant and the senstvty of the human vsual system may vary by crcumstances but the spectral reflectances of objects represent the physcal propertes of objects. Therefore the recovery of spectral reflectances of objects s mportant not only to reproduce color mages under varous llumnants [-3] but also to study computatonal color vson [4-6] and color constancy [7-9]. Several models have been proposed to recover the spectral reflectances vector r by use of the sensor response vector p by applyng a reconstructon matrx. There are manly two types of these models, the frst s the Wener estmaton, whch mnmzes the mean square errors (MSEs) between the recovered and the measured spectral reflectances [0-], and the second s the fnte-dmensonal lnear models of spectral reflectances [3-6]. The spectral senstvtes of the mage sensors S, the spectral power dstrbutons of the llumnants L and the learnng samples of the spectral reflectances vector r whch are smlar to those of the maged objects are requred for recoverng the spectral reflectances by these two models because a sensor response p s determned by p=slr+e, where e s an addtve nose vector. Snce t s dffcult to measure these spectral characterstcs, modfcatons of these models have been proposed that do not use pror knowledge of the spectral senstvtes of a set of sensors and the spectral power dstrbuton of the llumnant. The modfcaton of the Wener estmaton uses the regresson analyss [7] between the known spectral reflectances and the correspondng sensor responses [3,8-], and ths model s called the pseudonverse transformaton or the regresson model [8,3]. Another modfed lnear model also uses the regresson analyss between the weght column vectors for the orthonormal bass vectors to represent known spectral reflectances and the correspondng sensor responses [0,4]; n ths model the bass vectors are usually derved by the prncpal

14 component analyss of spectral the reflectances or derved by sngular values decomposton (SVD) [5]. The modfed model s called the Ima-Berns model [6,7]. The recovery of the spectral reflectances vector r from the sensor response p s an nverse problem. Thus, the accuracy of the estmates depends largely on the nose varance used n the Wener estmaton. Recently Shmano proposed a new model to estmate the nose varance of an mage acquston system and also showed that the spectral reflectances of objects beng maged are recovered accurately by usng the Wener estmaton through the use of the mage data from a multspectral camera wthout pror knowledge of the spectral reflectance of the objects and the nose present n the mage acquston system [8,9]. He also proposed an evaluaton model [30] based on the Wener estmaton to measure the accuracy of the estmaton of the reflectances by the mage acquston devce. In multspectral color scence, there are two methods for computng characterstcs of colors. The one s spectral and the other s colormetrc. In spectral color processng, every nformaton of color stmulus obtaned from the nput devce s used, on the other hand n colormetrc color processng, color stmul s dvded nto two parts, the fundamental and the resdual [3]. The fundamental s a projecton of a color stmulus onto the human vsual subspace (HVSS) spanned by the senstvtes of the cones of the retna and evokes color sensaton to a human vsual system. The resdual s an orthogonal part of the color stmulus and evokes no sensaton to a human vsual system. The concept for the spectral and the colormetrc s shown n Fg... Shmano also showed that hs model works well both n spectral and colormetrc cases. [3,33]. 3

15 Vector; Spectral r MSE; Spectral Fundamental Vector; Colormetrc rˆ MSE; Colormetrc P v r P v rˆ HVSS Fg... Concepts of spectral and colormetrc; vectors projected onto human vsual subspace (HVSS). Ths thess focuses on recoverng the reflectances of objects by use of mage acquston devces and on the effect of the nose to the accuracy of the recovery. Ths thess addresses that () the above mentoned spectral evaluaton model based on the Wener estmaton can be appled to other reflectance recovery models, () the above mentoned colormetrc evaluaton model can be appled to other reflectance recovery models and the evaluaton model s nose robust and the model stands n the case as nosy as 0nm samplng nterval and 6bt samplng bts and (3) the model to estmate the nose varance of an mage acquston system s extended to a comprehensve model and t s appled not only to the Wener model but also to the lnear model. Ths thess s organzed as follows. In chapter, the spectral evaluaton model for the mage acquston devce based on the Wener model s examned and the spectral measure of the 4

16 qualty (Qr) of the mage acquston devce for the recovery of the reflectances s studed and t s shown that the Qr can be appled to other reconstructon models wth expermental results and mathematcal proofs [34]. In chapter 3, t s shown that the above mentoned model stands n the subspace projected by the senstvtes of the human vson (HVSS). The colormetrc measure of the qualty (Qc) s also appled to multple reconstructon models and ts nose robustness for the quantzaton error and samplng error s examned Also the concept of the NIF (Nose Influence Factor) s proposed and the reason of the nose senstvty s analyzed [35]. In chapter 4, the model estmatng the nose varance based on the Wener model s extended and modfed to a comprehensve model wth a reconstructon matrx W and t was appled to the reconstructon matrces of the Wener and the lnear models. The accuracy of the estmaton for the nose varance by the proposed model s confrmed by experments. The ncreases n the mean square errors (MSE) of the reconstructon for the reflectances are examned n the Wener and the lnear models and t s shown that the effect of the regularzaton for the ncrease n the MSE s well descrbed by the proposed model. Also the mathematcal proof of the equvalence of the proposed model to the exstng model n some condtons s descrbed [36]. In chapter 5, the overall dscussons and conclusons are summarzed. 5

17 References. M. D. Farchld, Color Appearance Models (Addson-Wesley, 997).. Y. Myake and Y. Yokoyama, Obtanng and reproducton of accurate color mages based on human percepton, Proc. SPIE 3300, (998). 3. K. Martnez, J. Cuptt, D. Saunders, and R. Pllay, Ten years of art magng research, Proc. IEEE 90, 8.4 (00). 4. H. C. Lee, E. J. Breneman, and C. P. Schulte, Modelng lght reflecton for computer color vson, IEEE Trans. Pattern Anal. Mach. Intell., (990). 5. G. Healey and D. Slater, Global color constancy: recognton of objects by use of llumnaton-nvarant propertes of color dstrbutons, J. Opt. Soc. Am. A, (994). 6. G. J. Klnker, S. A. Shafer, and T. Kanade, A physcal approach to color mage understandng, Int. J. Comput. Vs. 4, 7.38 (990). 7. L. T. Maloney and B. A. Wandell, Color constancy: a method for recoverng surface spectral reflectance, J. Opt. Soc. Am. A 3, 9.33 (986). 8. J. Ho, B. V. Funt, and M. S. Drew, Separatng a color sgnal nto llumnaton and surface reflectance components: theory and applcatons, IEEE Trans. Pattern Anal. Mach. Intell., (990). 9. G. Iverson and M. D Zumura, Crtera for color constancy n trchromatc lnear models, J. Opt. Soc. Am. A, (994). 0. A. Rosenfeld and A. C. Kak, Dgtal Pcture Processng, nd ed. (Academc, 98).. G. Sharma and H. J. Trussell, Fgures of mert for color scanners, IEEE Trans. Image Process. 6, (997).. H. Hanesh, T. Hasegawa, N. A. Hoso, Y. Yokoyama, N. Tsumura, and Y. Myake, System desgn for accurately estmatng the spectral reflectance of art pantngs, Appl. Opt. 39, (000). 3. J. Cohen, Dependency of spectral reflectance curves of the Munsell color chps, Psychon. Sc., (964). 6

18 4. L. T. Maloney, Evaluaton of lnear models of surface spectral reflectance wth small numbers of parameters, J. Opt. Soc. Am. A 3, (986). 5. J. P. S. Parkknen, J. Hallkanen, and T. Jaaskelanen, Characterstc spectra of Munsell colors, J. Opt. Soc. Am. A 6, 38.3 (989). 6. J. L. Dannemller, Spectral reflectance of natural objects: how many bass functons are necessary? J. Opt. Soc. Am. A 9, (99). 7. A. A. Aff and S. P. Azen, Statstcal Analyss, (Academc, 97), Chap Y. Zhao, L. A. Tapln, M. Nezamabad, and R. S. Berns, Usng the matrx R method for spectral mage archves, n Proceedngs of The 0th Congress of the Internatonal Colour Assocaton(AIC 5) (Internatonal Colour Assocaton, 005), pp H. L. Shen and H. H. Xn, Spectral characterzaton of a color scanner based on optmzed adaptve estmaton, J. Opt. Soc. Am. A 3, (006). 0. D. Connah, J. Y. Hardeberg, and S. Westland, Comparson of lnear spectral reconstructon methods for multspectral magng, Proceedngs of IEEE s Internatonal Conference on Image Processng (IEEE, 004), pp Y. Zhao, L. A. Tapln, M. Nezamabad, and R. S. Berns, Methods of spectral reflectance reconstructon for a Snarback 54 dgtal camera, Munsell Color Scence Laboratory Techncal Report December 004, (Munsell Color Scence Laboratory, 004), pp D. Connah and J. Y. Hardeberg, Spectral recovery usng polynomal models, Proc. SPIE 5667, (005). 3. J. L. Neves, E. M. Valero, S. M. C. Nascmento, J. H. Andres, and J. Romero, Multspectral synthess of daylght usng a commercal dgtal CCD camera, Appl. Opt. 44, (005). 4. F. H. Ima and R. S. Berns, Spectral estmaton usng trchromatc dgtal cameras, n Proceedngs of Internatonal Symposum on Multspectral Imagng and Color Reproducton for Dgtal Archves, (Socety of Multspectral Imagng, 999), pp B. Noble and J. W. Danel, Appled Lnear Algebra, 3rd. ed. (Prentce-Hall, 988), pp

19 6. M. A. Lopez-Alvarez, J. Hernandez-Andres, Eva. M. Valero, and J. Romero, Selectng algorthms, sensors and lnear bases for optmum spectral recovery of skylght, J. Opt. Soc. Am. A 4, (007). 7. V. Cheung, S. Westland, C. L, J. Hardeberg, and D. Connah, Characterzaton of trchromatc color cameras by usng a new multspectral magng technque, J. Opt. Soc. Am. A, 3.40 (005). 8. N. Shmano, Recovery of spectral reflectances of an art pantng wthout pror knowledge of objects beng maged, n Proceedngs of The 0th Congress of the Internatonal Colour Assocaton (Internatonal Color Assocaton, 005), pp N. Shmano, Recovery of spectral reflectances of objects beng maged wthout pror knowledge, IEEE Trans. Image Process. 5, (006). 30. N. Shmano, Evaluaton of a multspectral mage acquston system amed at reconstructon of spectral reflectances, Opt. Eng. 44, (005). 3. J. B. Cohen, Color and color mxture: scalar and vector fundamentals, Color Res. Appl. 3, 5.39 (988). 3. N. Shmano, "Suppresson of nose effect n color correctons by spectral senstvtes of mage sensors," Opt. Rev. 9, 8-88(00). 33. N. Shmano, "Applcaton of a colormetrc evaluaton model to multspectral color mage acquston systems," J. Imagng Sc. Technol. 49, (005). 34. M. Hronaga, N. Shmano, "Evaluatng the qualty of an mage acquston devce amed at the reconstructon of spectral reflectances usng recovery models," J. Imag. Sc. Technol. 5, (008). 35. M. Hronaga, N. Shmano, "Nose robustness of a colormetrc evaluaton model for mage acquston devces wth dfferent characterzaton models," Appl. Opt. 48, (009). 36. M. Hronaga and N. Shmano, "Estmatng the nose varance n an mage acquston system and ts nfluence on the accuracy of recovered spectral reflectances," Appl. Opt. 49, (00). 8

20 Chapter Evaluatng a qualty of an mage acquston devce amed at the reconstructon of spectral reflectances by the use of the recovery models Accurate recovery of spectral reflectances s mportant for the color reproducton under a varety of llumnatons. To evaluate the qualty (Qr) of an mage acquston system amed at recovery of spectral reflectances, Shmano proposed an evaluaton model based on the Wener estmaton [Opt. Eng., 44, pp , 005.] and showed that mean square errors (MSE) between the recovered and measured spectral reflectances as a functon of Qr agreed qute well wth the predcton by the model. In ths chapter, the evaluaton model s appled to two dfferent reflectance recovery models and t s confrmed that the proposed model can be appled to dfferent models wth expermental results and mathematcal proofs. 9

21 . Introducton Colors are one of the most mportant characterstcs of the human vson and they have been heavly studed to acqure accurate nformaton from color mages. The acquston of the colormetrc nformaton s consdered as the acquston of accurate colormetrc values of objects through the use of sensor responses [,]. The accuracy depends on the spectral senstvtes of a set of sensors, the nose present n the acquston devce and the spectral reflectances of the objects etc [,]. Therefore the evaluaton of the set of sensors s mportant for the evaluaton of the colormetrc performance or optmzaton of the spectral senstvtes of the sensors. Several models have been proposed to evaluate a colormetrc performance of a set of color sensors [3-7], and the optmzaton of a set of sensors has been performed based on the evaluaton models [8,9]. However, the applcaton of the evaluaton models to real color mage acquston devces such as dgtal cameras and color scanners has not appeared because of the dffculty n estmatng nose levels. Recently Shmano proposed a new model to estmate the nose varance of an mage acquston system [0] and appled t to the proposed colormetrc evaluaton model and a spectral evaluaton model, and confrmed that the evaluaton model qute agrees well wth the expermental results by multspectral cameras []. On the other hand, there s an alternatve approach for the color mage acquston. It s the acquston of the spectral nformaton of the objects beng maged. The purpose of ths approach s the acquston of the spectral reflectances of the maged objects through the use of sensor responses.[3,-9]. The acquston of accurate spectral reflectances of objects s very mportant n reproducng a color mage under a varety of vewng llumnants [30]. The accuracy of the recovered spectral reflectances depends on the number of sensors, ther spectral senstvtes, the objects beng maged, the recordng llumnants, the nose present n a devce and a model used 0

22 for the recovery. Therefore the evaluaton of a camera amed at the recovery of spectral reflectances s mportant for the optmzaton of an mage acquston system and to get an ntutve understandng about the acquston of the spectral nformaton. Shmano already derved the evaluaton model based on the Wener estmaton [3]. The proposed model s formulated by MSE( σ ) = E ( Q ( σ )) max r, where MSE( σ ) s the mean square errors between the recovered and measured spectral reflectances wth the estmated nose varance σ, Emax represents a constant whch s determned only by spectral reflectances of objects and Qr ( σ ) s the qualty of the mage acquston system amed at recovery of spectral reflectances wth the estmated nose varance σ. It was shown that Qr ( ) σ s determned by the spectral senstvtes of the sensors, the spectral power dstrbuton of the recordng llumnant, the nose varance of the mage acquston devce and the spectral reflectances of the maged objects. The model was appled to the multspectral cameras and t was confrmed that the model agrees qute well wth the expermental results,.e., the MSE ( σ ) of the recovered spectral reflectances by the Wener estmaton as a functon of the qualty Qr ( σ ) of a set of sensors by takng account of the nose shows the straght lne. As Qr ( σ ) s derved from Wener estmaton, t s very mportant to confrm whether the model can be appled to other recovery models snce the qualty Qr ( σ ) s useful not only for the evaluaton of an mage acquston devce but also for the optmzaton of a set of sensors amed at recovery of spectral reflectances. In ths chapter, t s shown that Qr ( σ ) has a lnear relaton to the MSE ( σ ) of the reflectances recovered by the multple regresson analyss [8] and the Ima-Berns model [6] by experments. Mathematcal proofs of the equvalence of the Wener model, the multple regresson model and the Ima-Berns model are gven. It s shown that the Qr ( σ ) s also approprately formulated for these models. Once ths lnear relaton s confrmed, we can estmate Qr ( σ ) by the multple regresson model or the Ima-Berns model, wthout the spectral

23 senstvtes of the sensors, spectral power dstrbuton of recordng llumnant or estmatng the nose varance [3]. Ths chapter s organzed as follows. The outlne of the evaluaton model and the method to estmate the nose varance and the models tested are brefly revewed. In the followng sectons, the expermental procedures and the results to demonstrate the trustworthness of the proposal are descrbed. The fnal secton presents conclusons and mathematcal proofs are n the appendx secton.. Models for the Reconstructon of Spectral Reflectances In ths secton, the dervaton of the qualty Qr ( σ ) to evaluate the color mage acquston system and the models used for the experments are brefly revewed... Wener Estmaton Usng Estmated Nose Varance A vector space notaton for color reproducton s useful n the problems. In ths approach, the vsble wavelengths from 400 to 700 nm are sampled at 0-nm ntervals and the number of the samples s denoted as N. A sensor response vector from a set of color sensors for an object wth a N spectral reflectance vector r can be expressed by p = SL r + e, (.) where p s a M sensor response vector from the M channel sensors, S s a M N matrx of the spectral senstvtes of sensors n whch a row vector represents a spectral senstvty, L s an N N dagonal matrx wth samples of the spectral power dstrbuton of an llumnant along the dagonal, and e s a M addtve nose vector. The nose e s defned to nclude all the sensor response errors such as the measurement errors n the spectral characterstcs of senstvtes, an llumnaton and reflectances, and quantzaton errors n ths work and t s

24 termed as the system nose[0] below. The system nose s assumed to be sgnal ndependent, zero mean and uncorrelated to tself. For abbrevaton, let (MSE) of the recovered spectral reflectances rˆ s gven by { r rˆ } S L = SL. The mean square errors MSE = E, (.) where { } E represents the expectaton. If rˆ s gven by rˆ = W0p, the matrx W 0 whch mnmzes the MSE s gven by W T ( S RssS + σ ) T 0 RssSL L L ei =, (.3) where T represents the transpose of a matrx, Rss s an autocorrelaton matrx of the spectral reflectances of samples that wll be captured by a devce, and the estmaton. Substtuton of Eq.(.3) nto Eq.(.) leads to [0] MSE( σ e N ) = λ = N β λ b = j= j N β σ + = j= 4 + κ v e j v ( κ e ) j + σ σ e s the nose varance used for σ λ b, (.4) j where λ s the egenvalues of Rss, b j, v κ j and β represent -th row of the j-th rght sngular vector, sngular value and a rank of a matrx / S Λ, respectvely, LV σ s the actual system nose varance, V s a bass matrx and Λ s an N N dagonal matrx wth postve egenvalues λ along the dagonal n decreasng order. It s easly seen that the MSE s mnmzed when σ e = σ, and the MSE ( σ ) s gven by MSE( σ N ) = λ = N β λ b = j= j N β + = j= κ σ v j + σ λ b. (.5) j 3

25 Equaton (.5) can be rewrtten as N β N β σ Σ= Σ j= λbj Σ= Σ j= λb v j N κ + σ j MSE( σ ) = λ. (.6) N = Σ= λ Therefore the qualty of a set of color sensors n the presence of nose s formulated as Q ( σ r ) = Σ N = Σ β j= λ b j Σ N = Σ Σ N = β j= λ κ σ v j + σ λ b j. (.7) Hence, the MSE ( σ ) s expressed as ( Q ( σ )) MSE( σ ) = E, (.8) max r where E max = = λ. Ths equaton shows that the MSE ( σ ) has a lnear relaton to Qr ( σ ) and N the slope of the lne s N N = λ. The values of λ = are dependent only on the surface spectral reflectance of the objects beng captured. The MSE ( σ ) decreases as the Qr ( σ ) ncreases to one. If we let the nose varance σ 0 for the Wener flter n Eq. (.3), then the MSE ( 0) s derved e = as (by lettng σ 0 n Eq. (.4)) e = MSE N N N ( 0) = λ λ b + λ b. (.9) = β = j= j β = j= σ κ v j j The frst and second terms on the rght-hand sde of Eq. (.9) represent the MSE ( 0) at a noseless case. We denote ths MSE as MSE free, then the estmated system nose varance ˆσ can be represented by 4

26 σˆ = ( ) MSE 0 MSE N β λ Σ= Σ j= κ free bj v j, (.0) where MSE free s gven by MSE free N = N β = λ λ b. (.) = j= j Therefore, the system nose varance σ can be estmated usng Eq. (.0), snce the MSE free and the denomnator of Eq. (.0) can be computed f the surface reflectance spectra of objects, the spectral senstvtes of sensors and the spectral power dstrbuton of an llumnant are known. The MSE ( 0) can also be obtaned by the experment usng Eqs. (.) and (.3) applyng the Wener flter wth σ 0 to sensor responses. Therefore, Eq. (.0) gves a method to estmate e = the actual nose varance σ. [0] The qualty Qr ( σ ) and MSE ( σ ) can be computed by substtutng the estmated nose varance n Eq.(.7) and Eq.(.3), respectvely... Multple Regresson Analyss Let p be a M sensor response vector whch s obtaned by the mage acquston of a known spectral reflectance r of the -th object, where represents a number. Let P be a M k matrx whch contans the sensor responses p, p, and let R be a N k matrx whch p,, k contans the correspondng spectral reflectances r, r, where k s the number of the r,, k learnng samples. The pseudonverse model s to fnd a matrx W whch mnmzes R WP, where notaton represents the Frobenus norm[33]. The matrx W s gven by. + W = RP, (.) 5

27 where + P represents the pseudo nverse matrx of the matrx P. By applyng a matrx W to a sensor response vector p,.e., rˆ = Wp, a spectral reflectance s estmated. Therefore ths model does not use the spectral senstvtes of sensors or the spectral power dstrbuton of an llumnaton, but t uses only the spectral reflectances of the learnng samples...3 Ima-Berns Model The Ima-Berns model [6] s consdered as the modfcaton of the lnear model by usng the multple regresson analyss between the weght column vectors for bass vectors to represent the known spectral reflectances and correspondng sensor response vectors. Let Σ be a d k matrx whch contans the column vectors of the weghts σ, σ,, σk to represent the k known spectral reflectances r, r and let P be a M k matrx whch r,, k contans correspondng sensor response vectors of those reflectances p, p,where d s a p,, k number of the weghts to represent the spectral reflectances. The multple regresson analyss between these matrxes s expressed as Σ BP. A matrx B whch mnmze the Frobenus norm s gven by + B = ΣP. (.3) Snce a weght column vectors σ for a sensor response vector p s estmated by σˆ = Bp, the estmated spectral reflectance vector s derved from rˆ = Vσˆ, where a matrx V s the bass matrx whch contans frst d orthonormal bass vectors of spectral reflectances. Ths model does not use the spectral characterstcs of sensors or an llumnaton..3 Expermental Procedures A multspectral color mage acquston system was assembled by usng seven nterference flters (Asah Spectral Corporaton) n conjuncton wth a monochrome vdeo camera (Kodak 6

28 KAI-40M). Image data from the vdeo camera were converted to 6-bt-depth dgtal data by an AD converter. The spectral senstvty of the vdeo camera was measured over wavelength from 400 to 700 nm at 0-nm ntervals. The measured spectral senstvtes of the camera wth each flter are shown n Fg... The llumnant used for mage capture was the llumnant whch smulates daylght (Serc Solax XC-00AF). The spectral power dstrbuton of the llumnant measured by the spectroradometer (Mnolta CS-000) s presented n Fg... The GretagMacbeth ColorChecker (4 colors) and the Kodak Q60R (8 Colors), let us denote them CC and KK respectvely for abbrevaton, were llumnated from the drecton of about 45 degree to the surface normal, and the mages were captured by the camera from the normal drecton. The mage data were corrected to unform the nonunformty n llumnaton and senstvtes of the pxels of a CCD. The computed responses from a camera to a color by usng the measured spectral senstvtes of the sensors, the llumnant and the surface reflectance of the color dose not equal to the actual sensor responses snce the absolute spectral senstvtes of a camera depend on the camera gan. Therefore, the senstvtes were calbrated usng an achromatc color n the charts. In ths work, the constrant s mposed on the sgnal power as T gven by = Tr( S R S ) ρ, where relaton of ρ = was used so that the estmated system nose L SS L varance can be compared for dfferent sensor sets. By usng varous combnatons of sensors from the three to seven n Fg.., the system nose varance was estmated by the methods descrbed above for each combnaton of sensors. Then the estmated nose varance was used to recover the spectral reflectances by the Wener estmaton, and then the MSE ( σ ) of the recovered spectral reflectances was computed. The spectral reflectances were also recovered by the multple regresson model and the Ima-Berns model. By usng the estmated nose varance, the qualty Qr ( σ ) for each combnaton of sensors was computed usng Eq.(.7). 7

29 Fg... Spectral senstvtes of the sensors of the camera. Fg... Spectral power dstrbuton of the llumnaton. 8

30 Fg..3. MSEs of the recovered spectral reflectances by the Wener, Regresson, and Ima-Berns method for the GretagMacbeth ColorChecker (CC) and the Kodak Q60R (KK) are plotted as a functon of Qr ( σ )..4 Results and Dscussons The values of the MSE ( σ ) of the CC and KK as a functon of Qr ( σ ) for the 80 sets of sensors are shown n Fg..3. The lnes n the fgure ndcate the theoretcal relaton between the N MSE ( σ ) and Qr ( σ ) as gven by Eq. (.8) for two color charts, where Emax = = λ was used for the determnaton of the slopes of the lne for each color chart. The expermental results of the MSE as a functon of Qr ( σ ) by the multple regresson analyss and the Ima-Berns method agree well wth the theoretcal lnes. 9

31 To show the mportance of consderng the nose varance, let Qr ( 0) be the value of Qr ( σ ) when the nose varance σ = 0,.e., Σ = Σ λ bj Q (0) =. (.4) r N β j= N Σ= λ Note that the Eq. (.9) was used as the MSEs of the recovered reflectances by the Wener flter wth zero nose varance. The relaton between MSE and Qr ( 0) s shown n Fg..4. Fg..4. MSEs of the recovered spectral reflectances by the Wener, Regresson, and Ima-Berns method for the GretagMacbeth ColorChecker (CC) and the Kodak Q60R (KK) are plotted as a functon of Qr ( 0), whch s the value of Qr ( σ ) when the estmated nose varance s zero. Ths s the case wthout consderaton for the nose. 0

32 In Fg..4 plots not only dsagree wth the theoretcal lnes but also scatter more largely compared to the Fg..3. Especally the plots of the KK recovered by the Wener model scatter far above the theoretcal lne. These scattered plots ndcate the mportance of the accurate estmaton of the nose varance n the mages. Also we confrmed that plots of the CC scatter more largely n Fg..4 when the 6-bt AD converter s used nstead of the 6-bt converter to dgtze the sensor responses. The results n Fg..3 agree well wth the theoretcal predctons and t means that Qr ( σ ) s able to be used for the multple regresson analyss and the Ima-Berns model. As a matter of a fact, the multple regresson model and the Ima-Berns model are mathematcally equvalent to the Wener estmaton,.e., the matrxes W n Eq.(.) and B n Eq. (.3) are equvalent to the matrx of the Wener flter W 0 n Eq.(.3) and whch can be proved by the mathematcal analyss. For proofs of the equvalences, see the appendx. Typcal examples of the recovered reflectances and the reproduced color mages for three cases of the Qr ( σ ) are shown n Fg..5 through Fg..7. It s very clear that the error of the recovered spectral reflectances ncrease wth a decrease n the Qr ( σ ) and fathfulness of the reproduced colors decreases wth a decrease n the Qr ( σ ). Also the typcal examples of the maxmum and mnmum values of the Qr ( σ ) and MSE ( σ ) by the three models for each number (three to seven) of sensor sets are shown n Table.. It s very nterestng that a set of four sensors (sensor number 457 ) has a larger Qr ( σ ) than a set of sx sensors (sensor number 3456 ). It s not always true that the Qr ( σ ) ncreases when the number of the sensors ncreases. Though we confrmed that the Wener model recovers the reflectances most accurately of the three reflectance recovery models such as the Wener, the regresson and the Ima-Berns model n most cases, especally n the cases when the reflectances of the learnng sample and the test

33 sample are not smlar [34] and that the Wener model recovers the hghly accurate reflectances by selectng the approprate learnng samples from a group of samples or databases of reflectances of objects [35], we have to recover the reflectances by the multple regresson model or the Ima-Berns model when pror knowledge of the spectral senstvtes of a set of sensors or the spectral power dstrbuton of the llumnant are unknown. In other words, we have to evaluate the mage acquston devces by the multple regresson model or the Ima-Berns model when pror knowledge of the mage acquston devce s unknown. It s now confrmed that the MSE ( σ ) of the spectral reflectances recovered by the multple regresson model and the Ima- Berns model have a lnear relaton to the qualty Qr ( σ ) of the mage acquston system. Once ths lnear relaton s confrmed, we can estmate Qr ( σ ) by the multple regresson or the Ima- Berns model wthout the spectral senstvtes of sensors, the spectral power dstrbuton of the recordng llumnant or the nose present n the mage acquston system snce t (Qr ( σ ) ) can be easly estmated by markng the value of the MSE by the models on the theoretcal lne,.e., the correspondng Qr ( σ ) of the pont gves the estmate. Now t s possble to estmate the qualty Qr ( σ ) by the multple regresson model or the Ima-Berns model wth only the spectral reflectances and the captured mages of the object..5 Conclusons The evaluaton of an mage acquston system amed at recovery of spectral reflectances, whch s derved based on the Wener estmaton, was appled to the multple regresson analyss and the Ima-Berns method. The expermental results by multspectral cameras agree qute well wth the proposed model. From ths result, t s concluded that the proposed evaluaton model s approprately formulated and that the estmaton of the nose varance of an mage acquston system s essental to evaluate the qualty Qr ( σ ). Ths result also gves us an easy way to

34 estmate the qualty Qr ( σ ) and provdes us an easer way to evaluate an mage acquston system amed at reconstructon of spectral reflectances wthout the spectral senstvtes of sensors, the spectral power dstrbuton of the recordng llumnant or the nose present n the mage acquston system [36]. 3

35 (a) (b) Fg..5. (a) Typcal example of the recovered spectral reflectance of the color red at a large Qr ( σ ) (Qr ( σ ) = ). (b) Color reproducton of the GretagMacbeth ColorChecker by the recovered spectral reflectance at Qr ( σ ) =

36 (a) (b) Fg..6. (a) Typcal example of the recovered spectral reflectance of the color red at a mddle Qr ( σ ) (Qr ( σ ) = ). (b) Color reproducton of the GretagMacbeth ColorChecker by the recovered spectral reflectance at Qr ( σ ) =

37 (a) (b) Fg..7. (a) Typcal example of the recovered spectral reflectance of the color red at a small Qr ( σ ) (Qr ( σ ) = ). (b) Color reproducton of the GretagMacbeth ColorChecker by the recovered spectral reflectance at Qr ( σ ) =

38 Table.. Maxmum and mnmum Qr ( σ ) for each number of sensors of the Macbeth ColorChecker. Qr ( σ ) Number of sensors Sensors Qr(0) MSE ( σ ) (Wener) MSE(0) MSE (Regresson) MSE (Ima-Berns) ch ch ch ch ch ch ch ch ch

39 References. P.S. Hung, Colormetrc calbraton for scanners and meda, Proc. SPIE 448, (99).. H.R. Kang, Colormetrc scanner calbraton, J. Imagng Sc. Technol. 36(), 6-70 (99). 3. G. Sharma and H.J. Trussell, Fgure of mert for color scanners, IEEE Trans. Image Process. 6(7), (997). 4. N. Shmano, Suppresson of nose effect n color correctons by spectral senstvtes of mage sensors, Opt. Rev. 9(), 8-88 (00). 5. S. Quan, N. Ohta, R.S. Berns, X. Jang and N. Katoh, Unfed measure of goodness and optmal desgn of spectral senstvty functons, J. Imagng Sc. Technol. 46(6), (00). 6. H.E.J. Neugebauer Qualty factor for flters whose spectral transmttances are dfferent from color mxture curves, and ts applcaton to color photography. J. Opt. Soc. Am. 46(0),8-84 (956) 7. P.L. Vora and H.J. Trussell, Measure of goodness of a set of color-scannng flters, J. Opt. Soc. Am. A 0(7), (993). 8. N. Shmano, "Optmzaton of spectral senstvtes wth Gaussan dstrbuton functons for a color mage acquston devce n the presence of nose", Optcal Engneerng, vol.45()., pp , (006). 9. G. Sharma, H.J. Trussell and M.J. Vrhel, Optmal nonnegatve color scannng flters, IEEE Trans Image Process. 7(), 9-33 (998). 0. N. Shmano, "Recovery of Spectral Reflectances of Objects Beng Imaged Wthout Pror Knowledge" IEEE Trans. Image Process. vol. 5, no.7, pp , Jul. (006).. N. Shmano, Applcaton of a Colormetrc Evaluaton Model to Multspectral Color Image Acquston Systems J. Imagng Sc. Technol. 49(6), (005).. J. Cohen, "Dependency of spectral reflectance curves of the Munsell color chps", Psychon. Sc,, (964). 8

40 3. L. T. Maloney, "Evaluaton of lnear models of surface spectral reflectance wth small numbers of parameters", J. Opt. Soc. Am. A3, (986). 4. J. P. S. Parkknen, J. Hallkanen, T. Jaaskelanen, "Characterstc Spectra of Munsell Colors", J. Opt. Soc. Am. A6, 38-3(989). 5. J. L. Dannemller, "Spectral reflectance of natural objects: how many bass functons are necessary?" J. Opt. Soc. Am. A9, (99). 6. A. Rosenfeld and A. C. Kak, Dgtal Pcture Processng, nd ed., (Academc Press, 98). 7. H. Hanesh, T. Hasegawa, N. A. Hoso, Y. Yokoyama, N. Tsumura and Y. Myake, "System desgn for accurately estmatng the spectral reflectance of art pantngs", Appl. Optcs, 39,66-663(000). 8. Y. Zhao, L.A. Tapln, M. Nezamabad and R.S. Berns, Usng the Matrx R Method for Spectral Image Archves, The 0 th Congress of the Internatonal Color Assocaton (AIC 5), (Granada, Span), pp (005). 9. K. Martnez, J. Cuptt, D. Saunders and R. Pllay, "Ten Years of Art Imagng Research", Proc. of the IEEE, 90,8-4(00). 0. A. A. Aff and S. P. Azen, "Statstcal Analyss", Chap.3,(Academc Press,97).. H. L. Shen and H. H. Xn, "Spectral characterzaton of a color scanner based on optmzed adaptve estmaton", J. Opt. Soc. Am. A3, (006).. D. Connah, J. Y. Hardeberg, and S. Westland, "Comparson of Lnear Spectral Reconstructon Methods for Multspectral Imagng", Proc. IEEE's Internatonal Conference on mage Processng (IEEE,004),pp Y. Zhao, L. A. Tapln, M. Nezamabad, and R. S. Berns, "Methods of Spectral Reflectance Reconstructon for A Snarback 54 Dgtal Camera", Munsell Color Scence Laboratory Techncal Report December 004,pp.-36(004). 4. D. Connah and J. Y. Hardeberg, "Spectral recovery usng polynomal models Proc. SPIE 5667, (005) 5. J. L. Neves, E. M. Valero, S. M. C. Nascmento, J. H. Andrés, and J. Romero, "Multspectral synthess of daylght usng a commercal dgtal CCD camera", Appl. Optcs, 44, (005). 9

41 6. F.H. Ima and R.S. Berns, "Spectral Estmaton Usng Trchromatc Dgtal Cameras", Proc. Int. Symposum on Multspectral Imagng and Color Reproducton for Dgtal Archves, Chba, Japan. 4-49, (999). 7. M. A. López-Álvarez, J. Hernández-Andrés, Eva. M. Valero and J. Romero, "Selectng algorthms, sensors and lner bases for optmum spectral recovery of skylght" J. Opt. Soc. Am. A4,94-956(007). 8. V. Cheung, S. Westland, C. L, J. Hardeberg and D. Connah, "Characterzaton of trchromatc color cameras by usng a new multspectral magng technque", J. Opt. Soc. Am. A, 3-40(005). 9. M. Sh and G. Healey, Usng reflectance models for color scanner calbraton, J. Opt. Soc. Am. A 9(4), (00). 30. Y. Myake and Y. Yokoyama, "Obtanng and Reproducton of Accurate Color Images Based on Human Percepton", Proc. SPIE 3300, 90-97(998). 3. N. Shmano, "Evaluaton of a multspectral mage acquston system amed at reconstructon of spectral reflectances", Optcal Engneerng, vol.44, pp , (005). 3. M. Hronaga, K. Tera and N. Shmano, A model to evaluate color mage acquston systems amed at the reconstructon of spectral reflectances, Proc. Nnth Internatonal Symposum on Multspectral Color Scence and Applcaton, Tape, R.O.C. 3-8, (007). 33. G.H. Golub and C.F.V. Loan, "Matrx Computatons", The Johns Hopkns 34. N. Shmano, K. Tera, and M. Hronaga, "Recovery of spectral reflectances of objects beng maged by multspectral cameras," J. Opt. Soc. Am. A 4, 3-39 (007). 35. N. Shmano and M. Hronaga, "Recovery of spectral reflectances of maged objects by the use of features of spectral reflectances," J. Opt. Soc. Am. A 7, 5-58 (00). 36. M. Hronaga, N. Shmano, "Evaluatng the qualty of an mage acquston devce amed at the reconstructon of spectral reflectances usng recovery models," J. Imag. Sc. Technol. 5, (008). 30

42 Appendx A. Proof of the equvalence of the multple regresson model to the Wener model. The multple regresson model mnmzes R WP, (A.) where P s a M k matrx whch contans the sensor response vectors p, p,, pk and let R be a N k matrx whch contans the correspondng spectral reflectances vectors r, r,, rk, where k s the number of the learnng samples. The gven by N M matrx W whch mnmzes Eq.(A.) s + W = RP, (A.) where + P represents the pseudo nverse matrx of the matrx P. P P (PP ) + T T = (A.3) because M < k holds n the mage acquston devces and Rank (P) = M. Let P = SLR + E, (A.4) where S s a M N matrx of the spectral senstvtes of sensors n whch a row vector represents a spectral senstvty, L s an power dstrbuton of an llumnant along the dagonal, and E s a N N dagonal matrx wth samples of the spectral M N matrx whch contans the addtve nose vectors. For abbrevaton, let S L = SL. Substtuton of Eq. (A.3) and Eq. (A.4) nto Eq. (A.) leads to W = R(S. (A.5) T T L R + E) ((SLR + E)(SLR + E) ) 3

43 Hence W s rewrtten as T L T ( S RssS + σ I) W = RssS (A.6) L L e because t RR s an autocorrelaton matrx of R and t EE gves the nose varance and RE T T = ER = 0 as the spectral reflectances and the error have no correlaton. Thus the matrx W s equvalent to that of the Wener flter. B. Proof of the equvalence of the Ima-Berns model to the Wener model. Let Σ be a d k matrx whch contans the vectors of the weghts to represent the k known spectral reflectances, where d s a number of the weghts to represent the spectral reflectances and let P, R, S L and E be as defned n Appendx A. A d M matrx B whch mnmzes Σ = BP (B.) s gven by + B = ΣP. (B.) From the Eq. (A.) through (A.6), t s easly understood that VB + T T T T T T = VΣP = VΣ(R SL + E )(SLRR SL + EE ). (B.3) From the defnton of the method, R = VΣ, where V s an orthonormal bass matrx. Hence T L T ( S RssS + σ I) VB = RssS. (B.4) L L e Thus the Ima-Berns model s equvalent to the Wener Model. 3

44 Chapter 3 Nose robustness of a colormetrc evaluaton model for mage acquston devces wth dfferent characterzaton models A colormetrc evaluaton of an mage acquston devce s mportant for evaluatng and optmzng a set of sensors. Shmano proposed a colormetrc evaluaton model [J. Imagng Sc. Technol. 49(6), (005)] based on the Wener estmaton. The mean square errors (MSE) between the estmated and the actual fundamental vectors by the Wener flter and the proposed colormetrc qualty (Qc) agreed qute well wth the proposed model and he showed that the estmaton of the system nose varance of the mage acquston system s essental for the evaluaton model. In ths chapter, t s confrmed that the proposed model can be appled to two dfferent reflectance recovery models and these models provde us an easy method for estmatng the proposed colormetrc qualty (Qc). The system nose orgnates from the samplng errors of the spectral characterstcs of the sensors, the llumnatons and the reflectance and the quantzaton errors. The nfluence of the system nose on the evaluaton model s studed and t s confrmed from the expermental results that the proposed model holds even n a nosy condton. 33