Six-Band HDTV Camera System for Color Reproduction Based on Spectral Information

Transcription

1 IS&T's 23 PICS Conference Sx-Band HDTV Camera System for Color Reproducton Based on Spectral Informaton Kenro Ohsawa )4), Hroyuk Fukuda ), Takeyuk Ajto 2),Yasuhro Komya 2), Hdeak Hanesh 3), Masahro Yamaguch 4), and Nagaak Ohyama 4) ) Telecommuncatons Advancement Organzaton of Japan, Tokyo, Japan 2) Olympus Optcal Co., Tokyo, Japan 3) Chba Unversty, Chba, Japan 4) Tokyo Insttute of Technology, Tokyo, Japan Abstract The archtecture of 6-band HDTV camera system, whch s newly developed for accurate color reproducton of moton pcture based on spectral nformaton s presented. The 6- band HDTV camera s constructed by an optcal system connectng the objectve lens and two sets of conventonal HDTV cameras whose spectral senstvtes are modfed dfferently by nterference flters nserted between the optcal system and each camera. We evaluated the accuracy of the color estmatons acheved by the 6-band and a conventonal RGB (.e. 3-band) HDTV camera. The evaluatons are conducted usng smulated camera sgnals and those obtaned by expermental measurements. It s confrmed that the 6-band camera acheves the accurate color estmaton for the use n extensve applcatons. In the expermental evaluaton the average color dfference of de ab obtaned from the 6-band camera sgnals and the RGB camera sgnals became.43 and 4.2 respectvely. Introducton Recent advancement of multmeda technology ntroduced the possblty to realze electronc art museum, electronc commerce, telemedcne and so on. In such systems the mage dsplay wth accurate color s one of the most mportant technologes. Current color magng systems for both of stll mage and moton pcture are based on RGB (3-band) camera and RGB (3-prmary) dsplay, but t s qute dffcult to acheve the accurate color estmaton of objects under arbtrary llumnaton wth the 3-band acquston. The dffculty to estmate colormetrc values under the same llumnaton as that used for mage acquston s caused by the devaton between the spectral senstvtes of camera sensors and those of human vson. In other words the camera whose spectral senstvtes are equvalent to those of human vson realzes accurate color estmaton. However, even f the camera has such characterstcs, t s mpossble to estmate the color accurately under the dfferent llumnatons. Recently mult-spectral color reproducton systems whch estmate spectrum usng mult-band camera as nput devce have been actvely studed n the felds of stll mages. It was revealed that by ncreasng the band number of nput devce t becomes possble to estmate spectrum more accurately and accordngly to refne the estmaton accuracy of color under arbtrary llumnaton spectrum. ) In the case of spectral estmaton of natural objects the accurate estmaton can be acheved wth not so large number of camera bands takng the fact that the spectral reflectance s well represented by a lnear combnaton of small number of bass functons nto account. 2 Consderng the mult-band camera for moton pcture the tme-multplexed type such as one usng flter-wheel cannot realze the acquston of mult-band mages as moton pcture wth enough tme-resoluton n current technologes. So we newly developed a 6-band HDTV camera for ths am usng two sets of RGB HDTV cameras wth an addtonal optcal system for dvson of the ncdent lght and two knds of nterference flters to modfy the spectral senstvtes of the RGB cameras. In ths paper we present the archtecture of the 6-band HDTV camera and the results of the evaluatons regardng the accuracy of the color estmatons calculated by computer smulaton and those obtaned by expermental measurements. The spatal dstrbuton of color estmaton error caused by non-unformty of spectral characterstcs of optcal components ncludng the nterference flters and the camera sensor s also evaluated. Archtecture of the 6-Band Camera System The 6-band camera system s consttuted of the 6-band camera, camera control unt (CCU), and data storage. The CCU, whch controls the 6-band camera unt, s connected to the data storage by 2 sets of HD-SDI cables to send captured 6-band mage data from the 6-band camera wth relatve nformaton regardng camera settng. The archtecture of each unt s descrbed. 7

2 IS&T's 23 PICS Conference 6-Band Camera Archtecture The 6-band HDTV camera s constructed by the objectve lens, the L-shape branch connecton optcal unt and two sets of HDTV camera heads. The optcal unt connects the objectve lens and two sets of HDTV cameras. Fg. shows a pcture of the 6-band HDTV camera. As s shown n Fg. 2 the half mrror n the optcal unt dvdes the ncdent lght from the objectve lens to reflected lght and to transmtted one. Each of the dvded lght from the half mrror s focused on the RGB CCD sensors of the camera head. The two sets of the RGB sensors n HDTV cameras have same spectral senstvtes n desgn. They are modfed by dfferent knd of nterference flters to make 6 knds of ndependent spectral senstvtes. The two HDTV cameras capture the 3-band mages n sync to brng each frame of 6-band mage. The total spectral senstvtes of the 6-band camera are determned by the spectral characterstcs of the camera lens, IR cut flter, the 6-band separaton flters, the CCD sensors, and all the other optcal components of the camera system. Fgure 3 shows the spectral transmttances of the 6-band separaton flters and the spectral senstvtes of the HDTV cameras, whch are provded by ther manufactures. The spectral transmttances of the nterference flters for 6-band separaton are desgned to form the 6-band spectral senstvtes havng almost equal band-wdth. One of the nterference flters s composed of long-pass flter wth mnus flter (sold lne n Fg. 3(a)). Ths flter changes the orgnal RGB spectral senstvtes to those of the longer wavelength part of the R, the shorter wavelength part of the G, and the longer wavelength part of the B senstvtes. The other flter s composed of short-pass flter wth mnus flter (dotted lne n Fg. 3(a)). Ths flter changes the orgnal RGB spectral senstvtes to those of shorter wavelength part of the R, longer wavelength part of the G, and shorter wavelength part of the B senstvtes. The 6-band separaton flters are exchangeable dependng on the condton of the camera use. Fgure 4 shows the locaton for the settng of the flter n the optcal unt. The spectral transmttance of the 6-band separaton flter changes dependng on the angle of the ncdent lght. It ntroduces spatal non-unformty of the spectral senstvtes of the camera. The nfluence to the color estmaton accuracy wll be examned later. Fgure 2. Schematc dagram of 6-band HDTV camera archtecture (a) transmttance(%) (b) relatve senstvty wavelength Fgure 3. Spectral characterstcs (a) Spectral transmttance of 6- band separaton flters (b) Spectral senstvtes of the HDTV camera Fgure. 6-band HDTV camera Fgure 4. The 6-band separaton flter locaton n the 6-band camera 72

3 IS&T's 23 PICS Conference Camera Control Unt and Data Storage Archtecture The 6-band camera s controlled by two sets of CCUs for the use of conventonal HDTV camera n sync. The knee and gamma characterstcs set n the CCUs are set off to make the camera response lnear for the purpose of the accurate color characterzaton of the camera. The CCUs send the 6-band mage wth the camera settng data as notsub sampled 4:4:4 data n the form of HD-SDI Dual Lnk sgnals 4 by two sets of HD-SDI cables to the data storage unt. The data storage unt has 2.5 TB data capablty. The captured moton pcture data s recorded n ths storage unt as bts 4:4:4 data wth camera settng nformaton correspondng to each frame embedded n the two lnes followng the HDTV 92 by 8 mage data. Camera Characterzaton The color estmaton from the camera sgnals s based on the camera characterzaton, whch provdes the camera sgnals gven spectral reflectance of the object, the llumnaton spectrum, and the spectral senstvtes of the camera. The mathematcal camera model provdes a calculaton method of the camera sgnals usng these data. Measurement of Spectral Senstvty One of the most mportant factors n the camera characterzaton s the set of spectral senstvtes of the camera. However, the spectral data provded by the manufacture s not always avalable or accurate enough to be useful for color estmaton. So we measured t usng a narrow-spectrum lght emtter and a spectrophotometer. In the measurement, the dgtal counts of the 6-band camera are obtaned by capturng the narrow-spectrum lght and the lght energy s measured by the spectrophotometer at same condton as capturng the lght. The spectrophotometer used n ths measurement s TOPCON SR-2A wth the measurng wavelength nterval of 5 nm. The measurement lght ranges from 38 nm to 78 nm wth nterval of 5 nm. From these data the spectral senstvtes are calculated as the camera sgnals correspondng to the ncdent lght wth unt energy. The narrow-spectrum lght s reflected by a dffuser and the reflected lght s captured by the 6-band camera as mage and s measured by the spectrophotometer. The captured mage has non-unform sgnal dstrbuton n the mage. So some extent of the central part of the sgnal dstrbuton n the mage s used to calculate the average camera sgnals of -th camera band g λ( = ~ 6). The spectral senstvtes h ( ( = ~ 6) at the wavelength λ are calculated by the Eq. (). h ( ( g λ g )/ Iλ = () where Iλ represents the lght energy measured by the spectrophotometer and g represents the bas sgnal. In ths measurement the gamma and the knee s set off to make the camera response lnear. The spectral senstvtes of a conventonal HDTV camera are also measured same way. Fg. 5(a) and Fg. 5(b) show the spectral senstvtes of the 6-band camera and the conventonal HDTV camera respectvely. The 6-band camera has 6 narrow-band senstvtes as s expected by ts spectral characterstcs shown n Fg. 3. It should be noted that the gan of each band s adjusted ndependently and there are some optcal parts not common to the 6-band camera and the conventonal HDTV camera. relatve senstvty Fgure 5 (a). Spectral senstvtes of the 6-band camera relatve senstvty Fgure 5 (b). Spectral senstvtes of the RGB camera Mathematcal Camera Model The mathematcal camera model provdes a calculaton method of the camera sgnals usng the spectral senstvtes of the camera, the llumnaton spectrum, and the spectral reflectance of the object to be captured. We adopt the followng equatons as the mathematcal camera model. 78 g = h ( λ ) E( f ( ( = ~ 6) (2) λ = 38 g = γ + (3) ( g ) g where h ( ( = ~ 6) s the spectral senstvtes of the 6- band camera shown n Fg. 5(a), E( s the llumnaton spectrum, and f ( s the spectral reflectance of the object. γ ( ) n Eq. (3) represents gamma functon of the camera. Usually n the actual cameras the above model does not hold strctly. We evaluate the model accuracy 73

4 IS&T's 23 PICS Conference used n the camera characterzaton for the color estmaton. Theory for Accurate Color Reproducton The CIE colormetrc values, X, Y, and Z under the llumnaton spectrum ( are defned by the followng equatons. 78 X = x( ( f ( (4a) λ = Y = y( ( f( (4b) λ = Z = z( ( f ( (4c) λ = 38 where x (, y (, and z ( represent the CIE color matchng functons. When the L-band camera sgnals are ~ ~ ~ obtaned, the estmated colormetrc values X, Y, Z are gven by the 3 L matrx transformaton of the camera sgnals. ~ C = MG (5) ~ ~ ~ ~ t where C = [ X, Y, Z], t G = g, g,..., ]. [ 2 The matrx M s determned by Wener estmaton mnmzng ~ t C C, where C = [ X, Y, Z], assumng that Eq. (2) holds and rewrtten as M = AB (6) where the elements of the matrx A and B are represented by Eq. (7) and Eq. (8) respectvely. a k b jk = λ= 38 = = j λ= 38 = 38 g L c( ( < f( f( ) > h ( ) E( ) (7) k h( E( < f( f( ) > h ( ) E( ) (8) where represents the operator of ensemble average, and c ( = x(, c 2( = y(, and c 3( = z(. If the llumnaton spectrum E( s equal to (, t s possble for the colormetrc values of arbtrary objects to be estmated exactly by a lnear transformaton of the camera sgnals n the condton that the lnear combnatons of the spectral senstvtes of more than 3 bands of camera represent x (λ ), y (, and z (. The color reproducton of the current televson system s based on ths theory n the case that the L s equal to 3 and the spectral senstvtes of the camera are standardzed as ones satsfyng ths condton. However, when ( s not equal to E( but arbtrary llumnant, the above theory does not hold any more. In ths case t becomes effectve to ncrease the number of the camera bands for accurate color estmaton. k Results The estmated color s evaluated by the color dfference between the color of the object under the llumnaton for observaton calculated by Eq. (4) and the color estmated from the smulated camera sgnals, whch are calculated by the camera mathematcal model gven by Eq. (2) and Eq. (3). Another evaluaton s the color dfference between the color of the object measured under the llumnaton for observaton and the color estmated from the captured camera sgnals obtaned under the llumnaton for capturng the mage. The former evaluaton s denomnated smulaton and the latter one experment. The color dfference between the smulaton and the experment due to nadequate camera model s also evaluated n the experment. Smulaton The error s defned by the color dfference between the colormetrc values of each object estmated by Eq. (5) where the camera sgnals are smulated by Eq. (2) and Eq. (3) and those defned by Eq. (4). The objects are 24 color charts ncluded n GretagMacbeth Color Checker. The llumnatons used n the mage capturng and the observatons are a knd of xenon lamp (Day), a knd of ncandescent lamp (A), and a knd of fluorescent lamp (Fl). The color s estmated by the aforementoned method usng the above data. In the estmaton process the statstcal nformaton on the object < f ( λf ) ( ) > s also necessary a pror. We used the data calculated from the spectral reflectance of the 24 color charts ncluded n GretagMacbeth Color Checker measured by spectrophotometer as the statstcal nformaton. The XYZ values are calculated usng the CIE93XYZ color matchng functons and t s transformed to the CIE976 L*a*b* values assumng that the whte pont s the whte chart ncluded n GretagMacbeth color checker. The spectral reflectance of the color charts and the three knds of the llumnaton spectra used n the smulatons are shown n Fg. 6 to Fg. 9. The average and maxmum errors of the estmated colors from the smulated 6-band camera sgnals and 3- band camera sgnals are lsted n table. The descrpton on the llumnaton condton (e.g. Day-A) represents the llumnaton for capturng and that for the observaton n ths order. Comparng the results of the 6-band camera wth those of the 3-band camera, the 6-band camera acheves smaller errors than the 3-band for all the llumnaton condtons. The 6-band camera can estmate color wth the average error of approxmately n Eab unt. On the other hand the 3-band camera estmates color wth the average error of approxmately 2 to 4 n Eab unt. The estmaton errors depend whether the llumnaton spectrum for capturng and that for observaton are same or not. Most estmaton n the same llumnaton condton makes the average error mnmum among those n the other combnatons of the llumnaton condtons. 74

5 IS&T's 23 PICS Conference spectral reflectance Fgure 6. Spectral reflectance of the objects Table. Color Estmaton Errors n the Smulaton Dav-Dav Dav-A Dav-EI 6-band band A-Dav A-A A-EI 6-band band EI-Dav EI-A EI-EI 6-band band relatve radance relatve radance relatve radance Fgure 7. Spectrum of the daylght Fgure 8. Spectrum of the ncandescent lamp Experment The 6-band camera and the 3-band camera captured GretagMacbeth Color Checker n a lght booth. The lght used n ths experment s a knd of xenon lamp whose spectral dstrbuton s smlar to that shown n Fg. 7. The captured mage area of the checker ncludng 24 color charts s smaller than a half of the whole mage area. The 6-band and the 3-band sgnals of each color chart are averaged n the adequate regon of the color chart. The colormetrc values of the 24 colors are measured by the spectrophotometer TOPCON SR-2A wth the measurng wavelength nterval of 5 nm on almost same geometrcal condton as capturng mage. The color dfferences between the color values estmated from the camera sgnals and those measured by the spectrophotometer are lsted n table 2. The average error of the 6-band camera s.43 and the maxmum error s The average error of the 3-band camera s 4.2 and the maxmum error s The average and maxmum errors of the color estmated from the smulated camera sgnals, whch are calculated usng the spectral data measured n ths experment as descrbed n the smulaton method, are also shown n Table 2. The color dfference between the colors estmated from the captured camera sgnals and from the smulated ones ndcates how large the error of the camera model contrbutes to the expermental error. The average and maxmum errors due to ths model error are also shown n Table 2. Table 2. Color Estmaton Errors n the Experment Experment Smulaton Model Error 6-band band Fgure 9. Spectrum of the fluorescent lamp Dscusson In the smulaton we examned the capablty of the accurate color estmaton by the 6-band camera comparng wth a conventonal HDTV 3-band camera. It s confrmed 75

6 IS&T's 23 PICS Conference that under three knds of llumnants typcal to our daly lfe the 6-band camera realzes color estmaton wth hgh accuracy acceptable n large extent of applcatons. In the experments the errors ncreased due to those of the camera model. In ths experment large part of the color estmaton error s the contrbuton of the wrong camera model. One of the causes of the camera model error s flare nsde the camera. The camera sgnals due to the flare should be compensated by sophstcated model for mprovng the color estmaton performance. In the above dscusson t s assumed that the spectral senstvtes of the camera are spatally unform. However, n the case of the 6-band camera the spatal non-unformty of the spectral senstvty can be brought by the nserted nterference flter used for 6-band color separaton n addton to the nonunformty of the spectral characterstcs of the optcal components and the mage sensors. We examned the spatal unformty of the estmated color of the 6-band camera. A whte dffuser s captured by the 6-band camera 25 tmes movng the drecton of t so as to adjust the mage poston of the whte dffuser beng on predetermned 5 by 5 sampled grd ponts. The dfference of the spectral senstvtes brngs the dfference of the camera sgnals for the same object. Hence the estmated color also becomes dfferent from pont to pont when the same spectral senstvtes are used n the color estmaton over magng area. The color dfferences relatve to the color of the center n magng area evaluated n Eab unt are shown n Fg.. The farther the pont of the dstance from the center s larger, the larger the estmated color dfference becomes. The estmated color dstrbuton n a*- L* plane and a*-b* plane are plotted n Fg.. 92 pxels pxels L* a* b* -2-2 Fgure. Estmated color dstrbuton n a*-l* and a*-b* plane Concluson We developed a 6-bnd camera for accurate color reproducton of moton pcture based on spectral nformaton. The accuracy of the estmated color of the 6- band camera comparng wth a conventonal RGB (.e. 3- band) HDTV camera s evaluated. The results of the color estmatons from the smulated camera sgnals suggest the 6-band camera has apprecable accuracy. In the expermental evaluaton the average color dfference of de ab obtaned from the 6-band camera sgnals attaned.43 although t became worse than those of the smulaton due to the error of the camera model. References. M. Yamaguch, et al, Proc. SPIE, Vol. 4663, 5-26, L.T. Maloney, J. Opt. Soc. Am. A, Vol.3, No., , Colormetry 2nd ed., CIE Publcaton, No.5.2, Central Bureau of the CIE, Venna, SMPTE 372M PROPOSED SMPTE STANDARD for Televson - Dual Lnk 292M Interface for 92 x 8 Pcture Raster a* Fgure. Dstrbuton of the estmated color dfference relatve to center of the mage 76