Using Re-Eye to improve fe etetion in low qulity vieo imges Rihr Youmrn Shool of Informtion Tehnology University of Ottw, Cn youmrn@site.uottw. Any Aler Shool of Informtion Tehnology University of Ottw, Cn ler@site.uottw. Astrt This pper presents metho to improve fe etetion y loting eyes in n imge using infrre (IR) light. IR light proues Re-Eye effet mking the pupil to shine more thn in norml lighting onitions. The lotion of the eyes n the fe ontour re ompute from the IR imges using olletion of imge proessing tehniques. The lgorithm opertes on gry imges uner vriety of skin tones, eye olors, ngles, n illumintions. The evelope metho nlyzes two onseutive frmes where the first one is tken uner regulr illumintion with no IR n the seon one using IR in orer to filitte eye etetion for sujets wering glsses. Keywors Re Eye effet, fe etetion, infrre lighting. 1 Introution Reently, iometris hs een mjor fiel of reserh tht is inispensle for uthentition n ientifition of suspets n for inresing seurity. Biometris is mesurement of uniqueness of humn eing suh s voie, hnprint or fil hrteristis. The use of iometris s wy to uthentite users ientities hs een topi of isussion for yers. Fe etetion n reognition is one importnt rnh of iometris tht is employe in mny res suh s irport seurity n orer mngement. The nee of fully utomte systems tht nlyze the informtion ontine in fe imges is neessry n for this reson, roust n effiient fe etetion lgorithms re require [5]. Given single imge or sequene of imges, the gol of fe etetion is to ientify ll imge regions, whih ontin fe regrless of its three-imensionl position, orienttion n lighting onitions. Suh prolem is hllenging euse fes re non-rigi n hve high egree of vriility in size, shpe, olor, n texture. The ility to etet fes in sene is ritil to moern surveillne pplitions. While mny imge proessing lgorithms exist to etet fes in imges [5], their performne is not ompletely relile, espeilly in situtions with vrile lighting, n when eling with low resolution imges. In this pper, we explore new tehnology tht improves fe etetion using the Re-Eye effet sine we know tht humn eyes will shine with o-xil infrre illumintion ([7], [?]). By using IR illumintion, it is possile to get informtion from whih the eye positions in the imge n e lulte. The evelope lgorithm will nlyze two onseutive frmes where the first one hs no IR n the other is shine with IR illumintion in orer to inrese the eye region vliity t erly stges. The propose pproh will etet n ignore refletions use y glsses. A similr work ws omplete in [1] where ontrst n ege enhnement tehnique were use for eye etetion without using frme ifferentil tehnique. 2 Experimentl setup The imges were tken using single lk n white mer, sensitive to infrre light, with zoom lens of 2.5-75 mm n NTSC out to the frme grer. In orer to just the overll illumintion of the re where imges re tken, stnr 60W ul with vrile illumintions is instlle. A frequeny genertor is use to use the ioes to stroe llowing the quisition of experimentl t with ON n OFF IR. The omplete setup n e foun in [1]. The t is pture for ifferent users uner vrious experimentl vriles tht simulte rel life senrios. Eh volunteer is ple 1.5 m wy from the mer. 24 test smples (5 seons eh) per volunteer re quire with omintion of ON n OFF IR. The experimentl vriles re the following: 1) pose t 0, 15, 30 n 45, 2) sujet with n without glsses, 3) sujets with ifferent skin tone level, 4) sujets with ifferent eye olor. 3 Algorithm esign This setion evelops n lgorithm to utomte the etetion of the eyes in imges tken uner low illumintion with ON n OFF IR. The propose lgorithm is ivie into two steps n esigne to etet refletions rising from glsses using ifferentil frme tehnique (onseutive IR/Non-IR frmes). The tehnique n lso e pplie on fes rotte from 0 to 45. The imges ontin only one iniviul n they re tken in poor lighting onitions. 3.1 Step 1 The gol of this setion is to extrt the fe ontour from the IR imge using the following sequene of opertions. The originl IR imges of sujet re shown in Fig. 1. The imges re tken with fe rotte t 15 n with n without glsses. 3.1.1 Non-liner ontrst streth n imge smoothing The originl imge is low pss filtere with 5x5 Gussin filter with N = 4 itertions, using the non-liner ege n ontrst enhnement lgorithm esrie in [4]. We onsier the imge gry level igitl representtion in the
Figure 1. Originl IR imges of sujet with glsses () n () without glsses. We n see refletions use y glsses (), whih n misle the lssifition prolem if it is etete s n eye region. The orresponing imge histogrms (, ) show tht most pixels hve low intensity vlues ue to poor illumintion. [0,M) rnge, where M =256 for n 8-it imge. In orer to voi loss of informtion, rithmeti opertions on imge pixel vlues re efine in logrithmil mppe spe where the forwr mpping funtion etween the imge pixel spe (F ) n the rel numer spe (Ψ) is: Ψ(F ) = log((m F ) /F ). In [4], Deng n Chill use the symols, n to represent ition, multiplition n sutrtion, respetively, in the log spe. Sine vetor ition, sutrtion n multiplition re oune opertions n well efine in the log spe, it is possile to erive non-liner equtions tht overome the loss of informtion prolem use y liner methos [2]. The itertive tehnique shown in Fig. 2 overomes the limittions of liner methos y performing non-liner weighting opertion on the input pixels of the imge. This requires the seletion of prmeters s i to ontrol the mount of highfrequenies introue in the solution. If s i < 1, the solution will e smoothe otherwise, it mplifies eges. The output of this system results in n enhne imge with reue high-frequenies ontent n etter ontrst. 3.1.2 Histogrm streth The histogrm in Fig.3() is strethe in orer to fill the entire ville gry-sle rnge (Fig.3()). Threshol vlues t L1 n t H2 re lulte orresponing to 15% n 95% of the totl numer of pixels in the histogrm of Fig.3(). This results in more visully istintive imge with ro histogrm (Fig.3)(). Also, this opertion provies etter ege elinetion, whih filittes the extrtion of the fe ontour from the kgroun. Figure 3. () Noise reution n ontrst enhnement using the log-rtio pproh. The imge is righter sine the histogrm of the imge () is shifte towrs higher intensity vlues. The histogrm streth opertion(, ) presents etter ege elinetion roun the fe region ompre to (). The upper n lower threshols re hosen s 15% n 95% of ll pixels in the histogrm shown in () 3.1.3 Non-liner orse ege enhnement The imge otine in setion 3.1.2 is low pss filtere with 9x9 Gussin filter with N = 8 itertions, using the non-liner ege n ontrst enhnement lgorithm esrie in 3.1.1. The output of this system results in orsely enhne imge with well-efine fe ounry (Fig. 4). Figure 4. : () Corsely enhne imge using nonliner orse enhnement lgorithm, () inrize imge using t H1 s threshol. Figure 2. Multisle lgorithm lok igrm showing three stges. At eh stge, the input imge (F or A) is filtere using Gussin [5x5] low-pss filter. An imge ontining only high-frequenies H(x, y) is otine y sutrting the smoothe output from the input. The ege mplifition prmeter s i is selete t eh stge, i, se on the level of high frequeny noise in H i (x,y). is slr ontrolling the ontrst level in the enhne imge. 3.1.4 Imge erosion n Ege etetion Erosion is pplie on the inry imge in Fig. 4 to reue region expnsion use y the lur effet from the non-liner ege enhnement opertion in setion 3.1.3. A 3x3 isk-shpe struturing element is use for the erosion proess. Using the eroe inry imge, Soel opertor is onstrute to perform 2-D sptil grient mesurement on n imge n gives more emphsis to high-frequeny regions tht orrespon to eges. The Soel opertor onsists
200 180 160 140 120 100 80 60 40 20 0 300 of pir of 3x3 onvolution kernels, whih re esigne to fin horizontl n vertil eges in n imge. 3.1.5 Fe ontour extrtion The following re the neessry steps for fe ontour extrtion: (i) Compute ll points on the ontour in the imge of Fig. 5(), (ii) fin n ritrry point lote in the fe region y snning the imge row-wise n y tking the men of ll ompute ege points on the ontour, (iii) strting t the pproximte fe lotion foun in (ii), serh for ll points lote on the inner fe ounry (itertively), (iii) rete n intensity vetor y summing ll intensity vlues in Fig. 5 () olumn-wise. The intensity vlues orresponing to oth mxim on the grph in Fig. 5 () n tht re lote on the inner ontour re hosen s fe proximities (Fig. 5 ()). Figure 6. Ege mplifition n noise reution proess illustrting the output of the multisle lgorithm for N = 4 itertions. The first two imges were otine fter setting s i < 1 (high-frequenies ttenution) for the first two itertions while the lst two, orrespon to n ege mplifition proess where s i > 1. The histogrms of eh imge re lso shown to illustrte how the unerlying intensity vlues of eh imge hnge s numer of itertions inreses. reue the numer of possile eye nite regions in the finl imge n will spee up the proess. Figure 5. () Ege mp using the Soel opertor, () Inner ontour extrte y serhing for ll the first pixels on the inner ontour in ege mp, ()plot of pixel intensity summtion in the vertil iretion, () Peks orrespon fe sies lotion, 3.2 Step 2 The gol of this setion is to fin ll possile eye lotion nites in n IR imge using two onseutive frmes. First, the non-ir imge is use to etet n nel ll possile refletions use y glsses n then, use the IR imge to fin finl eye lotions. 3.2.1 Non-liner fine enhnement The IR n non-ir frmes re low pss filtere with 5x5 Gussin filter with N = 4 itertions, using the non-liner ege n ontrst enhnement lgorithm esrie in [4]. Reuing the numer of itertions results in signifint reution of lur in the imge. The output of this system results in two finely enhne imges. An exmple of the fine enhnement proess n e seen in Fig. 6. 3.2.2 Imge inriztion After the enhnement proess, imges re inrize y seleting threshol t the til of the histogrm of the enhne imges (Fig. 6). Refletions re seen s white pixels in the inry imge(fig. 7). If the non-ir enhne imge ontins white pixels (i.e. refletions use y glsses or oily skin), oth imges otine in setion 3.2.1re summe together n resultnt regions with pixel intensity vlues greter thn one re set to zero (i.e. lk). This opertion removes refletions rising from glsses n reue the numer of white pixels in the imge. This will Figure 7. Finely enhne imges fter thresholing (() non-ir, () IR). Lote refletions from glsses (lk spot) re seen in (). 3.2.3 Imge lssifition Any region in Fig. 7(,) with re greter thn 50 pixels is neglete sine it nnot orrespon to eye refletions. Also, only regions lote within oth fe sies ompute in setion 3.1.5 re onsiere sine we re uniquely looking for eyes. 3.2.4 Eye etetion Compute mtrix (Λ) of size ixj s ll possile istnes in the vertil iretion etween ll eye regions suh s Λ(i,j)= y i -y j if i j n 0 otherwise. Repet the proess y omputing mtrix (Θ) of Eulien istnes etween ll possile nite regions s (Θ(i, j)= (x i x j ) 2 + (y i y j ) 2 ) where x n y re the imge oorintes in the horizontl n vertil iretion, respetively. Serh through Λ n Θ for the two nite region tht hve the smllest istne (greter thn zero) in the vertil iretion n whih hve n Eulien istne eu lote within 25% n 75% of the fe with (0.25 sies < eu < 0.75 sies ). sies is ompute from setion 3.1.5. 4 Results The lgorithm ws pplie on imges of ifferent sujets with ifferent skin olor, fe orienttion, tilt n
with/without glsses. The s i prmeters were set to [0.1, 0.1, 1, 1] in setion 3.1.1, to [0.1, 0.1, 0.1, 0.1, 1, 5, 5, 5] in setion 3.1.3 n to [0.1, 0.1, 10, 100] in setion 3.2.1. As n e seen, the s i prmeters were set to vlues equl or less thn one in setion 3.1.1 sine the sole purpose ws to reue noise n to slightly righten the imge y shifting the histogrm of the imge towrs higher pixel intensity vlues. On the other hn, fine ege enhnement ws rrie on in setion 3.2.1 sine s i 1. Fig.( 8, 9) show eye loliztion results for sujet with n without glsses, respetively. Fig. 8(,,) represent urte eye etetion while Fig. 8(,e,f) show other possile solutions for sujet wering glsses. In the se where the sujet hs no glsses, 100% eye etetion rte ws hieve (Fig. 9). e f Figure 8. Eye positions (lk ots) extrte using the evelope lgorithm for sujet wering glsses with fe oriente t (, ) 15 (, e) 30 (, f) 45. In some ses, the lgorithm presents multiple solutions tht re onsiere s possile eyes lotion (, e, f). Figure 9. Eyes lote for sujet without glsses with fe oriente t () 15 () 30 () 45 5 Disussion This pper presente new lgorithm to extrt fe n eye positions from n IR imge ontining one iniviul n tken uner poor illumintion. Conseutive non-ir/ir frmes were use in orer to etet n remove ny refletions use y glsses, whih n misle the lgorithm in extrting the right informtion out eye lotion. If the sujet oes not wer ny glsses, the lgorithm will skip the ltter step in orer to reue omputtionl omplexity. In the se where mny refletions (white pixels) our, the lgorithm will fin ll possile eye lotions n presents ll possile solutions. In orer to reue the set of possile eye region nites, extrting the nose oorintes (i.e. enter)n mking use of fe symmetry will e neessry. This will improve the performne n ury of the evelope metho while eling with fes t ifferent orienttion. Currently, the lgorithm performs etter for fes oriente etween 0 n 30. The lgorithm performs poorly for fes oriente t n ngle igger thn 45 sine these imges ontin in generl only one eye n the lgorithm is esigne to etet oth of them. However, if we know the nose lotion n fe ontour, the lgorithm n proess this informtion n present unique solution set. Similr work ws presente in [1] where eyes lotion were extrte exept tht the propose metho i not el well with rotte fes n with sujets wering glsses. Also, the evelope lgorithm in [1] i not use fine enhnement to etet pupil ontour in imges with poor ontrst n illumintion. Furthermore, they i not mke use of the informtion tht n e otine using onseutive non-ir/ir frmes to preit refletions not ssoite with pupils or eye informtion. 6 Conlusion This pper proposes n lgorithm to utomtilly etet eye lotion in IR imges of one iniviul tken uner poor illumintion. The lgorithm etet fe region in the imge, extrt the fe ontour n provies set of ll possile eye lotions. In generl, the lgorithm preitions re goo for sujets with no glsses sine it presents unique set of solutions. However, it provies multiple solutions for sujets wering glsses sine mny refletions our in the imge. Fining nose lotion in the imge n using fe symmetry to extrt the optiml eye lotions n solve this prolem. The iffiulties enountere in proessing these imges re ssoite with the poor illumintion, poor ontrst n wek ege elinetion present in the non-ir/ir imges. For now, the lgorithm nnot el with fes oriente t n ngle igger thn 45. Referenes [1] Asfw, Y., Chen, B., Aler, A., Fe etetion using re-eye effet, Deprtment of Eletril engineering, University of Ottw, Cn, 2002. [2] Bovik, AL., Hnook of Imge n Vieo Proessing, Aemi Press Texs, 2000 [3] Deng, G., Chill, L.W., Multisle imge enhnement using the logrithmi imge proessing moel,eletronis Letters, 29:803-804, 1993. [4] Deng, G., Chill, L.W., Imge Enhnement Using the Log-rtio Approh, Signls Systems n Computers, 1:198-202, 1994. [5] Fromherz, T., Stuki, P., Bihsel, M., A Survey of Fe Reognition, MML Tehnil Report, No 97.01,Dept. of Computer Siene, University of Zurih, Zurih, 1997. [6] Hro, A.,Flikner, M., Irfn, E., Deteting n Trking Eyes By Using Their Physiologil Proper-
ties, Dynmis, n Apperne Computer Vision n Pttern Reognition Conferene, June 13-15, 2000. [7] Morimoto, C.H.,Koons, D.,Amir, A.,Flikner, M., Pupil Detetion n Trking Using Multiple Light Soures Tehnil Report RJ-10117, IBM Almen Reserh Center,, Sn Jose, C, 1998. http://omino.wtson.im.om/lirry/yerig.nsf/home.