JOURNAL OF SOFWARE, VOL. 9, NO., FEBRUARY 4 39 Eye Locaton Based on Hough ransform and Drect Least Square Ellpse Fttng Dngl Yang a, Quchan Ba a, Yuln Zhang a, Rendong J a,b, Huanyu Zhao a a Faculty of Electronc and Electrcal Engneerng; Huayn Insttute of echnology; Hua an 33, Chna; Emal: yangdngl@63.com b College of Scence, Nanjng Unversty of Aeronautcs and Astronautcs, Nanjng 6, Chna Emal: jrd7933@63.com Abstract Feature extracton and locaton of eye are very mportant n face expresson recognton, human-computer nteracton, face detecton and dentfcaton etc. hs paper presents a new effcent method for eye locaton. Frstly, gradent, ntegral mage, and Hough transform are used to obtan locaton and radus of the pupl. Secondly, CANNY operator and ellpse fttng of drect least square are used to obtan ellptc equaton of the eyeld. Fnally, the center of the ellpse s deemed to be the center of the eyeld, and the edge of fttng ellpse s deemed to be the edge of the eyeld. hs method s extremely robust and accurate n the condton that two eyes are horzontal and nclned. he average error s less than.8 pxels. Index erms Gradent; ntegral mage; Hough transform; CANNY operator; ellpse fttng of drect least square I.INRODUCION Currently, face recognton, whch s used as bologcal characterstcs, has great applcaton prospect. Face detecton s a pretreatment of the face recognton. Its detecton accuracy s very mportant to the accuracy of the face recognton [-3]. Moreover, the eye s an mportant part of the facal feature. hus, n facal expresson, human computer nteracton, dentfcaton, face detecton, locaton and feature extracton of the eye are extremely mportant. Now, the state-of-the-art eye locaton manly has two basc methods: algorthm based on template matchng [4, 5] and algorthm based on feature ponts of the eye [6, 7]. he algorthm based on template matchng can obtan not only the shape but also poston of the eye. But ts parameter selecton s more comple and calculaton speed s very slow. he method based on feature pont makes good use of feature pont to construct the eye contour. Its advantage s that t s robust, and ts calculaton speed s very fast. Its drawback s that the accuracy of the calculaton s relatvely low. In order to further mprove the accuracy of the calculaton wth the method based on the feature pont, a new method of eye locaton s proposed n ths paper. Frstly, the approxmate area of the eyeld s obtaned by utlzng gradent nformaton, ntegral mage, etc. hen the exact locaton and sze of the pupl are obtaned by utlzng Hough transform. hus, the approxmate rectangular regon of rght eye and left eye may be obtaned. Meanwhle, the length of the mnor axs of fttng ellpse, whch s the edge of the eyeld, can be obtaned by usng the radus of the pupl. At last, edge detecton of the left and rght eye regon s executed by CANNY operator. By searchng connected regon of the maxmum edge pxel, the edge mage of eyeld s obtaned. hen, ellptc equaton of the edge of eyeld s obtaned by utlzng the method of ellpse fttng. Fnally, the center of the ellpse s seen as the center of the eye. Contour of the fttng ellpse s regarded as the edge of the eyeld. he rest of the paper s organzed as follows. In secton Ⅱ, locaton and radus of the pupl are obtaned by utlzng gradent, ntegral mage, and Hough transform. hen, the eye regon can be obtaned. In secton Ⅲ, Hough transform s used to extract pupl. In secton Ⅳ, the new eye locaton method s proposed. Eyeld s obtaned wth ellpse fttng method, whch s very robust when two eyes are horzontal and nclned. Performance evaluaton and conclusons are presented n secton Ⅴ and secton Ⅵ. II. HE PREREAMEN OF HE EYE REGION In order to determne the poston of the eye, frstly, the method of face detecton n lterature [8] s adopted to extract face regon. Secondly, brghtness and chromnance nformaton of the face, from whch the background and har nformaton can be removed, are obtaned by the method of skn color detecton, corroson and expanson of mathematcal morphology. hey are Y, Cb, and Cr mage. In the experments, t s found that the Cb of the eye regon s larger than the other n Cb color space n Fg.. But the Cr of the eye regon s smaller than the other n Cr color space. So Cr s transformed wth Cr=55-Cr. After t s transformed, the Cr of the eye regon s larger than the other n Cr color space n Fg.. Manuscrpt receved January, 3; hs work was supported n part by the Natonal Natural Scence Foundaton of Chna GrantNo. 6356).Foundaton of Huayn Insttute of echnologygrantno. HGB) 4 ACADEMY PUBLISHER do:.434/jsw.9..39-33
3 JOURNAL OF SOFWARE, VOL. 9, NO., FEBRUARY 4 Fg. Cb mage Fg. Cr mage beng transformed In order to reduce the nfluence on the detecton accuracy of the eye due to the change of the lumnance, the gradent nformaton s calculated from the brghtness of the mage by the followng formula. f G = G x y f = f / x / y he modulus of the gradent vector s G ) = G A) G A) ) A x y In formula ), f s lumnance value of the pont A n the mage. G x A) and G y A) are the gradent component of the pont A. In order to calculate the gradent convenently, the gradent s normalzed frstly, then t s adjusted to a range from to 55. he eye dagram nformaton EyeMapC, j) can be calculated by chromnance and gradent nformaton: EyeMapC, j) =.5*Cb, j) Cr, j)).5*g, j) 3) After EyeMapC, j) s calculated wth formula 3), lumnance of the eye regon s brghter than the other regon. See Fg. 3. Durng the experments, the edge of the face has an effect on subsequent detecton. So the facal edge s removed from the face mage n experments. he eye dagram can be obtaned by processng wth threshold n the Fg. 3. See Fg. 4. Fg. 3 Eye dagram of the removed facal edge Fg. 4 Eye dagram beng processed through threshold A Fg. 5 Calculaton method of ntegral mage he approxmate locaton of the eye can be found from ntegral mage. Integral mage s the sum of pxels wthn the rectangular area from the upper left pont to the lower rght pont. For the pont n the mage, ts ntegral mage value s: = x, y ) 4) y < y x < x A C 3 B D Fg. 6 Calculaton method of rectangular ntegral mage Fg. 7 Image be processed wth ntegral mage In Fg. 5, the shaded porton s the value of the ntegral mage of the pont x-,. he ntegral mage value can be calculated teratvely by the followng formula: s = s y ) 5) x, s = 6) In formula 5), s s the sum of the gray value of pxels whch column ordnate does not exceed column ordnate of the pont. See Fg. 5. After ntegral mage s calculated, the sum of pxel value wthn any rectangular area of the mage can be obtaned through the ntegral mage. See Fg. 6, RectSum) s the sum of pxels value n the area A. RectSum) s the sum of pxels value n the area AB). RectSum3) s the sum of pxels value n the area AC). RectSum4) s the sum of pxels value n the area ABCD). So the sum of pxels value n the area D s: RectSumD) = RectSum4) RectSum) - RectSum) - RectSum3) 7) If the length and wdth of the eye are n the rato :, the mage of the eye n Fg. 4 can be calculated by method of the ntegral mage. See Fg. 7. In Fg. 7, the brghtest locaton of the mage s the area of eye. Although there are three relatvely brght postons, the postons of the two eyes can be obtaned accordng to the relatonshp of the x-axs and y-axs of the two eyes. See equaton 8), 9). After the postons of the two eyes are found, t s marked n the orgnal mage. See Fg. 8. hen the regon contanng two eyes can be extracted accordng to the dstance of the two eyes, and gradent nformaton of two eyes can be obtaned by CANNY operator. See Fg. 9. Fg. 8 Approxmate locaton of eye area ε x zuo x you ε 4 y zuo y you Fg. 9 Gradent mage of eye area ε s threshold 8) ε s threshold 9) III. EXRAC PUPIL WIH HOUGH RANSFORM 4 ACADEMY PUBLISHER
JOURNAL OF SOFWARE, VOL. 9, NO., FEBRUARY 4 3 Hough transform s a mappng relatonshp from the mage space to parameter space. It s to cluster pxels together by means whch have a certan relatonshp n the mage space. It s to look for cumulatve correspondng pont n parameter space, whch can lnk these ponts by certan analytcal form. It s very tolerant to fault and very robust when regon boundary s ntermttent, whch s nterfered by nose or covered by other object. hs transform has desred effect n the parameter space that s less than two-dmenson. But computaton complexty wll ncrease rapdly wth parameter space dmenson ncreasng [9]. he standard equaton of the crcle s: x x ) y y) = r. In order to reduce computaton, r s set to change from R to Rmax n standard equaton. So x s set to change from xstart xstart=x- to xend xend=x. y s set to change from ystart ystart=y- to yend yend=y. In practce, error of r s allowed. herefore, equaton 9) s used n the experments. x ε x 9) ) y y) = r ± ) In equaton 9),ε s used to control range of error. Step of the Hough transform of the crcle s: ) Set up a three-dmensonal array B. ) For an arbtrary pont whch value s not zero n Fg.9. Frstly, r s set to certan value. hen x and y are changed. If t meets the equaton 9), accumulator array B plus. 3) he value of r s changed accordng to the step. hen x and y are changed. If t meets the equaton 9), accumulator array B plus. 4) Accumulator arrays are voted alternately. Each pont whch value s not zero s calculated. If the peak of the accumulator array s found, then the center and the radus of the crcle are obtaned accordng to equaton 7), 8). he obtaned center and the radus of the crcle are the center and the radus of the pupl respectvely. See Fg.. hen approxmate range of mnor axs and center of the fttng ellpse of eyeld are obtaned. Fg. Pupl found wth Hough transform Fg. Image of left and rght eye Fg. Edge mage of two eyes Fg. 3 Maxmum edge pxel connected regon After center and radus of left and rght pupl are obtaned, approxmate regon of the left and rght eye can be got. See Fg.. It s processed wth method of edge detecton. See Fg.. In Fg., not only edge of eyeld contour, but also edge of non-eyeld contour s ncluded. So maxmum edge pxel connected regon must be found, whch s consdered to be the true contour of the eye. See Fg. 3. IV. DEERMINING EYELID WIH ELLIPSE FIING MEHOD here are two methods to fnd ellpse manly. he frst method s votng clusterng. he second method s optmal algorthm. Hough transform belongs to the frst class. he drect least square fttng method belongs to the second class. hs algorthm s that the smallest squared sum of algebrac dstance s used as fttng mathematcal prncple. he plane conc general equaton of ellpse can be represented by formula ) []. F B, X ) = B X = ax bxy cy dx ey f ) In formula ), B = [ a, b, c, d, e, f ], X = [ x, xy, y, y,], F B, X ) s algebrac [ x, y to quadratc curve F B, X ). If 4ac b s subject to 4ac b =, dstance from pont ] [ N the fttng curve s ellpse [,]. Smallest squared sum of algebrac dstance from pont x, y ] to quadratc curve F B, X ) s to fnd mnmal F B; X ). N s the number of fttng pont. = Supposng: H ] N 6 = [ X X... X N, = C ) he method of drect least square fttng can be transformed nto fndng coeffcent [ a, b, c, d, e, f ] to obtan mnmal B CB = HB under the condton of. hen Lagrange operator s ntroduced, and the same soluton equaton s obtaned by equaton, 3): H HB λ CB = ) B CB = 3) he soluton of equaton ) can be obtaned wth the 4 ACADEMY PUBLISHER
3 JOURNAL OF SOFWARE, VOL. 9, NO., FEBRUARY 4 generalzed egenvalue and generalzed egenvector. Its soluton s λ, u ). For any α, λ, α u ) s R also the soluton of equaton ). So λ, α u ) must meet α u Cu =.hen α = = u Cu Usually, elements of u λ H H H are postve for the m- u H Hu always are age data. hus, denomnator postve for all Hu 4) u. hus, the exstng condton of the square root n equaton 4) s λ. herefore, the soluton of the equaton ) must have a postve egenvalue. Because the egenvalue of the C s [-,-,,,,], there s only one unque generalzed egenvalue λ > and generalzed egenvector u that s used as soluton of ellpse fttng. hus, the soluton of fttng ellpse s B = α u. After sx parameters of the ellptc equaton are obtaned, equaton 5) can be obtaned when quadratc curve ax bxy cy dx ey f = s rotated and shfted. x ' x) M y ' y) N = > 5) he angle of nclnaton of the ellpse can be obtaned: theta =.5*atanb/a - c)) 6) he center coordnate of the ellpse n rotatng coordnate system s: x = round-.5*d*costheta).5*e sntheta))/a.5*b*tantheta))) * 7) y = round--.5*d*sntheta).5*e costheta))/c -.5*b*tantheta))) * 8) he center coordnate of the ellpse n mage s: xc = x*costheta) - y* sntheta) 9) yc = x* sntheta) y* costheta) ) he length of the major axs and the mnor axs of the ellpse are respectvely: M = -C /a.5*b* tantheta)) ) the center of the ellpse s deemed to be the center of the eyeld, and the edge of the fttng ellpse s deemed to be the edge of the eyeld. Fg. 4 Fttng ellpse of two eyes V. ANALYSIS OF EXPERIMENAL RESULS In experments, PC s dual-core PentumR) 4 CPU 3.6G, 3.7G, Memory s.5g, software s MALAB7.Rb). All procedures are programmed wth MALAB. Due to usng the chromnance nformaton n experments, ths method s only sutable for color mages. All color mages are obtaned from the nternet. he frst group s the face mage that two eyes are horzontal. he second group s the face mage that two eyes are nclned. he thrd group s the face mage that two eyes are closed. here are 4 mages each group. here are mages n all. he sze of each mage s 8 * 8. he frst experment s that the eye locaton and ellpse fttng s executed to the frst group, the second group, and the thrd group wth the method proposed n ths paper and the method n lterature []. See Fg. 5. In Fg. 5, the top row of the a), b), c) s the detectng result respectvely to three groups of mage wth the proposed method. he bottom row s the detectng result respectvely to three groups of mage wth the method of lterature []. he detecton accuracy and the detecton tme are shown n able and able. As can be seen from Fg. 5, able and able, the detecton accuracy s much hgher wth the proposed method than wth the method of lterature []. Especally to the nclned ellpse, the detecton accuracy s much hgher. But the detecton tme s much longer wth the proposed method than wth the method of lterature []. he reason s that ellpse s set to be horzontal and no nclned n lterature [], whle ellpse s assumed to be nclned and rotary n ths paper. he angle of nclnaton can be calculated accurately wth the proposed method. So the calculated tme wll be longer. he second experment s to fnd the center of fttng ellpse, calculate the length of the major axs and the mnor axs of the ellpse respectvely, and obtan the error of the center of the eye poston wth the proposed method and the method of lterature [].hree groups of mages are tested respectvely. he results are shown n able 3 and able 4. As can be seen from able 3 and able 4, the error of coordnate of the eye center and the error of the length of the major axs and the mnor axs measured wth the proposed method are smaller than wth the method of lterature []. N = -C /c.5*b*tantheta)) ) In above formula, C s the ntermedate varable. Edge mage n Fg. 3 s ftted to obtan fttng ellpse wth drect least square method. See Fg. 4. In Fg. 4, a) Detecton results when two eyes are horzontal b) Detecton results when two eyes are nclned 4 ACADEMY PUBLISHER
JOURNAL OF SOFWARE, VOL. 9, NO., FEBRUARY 4 33 REFERENCES c) Detecton result when two eyes are closed Fg. 5 Detecton results wth two methods ABLE HE DEECION ACCURACY AND HE DEECION IME WIH HE PRO- POSED MEHOD group detecton accuracy detecton tme /s Frst 97%.5 Send 9%. hrd 75%.4 ABLE HE DEECION ACCURACY AND HE DEECION IME WIH HE MEHOD OF LIERAURE [] group detecton accuracy detecton tme /s Frst 95%. Send 85%. hrd 73%. ABLE 3 AVERAGE ERROR WIH HE PROPOSED MEHOD PIXEL) he proposed method group ellpse center major axs mnor axs Frst.5.9. Second.6.3. hrd..5.5 able 4 Average error wth the method of lterature [] pxel) group ellpse center major axs mnor axs Frst.6.. Second.3.8.7 hrd.4.6.54 VI. CONCLUSIONS In ths paper, gradent nformaton, ntegral mage, Hough transform, CANNY operator edge detecton, the method of maxmum edge pxel connecton regon, the method of least squares ellpse fttng are used to obtan ellpse contour of the eyeld. hen the center of the ellpse s deemed to be the center of the eyeld. Fttng ellpse s deemed to be the edge of the eyeld. hereby, eye s located. In the experments, the proposed method s proved to be strongly robust and hghly accurate. Its average error s.7 pxels. he nadequacy of the algorthm s that detecton accuracy s low n the closed eye or nearly closed. he next step wll be to consder how to mprove t. [] Youja Fu, He Yan, Janwe L, Rux Xang, Robust Facal Features Localzaton on Rotaton Arbtrary Mult-Vew Face n Complex Background, Journal of Computers, vol. 6, no., pp.337-34,. [] D Wu, Je Cao, Jnhua Wang,We L, Mult-feature Fuson Face Recognton Based on Kernel Dscrmnate Local Preserve Projecton Algorthm under Smart Envronment, Journal of Computers, vol. 7, no., pp.479-486,. [3] Zqang Wang, Xa Sun, Optmal Kernel Margnal Fsher Analyss for Face Recognton, Journal of Computers, vol. 7, no. 9, pp.98-35,. [4] Yulle A L, Hallnan P W, Cohen D S, Feature extracton from faces usng deformable templates, Internatonal Journal of Computer Vson, vol. 8, no., pp.99-,99. [5] Yn L, Basu A, Realstc anmaton usng extended adaptve mesh for model based codng, Proceedng of the nd Internatonal Workshop on Energy Mnmzaton Methods n Computer Vson and Pattern Recognton. Sprnger, pp.69-84,999. [6] L Yunhan, Zhu Shanan, Hough transform for eye feature extracton, Journal of Zhejang Unversty Engneerng Scence), vol. 4, no. 7, pp.65-68,8. [7] Goto, Lee W S N, Facal feature extracton for quck 3D face modelng, Sgnal Processng: Image Communcaton, vol. 7, no. 3, pp.43-59,. [8] YANG Dng-l, BAI Qu-chan, ZHANG Yu-ln et al, A Improved Method of Fast Face Detecton, Journal of Huayn Insttute of echnology, vol., no., pp.-6,. [9] Gonzalez.R.C. Dgtal mage processng, Publshng House of Electroncs Industry, Pekng, 5. [] QIU Wesheng. Analytc Geometry, Pekng Unversty Press, Pekng, 999. [] JIN Junca, ONG Weqng, LIANG Xaon et al, Accurate eye locaton n near-nfrared mages based on ellpse fttng, Journal of East Chna Normal Unversty Natural Scence), no. 3: pp.3-,. Dngl Yang receved hs M.S. degree n electronc and nformaton engneerng from Southeast Unversty, Chna, n June 6.Hs current research nterest ncludes dgtal sgnal processng, mage processng, and pattern recognton. Quchan Ba receved hs M.S. degree n electronc and nformaton engneerng from Northwestern polytechncal Unversty, Chna, n June 6.Hs current research nterest ncludes mage processng, and pattern recognton. Yuln Zhang receved hs Ph.D. degree n communcaton and control engneerng, from Jangnan Unversty, Chna, n. Hs current research nterest ncludes pattern recognton. 4 ACADEMY PUBLISHER