A Novel Accurate Algorthm to Ellpse Fttng for Irs Boundar Usng Most Irs Edges Mohammad Reza Mohammad 1, Abolghasem Rae 2 1. Department of Electrcal Engneerng, Amrabr Unverst of Technolog, Iran. mrmohammad@aut.ac.r 2. Department of Electrcal Engneerng, Amrabr Unverst of Technolog, Iran. rae@aut.ac.r Abstract: Ellpse fttng for rs boundar s an mportant step n ee gaze estmaton and rs recognton applcatons. In most of the proposed algorthms onl the edge mage s used for ellpse fttng. In these algorthms, the undesred edges ma lead to erroneous rs ellpse. In ths paper, we propose a novel algorthm to ellpse fttng for rs boundar whch uses gra and edge mages smultaneousl. In ths algorthm, edges of ee regon are computed and refned b some proposed constrants. Then, remander segments merge together b an optmzaton process and the best ellpse s ftted for the merged segments. In ths algorthm, most edges of rs boundar are used for ellpse fttng and hence, ts accurac s hgh. The performance of the method has been evaluated on real mages. J Am Sc 2012;8(6):758-763]. (ISSN: 1545-1003).. 94 Kewords: Ellpse fttng, Splt ponts, Merge of segments. 1. Introducton In applcatons such as ee gaze estmaton [1-5] and rs recognton [6-9] an mportant step s fttng a shape for rs boundar. In some papers, t s assumed that contour of rs s a crcle (such as [3]). But, as shown n [1], contour of rs n a 2D mage s an ellpse. In [1,2,6] edges of the ee regon mage are computed b smple algorthms and then ellpse fttng algorthms are proposed to appl on t. In [1], wth respect to ee shape and eeld effects, t s assumed that two longest vertcal segments belong to rs boundar. Then, an ellpse s ftted for the pels on these segments b the least square method. Not usng horzontal pels lead to reduced accurac of ths algorthm. Also, n some mages, t s possble that an erroneous segment s selected for rs boundar and the error of the ellpse fttng becomes ver large. An mprovement of Randomzed Hough Transform (RHT) s proposed n [2] to ellpse fttng for rs boundar and n [6] RANSAC algorthm s used for t. A common problem n the presented algorthms s usng onl edge mage to fttng the desred ellpse. In some mages, because of eelds and other edges, the best ftted ellpse to the edge mage s not dentcal to the rs ellpse. To overcome ths problem, t s requred to use gra (or color) mage and edge mage smultaneousl. In ths paper, we propose a new algorthm to ellpse fttng for rs boundar whch utlzes gra and edge mages. In ths algorthm, we compute edges of ee regon and remove splt ponts (defned n secton 3) from t. In addton, we remove some segments, wth respect to rs center, whch cannot belong to the rs boundar. Fnal step s mergng the remanng segments and fttng the best ellpse for them. The remander of ths paper s structured as follows. In Secton 2, we propose a novel algorthm to compute and refne edges of ee regon whch elmnates a lot of undesred ones. In Secton 3, an optmzaton algorthm s ntroduced to merge the remander segments. Epermental results are presented n Secton 4. Fnall, Secton 5 concludes ths paper. 2. Edge Computaton and Refnement In ths secton, we present a novel algorthm for edge computaton and refnement formed from three steps. The frst step s to compute edges. Elmnaton of splt ponts and undesred segments are the net steps dscussed n the remander of ths secton. 2. 1. Edge Computaton of Ee Regon Because of the estence of a lot of undesred edges n the ee regon, most of them related to eelds and eelashes, t s requred that the ee regon edges be computed b hgh accurac and be refned. To compute the edges, we use Cann operator n a 2- level pramd. In other words, b Cann operator, edges on low-resoluton mage are computed. Then, edges on hgh-resoluton are computed and edges whch do not have equvalent pel n low-resoluton mage are removed. B ths method, a lot of small edges, most of them related to eelashes, are removed. Fgure 1 shows an eample of edge computaton b ths method. Some pels computed as edge n ee regon mage do not belong to rs boundar (undesred pels). To elmnate a lot of them, we propose new algorthms. Undesred pels are located on segments whch all or parts of them are undesred. Thus, for 758
an segment, we compute pels whch are the locaton of the ntersecton of two (or more) dfferent segments. After ths, all pels n a segment are undesred or not. Hence, we elmnate all pels of the segments whch ther probablt to be located on rs boundar s lower than the threshold. These operatons wll be dscussed n the net parts of ths secton. dfferental vector at the pel and the average dfferental vector for some pels before and after the pel to determne splt ponts of tpe 2. The frst parameter conssts of dfferences n dfferental vectors for current and net pels. In dscrete ponts, dfferental vector s obtaned from Equaton (1): df d ˆ d jˆ (1) where d and d can be calculated from Equatons (2) and (3). d 1 d 1 (2) (3) (c) (d) Fgure 1. Edge computaton b Cann operator n a 2-level pramd, man mage, edges of hghresoluton mage, (c) edges of low-resoluton mage, and (d) result of combnaton of two levels 2. 2. Elmnaton of Splt Ponts In the edge mage of ee regon, there are two tpes of splt ponts (Fgure 2). Tpe 1 s the locaton of the ntersecton of two (or more) dfferent segments and tpe 2 s the locaton n whch the end parts of two dfferent segments are dentcal. tpe 1 tpe 2 Fgure 2. Two dfferent tpes of the splt ponts n an edge mage To compute the locaton of tpe 1 splt ponts, we mae the edge mage thn b morphologcal thnnng operator and convolve the result b a 3 3 ones flter. Pels, n the resulted mage, whch have values greater than 2 are the locatons of splt ponts of tpe 1. The man parameter to localze splt ponts of tpe 2 s the change of dfferental vector. The change of dfferental vector n an ellpse cannot be greater than a threshold. So, we use the change of To quantf the changes of dfferental vectors, we use normalzed dot product as Equaton (4): d d d d d 1 1 2 2 2 2 d 1 d 1 d d (4) If the number of ponts for an ellpse s large, d should be appromatel 1 for all ponts. But, f the ellpse conssts of onl 20 pels and ts aspect rato s smaller than 2 (equvalent to θ = 60 degrees), the values of d change theoretcall between 0.8176 and 0.9875. Thus, we set T d = 0.8 as threshold of d. Dfferental vector at a pont has hgh senstvt to nose and other errors n the edge detecton phase. Therefore, we use the average dfferental vector as the second parameter. To obtan a good estmaton of the average dfferental vector, we ft a lne n the sense of MSE for ponts before the current pel. B some calculatons, Equaton (5) results the slope of the ftted lne: m 1 l 1 l1 2 l1 l 1 l1 2 l1 (5) where and are coordnates of the prevous ponts. In the same manner, Equaton (6) s obtaned for ponts after the current pel: 759
m 2 l 1 l1 2 l1 l 1 l1 2 l1 (6) The amount of dfference between these two slopes s computed b normalzed dot product as n Equaton (7): 1 mm m 1 2 2 2 1m1 1m2 (7) If m s close to 1, t means that the average slope changes slowl. Otherwse, the average slope changes rapdl and means that ths pel can be a splt pont of tpe 2. Based on the eperments, f all ponts of the ellpse are greater than 50, selectng 4% of them s suffcent for. Wth ths choce, for ellpses wth an aspect rato smaller than 2, m changes between 0.8800 and 0.9921. So, we select T m = 0.85 as threshold of m. Wth ths descrpton, summar of the algorthm s as follows: For all pels of an segment, value of d s computed from Equaton (4) and compared wth T d. If d < T d, then, value of m s computed from Equaton (7) and compared wth T m. If m < T m, then, a splt pont of tpe 2 has detected. Fgure 3 shows an eample of applng the proposed algorthm for elmnaton of the splt ponts. t as rs center. Two samples of rs center localzaton b ths method are shown n Fgure 4. For a pel located on permeter of an ellpse, the slope s appromatel perpendcular to the lne that connects center of ellpse to t. Moreover, concavt n the pels of an ellpse s appromatel towards the center of the ellpse. So, we ft two parametrc curves of order 3 for and coordnates of the segment. Then, we compute frst and second order dervatves of the ftted curve and choose them to call d t, d t, d 2 t and d 2 t, respectvel: where t = 1: n whle n s the number of pels of the segment. To quantf the frst and second features (perpendculart of the slope to and parallelsm of the concavt b the lne that connects center of ellpse to the pel) we compute normalzed dot product for each pel as Equatons (8) and (9): s d d (8) t1 t t t t s d d (9) 2 2 t 2 t t t t Fnal parameter s calculated from weghted average of s 1 and s 2 as n Equaton (10): 1 n 1 t1 2 t2 n t 1 (10) s s s Accordng to the performed eperments, we choose 1 = 5 and 2 = -1 and set threshold value to - 3. Fgure 4 shows the results of applng ths algorthm on the mages of Fgure 4. Fgure 3. Elmnaton of the splt ponts b the proposed algorthm, an edge mage of ee regon, and elmnaton of splt ponts 2. 3. Elmnaton of Undesred Segments Segments that belong to rs boundar have curvature towards the rs center. Therefore, to elmnate undesred segments, we estmate rs center, then compute the amount of curvature for an segments and compare t b a threshold value. In ths algorthm, accurac of the rs center localzaton s not essental. Thus, we use a smple and fast algorthm for t. In the ee regon, the darest parts belong to pupl and rs regons; therefore, we convolve the mage b a large Gaussan flter and compute the locaton of mnmum value n Fgure 4. Elmnaton of the undesred segments, rs center localzaton b a smple flterng, and remander segments 3. Mergng of the Remander Segments In prevous secton, we computed and refned the edges of ee regon mage. The output of these algorthms s several segments that onl some of them belong to rs boundar. In ths secton, we propose an algorthm to merge the segments that belong to rs boundar. In [1], two longest vertcal edges are selected as rs boundar. The performance of ths method, for heads wthout rotaton, has been 760
reported to be approprate. Senstvt to head rotaton and not usng horzontal edges are the man lmtatons of ths method. In ths paper, for mamum usng of rs edges and elmnatng lmtaton of head rotaton, we defne a ftness functon whose optmzaton results the segments that should be merged. In other words, the ftness functon s defned accordng to parameters of each segment. Then, onl segments that mprove the ftness functon merge together. We use four features n the ftness functon as dscussed n net parts. 3.1 Average Dstance from the Ftted Ellpse The frst feature to merge two segments s the average dstance of the ponts n the segments from ther ftted ellpse. Ths feature epresses ellptcal measure of the ponts n the segment. It s mportant that both segments be close to the ftted ellpse. So, we propose Equaton (11) as f 1 : f dff dff 1 2 (11) 1 1 2 mn dff 1, dff 2 ma dff, dff Where dff 1 and dff 2 are average dstances of the two segments from the ftted ellpse. For two segments that are located on an ellpse, the value of f 1 wll be 0. Ftted ellpses to four combnatons of two segments are shown n Fgure 5. For these combnatons, values 0.1942, 0.1723, 1.2855 and 1.6020 are computed respectvel as f 1 whch epresses hgh performance of ths feature. (c) (d) Fgure 5. Values of f 1 for 4 eamples, 0.1942, 0.1723, (c) 1.2855, and (d) 1.6020 3. 2. Matchng of the Ftted Ellpse b All Edges The second feature s the percent of ponts n all segments that are close to the ftted ellpse of the segments. In other words, ths feature determnes how man ponts of all segments match wth ths ellpse. So, we propose Equaton (12) as the second feature: f 2 L 1 dff L T (12) where L s the number of all ponts on the segments and dff s the dstance of the th pont from the ftted ellpse. Also, T s a threshold level whose value s 0.5. Fgure 6 shows an edge mage that conssts of 5 segments. Parts to (f) show 5 combnatons (out of 10 possble combnatons) whose f 2 values are 0.3988, 0.4104, 0.0809, 0.3873 and 0.1908, respectvel. The best value for f 2 s 1; so, the computed values show the hgh performance of ths feature. (c) (d) (e) (f) Fgure 6. Matchng of the ftted ellpse b all edges, an edge mage whch conssts of 5 segments, and to (f) fve combnatons for whch values 0.3988, 0.4104, 0.0809, 0.3873 and 0.1908 are obtaned as f 2 f 1 and f 2 features use onl the edge mage to merge the segments. In man cases these features are suffcent; however, n some cases the best ftted ellpse for the edge mage s not dentcal to the best ftted ellpse to rs boundar. Therefore, we defne the thrd and fourth features whch are based on gra mage of the ee regon. 3. 3. Adjacenc to the Eeball Regon An mportant feature of the rs edges s ther adjacenc to the eeball regon. Moreover, eeball s the brghtest regon n the ee mage. So, we can etract eeball regon b a smple thresholdng algorthm and gve hgh scores to the segments whose pels are close to ths regon. We propose (13) as f 3 : f N N (13) 3 j Where N and N j are the percent of ponts n each segment beng close to the ee regon. Fgure 7 shows a sample of computng N. Values 0.9268, 0, 0, 1 and 0 are computed as N for the 5 segments of ths fgure, respectvel. These values ndcate the robustness of ths feature. 761
(c) Fgure 7. Adjacenc to the eeball regon, orgnal mage of ee regon, thresholded mage followed b openng operator to remove small partcles, and (c), fve remander segments whose values of f 3 are 0.9268, 0, 0, 1 and 0, respectvel 3. 4. Dfference of Irs Color b Other Segments Another mportant feature for rs ellpse s color of pels located nsde and outsde of t. Thus, we defne the rato of average ntenst for around pels of the ftted ellpse located nsde and outsde of t as f4. Around rs boundar, nsde pels are darer than outsde ones; therefore, ths feature can remove segments whose ftted ellpse s not compatble wth ee propertes. Equaton (14) s proposed to compute f4: M f 4 (14) M o where M and M o are the average ntenstes for pels of nsde and outsde of the ftted ellpse. Fgure 8 shows some samples to demonstrate the performance of ths feature. Parts and are the gra mage and edge mage that conssts of 5 segments, respectvel. Parts (c) to (f) are 4 combnatons (out of 10 possble combnatons) for whch the values 0.5158, 0.5219, 0.8641and 0.5951 are computed as ther f 4. So, ths s a good dscrmnatve feature. 3. 5. Optmzaton of the Fttng Functon We defne the fttng functon as a weghted sum of the dscussed features. So, Equaton (15) s the fttng functon: f Af Bf Cf Df (15) 1 2 3 4 To optmze f, we merge onl the segments that ncrease the value of f. Wth respect to the mportance of the features and preformed eperments, we choose values -3, 2, 1.5 and -2 for A, B, C and D, respectvel. Implementaton results show that the accurac of ths method s ver hgh and t can manage the estence of man undesred edges. Fgure 10 shows dfferent steps of the proposed algorthm to ellpse fttng for rs boundar n a sample mage. In part (d) of ths fgure, merged segments are shown wth whte color and other segments wth gra color. Also, the ftted ellpse, whose parameters are computed for pels on the merged segments b algorthm of [10], s shown wth blac color. (c) (d) (e) (f) Fgure 8. Performance of the f 4 for some eamples, orgnal mage of ee regon, the edge mage whch conssts of 5 segments, and (c) to (f) four combnatons for whch values 0.5158, 0.5219, 0.8641 and 0.5951 are obtaned as f 4 (c) (d) Fgure 9. Dfferent steps of the proposed algorthm for ellpse fttng to rs boundar, orgnal mage of ee regon, edge computaton, (c) edge refnement, and (d) merge of remander segments and the fnal ftted ellpse 4. Epermental Results The man motvaton of ths paper s to propose a new algorthm that, ndependent of head rotaton, uses most of the estng rs edges to ft an accurate ellpse to the rs boundar. To evaluate the proposed algorthm, we get 72 mages from ee regons. For an mage, the number of all edges, the number of rs edges and the number of splt ponts (consstng of pels located on undesred segments) are counted. 762
For all mages, from 18201 edge pels, 1406 pels are elmnated as splt ponts and 2518 pels are removed as undesred segments. Moreover, 11860 pels dentfed as the rs edges. It should be noted that 2417 pels are removed n mergng phase. The error of the proposed algorthm was 112 pels whch erroneousl have been consdered as rs edges and 120 pels whch belong to rs boundar but are not consdered. Thus, FPR and FNR of the proposed algorthm are about 1%. 5. Concluson The man contrbuton of ths stud was the development of an algorthm to accuratel estmate the parameters of the ellpse of the rs boundar. The novel aspect of ths wor was proposng new constrants to splt the edge mage to some segments and proposng new features to merge these segments. One of the future wors s defnng some new features to use n the ftness functon. Moreover, to elmnate the splt ponts we use the change of dfferental vector. To mprove ths algorthm, we can use gra mage features n the future wors. Acnowledgements: The authors would le to epress ther sncere thans to Dr M. R. Eslam Khouzan for ther valuable comments. 6. D. Zhang, A. Jan, and X. L, Modelng Intra-class Varaton for Nondeal Irs Recognton, Advances n Bometrcs, Lecture Notes n Computer Scence, Sprnger Berln / Hedelberg, 2005; 419-427. 7. Bonne B, Ives R, Etter D, Yngz D. Irs pattern etracton usng bt planes and standard devatons. n Sgnals, Sstems and Computers, Conference Record of the Thrt-Eghth Aslomar Conference on, 2004;582-586. 8. Daugman J. New Methods n Irs Recognton. Sstems, Man, and Cbernetcs, Part B: Cbernetcs, IEEE Transactons on, 2007;37(5):1167-1175. 9. Plla J, Patel V, Chellappa R, Ratha N. Secure and Robust Irs Recognton usng Random Projectons and Sparse Representatons. Pattern Analss and Machne Intellgence, IEEE Transactons on, 2011;(99):1-17. 10. Ftzgbbon A, Plu M, Fsher R.B. Drect least square fttng of ellpses. Pattern Analss and Machne Intellgence, IEEE Transactons on, 1999; 21(5):476-480. 5/15/2012 References 1. Wang J.G, Sung E, Venateswarlu R. Estmatng the ee gaze from one ee. Comput Vs Image Underst 2005; 98(1):83-103. 2. Huang S.C, Wu Y.L, Hung W.C, Tang C.Y. Pont-of-Regard Measurement va Irs Contour wth One Ee from Sngle Image. n Proceedngs of the 2010 IEEE Internatonal Smposum on Multmeda, 2010. 3. Lang D.B.B, Lm Ko H, Non-ntrusve ee gaze drecton tracng usng color segmentaton and Hough Transform. n Communcatons and Informaton Technologes, 2007. ISCIT '07. Internatonal Smposum on, 2007;602-607. 4. Kohlbecher S, Bardnst S, Bartl K, Schneder E, Potsche T, Ablassmeer M. Calbraton-free ee tracng b reconstructon of the pupl ellpse n 3D space. n Proceedngs of the 2008 smposum on Ee tracng research & applcatons, Savannah, Georga, 2008. 5. M. R. Mohammad, and A. Rae, Robust Pose- Invarant Ee Gaze Estmaton Usng Geometrcal Features of Irs and Pupl Images, the 20th Iranan Conference on Electrcal Engneerng, ICEE2012, 2012. 763