2014 22nd Internatonal Conference on Pattern Recognton Face Recognton usng 3D Drectonal Corner Ponts Xun Yu, Yongsheng Gao School of Engneerng Grffth Unversty Nathan, QLD, Australa xun.yu@grffthun.edu.au, yongsheng.gao@grffth.edu.au Abstract In ths paper, we present a novel face recognton approach usng 3D drectonal corner ponts (3D DCPs). Tradtonally, ponts and meshes are appled to represent and match 3D shapes. Here we represent 3D surfaces by 3D DCPs derved from rdge and valley curves. Then we develop a 3D DCP matchng method to compute the smlarty of two dfferent 3D surfaces. Ths representaton, along wth the smlarty metrc can effectvely ntegrate structural and spatal nformaton on 3D surfaces. The added nformaton can provde more and better dscrmnatve power for obect recognton. It strengthens and mproves the matchng process of smlar 3D obects such as faces. To evaluate the performance of our method for 3D face recognton, we have performed experments on Face Recognton Grand Challenge v2.0 database (FRGC v2.0) and resulted n a ran-one recognton rate of 97.1%. Ths study demonstrates that 3D DCPs provdes a new soluton for 3D face recognton, whch may also fnd ts applcaton n general 3D obect representaton and recognton. Keywords3D drectonal corner ponts; 3D drectonal corner pont matchng; 3D face recognton I. INTRODUCTION Human target recognton has been an actve research area n the last decades, wth one of the maor topcs on automatc face detecton and matchng for the purpose of verfcaton and dentfcaton [1]. Sgnfcant achevements have been reached on two-dmensonal (2D) face recognton, but there are stll varous problems n handlng large amount of facal varances caused by changes n llumnaton, pose, expresson and age. Because the human face s a three-dmensonal (3D) obect whose 2D proecton s senstve to above changes. To overcome the nherent lmtatons assocated wth 2D face recognton technology, usng 3D face nformaton n face recognton has attracted ncreasng attenton, wth varous technques beng presented n recent years [2]. 3D face recognton s expected to be less senstve to llumnaton and pose varances because the 3D shape of a facal surface that s related to the nternal face anatomy nstead of external appearance and envronment. In addton, the geometrc nformaton avalable n 3D data s thought to provde more dscrmnatve features for face recognton. However, 3D face recognton technques also have ther own drawbacs. The conventonal methods [3] use holstc pont clouds and meshes on 3D face are computatonally expensve and n hgh storage demand. Therefore, t s crucal to fnd effcent and meanngful feature descrptors of 3D face Jun Zhou School of Informaton and Communcaton Technology Grffth Unversty Nathan, QLD, Australa Jun.zhou@grffth.edu.au structure to perform recognton. Mahoor and Mohamed [4] encoded the range data of 3D face nto a rdge mage, whch showed the locatons of rdge lnes around the mportant facal regons on the face (.e., eyes, nose, and mouth). Then teratve closet ponts (ICP) matchng method was utlzed to match the 3D ponts lyng on rdge mage of a gven probe to the created rdge mages of the subects n the gallery. In ther wor, only about 14% 2% of the total number of ponts on the range data were used, but acheved 91.8% accuracy n experments on FRGC v2.0 database. However, wthout consderng the nherent local structural characterstcs nsde such mages, ths method uses only the spatal nformaton of rdge mage. In ths paper, a novel 3D face descrpton and smlarty measurng technque s proposed, whch effectvely harnesses structural and spatal nformaton on a 3D surface, and reduces the storage requrement. Unle methods usng local operaton of solated ponts, the proposed approach employs 3D drectonal corner ponts (3D DCP) matchng n whch drectonal nformaton showng connectvty to ts neghbors s utlzed n the pont correspondence. Moreover, the storage load s further reduced by usng salent corner ponts nstead of all the ponts on those rdge and valley curves. The whole algorthm flowchart s llustrated n Fg. 1. In the followng, Secton 2 presents the proposed 3D drectonal corner ponts, whch ncorporates structural nformaton wth spatal features. Secton 3 presents our 3D drectonal corner ponts matchng method. Encouragng expermental results on a publc database are reported n Secton 4. Fnally, the paper concludes n Secton 5. Probe 3D Face Data Gallery Normalzed 3D Face Normalzed 3D Face Rdge and Valley Curves 3D Drectonal Corner Ponts Matchng Rdge and Valley Curves Fg. 1. Overvew of the proposed method 3D Drectonal Corner Ponts Identty 3D Drectonal Corner Ponts 1051-4651/14 $31.00 2014 IEEE DOI 10.1109/ICPR.2014.483 2802
Fg. 2. Descrbng 3D shapes usng 3D drectonal corner ponts. The red ponts are 3D DCPs on rdge curves and the blue ponts are 3D DCPs on valley curves. II. 3D DIRECTIONAL CORNER POINTS In ths secton, we frst gve a bref ntroducton on the concept of prncple curvatures, based on whch we then propose 3D drectonal corner ponts as a descrptor for 3D shapes. For a gven pont on 3D face surface S, the maxmal and mnmal prncpal are max and, and ther correspondng mn prncpal drectons are denoted as t max and t. The mn defnton of ponts on rdge or valley curves n dfferental geometry are characterzed by e / t, e / t max max max mn mn mn e 0, e / t 0, ( rdges) max max max max mn e 0, e / t 0, ( valleys) (1) mn mn mn mn max As a curvature-based feature descrptor, rdge and valley curves on a 3D face surface along the eyes, the nose, and the mouth where the surface bends sharply are geometrcally and perceptually salent surface features, so they are expected to contan enough dscrmnatve nformaton for face recognton. In addton, codng 3D face nto rdge and valley can decrease the storage demand and computatonal expense. However, rdge (or valley) mage utlze spatal nformaton of 3D face but lac structural representaton. In ths study, we propose a 3D face feature descrptor, 3D drectonal corner ponts to solve ths problem, whch combnes the structural connectvty nformaton wth spatal nformaton of 3D faces. After detectng rdge and valley curves on 3D surface, a corner pont detecton process, whch s based on Douglas-Peucer algorthm [5], s then appled to generate 3D DCPs of the face. A 3D DCP, represented as P( x, y, z, n, n ), consst of Cartesan coordnates ( x, y, z ) and two unt drectonal vectors n 1 and n. n s the unt vector that ponts 2 1 to ts front neghborng corner pont. Smlarly, n 2 s the unt vector that ponts to ts rear neghborng corner pont. If a 3D DCP s a start pont of a curve, a null s assgned to n. If a 3D 1 DCP s an end pont of a curve, a null s assgned to n. A 3D 2 DCP s a two-drectonal corner pont wth two unt drectonal vectors pontng to ts two neghborng 3D DCPs or a onedrectonal corner pont (start/end pont) of the valley (or rdge) curves wth a sngle unt drectonal vector pontng to ts neghborng 3D DCP. These drectonal vectors provde solated feature ponts wth addtonal structural nformaton about the connectvty to ther neghbors, whch can enhance the dscrmnatve power of the descrptor. Moreover, the 3D DCP descrptor, usng spare ponts, further reduces the storage demand of a 3D shape representaton. Fg. 2 shows the 3D DCPs of an example 3D face n FRGC v2.0 database. 2803
Fg. 3. An llustraton of 3D DCP converson: (a) two 3D DCPs before convertng, (b) after translaton operaton, (c) after rotaton 1 operaton, (d) after rotaton 2 operaton, (e) after open/close operaton III. 3D DCP MATCHING After utlzng the 3D DCP detectng process above, a 3D face s encoded nto a set of 3D DCPs along rdge and valley curves, whch contan both poston and drecton features. A pont-to-pont convertng process s developed to calculate the dfference between two 3D DCPs from a face n probe database and a face n gallery database. The dssmlarty between the two faces s then calculated through a global convertng process between the two 3D DCP sets. A. Pont-to-pont Correspondence A A A A A B B B B B Let A( x, y, z, n, n ) and B( x, y, z, n, n ) be two 3D DCPs. The cost of convertng A to B (vce verse) s calculated through a four-step process that conssts of translaton operaton related to locaton feature, and rotaton and open/close operatons related to drecton feature. Fg. 3 llustrates the whole process where unt vector A A A A A B B B B B n ( n n ) / n n and n ( n n ) / n n are consdered as the prncpal drectonal vector of 3D DCPs A and B, whch are plotted n red and blue dashed lnes. To help demonstrate the whole convertng process, there are several B B A A planes plotted n Fg. 2: n n, n n, n n n A n B n B *, //., 1) Translaton operaton: A translaton operaton from A to B, denoted as T A B, moves A to the locaton of B, then x x, y y and z z (Fg. 3(a) and (b)). The cost functon for a translaton operaton from A to B s defned as 2 2 2 ( ) ( ) ( ) (2) C T x x y y z z 2) Rotaton 1 operaton: A rotaton 1 operaton from A to B, denoted as R1 A B, rotates w.r.t to n A tll (Fg. 3(b) and (c)). The cost functon for a rotaton 1 operaton from A to B s defned as: C R A B 1 90 arccos( n n ) 90 arccos( ( n n ) ( n ( n n )) ) A A B where n and n are the normal vectors of planes and respectvely. 3) Rotaton 2 operaton: A rotaton 2 operaton from A to B, denoted as R2 A B, rotates w.r.t to n A tll (Fg. 3(c) and (d)). The cost functon for a rotaton 2 operaton from A to B s defned as: arccos (3) C R n n 2 (4) 2804
4) Open/Close operaton: An open (or close) operaton from A to B, denoted as O / C A B, opens (or closes) the two drectonal vectors of A untl the two drectonal vectors concde wth the correspondng drectonal vectors of B (Fg. 3(d) and (e)). The cost functon for an open/close operaton from A to B s defned as: B B A A C O / C arccos n n arccos n n (5) Let A B denote a convertng operaton from A to B. Equaton (6) defnes the cost functon for convertng A to B as an ntegrated cost of above four operatons. C R A B 1 2 2 C A B C T A B f C R2 A B (6) C O / C A B where f ( x) s a non-lnear functon to penalze large angle devaton resulted from nter-class dfference, but gnore small varaton derved from segmentaton error or ntra-class dfference. In ths paper, a quadratc functon x f ( x) (7) W s used, where W s the weght to be determned by a tranng process and the delmt of f ( x ) s [0,450 ) as llustrated n Fg. 2. For converson between two one-drectonal 3D DCPs A A A A A( x, y, z, n ) and B( x B, y B, z B, n B ) ( = 1 or 2, = 1 or 2), the translaton operaton s the same as two-drectonal 3D DCPs, but the rotaton and open/close operatons are dfferent. A rotaton operaton from A to B rotates n A to n B. The cost functon for a rotaton operaton between two sngledrectonal 3D DCP s defned as arccos 2 C R n n (8) To arrange the drecton related operaton cost n the same range [0,450 ) as that of two-drecton 3D DCPs. The cost functon between two one-drectonal 3D DCPs s defned as 2 2 2.5 C C T f C R (9) B. Set-to-set Correspondence We start from the defnton of two fnte 3D DCP sets r v r r r v v v G G G G{ A, A,, A, A, A,, A } and p q r v r r r v v v P P P P{ B, B,, B, B, B,, B } that represent a s t gallery and a probe n the 3D face database respectvely, where superscrpts r and v stand for rdge and valley. r v Therefore, G conssts of two subsets G and G that correspond to 3D DCPs along rdge and valley curves, and smlar settng apples to P. A 3D DCP set to set convertng process s proposed to establsh every 3D DCP correspondence between the two 3D DCP sets by mnmzng the global converson cost. For each 3D DCP A n G, ts correspondng 3D DCP B n P s dentfed as the one wth mnmum convertng cost from A to B among all B P ( represents r or v). The cost for establshng the correspondence for can be calculated by A C( A ) mn C( A B ) (11) B P Smlar to least trmmed square - HD (LTS - HD) [6] to handle the outler nose derved from the 3D scannng and corner detectng process, the cost for convertng the whole set G and P, denoted as G P, s defned as 1 1 C( G P ) C( A ) mn C( ) where H H H B 1 1 P H (12) H h N (0 h 1), N G G A G s the number of 3D DCP n G, and C( A ) are sorted n sequence C( A ) C( A ) C( A N ). The measure s calculated by elmnatng the large converson cost values and only eepng the h fracton of the smallest convertng cost. In ths study, the value of h s set as 0.8, whch results the best recognton performance. Fnally, the dssmlarty between G and P s defned as the maxmum of the two mnmum costs to establsh correspondences from G to P and vce versa. In (13), the dssmlarty costs between rdge and valley subsets are calculated frst, then the fnal result s merged from the two subset dssmlartes. Ths fuson process can further mprove the dscrmnatve power. For convertng between a one-drectonal 3D DCP and a two-drectonal 3D DCP, the translaton operaton s the same, but the maxmum value 450 s set as the drecton related operaton cost. Because t s desrable to prohbt convertng between two 3D DCPs of dfferent types. The cost functon between two one-drectonal 3D DCPs s defned as 2 2 450 (10) C C T f r r v v D( G, P) D( G, P ) D( G, P ) r r r r max[ C( G P ), C( P G )] v v v v max[ C( G P ), C( P G )] (13) 2805
Fg. 5. The effect of W on recognton rate. Fg. 4. Sample 3D faces used n our experments. (a) Normalzed faces, (b) Rdge data, (c) Valley data. IV. EXPERIMENTS AND RESULTS The FRGC v2.0 [7] 3D face database was used n our experments to evaluate the feasblty and effectveness of the proposed approach. The FRGC v2.0 database contans 4950 face texture and range mages, dvded nto three sets, namely Sprng2003, Fall2004 and Sprng2004. In lne wth FRGC v2.0 protocol, n our experments, the tranng set was generated from Sprng2003 and the test mages were generated from the other two sets. The face mages are extracted, normalzed and cropped n the same manner as n [8]. Spes n the range maps are removed and holes are flled. Fg. 4 llustrates several samples of normalzed and cropped face mages n the FRGC v2.0 dataset. A. Determnaton of Parameter W In ths secton, we nvestgate the effect of parameter W n (7) on the recognton accuracy. In ths experment, 100 people wth 2 neutral 3D faces per person from the FRGC v2.0 Sprng2003 were used to create a tranng dataset. The neutral 3D face n sesson one were used to construct the gallery database, and the neutral 3D face n sesson two were used as probe mages. The recognton rate s plotted aganst the values of W n Fg. 5. It s observed that the algorthm performed badly wth a low value of W and only acheved 7.2% when W = 10. The performance ncreased qucly and reached the optmal value of 98% when W was 600 and remaned stable tll 1200. In the rest of experments n ths study, W was set as 1000. Fg. 6. CMC curves of the proposed approach B. Face Recognton In ths experment, we nvestgated the effectveness of the proposed approach on the neutral 3D face mages of the FRGC v2.0 database. There are 388 subects whch have at least two neutral mages captured n dfferent sessons. However, some mages were found lost or corrupted after downloadng through Internet. 385 subects are complete and can be used. For each subect, one neutral 3D face was used as a gallery whle the other was used as a probe. The performance s measured n terms of the Cumulatve Match Characterstcs (CMC) [9] and the ran-1 recognton rate. In order to demonstrate the performance mprovement resulted from the fuson process, we also presented the fnal fuson result, as well as the result based on rdge (or valley) data only, as llustrated n Fg. 6. It s encouragng to fnd that the fuson process mprove the recognton performance greatly. The ran-1 recognton rate of the methods gven n Fg. 6 above are tabled n Table 1 together wth the reported results of Mahoor and Mohamed [4]. Note our methods perform consstently superor to the benchmar method based 2806
TABLE 1. RECOGNITION ACCURACIES ON 3D FACE SCANS Method Recognton Result Pont Number* 3D DCP (fuson result) 97.1% 5.9% 3D DCP (rdge data) 95.1% 3.7% 3D DCP (valley data) 92.7% 2.2% Mahoor and Mohamed [4] (rdge mage) Mahoor and Mohamed [4] (entre surface) 91.8% 14% 93.7% 100% * Pont number s calculated through the percentage of the entre 3D face scans. [5] D. H. Douglas and T. K. Peucer, "Algorthms for the reducton of the number of ponts requred to represent a dgtzed lne or ts carcature," Cartographca: The Internatonal Journal for Geographc Informaton and Geovsualzaton, vol. 10, pp. 112-122, 1973. [6] D. G. Sm, O. K. Kwon, and R. H. Par, "Obect matchng algorthms usng robust Hausdorff dstance measures," Ieee Transactons on Image Processng, vol. 8, pp. 425-429, 1999. [7] P. J. Phllps, et al., "Overvew of the face recognton grand challenge," n Proceedngs of IEEE Computer Socety Conference on Computer Vson and Pattern Recognton, pp. 947-954, 2005. [8] A. S. Man, M. Bennamoun, and R. Owens, "An effcent multmodal 2D-3D hybrd approach to automatc face recognton," IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 29, pp. 1927-1943, 2007. [9] S. A. Rzv, P. J. Phllps, and H. Moon, "The FERET verfcaton testng protocol for face recognton algorthms," n Proceedngs of the 3rd Internatonal Conference on Automatc Face and Gesture Recognton, pp. 48-53, 1998. on rdge mage, whch demonstrates that the added structural nformaton maes 3D DCP a more dscrmnatve measure than tradtonal feature pont based method. The fuson result of our method s also hgher than Mahoor and Mohamed [4] on the entre surface by 3.4%. In addton, compared wth the benchmar that usng entre surface, our 3D DCP methods based on rdge data, valley data and fuson process, requre only about 3.7%, 2.2% and 5.9 % storage space, whch further decreases the storage demand. V. CONCLUSIONS Ths paper presents a new 3D face recognton method usng 3D drectonal corner ponts (3D DCPs), whch employs both spatal and structural nformaton of rdge and valley curves on 3D surface. In order to represent the 3D shape effcently, we extract 3D drectonal corner ponts from rdge and valley curves. Such sparse pont representaton can further reduce the storage demand and the drectonal attrbutes can effectvely enhance the dscrmnatve power. The proposed method has been evaluated on the FRGV v2 dataset and been compared wth a benchmar approach based on rdge mage. It s very encouragng to fnd that the 3D DCP method performed superor to the benchmar approach n terms of hgher recognton accuracy and less storage space demand. Ths study reveals that 3D DCPs provdes a new soluton for 3D face recognton, whch may also fnd ts applcaton n general 3D obect representaton and recognton. REFERENCES [1] W. Zhao, R. Chellappa, P. J. Phllps, and A. Rosenfeld, "Face recognton: A lterature survey," ACM Computng Surveys, vol. 35, pp. 399-459, 2003. [2] K. W. Bowyer, K. Chang, and P. Flynn, "A survey of approaches and challenges n 3D and mult-modal 3D+2D face recognton," Computer Vson and Image Understandng, vol. 101, pp. 1-15, 2006. [3] G. Medon and R. Waupottsch, "Face modelng and recognton n 3- D," n Proceedngs of IEEE Internatonal Worshop on Analyss and Modelng of Face and Gestures, pp. 232-233, 2003. [4] M. H. Mahoor and M. Abdel-Mottaleb, "Face recognton based on 3D rdge mages obtaned from range data," Pattern Recognton, vol. 42, pp. 445-451, 2009. 2807