Face Authentication Based on Multiple Profiles Extracted from Range Data

Transcription

1 Face Authentication Based on Multiple Profiles Extracted from Range Data Yijun Wu, Gang Pan, and Zhaohui Wu Department of Computer Science and Engineering Zhejiang University, Hangzhou, , P. R. China Abstract. In this paper we address face authentication based on profiles extracted from range data. Three kinds of profiles are defined, then extracted, and are combined to classify. For obtaining central profile, a novel robust symmetry plane detection method is proposed. A global profile matching approach based on the partial Hausdorff metric is presented to align and compare profiles, without detection of fiducial points that is often unreliable. To utilize more information in range data, we extract nose-crossing profile and forehead-crossing profile,more than only central profile. The experiments are carried out on a low-quality database with 180 pieces of range data of 30 individuals acquired by structured light system. Based on the experimental results, we observe that the presented scheme can cope with limited quality of facial range data. 1 Introduction Even though numerous techniques for face recognition or authentication have been explored over the past two decades, most research has primarily focused on recognition from 2D image. Current methods work very well under conditions similar to those of the training images. However, either of the illumination variation and pose change may cause serious performance degradation for most existing systems. Recent advances in modelling and digitizing techniques have made acquisition of 3D data much easier and cheaper [9]. The 3D data have the potential to overcome these problems, whose advantage is the explicit representation of 3D shape. For applied point of view, an economical 3D face recognition system is preferred, whereas the data acquired by an economical fully automatic 3D acquisition system are often range data of limited quality and contains inaccurate and noisy data. 3D face recognition method tolerant to relatively coarse range data is highly expected. Because of the noise such as those caused by beard and spectacle, and the expression which makes the range data sets of same individual differ from each This work is partly supported by NSFC ( ), 863 High-Tech Programme (2001AA4180) and Zhejiang NSF for Young Scientist(RC01058).The authors would like to thank Dr. Charles Beumier for the supply of 3D RMA database. J. Kittler and M.S. Nixon (Eds.): AVBPA 2003, LNCS 2688, pp , c Springer-Verlag Berlin Heidelberg 2003

2 516 Yijun Wu et al. other, We must select some reliable and discriminative region for the recognition to get an robust approach. The central profile curve, which are stable, can give fairly high discriminability. And it is easier and faster to deal with 2d curves than 3d range data. We try to find some other profiles to utilize in the face authentication to get better performance. This paper presents a effective scheme for facial profile extraction from low-quality range data and an method for face authentication based on multiple profiles. For recognition using facial profile, many methods have been done. Most of previous work is based on extraction of fiducial points. [6, 10, 11, 5] extracted fiducial marks from the profile by heuristic rules, and a set of features was detected in terms of the positions of these fiducials. The distance between the features of a test profile s fiducial points and that of the model s fiducial points was calculated. Harmon et al. [5] manually drew the outlines from profile photos of 256 males. Nine fiducial points were selected. A set of 11 features was derived from these fiducial points. After aligning the two profiles to be matched by two selected fiducial marks, the matching was achieved by measuring the Euclidean distance of the feature vectors derived from the outlines. Wu et al. [10] developed a face profile recognition procedure based on 24 fiducial points. The outline curves were automatically obtained instead of by an artist s drawing. Then, they used a B-spline to extract turning points on the outline curve. Subsequently, six interesting points and 24 features were derived from these points. Yu et.al [11] used a tuning method to get more precise position of the fiducial points. They define a number of small steps around the determined positions of the fiducial points. For each combination of the new positions of the fiducial points, the matching was performed,and the one with the best matching score is chosen. Here we presents a face authentication method based on multiple profiles. A novel and effective method was proposed to detect the symmetry plane of face to extract the central profile. For each pair of range data sets, we use the symmetry planes and profiles to align them together, then we get the horizontal profile curves. We minimize the Hausdorff distance between two profiles to measure their difference. In the rest of this paper, we introduce our method to extract profiles from range data in section 2. In section 3 we introduce our approaches for profile matching base on partial Hausdorff distance. Our experimental result is reported in section 4. And Section 5 concludes the paper. 2 Extracting Profiles from Range Data 2.1 Symmetry Plane Detection Given range data for facial profile recognition, we should first extract the profile from range data or detect the symmetry plane. Cartoux et al [2] proposedan approach which extracts the profile by looking for the vertical symmetry axis of Gaussian curvature values of the facial surface. However computation of Gaussian curvature needs sufficiently accurate range data, which can not usually be provided by automatic 3D face acquisition system. Here we present an effec-

3 Face Authentication Based on Multiple Profiles Extracted from Range Data 517 (a) (b) (c) Fig. 1. Finding symmetry plane with alignment. (a) The original facial surface, (b) The mirrored facial surface with respect to an initial symmetry plane, (c) Alignment of both surfaces tive approach based on alignment, which is not involved in curvature and can robustly extract the symmetry plane from range data of low quality. Assume that the 3D facial surface is symmetrical and continuous. After setting the facial surface an initial but inaccurate symmetry plane, its mirrored surface can be easily obtained, as shown in Fig. 1, wherepointa in Fig. 1(b) is symmetrical to point A in Fig. 1(a) with respect to the initial symmetry plane. Once we align the two surface accurately, segment AA must be vertical to the true symmetry plane and its central point must lie on the true symmetry plane. That is the segment AA determines the true symmetry plane. The true symmetry plane can be written as n x + k =0 (1) where n is the normal vector of the plane and k is a constant. After alignment of two surfaces, for point A denoted by a (x 1,y 1,z 1 )andpointa denoted by a (x 2,y 2,z 2 ), the symmetry plane can be obtained by solve: n =( / a a ) a a n ( a + a )/2+k =0 (2) Considering error from range data acquisition and non-exact-symmetry of facial surface, in addition, for robustness purpose, actually we solve Eq. 2 for each point in facial surface and then perform least-mean-square method for final solution. The algorithm we have used for alignment in our system is a variant of ICP (Iterative Closest Points)[8], which is widely used for geometric alignment of three-dimensional models when an initial estimate of the relative pose is known. After some experimentation, we choose a variant similar to [9], which can complete alignment in one second. Some results are shown in Fig. 2. Thesample in the first row is seriously incomplete around eye regions. The second row is a sample whose data on the right side is obviously less than that on the left side. The third row shows a sample with depth rotation. However, in these cases, symmetry planes are all detected correctly.

4 518 Yijun Wu et al. (a) (b) (c) (d) Fig. 2. Profile extracting with symmetry plane detection. (a) the test model after triangle-based linear interpolation of the original range data, and its initial symmetry plane denoted by a gray line, (b) the mirrored model after aligning, (c) the symmetry plane detected, (d) the profile extracted 2.2 Horizontal Profiles To utilize more information, we considered to extract more profiles for the authentication. These profiles should be robust, insensitive to the facial expression, and have some discriminability. We choose the horizontal profile located on nose and forehead. It must be guaranteed that the positions we get on different range data sets of same individual are the same, otherwise, the horizontal profiles would be quite different. We achieved this by aligning range data sets, denoted by A and B, first by the central profile. We then use a line to fit one of central profiles, followed by rotation of the two range data sets so that the line gets upright. We then detect the nose saddle and nose tip by curvature and distance from the line. denote the vertical distance between nose saddle and nose tip as l, we get the horizontal profiles at the position l/2 belowandabovethe nose saddle, named nose-crossing profile and forehead-crossing profile. Fig. 3(a) shows the method and the position of two horizontal profiles. Fig.3(b,c) shows the nose-crossing profile and forehead-crossing profile. 3 Facial Profile Matching As described in Sect. 1, most existing methods for profile recognition are based on fiducial points, which involve in inconsistency problem of feature point detection. One solution to this issue is to match the whole profile without detection of

5 Face Authentication Based on Multiple Profiles Extracted from Range Data 519 Fig. 3. Horizontal profiles extraction. (a) the positions of two horizontal profiles, (b) nose-crossing profile, (c) forehead-crossing profile fiducial points. We propose a global matching approach to align and compare two profiles. 3.1 Similarity Metric To be tolerant towards the noise and lack of features, the function measuring the difference between two profiles should be robust enough. It should be insensitive to the small difference and mainly measure the global difference between the profiles. The partial Hausdorff distance is a function measuring the distance between two point sets [7]. Given two sets of points A = {a 1,..., a m } and B = {b 1,..., b n }, the partial Hausdorff distance is defined as where H k (A, B) =max(h k (A, B),h(B,A)) (3) h k (A, B) =kth min a A b B a b (4) where kth denotes the k-th ranked value (or equivalently the percentile of m values). If user specifies the fraction f, 0 f 1,k can be determined by k = f m. The partial Hausdorff distance is insensitive to small perturbations of the point sets and allows for small positional errors in point sets.in terms of set containment, h k (A, B) δ if and only if there is some A k A such that A k B,whereA k contains k points of A. Thuswecanthinkofh k (A, B) as partitioning A into two sets, A k which is close to (within δ of) B and the outliers A A k. It separately accounts for perturbations (by the distance δ) and for outliers (by the rank k). 3.2 Matching by Optimization For alignment of profiles, the dimension of transformation space is 3, they are rotation angle θ and translation vector (t x,t y ). After collecting the parameters

6 520 Yijun Wu et al. Fig. 4. Profiles at the initial position and the matching result into a parameter vector a =(θ, t x,t y ), given a point x in a profile, the transformation is: T 2d ( ( ) ( ) cos θ sin θ tx a ; x) =T (θ, t x,t y ; x) = x + (5) sin θ cos θ t y Thus, the alignment of profile L 1 = p i by L 0 = q j can be formalized as: argmin a H lk (L 0,T 2d ( a ; L 1 )) (6) As the wide variation of profiles and the characteristic of Hausdorff distance, many local minimums occur in the state space, with the result that many conventional optimization methods like Newton algorithm usually can not converge at the global minimum of the function. Here we use the simulated annealing method [4] to solve this optimization problem. Before optimization, we used a line to fit the profile curve, followed by rotation and translation of the profiles so that the two lines and their centroid overlapped each other. Fig. 4 demonstrates the profiles at the initial position and the final position converged. 4 Experimental Results We performed our experiment on the facial range data from 3D RMA database, which is a part of M2VTS project. The range data were obtained by a 3D acquisition system based on structured light, in xyz form. There are about 3000 points in a model. It consists of four parts (DBs1m, DBs2m, DBs1a, DBs2a) that were build up from two sessions taken in different time separated by several months. See [1] for more details. In each part a person has exactly three shots with different orientation of head: straight forward, left or right, upward or downward, and some people smiled in some shots. Since spectacles, beards and moustaches may be present, some facial features are often incomplete, like nose, eye. Fig. 5 shows two examples, two views for each. The proposed system is implemented on the P4 2.0GHz. The percentile of the parital Hausdorff distance is set to 0.8. It takes about 2.2 seconds to compare two range data sets,including symmetry plane detecting and profile matching. To combine the three scores from three kinds of profiles, we use Fisher discriminant analysis to transform the 3-dimensional score vector into a new subspace that maximizes the between-class separation.

7 Face Authentication Based on Multiple Profiles Extracted from Range Data 521 Fig. 5. Two sample models from 3D RMA, two views for each model Fig. 6(a) shows the ROC curves using central profile, nose-crossing profile, forehead-crossing profile for DBs1m and Fig. 6(d) shows the ROC curve of fusion approach for the same session. Although the accurate rate of nose-crossing profile and forehead-crossing profile are rather low, the fusion can still make the performance improved that error rate decreased to half of which using only central profile. More detailed results on six databases are shown in Tab.1. The EER(Equal Error Rate) of fusion get notably decreased from the EER of central profile. The best EER performance, tested for DBs1m, reaches 1.11%. 5 Conclusions We have proposed a robust face authentication method based one multiple profiles. we present a novel and effective method to extract symmetry plane of facial surface based on ICP. A global profile matching approach based on partial Hausdorff metric is also presented to align and compare profiles. It does 25 Receiver Operating Characteristic Central Profile Nose crossing Profile Forehead crossing Profile 25 Receiver Operating Characteristic False Rejection Rate(%) False Rejection Rate(%) False Acceptance Rate(%) False Acceptance Rate(%) (a) (b) Fig. 6. ROC curves of profile matching for DBs1m(30 persons). (a) central profile,nose-crossing profile,forehead-crossing profile, (b) fusion

8 522 Yijun Wu et al. Profile matching EER on six databases, each database has 30 indi- Table 1. viduals Databases central profile nose-crossing profile forehead-crossing profile Fusion DBs1m 2.22% 14.08% 18.85% 1.11% DBs2m 4.44% 14.08% 19.89% 2.22% DBs1m+s2m 6.67% 16.56% 21.40% 4.44% DBs1a 5.56% 16.56% 19.89% 3.33% DBs2a 7.78% 17.67% 21.40% 4.44% DBs1a+s2a 8.89% 18.77% 23.10% 5.56% not need to detect the fiducial points. By adding the nose-crossing profile and forehead-crossing profile, we improved notably the performance of the face authentication. The experiments on the 3D RMA database demonstrate that the presented scheme can cope with limited quality of facial range data and method based on profiles can get fairly high performance utilizing limited information in 3d range data. References [1] C. Beumier and M. Acheroy. Automatic 3d face authentication. 18(4): , [2] J.Y. Cartoux, J.T.Lapreste, and M.Richetin. Face authentification or recognition by profile extraction from range images. In Workshop on Interpretation of 3D Scenes, pages , Nov [3] Yongsheng Gao and Maylor K.H. Leung. Human face profile recognition using attributed string. Pattern Recognition, 35: , [4] William L. Goffe. Global optimization of statistical functions simulated annealing. Journal of Econometrics, 60:65 99, [5] L. D. Harmon and W. F. Hunt. Automatic recognition of human face profiles. Comput. Graphics Image Process, 6, [6] L. D. Harmon, M. K. Khan, R. Larsch, and P.F. Raming. Machine identification of human faces. Pattern Recognition, 13:97 110, [7] D. P. Huttenlocher, G. A. Klanderman, and W. J. Rucklidge. Comparing images using the hausdorff distance. 15(9): , September [8] Neil D. Mckay Paul J. Best. A method for registration of 3-d shapes. 14(2): , [9] Szymon Rusinkiewicz, Olaf Hall-Holt, and Marc Levoy. Real-time 3d model acquisition. In SIGGRAPH 2002 Proceedings, pages , July , 517 [10] C.J. Wu and J.S. Huang. Human face profile recognition by computer. Pattern Recognition, 23: , [11] K. Yu, X. Y. Jiang, and H. Bunke. Robust facial profile recognition. In Proc. IEEE Int. Conf. on Image Processing, volume 3, pages ,