3D FACE RECOGNITION FROM RANGE DATA

Transcription

1 International Journal of Image and Graphics Vol. 5, No. 3 (2005) c World Scientific Publishing Company 3D FACE RECOGNITION FROM RANGE DATA GANG PAN and ZHAOHUI WU Department of Computer Science, Zhejiang University Hangzhou, Zhejiang , P.R. China gpan@cs.zju.edu.cn Received 8 March 2003 Revised 24 December 2003 Accepted 10 March D facial data has a great potential in overcoming the problems of illumination and pose variation in face recognition. In this paper, we investigate face recognition from range data by facial profiles and surface. An efficient symmetry plane detection method for facial range data is presented to help extract facial profile. A global profile matching method is then exploited to align and compare the two profiles without detecting fiducial points that is often unreliable. The central profile and two kinds of horizontal profiles nose-crossing profile and forehead-crossing profile are employed in recognition. For each individual, a statistical model is built to represent the distinct discriminative capability of the different regions on the facial surface. It is then incorporated into a weighted distance function to measure for the similarity of surfaces. The comparable experimental results are achieved on a facial range data database with 120 individuals. Keywords: 3D face recognition; surface matching; facial profile; symmetry plane detection. 1. Introduction The automatic face recognition based on 2D images has been actively researched during the past three decades, and various techniques have been presented, 1,2 such as Eigenface, 3 Fisherface, 4 elastic bunch graph matching 5 and Kernel method. 6 Great strides have been made in recent years, and the existing methods usually work very well under well-controlled condition. However, variations in illumination, facial expression and pose may cause serious degradation in performance for most existing systems. Despite much effort made for these problems, e.g., modeling illumination, 7 SFS based view synthesis 8 and employing 3D morphable model to correct the pose, 9 robust face recognition is still an uphill task. Recent advances in 3D modeling and digitizing techniques have made the acquisition of 3D human face data much easier and cheaper. 10,11 The advantages of 3D range data are: (i) Explicit representation of 3D shape, (ii) representation of face shape with real size. Recognition using 3D data has the potential to overcome these problems. 573

2 574 G. Pan & Z. Wu In this work, we focus on automatic face recognition from range data by facial profiles and surface. For facial surface, three kinds of profiles (central profile, nosecrossing profile and forehead-crossing profile) and their fusion are explored for recognition. The experiments are carried out on a database with 120 individuals, and the competitive results are obtained. In the following section we will review the previous work. Section 3 gives the recognition from range data via multiple profiles, and Sec. 4 presents the surface matching method based on statistical discriminative model. The experimental results are reported in Sec. 5 and conclusions are drawn in Sec Related Work 2.1. Recognition based on 3D data The activity to exploit 3D data to improve the accuracy and robustness of face recognition system is still weakly addressed. Only a few works on the use of 3D data have been reported. These methods can be categorized into four groups: Methods based on curvature analysis, methods by shape representation, methods by model fitting and image synthesis, and other methods. Many of the early studies concentrate on curvature analysis The seminal work by Gordon 13,14 presents a template-based recognition system using descriptors derived from range image. The sensed surface regions are classified as convex, concave and saddle by calculating the minimum and maximum principal curvature, then the locations of nose, eyes, mouth and other features are determined, which are used for depth template comparison. Lee et al. 12 propose a method to detect corresponding regions in two range images by graph matching based on Extended Gaussian Image (EGI). An approach to label the components of human faces is proposed by Yacoob et al. 15 Its preprocessing stage employs a multistage diffusion process to identify convexity and concavity points. These points are grouped into components. Qualitative reasoning about possible interpretations of the components is performed, followed by consistency of hypothesized interpretations. Tanaka et al. 16 also use the Extended Gaussian Image. For each face, two EGIs are constructed from maximum principal curvature and minimum principal curvature. The EGI similarity is measured by Fisher s spherical correlation. However, because they are involved in computing curvature, all these techniques require high resolution of the range data, otherwise the computation of curvature will be inaccurate and unreliable. References attempt to use a shape representation to analyze the 3D facial data. Chua et al. 17 describes a technique for 3D face recognition based on Point Signature a representation for free-form surfaces, which is also highly dependent on the quality of facial range data. In the method, the rigid parts of the face of one person are simply extracted to deal with different facial expressions. Their subsequent work 18 combines Point Signature on 3D range data and Gabor filter response on 2D grayscale image for facial feature detection and recognition. Pan

3 3D Face Recognition from Range Data 575 presents a novel signature Curgram for pose-invariant detection of facial feature from range data. 19 The third kind of approach is to use model fitting or image synthesis to cope with the influence of illumination and pose. For example, Blanz et al. 9,20 utilize a 3D morphable model 21 to fit the input facial image to tackle variation of pose and illumination. For this approach, the shape and texture fitting procedure is hugely time-consuming. In Ref. 22, Lee et al. employ an edge model and a color region model to analyze face image, and a wireframe model to synthesize the face image in virtual view for recognition. And Zhao et al. present a method to synthesize the virtual image with SFS-based 3D shape recovery. 8 For this kind of approach, the input is a 2D face image but not 3D data. Because an image is essentially the projection from 3D space to 2D space due to the nature of 2D image, there difficulties in accurate recognition across pose and illumination. Other 3D face recognition approaches include those mentioned in Refs Beumier et al. 23 propose two 3D comparison methods based on surface and profiles matching respectively. In Ref. 24, authors do the task of face recognition via feature vector that is generated from depth information of the area in some contour line. Recently Bronstein et al. propose a novel 3D face recognition method. 25 It converts facial shape and texture to the special images by a bending-invariant mapping scheme, then perform eigenface decomposition on the special images to do the recognition task Recognition from facial profile For recognition using facial profile, many methods have been performed. Many of the previous work carried out depends on fiducial points extracted by heuristic rules In Ref. 26 by Harmon et al., the outlines of 256 profile photos are manually drawn. Nine fiducial points are selected, and a set of 11 features is derived from these fiducial points. Then two profiles are aligned to be matched by two selected fiducial marks, and the matching is achieved by measuring the Euclidean distance of the feature vectors derived from the outlines. In Ref. 27 they extend their work, where eleven features are reduced to ten. Set partitioning technique is used to reduce the number of candidates to be included in the Euclidean distance measure. In their subsequent work, fiducial points are defined to generate 17-D feature vector, and searching space is pruned by thresholding windows. Kaufman et al. 29 develop a face recognition system based on profile silhouettes. The feature vector is a set of normalized autocorrelations expressed in polar coordinates, and is classified by a distance weighted k-nearest neighbor rule. Wu et al. 30 report a face profile recognition system based on 6 fiducial points. They use a B- spline to extract turning points on the outline curve, and six fiducial points and 24 features are derived from these points. The stored features are obtained in a training process that use three profiles per person. Yu et al. 31 give a tuning method to get more precise position of the fiducial points. They define a number of small steps

4 576 G. Pan & Z. Wu around the determined positions of the fiducial points. For each combination of the new positions of the fiducial points, the matching is performed. An attributed string matching method for profile recognition recently is proposed to tackle the inconsistency problem of feature point detection, in which a quadratic penalty function is proposed to prohibit large angle changes and over-merging Facial Profile Matching As described above, most current methods for profile recognition are based on fiducial points, which involved in inconsistency problem of feature point definition and detection. One solution to this issue is to match the whole profile without detection of fiducial points. In this section we present a robust symmetry plane detection method to extract profile from range data and a global matching approach to align and compare two profiles Detection of the bilateral symmetry plane Given range data for facial profile recognition, we should first extract the central profile from range data. Assume that the 3D facial surface is bilateral symmetrical and continuous. Extraction of central profile can be achieved by detecting the bilateral symmetry plane. Detecting symmetry is a well studied problem in the computer vision area However, Shen s method 35 only works with 2D image, Kazhdan s symmetry descriptor 36 is largely time-cosuming, accurate point correspondences are required by Zabrodsky et al., 33 and EGI (Extended Gaussian Image)-based method 34 needs high quality of range data. Cartoux et al. 37 proposed an approach which extracts the profile by looking for the vertical symmetry axis of Gaussian curvature values of the facial surface, while computation of Gaussian curvature also needs sufficiently accurate range data. Actually, for a practical 3D face recognition system, in most cases, although the head may turn left, right, upward, downward during acquisition of range data, the rotation is slight and its angles corresponding to three axes are usually less than 30. Under this condition, we present a simple but effective method using alignment to robustly extract symmetry plane from the symmetrical object, which does not need computation of curvature. The idea of our symmetry plane detection algorithm is simple. Suppose that an initial but inaccurate symmetry plane of facial surface is given, in Fig. 1(a). Its mirrored surface can be easily obtained, as shown in Fig. 1(b), where point A in Fig. 1(b) is reflectively symmetrical to point A in Fig. 1(a) with respect to the initial symmetry plane. Once we align two surfaces accurately, segment AA must be vertical to the true symmetry plane and its central point must lie on the true symmetry plane. That is the segment AA determines the true symmetry plane.

5 3D Face Recognition from Range Data 577 (a) (b) (c) Fig. 1. Finding symmetry plane by alignment. (a) The original facial surface, (b) The mirrored facial surface with respect to an initial symmetry plane, (c) Alignment of both surfaces. The bilateral symmetry plane can be formalized as n x + k =0, (1) where n is the normal vector of the plane and k is a constant. After the alignment of two surfaces, for point A denoted by a (a x,a y,a z )andpointa denoted by a (a x,a y,a z), the true symmetry plane can be obtained by solving: { n =( a a )/( a a ) n ( a + a. (2) )/2+k =0 Considering error from range data acquisition and non-exact symmetry of facial surface, we solve Eq. (2) for each point in facial surface and perform least squares method 38 to get the final solution. The algorithm we have used for alignment in our system is a variant of ICP (Iterated Closest Point). 39 For our needs, we are interested in an algorithm that offers the highest possible performance. After some experimentation, we employ a scheme similar to that in Ref. 11, which can attain complete alignment in one second. ICP is attractive because of its simplicity and its good performance Horizontal profiles If the facial symmetry plane is available, the central profile can be easily extracted. To utilize more discriminative information in range data, we consider extracting more profiles for recognition. These profiles should be robust and less sensitive to the facial expression. We choose two horizontal profiles located on nose and forehead, which we call nose-crossing profile and forehead-crossing profile respectively. The positions on which these profiles are extracted from different range data of same individual should be extremely similar, otherwise the horizontal profiles would

6 578 G. Pan & Z. Wu be quite different for same individual. We achieved this by setting nose-crossing profile 1.5 cm above nose tip and setting forehead-crossing profile 4.2 cm above nose tip after facial data registration described in Sec. 4.1, since profiles in these positions are less sensitive to facial expression and are easy to determined. Figure 2(a) illustrates the positions of two kinds of horizontal profile. Figures 2(b) and (c) show samples of extracted nose-crossing profile and forehead-crossing profile. (a) (b) (c) Fig. 2. Horizontal profiles. (a) The positions of two kinds of horizontal profiles, (b) nose-crossing profile, (c) forehead-crossing profile.

7 3D Face Recognition from Range Data Similarity metric for profiles To be tolerant towards the noise, the function measuring the difference between two profiles should be robust enough. It should be insensitive to the small difference and should better measure the global difference between the profiles. Hausdorff distance is a distance function between two point sets. Regarding the profile as a point set, Hausdorff distance between two profiles gives a measure of their difference. Given two sets of points A = {a 1,...,a m } and B = {b 1,...,b n }, the Hausdorff distance is defined as H(A, B) =max(h(a, B),h(B,A)), (3) where h(a, B) =max a b. (4) min a A b B But the Hausdorff distance is very sensitive to even a single outlying point of A or B. 40 A generalization of the Hausdorff distance is formalized as the following, which is insensitive to small perturbations of the point sets and allows for small positional errors in point sets: h k (A, B) =kth a b, (5) min a A b B where kth denotes the kth ranked value (or equivalently the percentile of m). If user specifies the fraction f, 0 f 1, k can be determined by k = fm. In terms of set containment, h k (A, B) δ if and only if there is some A k A such that A k B, wherea k contains k points of A. Thuswecanthinkof h k (A, B) as partitioning A into two sets, A k which is close to (within δ of) B and the outliers A A k.thisisinpracticeanimportant aspect of the generalized Hausdorff measure: It separately accounts for perturbations (by distance δ) and for outliers (by the rank k) Matching by optimization For alignment of profiles, the dimension of transformation space is three, they are rotation angle θ and translation vector (t x,t y ). Putting the parameters into a parameter vector a =(θ, t x,t y ) together, given a point x in a profile, the transformation is: T 2d ( a; x) =T (θ, t x,t y ; x) ( ) ( ) cos θ sin θ tx = x +. (6) sin θ cos θ t y Thus, the alignment of profile L 1 = p i by L 0 = q j can be written as: argmin a H lk (L 0,T 2d ( a; L 1 )). (7) During the optimization process, local minimum may occur in the state space. As a result, many conventional local optimization methods like Newton algorithm

8 580 G. Pan & Z. Wu usually cannot converge at the global minimum of the function. Here we use the simulated annealing method 41 to solve this optimization problem. A good choice of the initial parameter can notably reduce the number of iterations needed for the convergence of the simulated annealing. To obtain the initial position, we use a straight line to fit the profile curve, followed by rotation and translation of the profiles so that their centroid and the two lines coincide with each other. 4. Facial Surface Matching 4.1. Facial data registration Surface matching based method can be split up into two critical parts: Data alignment and data comparison. The accuracy of alignment will greatly impact on the result of following comparison. Although Blanz 21 gives a nice solution to register 3D facial data, huge time cost makes it hard to be incorporated into a practical recognition system. Assuming that facial range data is not subject to projective scaling, registration of facial data is an optimization problem in a 3D rigid transformation parameter space consisting of three degrees for translation and three degrees for rotation. Given two sets of facial range data, probe data S = {s 1,...,s n } and gallery data M = {m 1,...,m k }, the task of 3D registration is to find the rigid transformation which will optimally align the regions of S with those of M. The transformation T 3d include a rotation around the axes X, Y, Z with angles φ, γ and θ respectively, and a translation t 3d. So the result of this transformation of 3D point s i is T 3d (s i )=R φ R γ R θ s i + t 3d. (8) Alignment error can be measured by a function ɛ 2 ( x ), for which, a typical choice is to define: ɛ 2 ( x ) = x 2 = T 3d (s i ) m ψ(i), (9) where m ψ(i) is the corresponding model point for s i. Thus, the estimate of the optimal registration is given by minimizing the error: ˆT 3d = argmin T 3d (s i ) m ψ(i) T 3d i = argmin R φ R γ R θ s i + t 3d m ψ(i). (10) T 3d i Our system solves this linear numerical problem using ICP. To speed up registration procedure, the following steps are performed to obtain an appropriate initial position, in which aprioriknowledge of the human face and facial features is exploited. (i) A plane is fitted to probe S, and frontal view and back view are detected on the basis of point distribution on both sides of the plane

9 3D Face Recognition from Range Data 581 (ii) Approximately estimate the location of nose tip (iii) Translate, rotate according to the parameters obtained by steps 1 2. When computing the point correspondence for models using nearest neighbor rule, we reject the worst 10% pairs based on point-to-point distance. Its purpose is to eliminate outliers which may have a large effect when performing the least squares minimization The statistical discriminative model For recognition using surface matching, different region on facial surface usually has different discriminative distribution for classification, which is not explicit. Varying from one person to other person, those most discriminative regions are difficult to heuristically define and evaluate. For this reason, we present a point-based statistical discriminative model for each subject to describe discriminative capability of each point. Given models (or range data) labelled subject A: {A i,i=0,...,m}, A i = {a ik, k =1,...,N a } And models labelled non-subject A: {B j, j =1,...,n}, B j = {b jk }. We assume that {A i } and {B j } have to be registered by A 0 where each point a ik or b jk corresponds to the point a 0k with the same index. Actually, the correspondence between each pair of model can be built by the nearest neighbor rule. We define the within-class scatter S w and between-class scatter S b for each point in A 0 : Sw k = 1 m a ik m 1k, (11) m +1 i=0 Sb k = m 1k m 2k, (12) where m ik is the mean given by m 1k = 1 m a ik, m +1 i=0 (13) m 2k = 1 n b jk. n (14) j=1 Therefore, the statistical discriminative model (SDM) for A 0 is defined as follows SDM(A 0 )= {( Sw k ) },Sk b,k =1,...,Na (15) In which the discriminative capability of point a 0k scatter Sw k and between-class scatter Sb k together. is described by within-class

10 582 G. Pan & Z. Wu 4.3. Similarity measurement for surface Given two pieces of range data A = {a i }, B = {b i } where each point b j corresponds to the point a j with the same index, and the statistical discriminative model of SDM(A) = {( )} Sw k,sk b, the weighted directed distance is defined as follows, to measure similarity from B to A: Sim d (A, B) = 1 Sb i min N a i b. (16) a b B i S i w Note that the worst 10% pairs are rejected when the correspondences are built. 5. Experimental Evaluation In this section, we measure the system performance by EER (Equal Error Rate), which means at this location the false acceptance rate and the false rejection rate are equal. Since EER is derived from the ROC curve (Receiver Operator Characteristic curve), it is not exactly multiples of 1/n, wheren is the number of probe samples Facial range database Our experiments were carried out on the facial range database 3D RMA, 23 which is the biggest 3D face database publicly available. Each face in 3D RMA is represented by scattered 3D point cloud, obtained by a 3D acquisition system based on structured light. The database consists of 120 individuals and two sessions (session1: Nov 97 and session2: Jan 98). For each session, three instances were taken, corresponding to three poses neutral, limited up/down and left/right. During acquisition, people sometimes wore their spectacles, and some people smiled. Beards and moustaches also were of presence. In 3D RMA, two databases were built up from the two sessions: (i) automatic DB, reconstructed automatically, 120 individuals; and (ii) manual DB, reconstructed interactively, only the first 30 individuals in alphabetical name order. We denote these four datasets as DBs1a, DBs2a, DBs1m and DBs2m, shown in Table 1. The facial range data in 3D RMA database are of limited quality. Its resolution is relatively low, since there are only about 3000 points in each face model, compared with the experimental data in other literature, e.g. still having points Table 1. The four datasets in 3D RMA. Session 1 Session 2 (3 instances/person) (3 instances/person) Automatic DB reconstructed automatically DBs1a DBs2a (120 persons) Manual DB reconstructed manually DBs1m DBs2m (the first 30 persons out of 120)

11 3D Face Recognition from Range Data 583 Fig. 3. Four samples of facial range data for our experiment, two views for each model. after model simplification in Ref. 9, range image in Ref. 18, range image in Ref. 25. Additionally, some facial features are often incomplete, like nose, eye. In our implementation, the points below the chin are cropped manually. Figure 3 shows several examples, two views for each Time cost We implemented the proposed system on the Pentium IV 2.0 GHz. The mostly time-consuming stages are symmetry plane detection, facial profile matching and facial surface registration. Their average computational time is shown in Table 2. Table 2. Average computational time for three stages. Stage Symmetry plane detection Matching two profiles Facial surface registration Average computational time 0.96 s 2.17 s 0.93 s

12 584 G. Pan & Z. Wu Fig. 4. Some results of symmetry plane detection carried out on 3D RMA database. Models in the first row are the initial position for symmetry plane detection and models in the second row are the corresponding detection results. The symmetry plane is marked by the white line Results of symmetry plane detection Figure 4 shows some results of the symmetry plane detection on 3D RMA database. The top row illustrates the models with the initial position for symmetry plane detection, and the bottom row exhibits their detection results, where symmetry plane is marked by the white line. Some models are obviously incomplete, yet the algorithm found the bilateral symmetry plane correctly Results by profile matching Examples of the extracted central profiles are shown in Fig. 5. The facial data in the first row in Fig. 5(a) is seriously incomplete around eye regions. The second row in Fig. 5(a) is a sample whose data on the right side obviously less than that on the left side. The third row in Fig. 5(a) shows a sample in depth rotation. In these cases, the central profiles are all successfully extracted by our symmetry planes detection algorithm. Figure 6 demonstrates the initial position and the final position converged during profile matching for the three kinds of profiles. ROC curves by central profile matching carried out on DBs1m and DBs1a are shown in Fig. 7. To evaluate the effect of reconstruction error, we use the first 30 persons in DBs1a, where the individuals are similar to DBs1m. The equal error rate on DBs1m is 2.22%, while that on DBs1a is 5.56%.

13 3D Face Recognition from Range Data 585 (a) (b) (c) (d) Fig. 5. Profiles extracted by the symmetry plane detection algorithm. (a) the input model after triangle-based linear interpolation of the original range data, and its initial symmetry plane denoted by a gray line, (b) the mirrored model after aligning, (c) the symmetry plane detected, (d) the profile extracted. To determine the percentile parameter k in the partial Hausdorff distance, experiment with different k was conducted. Figure 8 suggests that the best result is achieved around k =0.8. EER by our global profile matching on different data sets of 3D RMA are demonstrated in Table 3. Results of the robust profile recognition method presented by Yu 31 are also reported.

14 586 G. Pan & Z. Wu Fig. 6. Profiles at the initial position and the matching result. 50 Receiver Operating Characteristic 40 Manual DB Automatic DB False Rejection Rate(%) False Acceptance Rate(%) Fig. 7. ROC curves of central profile matching for DBs1m and DBs1a (30 persons).

15 3D Face Recognition from Range Data DBs1a DBs1m DBs2a DBs2m EER (%) k Fig. 8. EERs of central profile with different percentile of k-hausdorff distance performed on the four data sets (only 30 persons) Results by 2D methods Evaluation of the recognition performances from range data by the appearancebased face recognition methods is given here. The 2D appearance-based methods cannot be directly used in 3D data. A simple scheme is to convert the range data to 2D depth image in frontal view as the input. Since the range data in 3D RMA is described by point cloud, generation of 2D depth image needs interpolation. Delaunay triangle-based linear interpolation is employed to convert the facial range data to the 2D depth image. Firstly, the irregular range data is Delaunay triangulated. Then, for each regular grid point, the closest triangle is selected, and the depth value of the regular point are computed by triangle-based linear interpolation of the depth values at the three vertices. 42 Several samples after interpolation are shown in Fig. 9. To make a comparison, we implement three appearance-based face recognition algorithms, Eigenface (reduced to 60 principal components), 3 Fisherface (reduced to 29 dimension for manual DB andto60dimensionforautomatic DB), 4 Kernel Fisherface (reduced to 29 dimensions for manual DB and to 60 dimension for automatic DB, polynomial kernel). 6 The distance metric in reduced subspace is L 2. Results are shown in Table Results by surface matching Results by surface matching are shown in Table 3. The best EER performance is 3.33% obtained on DBs1m. When carried out on 120-person data set DBs1a and

16 588 G. Pan & Z. Wu Table 3. Error rates on 3D RMA. Data set DBs1m (30 3) DBs2m (30 3) DBs1m + s2m (30 6) (persons instances/person) DBs1a (120 3) DBs2a (120 3) DBs1a + s2a (120 6) Central profile by Yu % 8.89% 13.33% 14.62% 15.04% 18.95% Proposed central 2.22% 4.44% 6.67% profile matching 9.83% 10.83% 13.76% Nose-crossing profile 8.89% 11.11% 12.78% 12.96% 15.72% 16.71% Forehead-crossing profile 13.33% 13.33% 15.0% 16.67% 18.05% 20.38% Eigenface (60 PCs) % 6.67% 7.78% 17.47% 19.31% 22.03% Fisherface (29/60) % 6.67% 7.78% 18.26% 19.54% 15.83% Kernel Fisherface % 5.56% 7.78% (29/60, polynomial kernel) 16.11% 17.97% 15.09% Gordon91, 13 but facial 13.33% 15.56% 19.44% landmarks labelled manually Beumier % 7.0% 9.5% Our SDM-based method 3.33% 5.56% 6.67% 6.73% 6.94% 8.79% Fusion of profile and surface 1.11% 2.22% 4.44% 5.64% 6.15% 7.93% Fig. 9. 2D depth images by triangle-based linear interpolation. DBs2a, the proposed method still has a low EER, exceeding all other methods listed in Table 3. Experimental result by seminal approach of Gordon 13 is conducted and reported in Table 3. Because of the low resolution of the range data and in the presence of noise, Gordon s method cannot correctly detect facial features like eyelid, eyeball, corners of eye. We have to manually label these facial features for this method. Only the data in manual DB is labelled. The EER on manual DB has already shown its inferiority.

17 3D Face Recognition from Range Data 589 The performance of Beumier s surface matching method 23 is also listed here, which is from Beumier s paper. Result on DBs1a and DBs2a is not reported in literature Results for fusion The main motivation of the profile analysis is to access the complementary information in range data to be combined with the surface analysis in order to improve the recognition performance and robustness of the system. The performance of various decision level fusion schemes will depend on the nature of input data. We conducted the experiment with several commonly used fusion schemes, including rules of Max, Min, Sum, Product, Median and Majority Vote, 43 to make fusion of the four experts (three profile experts and one surface expert). The SUM rule achieves the best performance, and is chosen in our approach. The result by SUM rule is demonstrated in Table 3. Error rates on 3D RMA are obviously decreased after fusion. It also shows that there is compensation in discrimination between surface the global measure of shape, and profile the partial sampling of shape. 6. Conclusions This paper has investigated face recognition from range data by facial profile and surface matching. A simple but effective symmetry plane detection method is presented to extract the central profile. And a robust global profile matching method is developed, discarding detection of feature points on the profile. The authors also have explored the two horizontal profiles for recognition. To describe discriminative capability of different regions in facial surface, a statistical discriminative model is proposed for surface-based recognition. Their effectiveness has been demonstrated by comparable experimental results. The proposed methods are carried out on the 3D RMA, a facial range data database with limited quality. The experimental results show that the proposed surface matching and profile matching are competitive, and outperform several excellent appearance-based face recognition methods. The fusion result gives a experimental proof for the presence of discriminative compensation between surface and profile. Recognizing faces across pose and illumination still a hard problem. The pose and illumination variation are two challenges in 2D image-based face recognition. 44 Recognizing from 3D data is likely to solve the two problems. Despite variant of head pose, the viewpoint is more easily recovered from range data rather than from 2D images. Moreover, if acquisition of range data is not considered, lighting condition has no effect on our approach. 3D Face recognition from range data is promising. In the future, experiments on a more accurate database are expected. It is also important to characterize the noise in range data produced by different 3D acquisition systems like laser range scanners and photometric stereo techniques,

18 590 G. Pan & Z. Wu and to study the effect of the noise on the face recognition algorithms. Tackling facial expression problem is another work in the future. Acknowledgments The authors are grateful for the grants from the National Science Foundation of China ( ), National 863 High-Tech Programme (2001AA4180), and Zhejiang Provincial Natural Science Foundation for Outstanding Young Scientist of China (RC01058). They would like to thank Signal and Image Center at Royal Military Academy of Belgium and Dr. Charles Beumier for providing 3D RMA facial range database, and we thank the anonymous reviewers for their valuable comments in the improvement of this manuscript. References 1. R. Chellappa, C. L. Wilson and S. Sirohey, Human and machine recognition of faces: A survey, Proc. IEEE 83(5), (1995). 2. W. Zhao and R. Chellappa, Face recognition: A literature survey, University of Maryland, Tech. Rep. CS-TR4167 (2002). 3. M. Turk and A. Pentland, Eigenfaces for recognition, Journal of Cognitive Neuroscience 3(1), (1991). 4. P. N. Belhumeur, J. P. Hespanha and D. J. Kriegman, Eigenfaces vs. fisherfaces: Recognition using class specific linear projection, IEEE Trans. Pattern Anal. Machine Intell. 19(7), (1997). 5. L. Wiskott, J.-M. Fellous and N. Kruger, Face recognition by elastic bunch graph matching, IEEE Trans. Pattern Anal. Machine Intell. 19(7), (1997). 6. M.-H. Yang, Kernel eigenfaces vs. kernel fisherfaces: Face recognition using kernel methods, in Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp (2002). 7. A. S. Georghiades, P. N. Belhumeur and D. J. Kriegman, From few to many: Illumination cone models for face recognition under variable lighting and pose, IEEE Trans. Pattern Anal. Machine Intell. 23(6), (2001). 8. W. Zhao and R. Chellappa, Illumination-insensitive face recognition using symmetric shape-from-shading, in Proc. IEEE International Conference on Computer Vision, pp (2000). 9. V. Blanz, S. Romdhani and T. Vetter, Face identification across different poses and illumination with a 3D morphable model, in Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp (2002). 10. O. Hall-Holt and S. Rusinkiewics, Stripe boundary codes for real-time structuredlight range scanning of moving objects, in Proc. IEEE International Conference on Computer Vision (2001). 11. S. Rusinkiewicz, O. Hall-Holt and M. Levoy, Real-time 3D model acquisition, in Computer Graphics Proceedings SIGGRAPH 2002, pp J. C. Lee and E. Milios, Matching range images of human faces, in Proc. IEEE International Conference on Computer Vision, pp (1990). 13. G. G. Gordon, Face recognition based on depth maps and surface curvature, in Geometric Methods in Computer Vision, SPIE Proceedings, 1570, (1991).

19 3D Face Recognition from Range Data G. G. Gordon, Face recognition based on depth and curvature features, in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp (June 1992). 15. Y. Yacoob and L. S. Davis, Labeling of human face components from range data, CVGIP: Image Understanding 60(2), (September 1994). 16. H. T. Tanaka, M. Ikeda and H. Chiaki, Curvature-based face surface recognition using spherical correlation principal direcions for curved object recognition, in Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp (1998). 17. C. S. Chua, F. Han and Y. K. Ho, 3D human face recognition using point signature, in Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp (March 2000). 18. Y. Wang, C.-S. Chua and Y.-K. Ho, Facial feature detection and face recognition from 2D and 3D images, Pattern Recognition Letters 23, (2002). 19. G. Pan, Y. Wang and Z. Wu, Pose-invariant detection of facial features from range data, in Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, Hyatt Regency, Washington D.C., pp (2003). 20. V. Blanz and T. Vetter, Face recognition based on fitting a 3D morphable model, IEEE Transactions on Pattern Analysis and Machine Intelligence 25(9), (2003). 21. V. Blanz and T. Vetter, A morphable model for the synthesis of 3D faces, in Computer Graphics Proceedings SIGGRAPH 99, pp (1999). 22. M. W. Lee and S. Ranganath, Pose-invariant face recognition using a 3D deformable model, Pattern Recognition 36(8), (2003). 23. C. Beumier and M. Acheroy, Automatic 3D face authentication, Image and Vision Computing 18(4), (2000). 24. Y. Lee, K. Park, J. Shim and T. Yi, 3D face recognition using statistical multiple features for the local depth information, in Proc. 16th International Conference on Vision Interface, Halifax, Canada (2003). 25. A. M. Bronstein, M. M. Bronstein and R. Kimmel, Expression-invariant 3D face recognition, in Proceedings of International Conference on Audio- and Video-based Biometric Person Authentication, Lecture Notes in Computer Science 2688, (2003). 26. L. D. Harmon and W. F. Hunt, Automatic recognition of human face profiles, Computer Graphics Image Processing 6 (1977). 27. L. D. Harmon, S. C. Kuo, P. F. Raming and U. Raudivi, Identification of human face profiles by computer, Pattern Recognition 10, (1978). 28. L. D. Harmon, M. K. Khan, R. Larsch and P. Raming, Machine identification of human faces, Pattern Recognition 13, (1981). 29. G. J. K. Jr and K. J. Breeding, The automatic recognition of human faces from profile silhouettes, IEEE Trans. Syst., Man, Cybern., pp (1976). 30. C. J. Wu and J. S. Huang, Human face profile recognition by computer, Pattern Recognition 23, (1990). 31. K. Yu, X. Y. Jiang and H. Bunke, Robust facial profile recognition, in Proc. IEEE International Conference on Image Processing 3, (1996). 32. Y. Gao and M. K. Leung, Human face profile recognition using attributed string, Pattern Recognition 35, (2002). 33. H. Zabrodsky, S. Peleg and D. Avnir, Symmetry as a continuous feature, IEEE Trans. Pattern Anal. Machine Intell. 17(12), (1995).

20 592 G. Pan & Z. Wu 34. C. Sun and J. Sherrah, 3D symmetry detection using the extended gaussian image, IEEE Trans. Pattern Anal. Machine Intell. 19(2), (1997). 35. D. Shen, H. H. S. Ip, K. K. T. Cheung and E. K. Teoh, Symmetry detection by generalized complex (gc) moments: A close-form solution, IEEE Trans. Pattern Anal. Machine Intell. 21(5), (1999). 36. M. Kazhdan, B. Chazelle, D. Dobkin, A. Finkelstein and T. Funkhouser, A reflective symmetry descriptor, in Proc. European Conference on Computer Vision 2, (2002). 37. J. Cartoux, J. T. Lapreste and M. Richetin, Face authentification or recognition by profile extraction from range images, in Workshop on Interpretation of 3D Scenes, pp (November 1989). 38. W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery, Numerical Recipes in C, 2nd ed., Cambridge University Press (1992). 39. P. J. Best and N. D. Mckay, A method for registration of 3D shapes, IEEE Trans. Pattern Anal. Machine Intell. 14(2), (1992). 40. D. P. Huttenlocher, G. A. Klanderman and W. J. Rucklidge, Comparing images using the Hausdorff distance, IEEE Trans. Pattern Anal. Machine Intell. 15(9), (September 1993). 41. S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi, Optimization by simulated annealing, Science 220, (1983). 42. D. F. Watson and G. M. Phillip, Triangle based interpolation, Mathematical Geology 8(16), (1984). 43. J. Kittler, M. Hatef, R. P. W. Duin and J. Matas, On combining classifiers, IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), (1998). 44. P. J. Phillips, P. Grother, R. J. Micheals, D. M. Blackburn, E. Tabassi and M. Bone, Face recognition vendor test 2002: Evaluation report, Tech. Rep. (March 2003). Gang Pan received the BS and PhD degrees in computer science from Zhejiang University in 1998 and 2003, respectively. He is a recipient of Microsoft Fellowship Award From September to December 2000, he was a visiting student at Microsoft Research Asia. He has published over ten papers on computer vision and image processing. Currently, he is an assistant professor of computer science at Zhejiang University. His research interests include computer vision, image processing, statistical pattern recognition and computational learning theory. Dr. Pan is a member of the IEEE. His homepage is: cs.zju.edu.cn/ gpan. Zhaohui Wu received the PhD degree in computer science from Zhejiang University in From 1991 to 1993, he was with The German Research Center for Artificial Intelligence (DFKI) as a joint PhD student, where he was working in the area of knowledge representation and expert system. He joined the Computer Science Department at Zhejiang University in 1993, and is currently a professor and the vice-director of the College of Computer Science and Technology.

21 3D Face Recognition from Range Data 593 He served as the PC member of various international conferences and is on the editorial board of several journals. He also is the author of more than 100 refereed papers. His current research interests mainly include biometrics, knowledge grid computing, artificial intelligence and pattern recognition. He is a member of the IEEE and the IEEE Computer Society.