3D FACE RECOGNITION BY POINT SIGNATURES AND ISO-CONTOURS

Transcription

1 3D FACE RECOGNITION BY POINT SIGNATURES AND ISO-CONTOURS Iordanis Mpiperis 1,2, Sotiris Malasiotis 1 and Michael G. Strintzis 1,2 1 Informatics and Telematics Institute 2 Information Processing Laboratory Center for Research and Technology Hellas Electrical and Computer Engineering Department Thermi-Thessaloniki Greece Aristotle University of Thessaloniki Thessaloniki Greece {iordanis, malasiot, strintzi}@iti.gr ABSTRACT The paper addresses the problem of face recognition from range images. A novel technique based on matching of level contours is proposed and compared with a variant of the point signatures algorithm. Their efficiency is investigated on conditions of changes in expression and pose, and presence of glasses. Using a large database of range images, comparative experimental results are presented, showing that iso-contours outperform point signatures both in computational efficiency and in recognition rates. KEY WORDS Face recognition, iso-contours, point signatures. 1 Introduction Automatic face recognition has always attracted scientific research interest. Face recognition can be employed in a variety of applications, such as biometric security systems, access control systems, digital rights management systems, criminology and human-computer interaction. In the last decade many algorithms have been proposed and great advances have been reported. However, it remains an open issue since the majority of the proposed methods fails to achieve the level of performance that most of application systems desire. Although the 3D structure of the human face intuitively provides highly discriminatory information and is insensitive to environmental conditions, only a few techniques have been proposed employing range images. This is mainly due to the high cost of available 3D digitizers that makes their use prohibitive in real-world applications. Furthermore, these devices often do not operate in real time (e.g. time of flight laser scanners) or produce inaccurate depth information (e.g. stereo vision). The work presented in this paper is partly motivated by the recent development of low cost 3D sensors that are capable of real-time 3D acquisition [1]. A common approach adopted towards 3D face recognition is based on the extraction of 3D facial features by means of differential geometry techniques. Facial features invariant to rigid transformations of the face may be detected using surface curvature measures [2], that are subsequently used to extract higher level facial features [3 5]. Recently point signatures have been proposed for 3D surface feature extraction and have also been applied for 3D face recognition [6, 7]. A few techniques [7, 8] employ grayscale images as well, mainly for the detection of features such as the eyes which are harder to detect on the range image. The most important argument against techniques using a feature-based approach is that they rely on accurate 3D maps of faces, usually extracted by expensive off-line 3D scanners. In practice, 3D digitizers produce range images containing noise and missing pixels (e.g. over areas with low reflection). The applicability of feature based approaches when using such data is questionable, especially if computation of curvature information is involved. Also, the computational cost associated with the extraction of the features (e.g. curvatures) is significantly high. This hinders the application of such techniques in real-world security systems. The recognition rates claimed by the above techniques were estimated using databases of limited size and without significant variations of the faces. Appearance-based techniques for 3D face recognition were also investigated in [9] and [10]. This paper proposes a variant of the point signatures algorithm and a novel technique based on iso-contours. As concerns point signatures, in this paper we introduce a simplified version of the algorithm, that has significantly lower computational complexity from that of the original algorithm [11]. This is achieved by extracting signatures on a limited set of suitable selected points over the facial surface rather than all points. Also for each point several radii are used. The Fisher s Linear Discriminant function is introduced for finding best combinations of points and radii according to their discriminatory power. The greatest shortcoming of this method was shown to be the sensitivity to the misalignment of the compared images. In order to overcome this obstacle we propose a novel technique based on iso-contours. 3D information of the face surface is represented by a set of planar curves formed by the intersection of the surface with equidistant parallel planes. Thus, the surface matching problem is reduced to a contour matching problem that is better studied.

2 (a) (b) Figure 1. Point signature definition: (a) face surface and sphere, (b) function d(θ). All experiments are conducted on a large database of noisy range images. The performance of the proposed algorithms is tested on three types of images: images depicting facial expressions, images with and without glasses, and frontal images. The sequel of the paper is organized as follows: The point signatures algorithm variant is described in Sec.2. Sec. 3 introduces the iso-contours scheme for 3D face recognition. Comparative experimental results are in Sec. 4 and conclusions are drawn in Sec Point Signatures In this section we briefly present the point signatures algorithm and its application to 3D face recognition. A point signature over a surface point is a 1D function representing the local structure of the surface in the neighborhood of this point. The basic idea behind this approach was motivated by the concept of radial decomposition proposed by Radack and Badler (1989) and that of splash, proposed by Stein and Medioni (1992). However, the form in which it is used in this paper was first proposed by Chua and Jarvis in [11]. 2.1 Definition of point signature For a given 3D point p on the object surface, we place a 3D sphere of radius R centered at p. The intersection of the surface and the sphere is a space curve C. A plane P is fitted to C. Let n 1 be the normal vector of the plane. The plane P is translated to point p in the direction of n 1, thus defining a new plane P serving as a projection plane. The space curve C is projected onto the projection plane and a new planar curve C is formed. Each point of C is characterized by the signed distance from the corresponding point of C. A reference vector, n 2 is defined as the unit vector from point p to the point of the planar curve C with the largest positive distance. By definition, vector n 2 is orthogonal to vector n 1. Now, each point of C is also characterized by a clockwise rotation angle θ about n 1 from the reference vector n 2. Using the signed distance and the angle θ of each point of C, we can form a function d(θ), 0 < θ < 360 degrees. This function is the point signature at p. More details are given in [11]. In practice, the signature is sampled over θ for N points and function d(θ) becomes a sequence d(θ i ), i = 1,..N. 2.2 Point signatures matching Let d s be a given signature of a point on the probe surface S and d m the corresponding signature extracted from the gallery surface M. The establishment of a difference measure between them starts by estimation of the phase shift φ between the two signatures. This is achieved by registration of the largest distance maxima in the two signatures. If no such registration may be established, then the Euclidean distance between the two signatures is used as a matching score. Otherwise, the difference measure is given by: D sm = min N (d s (θ i φ) d m (θ i )) 2 i=1 2.3 Depth maps matching For 3D face recognition we assume that probe and gallery images were aligned in advance. Therefore, signatures can be extracted at the same points for all depth maps. Moreover, we allow more than one signatures at each point with different radii. The points and the corresponding radii used for recognition are selected according to their discriminatory power. The discriminatory power is measured by a coefficient, DP C [12], which is given by: DPC = (µ w µ b ) 2 σ 2 w + σ2 b

3 (a) (b) (c) Figure 2. (a) iso-contours, (b) images of the same person with different facial expression, (c) image of another person µ w = E{D w ij } σ2 w = V ar{dw ij } µ b = E{D b ij } σ2 b = V ar{db ij } i, j are signatures coming from depth maps I and J respectively. w stands for within-class, that is I and J are range images of the same person, and b stands for between-class, that is I and J are range images of different persons. The combinations of points and radii which are used for recognition, are those with large DPC coefficients. Now the establishment of the difference measure between a given depth map G and a candidate depth map C can be done using the formula: M{G, C} = K k=1 D g k c k where g k and c k are signatures of depth maps G and C respectively, with the k-th combination of point and radius as parameters. 3 Face Classification by Iso-Contour Matching One of the main shortcomings of the point signature algorithm for object classification is its sensitivity to misalignments between the probe and gallery images, since even small misalignments may introduce significant variations of the extracted signature especially over high curvature regions. Other disadvantages include the sensitivity to image noise, large computational complexity and difficulty in selecting the appropriate points and associated radius for signature extraction. The above shortcomings prohibit the application of this approach for online 3D face recognition where noisy and ill-registered images are usually the case. In this paper we propose a simple yet powerful approach that overcomes most of the above problems thus leading to even better recognition rates. It is based on the representation of the object surface by means of its level contours or iso-contours. These are defined as L i = {(x, y) : I(x, y) = d i }, i = 1,...,N where d i are depth values uniformly sampled over the range of depth values in the images (in our experiments a contour is extracted every 2.5mm). In practice, since I(x, y) is not a continuous surface, its set of points L i will contain a loosely connected cloud of points and it is difficult to extract continuous iso-contours as required by our algorithm. Therefore we alternatively define iso-contours as the boundaries of the objects defined by B i = {(x, y) : I(x, y) d i }, i = 1,...,N. In practice each set of points B i may be represented by a binary mask. We process these masks by applying connected-component analysis and subsequently eliminating small isolated objects and holes which are due to noise. Then, contour following is applied to the processed mask and a continuous, closed chain of pixels L i is extracted (see fig. 2). Note that for concave surfaces there may be several contours for each level. Since faces are approximately convex we assume for simplicity a single contour for each level. This assumption was experimentally shown not affecting the overall efficiency of the algorithm. Assuming that images are approximately aligned, the discrepancy between two face images is computed by extracting the iso-contours for each of the images and subsequently measuring the distance between iso-contours of the same level. The problem of 2D contour matching has been extensively investigated in the literature. We have experimented with three contour matching algorithms:hu moments, Elliptic Fourier descriptors [13] and Curvature Scale Space (CSS) [14]. All three algorithms are invariant to translation, rotation, scaling and starting point of the contour chain. Hu moments correspond to eleven global shape parameters. The Elliptic Fourier algorithm approximates a 2D closed contour by a set of elliptic basis functions. By increasing the number of model parameters the approximation error decreases but also becomes more sensitive to noise (20 such descriptors were used in our experiments). For the above algorithms comparison between two contours is based on the Euclidian distance between the corresponding feature vectors. Finally in the CSS approach the inflection points (or curvature zero-crossing points) of

4 the contour over different scales are computed. The features recovered from a CSS image for matching are the maxima of its zero-crossing contours. The matching of two CSS images consists of finding the optimal horizontal shift of the maxima in one of the CSS images that would yield the best possible overlap with the maxima of the other CSS image. The matching cost is then defined as the sum of pairwise distances (in CSS) between corresponding pairs of maxima. Given a distance function D between two contours the matching score between two range images is computed as N D(L A i, LB i ) i=1 where L A i, LB i are the iso-contours for the images A and B respectively. 4 Experimental Results The experiments in this section were conducted using a newly recorded face database. 3D face images are recorded using a novel real-time 3D sensor based on the color structured-light approach [1]. The sensor is based on low cost devices, an off-the-shelf CCTV-color camera and a standard slide projector. The average depth accuracy of the system optimized for an access control application is about 0.5mm. The spatial resolution of the range images is approximately equal to the color camera resolution. For each subject several images depicting different appearance variations were acquired: images depicting facial expressions, images with and without glasses, and frontal images. The database contains 20 persons and about 40 images per person. For the training of the system one frontal image and two images depicting facial expressions, other than those used for testing, were used. All images are automatically aligned by applying the pose compensation algorithm described in [15]. Table 1 demonstrates the recognition rates achieved by the two methods for each type of input images. As shown, iso-contours algorithm has a better performance. This is mainly due to the fact that contour matching is independent of translation and planar rotation around z axis. The main shortcoming is the remanent dependence upon non-planar rotations. On the other hand, any rigid transformation of the surface, as the result of imperfect alignment which is the case for the majority of automatic systems, corrupts point signature. Finally, both algorithms are shown to be sensitive to facial expressions. 5 Conclusion In this paper we demonstrated that iso-contours are capable of capturing the discriminative information of the facial surface structure suitable for recognition and we compared I.C. P.S. EFD CSS Hu Frontal Expression Glasses Total Table 1. Recognition rates (%) for point signatures and isocontours methods. P.S.: point signatures, I.C.: iso-contours our algorithm with a simplified yet improved in computational complexity version of point signatures algorithm. As shown by experimental results, iso-contours outperforms point signatures as the proposed algorithm is not sensitive to translation and planar rotation between the images compared. Still, the proposed algorithm is not robust enough for images depicting facial expressions. Overcoming this problem is the target of our future work. Acknowledgement This work is funded by research project PASION IST (Psychologically Augmented Social Interaction Over Networks) and 3DTV FP (3DTV- Integrated Three-Dimensional Television - Capture, Transmission, and Display), under Information Society Technologies (IST) priority of the 6th Framework Programme of the European Community. Iordanis Mpiperis is funded by the Greek Secretariat of Research and Technology (PENED Ontomedia 03 ED 475). References [1] F. Forster, P. Rummel, M. Lang, and B. Radig, The hiscore camera: a real time three dimensional and color camera, in Proc. Int. Conf. Image Processing, Oct. 2001, vol. 2, pp [2] E. Trucco and A. Verri, Introductory Techniques for 3-D Computer Vision, (Prentice-Hall, NJ 07458, 1998). [3] J. C. Lee and E. Milios, Matching range images of human faces, in Proc. IEEE Int. Conf. on Image Processing, 1990, pp [4] G.G. Gordon, Face recognition based on depth and curvature features, in Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, CVPR 92, 1992, pp [5] H. T. Tanaka, M. Ikeda, and H. Chiaki, Curvaturebased face surface recognition using spherical correlation. principal directions for curved object recognition, in Proc. 3rd IEEE Int. Conf. on Automatic Face and Gesture Recognition, 1998, pp

5 [6] C.-S. Chua, F. Han, and Y.-K. Ho, 3d human face recognition using point signature, in Proc. 4th IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2000, pp [7] Y. Wang, C.-S. Chua, Y.-K. Ho, and Y. Ren, Integrated 2d and 3d images for face recognition, in Proc. 11th Int. Conf. on Image Analysis and Processing, 2001, pp [8] S. Tsutsumi, S. Kikuchi, and M. Nakajima, Face identification using a 3d gray-scale image-a method for lessening restrictions on facial directions, in Proc. 3rd IEEE Int. Conf. on Automatic Face and Gesture Recognition, 1998, pp [9] F. Tsalakanidou, S. Malassiotis, and M. G. Strintzis, Face authentication using color and depth images, IEEE Trans. Image Processing, submitted. [10] K. Chang, K. Bowyer, and P. Flynn, Face recognition using 2d and 3d facial data, in Proc. Multimodal User Authentication Workshop, Santa Barbara, December 2003, to appear. [11] Chin Seng Chua and Ray Jarvis, Point signatures: A new representation for 3d object recognition, International Journal of Computer Vision, [12] R. J. Schalkoff, Statistical, structural and neural approaches., Pattern Recognition, [13] O. D. Trier, A. K. Jain, and T. Taxt, Feature extraction methods for character recognition: A survey, Pattern Recognition, vol. 29, no. 4, pp , [14] F. Mokhtarian and A. K. Mackworth, Scale-based description and recognition of planar curves and twodimensional shapes, IEEE Trans. Pattern Anal. and Mach. Intell., vol. 8, no. 1, pp , [15] S. Malassiotis and M. G. Strintzis, Pose and illumination compensation for 3d face recognition, in Proc. Int. Conf. Image Processing, Singapore, 2004.