The BJUT-3D Large-Scale Chinese Face Database

Transcription

1 The BJUT-3D Large-Scale Chinese Face Database MISKL-TR-05-FMFR-001 Aug 2005 Multimedia and Intelligent Software Technology Beijing Municipal Key Laboratory Beijing University of Technology 1

2 The BJUT-3D Large-Scale Chinese Face Database Multimedia and Intelligent Software Technology Beijing Municipal Key Laboratory Beijing University of Technology, Beijing , China Abstract The Multimedia and Intelligent Software Technology Beijing Municipal Key Laboratory in Beijing University of Technology has constructed the BJUT-3D large-scale Chinese face database under the joint sponsor of National Natural Science Foundation of China, Beijing Natural Science Foundation Program, Beijing Science and Educational Committee Program. The goals to create the database include (1) providing the worldwide researchers of Face Modeling community a large-scale 3D face database for building 3D face model; (2) providing the worldwide researchers of Face Recognition community a large-scale 3D face database for training and evaluating their algorithms; (3) advancing the state-of-the-art Face Modeling and Face Recognition technologies aiming at practical applications especially for the oriental. The BJUT-3D database includes 500 Chinese people. There are 250 females and 250 males in the database. Everyone has the 3D face data. We acquire original high-resolution human 3D face data by the CyberWare 3D scanner in special environment. To build 3D face database, the original 3D data should be preprocessed, and cut the redundant parts. This face database is now made available (named by BJUT-3D-R1) for research purpose only on a case-by-case basis only. The Multimedia and Intelligent Software Technology Beijing Municipal Key Laboratory in Beijing University of Technology is serving as the technical agent for distribution of the database and reserves the copyright of all the data in the database. 2

3 CONTENTS Abstract Introduction Data capturing settings Laser scanner Illumination Accessories Description of the released BJUT-3D face database: BJUT-3D-R Contents of the BJUT-3D-R File naming convention Data formats and directory structure Application of 3D face database Conclusion and future work Obtaining the database Acknowledgements Reference

4 1. Introduction With the development of computer vision, computer graphic and pattern recognition and so on, especial with the deep research of face detection and face animation, many research institutes have built kinds of face databases to make experiment about correlative research and test algorithms. The existing face databases almost are 2D images or videos applied in research of face detection, face recognition, face tracing, face feature selection, face expression and illumination. Faces in those databases vary in race, age, sex, pose, expression and illumination. The following are some typical face databases. MIT face database [1] It includes 16 people s multi-resolution images. Everyone has 27 face images of different poses and illumination. Every image has 6 kinds of resolutions, and the highest resolution image is gray image of size Olivetti face database [2] It is one of the earliest face databases. It includes 400 images of 40 people, and everyone has 10 images nearly just face. The images vary in time, illumination, expression and some are even with beards and glasses. The face images are gray images in size of The Yale Face Database B [3] It varies in poses and illumination. There are 10 people in the database. The variation includes 9 kinds of poses and 65 kinds of illumination including 64 kinds of artificial illumination and the circumstance. There are =5850 images in the database. Every image is a gray image in size of CMU PIE (Pose, Illumination, and Expression) [4] It is a large scale face database including images varied in expression, poses and illumination. There are color images of 68 people s 4 kinds of expression, 13 poses and 43 kinds of illumination. The size is JAFFE Facial Expression Image Database [5] It includes Japanese mature female s 213 face images of 7 kinds of expression. 4

5 There are expression marks about every image. Chinese have do some work in this area. The Chinese academic institute built a large scale Chinese face image database with the name of CAS-PEAL Face Database [6]. It varies in illumination, poses, expression and so on. Different kinds of face database build a strong basis for research of face information processing, and make a uniform research and test platform. But, with the development of the research, especial research about face model construction, face animation, we believe that it is very argent to build 3D face databases in face research area considering the following reasons: (1) The traditional method based on analysis of 2D images has many difficulties in processing the face 3D structure because of the absence of 3D information. (2) Some of them are problems about poses and illumination. Some new face recognition methods use 3D model to solve these problems [7 10]. (3) The 3D face model has obvious advantages in analysis of 3D information. The 3D face database is a basis to create a 3D face model. Some little scale 3D face databases have appeared out of China, and research on face recognition, face animation based on 3D face databases has introduced. The following are some 3D face databases. 3D_RMA [11] It was created in the Signal and Image Center (SIC) of Brussels (Belgium). The 3D acquisition system was based on structured-light, being constructed using a camera and a projector, and generating the 3D coordinates of the surface points with a high precision. It contains 120 individuals. GavabDB [12] It contains 427 three-dimensional facial surface images corresponding to 61 persons (45 male and 16 female), and there are 7 different images each person. Each image consists in a three-dimensional mesh representing a face surface. There are systematic variations over the pose and facial expression of each person. 3D face database of York University [13] This database has images corresponding to 97 individuals. It contains 10 captures per individual including different poses. However, only 2 of these views of each 5

6 individual present light facial expressions (happiness and frown), and one presents face occlusion. XM2VTS database [14] It is a large multimodal database supported by European ACTS projects Multi Modal Verification for Tele services and Security applications. It includes lots of face images, face videos and 3D face data of 295 persons, The 3D data was generated using an active stereo system and was converted to VRML format. This is a commercial database. Notre Dame Biometric Dataset [15] It contains near-frontal range images of 277 individuals. For each individual, there are between three to ten range-images, all taken at different times. The database contains considerable variations in hairstyles of individuals. MPI face database [16] This database contains images of 7 views of 200 laser-scanned (Cyberware TM) heads without hair. The 200 head models were newly synthesized by morphing real scans. Currently, there are 5 sets of full 3D head models available. These 3D face databases have some shortcomings. Fisrt they are not large scale,second individuals in these databases are western not suitable for oriental. The goals to create the BJUT-3D Face Database include: (1) providing the worldwide researchers of Face Modeling community a large-scale 3D face database for building 3D face model; (2) providing the worldwide researchers of Face Recognition community a large-scale 3D face database for training and evaluating their algorithms; (3) advancing the state-of-the-art Face Modeling and Face Recognition technologies aiming at practical applications especially for the oriental. The 3D face information processing in China is lag. There are no reports about the construction of 3D face databases. 3D face databases out of China are not suitable with oriental features. At the same time, there are many problems about data processing and standardizing the data [17]. We built a large-scale Chinese 3D face database. Firstly, we got the original data by the CyberWare 3D scanner. Secondly we preprocessed the original face data. Finally, we cut the redundant parts and get the 3D face data what we need. The following are the 6

7 capturing system, the contents and description of the database, and some applications based on the database. 2. Data capturing settings The capturing settings are the necessary hardware conditions, including the 3D scanner, the layout of the controlling apparent, illumination condition and so on. The organization and application of the hardware play an important role on the feature and quality of the face database. We paid much attention to the designation of the capturing system before the construction of the database. We built a special data capturing room, to get the best effect under the current hardware condition. 2.1 Laser scanner We get the 3D face original data through the CyberWare 3030RGB/PS laser scanner [18], and it can get the precise shape and color texture at one time. The scanner records the shape information in cylinder coordination, and there are 489 sampling points in the circle direction ( ϕ, 0 ϕ < 2π ), and 478 sampling points in axis direction ( h 0 < h < 300mm). The scanning radius ranges from 260mm to 340mm, and every sampling point is corresponding with a 24-bit texture point which is saved as a texture image of points. The data is very precise captured by the scanner. Everyone s original data is made up of 200,000 points and 400,000 triangle faces. The CyberWare scanner and the captured face data are showed in figure-1. (a) 3Dscanner (b) 3D face (c) shape data (d) texture image Figure-1 CyberWare scanner and the captured 3D face data 7

8 2.2 Illumination The CyberWare3030RGB/PS scanner has a reasonable illumination system. We shielded all the rays out of the room to get consistent and uniform ambient rays and to control the system easily. We designed the illumination system of the data capturing room to simulate the ambient rays. At the same time we kept the illumination system of the scanner, so we can assure the illumination condition is reasonable and consistent at each time. 2.3 Accessories The effect of the scanner is not good when it scans complex structure objects such as hair and black objects, and we only need the face information, so everyone has a swimming hat without addressing, glasses or other accessories. The hat is blue or red which is very different from the skin color. The hat shields the hair above the forehead and the ears, and keeps the face information at the most. 3. Description of the released BJUT-3D face database: BJUT-3D-R1 3.1 Contents of the BJUT-3D-R1 We have got more than 500 Chinese 3D face original data by the special face scanner in the designed data capturing room. The database includes 500 Chinese people. There are 250 females and 250 males in the database that aged between 16 and 49. They expression is natural without glasses or other accessories. The face database is now made available (named by BJUT-3D-R1) for research purpose only on a case-by-case basis only. The Multimedia and Intelligent Software Technology Beijing Municipal Key Laboratory in Beijing University of Technology is serving as the technical agent for distribution of the database and reserves the copyright of all the data in the database. The scanning lamp-house is the lamp-house of the scanner. We preprocess the original data after we get them. Then we move the redundant parts of the original data and get the 3D face data for the database. The 3D face data include shape data and texture 8

9 data. At the preprocessing step, we smooth the capturing data, move burrs, fill holes (missing data), and rectify coordinates. Then we cut the data to get the data (texture data and shape data) of the database and get rid of the redundant parts. The 3D face cutting separates the face from the whole scanning data of the head, and moves away the 3D data of the hair and the shoulder parts. Figure-2 shows the separated face data after cutting, and the first is the shape, the second is the texture, the third and the fourth are the 3D face images from different viewpoints. Figure-2 3D face after cutting 3.2 File naming convention Naming convention of files in the database is showed as following: x_xxxx_ax_ex_cxxxx_rx.ext (1) Sex field (1 bit). M indicates male, and F indicates female. (2) ID (4 bits). The four digital number sequences indicate the identification of the subject. We fill the blank with 0. (3) Age (2 bits). A represents age variation. The following bit indicates different age. There are 4 conditions. The meaning is showed in the following table. Symbol Age(year) (4) Expression (2 bits). E represents expression variation. The second bit indicates different expression. The meaning is showed in the following table. All people s expression is natural in the current database. 9

10 Symbol n h p a Expression natural happy surprise anger (5) Content (5 bits). C represents content. The following 4 bits: trim indicates that the data is the face after preprocessing and moving the redundant parts of the 3D face data. (6) Publicity (2 bits). R represents publicity. The second bit indicates whether the data is published. 0 : unpublished. 1 : published. (7) Extend name (3 bit). They indicate file formats. 3.3 Data formats and directory structure The directory of the database is as following: <BJUT-3D-R1> //The root directory of BJUT-3D-R1 face database - F0001A1EnCtrimR0 // the first data, it contains the texture and shape data - F0002A1EnCtrimR0. // the second data //other data 4. Application of 3D face database To capture 3D face data needs special costly apparatus, and it is a complex work to organize people and capture lots of original 3D face data, and we need process the original data when a database is built. So it is a difficult work to build a 3D face database. We built a large-scale Chinese 3D face database which is the first 3D face database in China. The database is a good resource and platform for Chinese face information processing. It will have good application values in the research of realistic face synthesis, face animation, face detection, face recognition and face pose estimate, illumination estimate and so on. The most direct application of the 3D face database is face synthesis. It is very clear that we can get a new face through the computation of the face vectors in the database. It is very different from the conventional methods to synthesis faces based on multi-viewpoints [19, 20], and it is a face synthesis method based on knowledge. For example, the simplest application is to make a new face from the linear combination of 10

11 the database faces. In fact, the model that Vetter etc [21] made is a linear model. But the linear model is not very useful for high resolution face data because of the large quantity of data. At the same time, we must think of the problem how to match a 3D face model with a 2D image. Much research should be done in the application of the 3D face model in face synthesis. Face detection in images and videos is the most active research hot point in biologic identification area recently. But passing methods are not ideal to the processing of face illumination and poses, and it is the main bar to the deep application of face detection and face recognition in practice. It is a good method to employ 3D face to solve the problem of face poses and illumination. The 3D face database provides rich experiment data for those researches. 5. Conclusion and future work It is a basic work in face detection, face recognition, face modeling, and face animation and so on to build a 3D face database. It is very important to the deep research in face information processing in China. We describe the whole capturing system and the content of the database, and make a brief introduction to the database application. The following work is to add more data to the database, such as expression face data, and make deep research in the application of 3D face database. 6. Obtaining the database To get a copy of the BJUT-3D-R1 face database, please download the release agreement, print and fill in the agreement appropriately, and fax it back to We will contact you on how you can get a copy either by posting a CD package (some CD fee and postage will be needed though the database itself is free.) or downloading through the Internet. 7. Acknowledgements This research is partially sponsored by National Natural Science Foundation of China ( ), Beijing Natural Science Foundation Program ( ) and Beijing 11

12 Science and Educational Committee Program (KZ ). Reference 1. M. Turk and A. Pentland. Eigenfaces for Recognition, J. Cognitive Neuroscience, 1991, 3(1): Ferdinando Samaria, Andy Harter. Parameterisation of a Stochastic Model for Human Face Identification. In Proceedings of 2nd IEEE Workshop on Applications of Computer Vision, Sarasota FL, December 1994: A.S.Georghiades, P.N. Belhumeur, D.J.Kriegman. From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose. IEEE Trans. Pattern Anal. Mach. Intelligence, 2001, 23(6): T. Sim, S. Baker, and M. Bsat. The CMU Pose, Illumination, and Expression Database. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(12): J.Michael. Lyons, Shigeru Akamatsu, Miyuki Kamachi, Jiro Gyoba. Coding Facial Expressions with Gabor Wavelets. In Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara Japan, IEEE Computer Society, April : CAS-PEAL Face Database. 7. J.Hunag, V.Blanz, B.Heisele. Face Recognition Using Component-Based SVM Classification and Morphable Models. Lecture Notes on Computer Science 2388, Springer-Verlag, Berlin Heidelberg, 2002: S. Romdhani, V.Blanz, T. Vetter. Face Identification by Fitting a 3D Morphable Model using Linear Shape and Texture Error Functions. Computer Vision - ECCV 2002, 2002, LNCS 2353: V.Blanz, S.Romdhani, T.Vetter. Face Identification across Different Poses and Illuminations with a 3D Morphable Model. In Proceeding of the 5th Int. Conference on Automatic Face and Gesture Recognition, 2002: Gu Chunliang, Yin Baocai, Kong Dehui, Hu Yongli. Face Recognition based on 3D Multi-resolution Model and Fisher Linear Discrimination. Chinese Journal of Computers, 2005, 1(1):97-104(Chinese). 12

13 Hu Yongli, Yin Baocai, Gu Chunliang, Cheng Shiquan, Liu Wentao. Chinese 3D Face Database Construction Key Technology Research. Journal of Computer Research and Development, 2005, 42(4): (Chinese). 18. Cyberware Laboratory Inc F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, D. H. Salesin, Synthesizing Realistic Facial Expressions from Photographs. In Proceedings of SIGGRAPH 98, 1998: B. Guenter, C. Grimm, D. Wood, H. Malvar and F. Pighin. Making faces. In Proceedings of SIGGRAPH 98, 1998: V.Blanz, T.Vetter. A morphable model for the synthesis of 3D faces. Proc. of SIGGRAPH 99, Los Angeles: 1999: