Face Recognition for Mobile Devices Aditya Pabbaraju (adisrinu@umich.edu), Srujankumar Puchakayala (psrujan@umich.edu) INTRODUCTION Face recognition is an application used for identifying a person from a digital image or a video frame. This technology has received much attention because of the broad interdisciplinary nature of interest involved in it. It can be used as a biometric system for authentication of the user, recognize someone at a conference or other large gathering and tag him to identify on social networking sites, recognize people and remember their preferences and peculiarities, etc. Adapting such kind of systems to mobile devices would potentially benefit because of the portability of mobile devices. Hence, creating a stand alone mobile application that does face recognition on captured images is an interesting avenue to explore. In this project we try to address the issues involved in the transformation and propose the ways to overcome those. The incorporation of face recognition algorithms in to mobile devices has been a challenging problem due to the constraints on processing power, limited storage of the mobile device, limited network bandwidth and connection instability, privacy and security concerns. Hence a client server architecture needs to be developed for this purpose. The client side performs facial detection based on color segmentation, template matching, etc on the captured image and extracts the informational features. These features are then sent to the server which does the computationally intensive task of comparison with the database image set that would help in the face recognition and then sends the information back to the client. The client then displays the required information to the user. TECHNICAL DETAILS Cropping and detection of the face has to be done before training and testing can be done. In face detection we try to find a face in the image and return the location of the image and the extent to each face. First we will delve into face detection before face recognition can be discussed. The diagram below shows the basic model used in our algorithm. Face Detection: Some of the common approaches used for face detection are Haar cascades and scan images using an increasing window, skin color segmentation, template matching and morphological processing algorithms. An impressive achievement in face detection would be efficient face detection at frame rate.
Haar cascades algorithm: This algorithm is popularly referred to as Viola Jones face detector[4] and its effectiveness is due to the robust real time processing capabilities. Training of the system is done by using a set of labeled images containing faces using the AdaBoost learning algorithm by Freund and Schapire[5]. For rapid processing, an idea called integral images is used. Skin color segmentation: In isolating skin color region normalized RGB, HIS and YCrCb color space transformations are commonly used. Based on the color component values, each pixel is classified as skin or non skin. The number of pixels used to represent the face in mobile phone photos dominates the pixels used to represent other part of the body. Therefore these approaches are likely to give good results on handsets. To reduce the effects of illumination variations scale by max color balancing is done. Many complex methods have been used in the literature to do the processing of images in extreme illumination conditions. One of the simplest methods to implement on a mobile phone could be do the calculations in RGB space. A pixel is classified as skin if its (R, G, B) values follow these conditions. R > 95 and G > 40 and B > 20 R > G and R > B R G > 15 Image showing skin detection Morphological Image Processing: This process helps to regroup the skin pixels by eliminating the nonskin visible pixels. Sparse non skin pixels can be removed by performing erosion. Smoothing of the contours and regrouping the skin regions is attained by doing dilation. Template Matching: This approach is for finding an object represented by the template in a given image. Using FFT and IFFT, cross correlation can be implemented in the frequency domain as it is computationally more efficient. Color segmentation and morphological processing will ideally leave only the face portion. But template matching will be helpful in cases where skin is exposed like hands, neck, etc. Another way of implementing face detection is by using the face detection class from the Android API.
Face Recognition: The commonly used schemes for face recognition are Eigenfaces and Fisherfaces. Eigenfaces: Eigenfaces tries to find out which features are important for classification by using principal component analysis of the images of the face. It is also known as Karhunen Loeve method. In this technique, the direction having the largest variance in the data is found and the data is projected onto it. The main motive in representing the sequence by principal components has been dimensionality reduction. A set of M training images are taken and the average face is calculated as The difference between the average face and the training image is The covariance matrix of the face images ф i is where. PCA leads to find the projection W opt that maximizes the determinant of the total scatter matrix of the projected images. where w i s are eigenvectors corresponding to the p largest eigenvalues. The p eigenfaces are represented by these p eigenvectors. Calculation of the weights of the training set images and test
images can be calculated. Euclidean distances are also found out. We, then recognize a test face as the face in the training set with the closest distance, using a threshold. Fisherfaces: PCA does not provide us with the opportunity of discriminating between the variance of the required object and the variance of unnecessary noise. Discriminant Analysis is a classic method of classification developed by Fischer in 1936. LDA provides dimensional reduction as PCA and also preserves much of the class discriminatory information. This method maximizes the ratio of betweenclass variance to the within class variance thus ensuring maximum separability. The variables are projected on a space of smaller dimension by preserving the class information. Classification is done based on the scatter within classes and scatter between classes. This analysis is help in the cases where image identity is needed to be differentiated from the variation of other sources such as illumination and expression. Classification of objects into one of two or more groups is done based on a set of features that describe the objects. The ratio of between class variance to within class variance needs to be maximized. The criterion function used is The rank of S B is c 1 or less because S B is the sum of 'c' matrices of rank one or less and c 1 of these are independent. c represents the number of classes in the training set. Therefore the original data can be represented by c 1 principal eigenvectors. Again the eigenvectors are found for J(w) and are used as Fisherfaces. Client Server Model:
MILESTONES Achieved so far: 1. Implemented a prototype face detection algorithm on Android. 2. Checked for the efficiency of LDA vs PCA by coding in MATLAB. Remaining: 1. Implement the face recognition algorithm on Android for a small database of images. (Target Date: November 23 rd ) 2. Transfer the face recognition algorithm on to the server to make it compatible with a larger database. (Target Date: December 5 th ) REFERENCES [1] M. Rahman, N. Kehtarnavaz, and J. Ren, A Hybrid Face Detection Approach for Real time Deployment on Mobile Devices, ICIP, pages 3233 3236, Nov. 2009. [2] Jung, S.U.[Sung Uk], Chung, Y.S.[Yun Su], Yoo, J.H.[Jang Hee], Moon, K.Y.[Ki Young], Real Time Face Verification for Mobile Platforms, ISVC08. [3] J. Kovac, P. Peer, and F. Solina, Human Skin Color Clustering for Face Detection, Proc. of Int. Conf. on Computer as a Tool, Vol.2, 2003. [4] Viola P, Jones M, Rapid Object Detection using a Boosted Cascade of Simple Features, In: Proc. of IEEE Int. Conf. on Computer Vision and Pattern Recognition, Hawaii. [5] Yoav Freund and Robert E. Schapire. Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, pages 148 156, 1996.