Smart Attendance System using Computer Vision and Machine Learning

Transcription

1 Smart Attendance System using Computer Vision and Machine Learning Dipti Kumbhar #1, Prof. Dr. Y. S. Angal *2 # Department of Electronics and Telecommunication, BSIOTR, Wagholi, Pune, India 1 diptikumbhar37@gmail.com, 2 yogeshangal@yahoo.co.in Abstract This document Face is the major part of the human which provides the identification of the person. With the help of characteristics of the face, the face recognition system can be implemented. In traditional attendance system, the teachers call student and mark according to present and absent. These traditional techniques are time-consuming. In this paper, the smart machine learning based face recognition approach has been proposed. The database is created by capturing the faces of the authorized students. The face is detected using deep learning based approach. The cropped images then stored as a database with respective labels. The features are extracted using PCA algorithm and classified using SVM and KNN. The proposed approach achieves the recognition rate of --98% for SVM and 92% for the KNN algorithm. The proposed system is implemented in real time using Raspberry pi 3 boards Keywords Face detection, Face recognition, KNN, PCA, SVM I. INTRODUCTION The attendance of the student or employees in their institute is important for the knowing the performance and making the attendance record. This plays the important role for the student and institutes because on the basis of the attendance of the students their final grades. This is also played importance to improve the standard of education. Most of the existing attendance systems are a manual system where the teacher has to mark the present and absent student manually on the sheet. In another system, the sheet is provided to the student and they have to sign on the sheet. But this system may get failed and time-consuming. Another disadvantage is that students may put a proxy sign. These manual systems are time-consuming when a number of students are more. There are many automatic attendance systems are available. To track the attendance of the student in the class the many attendance management systems are implemented hence bunking the classes without teachers knowledge has become difficult. Some of the colleges were used the RFID based system, punching card systems, swipe card systems and biometric systems based on the fingerprint. But every system has its own limitations like using RFID card anyone can give the attendance by simply tagging the card. Hence there is a strong requirement of the smart, secure attendance system. In this paper, a facial feature based attendance system has been proposed. In this approach, the faces are detected using deep neural network-based approach. The detected faces were cropped and stored in the databases with respective student label. The faces were captured in the various environments to improve the accuracy. The features were extracted using Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) algorithms. Finally, the extracted features were trained and test using two different machine learning algorithm such as Support Vector Machine (SVM) and K-Nearest Neighbor (KNN) algorithm. Page No:297

2 A prototype face recognition system is developed using Raspberry Pi module. This module has 1.2 GHz quad-core ARM Cortex A53, wireless LAN etc. The database is created by taking videos of different persons facing in different directions. In this system, the image is captured through the webcam and then face is detected. If the face image is matched with one of the database images then and then the only person gives the authentication of this system. II. LITERATURE SURVEY Xiaoguang Lu [1] proposed a number of algorithms which are divided into appearance and model-based approaches. In appearance-based methods, three linear subspace analyzes are proposed and non-linear manifold analysis for facial recognition is described. S. T. Gandhe [2], presents the face recognition approach to identify the person using different experimentation. This system provides the authentication to the system by face as a biometric. This system suggested different applications like identification system, access control, and document control. Anil Kumar Sao et al. [3] proposed a template matching algorithm for face recognition. This approach addresses the posed problem in face recognition. First, the faces are representing in edge view. The template matching is applied over the image. Edginess based approach represents the image in 1 dimension. The person identification is performed based on the matching score. Sujata G. Bhele [4] presents face detection systems reviews. This paper is mostly focused on the soft computing methods like SVM, ANN etc. to detect the face. These approaches may give better results. This paper discussed the different features extraction algorithms like PCA, LDA, and ICA. This paper address the problems of face recognition like illumination changes, pose variation and various facial expression etc. Riddhi Patel [5] proposed a summary of face recognition & discusses the method and it s working. It also represents the most modern face recognition techniques and their pro and cons. Some techniques specified here also develop the efficiency of face recognition under various illumination changing and expression condition of face images. In [6], Abdul Matin proposed two stages of authentication for recognizing a candidate. In the first stage, the candidate's face is matched with all stored faces and only a few best-matched samples are isolated to use as second stage training samples. Here in both stages, PCA is used for extracting significant features of the face. Proposed approach shows 1.5% better accuracy for ORL face database In [7], Adrian Rhesa Septian Siswanto done research to get the best face recognition algorithm. The algorithms provided by Open CV i.e. PCA and LDA (eigenface and fisherface) are implemented and the ROC curve is compared. Based on the experimentation carried out by the researcher, it is concluded that the Eigenfaces approach is superior to the Fisherface approach. This approach achieved 70% to 90% accuracy for face images. In [8], Nirmalya Kar describes a method for the student assistance system that will be integrated with facial recognition technology using the Principal Component Analysis (PCA) algorithm. The system will automatically record student attendance in the classroom environment and provide services to the faculty to easily access student information while maintaining an entry and exit record. Page No:298

3 In [9], A.H. Boualleg proposes a new "hybrid" method for facial recognition that combines neural networks with of the principal components analysis. Using the geometric approach, they conducted a preliminary facial classification using PCA before using a neural classifier (PMC). The results, compared to PCA and PMC classifiers, they provide a markable improvement in the classification period. To perform this test, use a very rich database created in the "LAIG" Guelma automation and computer laboratory. III. PROPOSED SYSTEM In the proposed system, the face recognition is used as a biometric for the student attendance. This approach is mainly divided into two steps: Face registration and Face recognition. The block diagram of the proposed system is represented in Fig. 1. Input Image Face Detection Feature Extraction Face Database Classification Person Recognition Fig. 1 Block diagram of the attendance system using face recognition The technical details of each stage of the system are as explained as below. A. Image recognition Image registration is a process where the facial images of the user are captured and stored in the database. The image registration process is explained as follow. a. Image acquisition The image is captured through the webcam. The USB webcam of 5 MP is used for this purposed. The captured image contains facial images with some background information. But for the registration of the data, only face image is necessary. This task is achieved by face detection algorithm. b. Face detection Face detection is the basic step of the face recognition system. There are many algorithms were available for face detection. In this method robust deep neural network based face detection algorithm is used to detect the face. It is Open CV's new Deep Neural Network (DNN) module which mainly consists of cv2.dnn.blobfromimage and cv2.dnn.blobfromimages preprocessing function. The functions perform Mean subtraction, scaling and channel swapping process. Mean subtraction is used to compensate for the illumination changes in the images. Scaling is a process where the face image is scaled with the different scaling factor. This is important to match the facial image of the different scale. The detected faces were cropped according to the height and width of the detected bounding box. Page No:299

4 c. Feature Extraction In this approach, the feature is extracted using the PCA algorithm. The face image then resizes to 128x128 pixels size. These face needs to convert into vectors because the PCA algorithm is working on the vector hence each image is converted into the vector. In most of the approach, the PCA is used as a feature reduction technique but PCA is one of the methods which represent the faces economically. In PCA, the face images are first converted into Eigenfaces and corresponding projection. The PCA sorted only meaningful features vectors from the vector of Eigenvectors. d. Feature Extraction: In this approach, the feature is extracted using the PCA algorithm. The face image then resizes to 128x128 pixels size. These face needs to convert into vectors because the PCA algorithm is working on the vector hence each image is converted into the vector. In most of the approach, the PCA is used as a feature reduction technique but PCA is one of the methods which represent the faces economically. In PCA, the face images are first converted into Eigenfaces and corresponding projection. The PCA sorted only meaningful features vectors from the vector of Eigenvectors. The PCA is mathematically expressed as: (1) = + Where X is the face vector, Y is a vector of Eigenfaces, W is the feature vector, and μ is the average face vector. In order to extract features principal component analysis (PCA) or Eigenfaces algorithm is chosen. Let X=(x 1, x 2,,x n ) be a random vector with observations Xi Rd. 1. Compute the mean (μ):- (2) = 2. Compute the Covariance Matrix (S):- (3) = ( )( ) 3. Compute the eigenvalues (λi) and eigenvectors (vi) of S :- = = 1,.., (4) 4. Order the eigenvectors descending by their eigenvalue. The k principal components are the eigenvectors corresponding to the k largest eigenvalues. 5. The k principal components of the observed vector x are then given by: (5) = ( µ) Page No:300

5 Where W = (v 1,,v k ) 6. The reconstruction from the PCA basis is given by: Where W=(v 1,,v k ) (6) = + B. Image Recognition Image recognition is the process where the given samples are classified using machine learning algorithm. In the proposed approach, SVM and KNNN are used for classification. Each classifier is explained in detail below. a. SVM Support vector machine is a flawless method to find out the hyperplane between two different particular classes in high dimensional feature space which can be used for classification. SVM is a supervised algorithm of machine learning [10]. Supervised learning method processed through two steps: Training and Testing. Fig. 2 Optimal hyperplane margin SVM classified into linear and Non-linear classification. The linear SVM classifier is worthwhile for the nonlinear classifier to map the input pattern into higher dimensional feature space. The data which can be linearly separable can be examined using hyper-plane and the data which is linearly non-separable those data are examined methodically with kernel function like higher order polynomial. SVM classification algorithm is based on different kernel methods i.e. Radial basic function (RBF), linear and quadratic kernel function. The RBF kernel is applied on two samples x and x', which indicate as feature vectors in some input space and it can be defined as, (7) (, ) = exp ( ) The value of the kernel function decreases according to the distance and fluctuates between zero (in the limit) and one (when x = x'). Page No:301

6 b. KNN KNN is a supervised algorithm which is working on the distance between the test sample with database sample. The steps of the KNN algorithm are explained as follow. 1. Determine parameter K= number of nearest neighbors. 2. Calculate the distance between the query-instance and all the training samples (8) (, ) = ( ) 3. The Euclidean distance between X = (x1, x2, x3,..xn) and Y = (y1,y2, y3,yn) is defined as: 4. Sort the distance and determine nearest neighbors based on the k-th minimum distance. 5. Gather the category of the nearest neighbors. 6. Use the simple majority of the category of nearest neighbors as the prediction value of the query instance. IV. IMPLEMENTATION AND RESULTS The proposed system is implemented using Open CV, an image and computer vision library. Open CV is the open source image processing library which includes different image processing and computer algorithms useful for the real-time applications. The feature extraction algorithm like PCA and machine learning algorithm like SVM and KNN are implemented using the sci-kit-learn library. The whole system is deployed in the small, card size, low power raspberry pi 3 hardware modules. In this approach, the face images are captured through the phone camera. The camera is connected with the raspberry pi using IP. The database of each candidate is captured and stored in the database is called the registration process. The features of each registered student were extracted using PCA algorithm and provide respective labels. The data is trained using SVM and KNN supervised algorithm. The testing frames are tested with a trained model in real time which marks the attendance of the respective student with time. The result of the proposed system is as shown below with qualitative and quantitative analysis. While checking result on own database in real time (actually taking input from the camera) there are many factors like camera, image quality, illumination, etc. Hence, these factors affect the accuracy of the output. In the qualitative analysis, the results of the recognized faces are shown in Fig. 3. (a) (b) Page No:302

7 (c) (d) (e) (f) Fig. 2 Qualitative analysis of the proposed method (a) (c) (e) Input face image (b)(d)( f) Recognized face Also, the attendance is marked in the excel sheet with respective time which is shown in Fig 3. The quantitative analysis of the proposed system is calculated using the accuracy parameter. The accuracy of the face recognition system is given as (Eq.2) = (9).. Page No:303

8 Table 4.1 present the cross-validation accuracy of presented algorithms. Sr. No. Table 4.1: Performance Evaluation Feature Training Testing Classifier Extraction Accuracy Accuracy technique 1 SVM PCA 98% 83% 2 KNN PCA 92% 80% V. CONCLUSION In this paper, the facial recognition system using SVM and KNN machine learning algorithm has been implemented. The faces are converted into a vector and meaningful features were sorted using principal component analysis. The extracted features are classified using SVM and KNN. The system is implemented in the real-time environment using Raspberry Pi 3 hardware platform. The proposed system achieved 98% crossvalidation and 83% testing accuracy using SVM classifier while 92% cross-validation and 80% testing accuracy using KNN classifier. In future, the system can be made more robust by increasing the database images in different light conditions. The accuracy of the system can be improved by deep learning method. REFERENCES [1] Xiaoguang Lu, "Image Analysis for Face Recognition" published by Dept. of Computer Science & Engineering Michigan State University [2] S. T. Gandhe, K. T. Talele, A. G. Keskar, Intelligent face recognition techniques: A comparative study published in IAENG International Journal of Computer Science. [3] Anil Kumar Sao and B. Yegnanaarayana, Template matching Approach for Pose Problem in Face Verification Speech and Vision Laboratory Department of Computer Science and Engineering, Indian Institute of Technology Madras [4] Sujata G. Bhele and V. H. Mankar, A Review Paper on Face Recognition Techniques published in International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) [5] Riddhi Patel and Shruti B. Yagnik, A Literature Survey on Face Recognition Techniques published International Journal of Computer Trends and Technology (IJCTT) [6] Abdul Matin, Firoz Mahmud, Most Tasnim Binte Shawkat, Recognition of an Individual using the Unique Features of Human face, 2016 IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE) December [7] Adrian Rhesa Septian Siswanto, Anto Satriyo Nugroho, Maulahikmah Galinium, Implementation of Face Recognition Algorithm for Biometrics-Based Time Attendance System, Bandung, Indonesia, 19 January [8] Nirmalya Kar, Mrinal Kanti Debbarma, Ashim Saha, and Dwijen Rudra Pal, Study of Implementing Automated Attendance System Using Face Recognition Technique International Journal of Computer and Communication Engineering, Vol. 1, No. 2, July 2012 [9] A.H. Boualleg, Ch. Bencheriet, and H. Tebbikh, Automatic Face recognition using neural network-pca Laboratory of Automatics and Informatics of Guelma - LAIG University of Guelma, IEEE [10] V. Vapnik, The Nature of Statistical Learning Theory, Springer, N.Y., ISBN Page No:304