Implementation of Face Detection System Using Haar Classifiers

Transcription

1 Implementation of Face Detection System Using Haar Classifiers H. Blaiech 1, F.E. Sayadi 2 and R. Tourki 3 1 Departement of Industrial Electronics, National Engineering School, Sousse, Tunisia 2 Departement of Electronic Engineering, Higher Institute of Applied Science and Technology, Sousse, Tunisia 3 Departement of Physical Sciences, Faculty of Science, Laboratory of Electronics and Microelectronics, Monastir, Tunisia Abstract - This paper presents a hardware implementation of face and eyes detection algorithm. To become we must go through several stages. First we have implemented an algorithm for detecting and tracking eyes written in C using an open source library of image processing and computer vision the OpenCV. Then a profiling on HW/SW (hardware/software) partition was done. The hardware part solves the part of the algorithm with higher computational costs. The system has been implemented on Spartan3A DSP FPGA board. In order to visualize the results on real images, we have made a co-simulation of this bloc using the Simulink tool of Matlab. Keywords: Face; detection; Haar; Adaboost; FPGA. 2 Software implementation In this section we describe the algorithm for detecting and tracking eyes employed [1], we implement this algorithm in C using the library OpenCV [2]. The main steps of the algorithm are described as follows Figure 1 : 1 Introduction A great deal of computer vision research is dedicated to the implementation of systems designed to detect user movements and facial gestures. In many cases, such systems are created with a specific goal providing a way for people with disabilities or limited motor skills to be able to use computer systems. These systems are named Human-computer interaction system. Among the main biometric characteristics that can serve as a means of identification in these systems of human computer interaction, we quote: the shape of the ear, the hand print, the voice, the eyes, the hand shape and the faces 2D and 3D. In this paper we will focus on the detection by the face 2D and the eyes. The paper is organized as follow: Section 2 gives a software implementation of the algorithm of detecting and tracking eyes. An improvement of this algorithm was done by adding a bloc for face detection, and then a profiling on hardware/software partition was done. The hardware part solves the part of the algorithm with higher computational costs. Section 3 outlines the face detection method based on Haar-Like descriptors. Section 4 presents the results of hardware implementation of the face detection bloc on Spartan 3A DSP Board, followed by a simulation and a co-simulation of this bloc using the Simulink tool of Matlab. Figure 1. Overview of the main stages in the system Our problematic in this system is the detection of the eyes from the whole image. Hence there is a possibility to have two components that are near and very similar to the characteristics of the eyes, but in reality they are not eyes as shown in the Figure 2. So to solve this problem and to increase the efficiency of our algorithm, we added a step with the aim of restricting the search box of the eye, to the face area of the user. Figure 2. Problematic in the detection of the eyes from the whole image

2 2.1 Eye detection through face detection This step consists in detecting the face by applying the method based on Haar descriptors, which will start the steps described above. This method is defined in the library OpenCV in C as "CVHaarDetectObjects. So the new faces of tests across the various stages of algorithm for detecting and tracking eyes are described below Figure 3 : whatever the scale adopted, so a loop of scale is applied to the input image for detecting all current faces in the image as shown in the figure below. Our contribution is to translate the bloc of face detection to VHDL working on one scale as shown in the figure below Figure 5. Figure 3. Detection of the eyes from the frame of the face 2.2 Profiling and hardware partition We calculated the runtime of each bloc of the detecting and tracking eyes algorithm. We note that the initialization bloc runs only once time, while the rest of the blocs run repeatedly until we get the position of the eyes and face. So leaving aside the initialization bloc, we note that the bloc with the higher computational costs in our algorithm is the face detection bloc. Therefore we decided to implement it on a hardware platform in order to reduce its runtime Figure 4. Figure 5. Diagram of the Haar face detection method 4 Hardware Implementation of the Face Detection Bloc on FPGA 4.1 Architecture of the Algorithm The previous algorithm has been translated into VHDL, it has one entity "Classifier_kernel" composed of 5 input and 7 output. The architecture of this algorithm Figure 6 called "code" includes 7 components (Feat_rect0_rom, Feat_rect1_rom,...) that contain the values of the classification functions of AdaBoost [7], [8]. Our architecture also includes 7 processes. We applied our detection algorithm on an Image 20x20. Figure 4. Comparison of runtime of different blocs 3 Description of the face method detection We begin by detailing the method of face detection which will be subsequently translated to VHDL. This method is based on Haar-Like descriptors [3], [4], [5], [6]. An Image will be the input of a tree of classification functions. This tree is based on the algorithm AdaBoost, it is composed of 22 cascaded stages. A positive result of the first classification function triggers the assessment of a second classification function, and so on. A negative result of one of the classification functions leads to a rejection of the sub-window. This tree of functions is applied to the input image at any point of the image and Figure 6. Architecture of the face detection bloc

3 4.2 Simulation Results Test of Positive Examples We tested a set of positive images Figure 7. the existence of the face in the Image. We note that our VHDL code is suitable for face detection on a single scale. For this, it requires a good positioning to properly detecting the true face. Example1: This image Figure 10 includes a bit of face. The simulation results gave a negative result. The image failed to pass the stage number 6, which has interrupted the process of detection, and the signal Failed is set to 1 Figure 11. Figure 7. Test images for positive images These images are named positives as they have successfully passed the 22 stages of the cascade by applying all the descriptors of Adaboost Figure 8. The behavior and the number of cycles (2580,5 cycles) for each positive example is the same since the Image will successfully pass all the stages of the AdaBoost algorithm Figure 9. Figure 10. Negativee example n 1 The image has failed in the 6th stage Test interrupted Figure 8. Processing for face detection From the first descriptor of AdaBoost testing whether the eye area is darker than the cheek region, we were able to extract the coordinates of the eyes. And from the positions of eyes, we have localized the face. The image has passed all the stages (22 stages) Face Detected Figure 11. Simulation of negative example n 1 Example2: This image Figure 12 does not contain faces. The detection process has failed from the first stage and the signal Failed is set to 1indicating that there is no face in the picture Figure 13. Figure 12. Negativee Example n 2 The image has failed in the 1st stage Test interrupted Figure 9. Simulation of positive examples The program that we implemented into VHDL and we synthesized with Xilinx ISE, gives good results Test of Negative Examples Secondly, we tested a set of images either not containing faces, or not having a good taken face. The simulation results vary depending on the position and / or Figure 13. Simulation of the negative example n 2

4 4.3 Co-Simulation Bloc VHDL of «ModelSim» Display results classifiers, and thresholds stages. Each bloc ROM is an IPCore that will be implemented on FPGA Spartan. Logic Utilization Table I. Synthesis Results on Spartan3A Used Available % Number of Slices % Number of Slices Flip Flops % Number of 4 input LUTs % Number of bonded IOBs Number of BRAMs Number of MULT18X18SIOs Number of GCLKs % 156% 18% 12% Following synthesis on the Card Spartan3A Table I, we observed that there's an overflow in the number of slices, LUTs and block RAMs used, for that we decided to use another FPGA board Spartan3A DSP. 4.5 Implementation on Spartan3A DSP Board We repeated the same work on a more recent and larger capacity card the Spartan3A DSP of the type XC3SD3400A, from the package FG676. As we worked with signals, it is always difficult for a user ignoring the VHDL, to interpret them. For these reasons we have achieved a co-simulation between Matlab and ModelSim using the Simulink tool. The following figure presents an overview of the Co- simulation Figure 14 : Converter data type Bloc «Locate» Figure 14. Co-simulation Similarly, we visualized the position of the face and eyes in other images taken with different positioning Figure 15. Figure 15. Localization of the face and the eyes in the images of test So Simulink proved that is a good simulation tool since it allows to simulate the code MATLAB and the code VHDL simultaneously. Its use in this work has allowed a good visualization of the results on images. Then, integrating this code on FPGA ensures automatic and effective detection on real time. 4.4 Implementation on Spartan3A Board The synthesis of the architecture is performed under cost constraint (time-to-market, area, power consumption, etc.). We will achieve our synthesis on Xilinx Spartan 3A type XC3S200A, FG320 package. To optimize our detection algorithm into terms of resources used and time of design, we use blocks as IP ROMs to store the predefined values for AdaBoost. To become we will build 6 blocks of ROM memory to store the following values: the values of the characteristics of descriptors (width, height, weight,...), the values left and right of the weak classifiers, the thresholds for weak Logic Utilization Table II. Synthesis Results on Spartan3A DSP Number of Slice Flip Flops Number of 4 input LUTs Number of Slices occupied - Number of Slices that contains only related logic - Number of Slices that contains unrelated logic Number total of 4 input LUTs - Number used as logic - Number used as route-thru Number of bonded IOBs Number of BUFGMUXs Number of DSP48As Number of RAMB 168WERs Average Fanout of Non-Clock Nets The period of the clock CLK MHz) By analyzing the results Table II, we note that the values are positive i.e we haven t an overflow in the resources used. The use of prefabricated IPs, we saved Time of Design and Time to Market. Table III. Comparison of Results Estimated Runtime in C 8,077 ms Used Available % % % % % % % By comparing the results of implementation of the bloc face detection in C and VHDL, by adopting a single scale, we notice that we have gained in terms of runtime and also in design time through the use of IPs Table III % % % % 4.00 is: ns (frequency: Estimated Runtime in VHDL on FPGA 0,036 ms

5 5 Conclusion In this paper, we described the state of the art of the principal methods of detection and tracking used recently in human-computer interaction systems. Then we detailed the face detection method used. We described the basic functioning of a human-computer interaction system that consists on tracking the eyes and detecting the blink with a USB camera. We performed an optimization in this system by adding a step of face detection for making the detection and location of the eyes more efficient. We also studied the runtime of different stages of the system, and we decided to implement the phase of face detection on an FPGA although it takes the most time in the system. So we wrote the code VHDL of the method and we implemented it on the Card Spartan with an embedded processor the MicroBlaze, using IPs Cores to store the values of the learning algorithm AdaBoost. This implementation has saved us the runtime, but also the design time "Time to Market" using predefined IPs. Moreover, we proved the effectiveness of our algorithm for detecting face and eyes that we developed in VHDL, by visualizing the results on a set of images by a cosimulation using the tool Matlab Simulink. Among our nearest perspectives, we want to improve our face detection algorithm to operate on multiple scales in order to scan the entire Image and detect faces in image. Also we'll use our hardware implementation of the method of face detection to achieve a Co-Design in order to build a HW/SW implementation responding to the real time constraint. Also we intend to build our own training base AdaBoost based on positive and negative images, and extend it to recognize the emotions and the attitude of the persons present in the image. thesis, University Pierre et Marie Curie - Paris VI, Institute of Intelligent Systems and Robotics, [7] S. Zhao, Apprentissage et Recherche par le Contenu Visuel de Catégories Sémantiques d'objets Vidéo, master, University Paris Descartes, laboratory of teams Image and Signal processing CNRS France, [8] J.U. Cho, S. Mirzaei, J. Oberg, R. Kastner, FPGA- Based Face Detection System Using Haar Classifiers, International Symposium on Field Programmable Gate Arrays (FPGA), References [1] M. Chau, M. Betke, Real Time Eye Tracking and Blink Detection with USB cameras, Boston University Technical Report No , [2] OpenCV library, sourceforge.net/ projects/ opencvlibrary [3] P. Viola, and M. Jones, Rapid object detection using a boosted cascade of simple features, in Proceedings IEEE Conf. on Computer Vision and Pattern Recognition [4] P. Viola, and M. Jones, Robust real-time object detection. Intl. J. Computer Vision, 57(2): , [5] N. Besbes, Indexation en intervenant d un document vidéo par identification du visage, master, University Paul Sabatier Toulouse III, Doctoral School of Computer Science and telecommunications, [6] P. Negri, Détection et Reconnaissance d objets structurés : Application aux Transports Intelligents,