CHAPTER 3 FACE DETECTION AND PRE-PROCESSING

Transcription

1 59 CHAPTER 3 FACE DETECTION AND PRE-PROCESSING 3.1 INTRODUCTION Detecting human faces automatically is becoming a very important task in many applications, such as security access control systems or contentbased indexing video retrieval systems. The automatic detection of human faces provides users with powerful indexing capacities of the video material. Although face detection is closely related to face recognition as a preliminary required step, face recognition algorithms have received most of the attention in the academic literature compared to face detection algorithms. Considerable progress has been made on the problem of face recognition, especially under stable conditions such as small variations in lighting, facial expression and poses. The illumination problem is basically the variability of an object s appearance from one image to another with slight changes in lighting conditions. Psychophysical experiments show that the human visual system can identify faces of the same person from novel images despite considerable variations in illumination. Lighting variations is a challenging problem in a face recognition research. It is regarded as one of the most critical factors for robust face recognition. The same person, with the same facial expression and view, seems very different under varying lighting conditions as in Figure 3.1. Changes in lighting conditions produce a considerable decrease in the performances of the recognition system.

2 60 Figure 3.1 Face image with variation in pose, expression, illumination shown in Figure 3.2. The general block diagram of the proposed face detection system is Figure 3.2 Block diagram for face detection 3.2 COLOR MODELS FOR SKIN COLOR CLASSIFICATION The skin color classification has gained increasing attention in recent years due to the active research in content-based image representation. For instance, the ability to locate image object as a face can be exploited for image coding, editing, indexing or other user interactivity purposes. Moreover, face localization also provides a good stepping stone in facial expression studies. It would be fair to say that the most popular algorithm to

3 61 face localization is in the use of color information, whereby estimating areas with skin color is often the first vital step of such strategy. Hence, skin color classification has become an important task. Much of the research in skin color based face localization and detection is based on RGB, YCbCr and HSI color spaces. In this section the color spaces are described RGB Color Space The RGB color space consists of the three additive primaries: red, green and blue. Spectral components of these colors combine additively to produce a resultant color. The RGB model is represented by a 3- dimensional cube with red green and blue at the corners on each axis Figure 3.3. Black is at the origin. White is at the opposite end of the cube. The gray scale follows the line from black to white. In a 24-bit color graphics system with 8 bits per color channel, red is (255, 0, 0). On the color cube, it is (1, 0, 0). The RGB model simplifies the design of computer graphics systems but is not ideal for Figure 3.3 RGB color cube all applications. The red, green and blue color components are highly correlated. This makes it difficult to execute some image processing algorithms. Many processing techniques, such as histogram equalization, work on the intensity component of an image only.

4 YCbCr Color Space In response to increasing demands for digital algorithms in handling video information YCbCr color space has been defined, and has since become a widely used model in a digital video. It belongs to the family of television transmission color spaces. The family includes other color space models such as YUV and YIQ. YCbCr is a digital color system, while YUV and YIQ are analog spaces for the respective PAL and NTSC systems. These color spaces separate RGB (Red-Green-Blue) into luminance and chrominance information and are useful in compression applications but the specification of colors is somewhat unintuitive. The Recommendation 601 specifies 8 bit (i.e. 0 to 255) coding of YCbCr, whereby the luminance component Y has an excursion of 219 and an offset of +16. This coding places black at code 16 and white at code 235. In doing so, it reserves the extremes of the range for signal processing footroom and headroom. On the other hand, the chrominance components Cb and Cr have excursions of +112 and offset of +128, producing a range from 16 to 240 inclusively HSI Color Space Hue, saturation and intensity are the three properties used to describe color. So, it is logical that the corresponding color model, HSI is used. In using the HSI color space, a percentage of different colors need not known to produce a color. Adjustment in hue gets the color required. A dark red is changed into pink by adjusting the saturation. In the same way the intensity is altered to bring a change in contrast. Many applications use the HSI color model. Machine vision uses HSI color space in identifying the color of different objects. Image processing applications such as histogram operations, intensity transformations and convolutions operate only on an intensity image. These operations are

5 63 performed with much ease on an image in the HIS color space. The hue (H) is represented as the angle, varying from 0 o to 360 o. Saturation (S) corresponds to the radius, varying from 0 to 1. Intensity (I) varies along the z axis with 0 being black and 1 being white. When S = 0, color is a gray value of intensity 1. When S = 1, color is on the boundary of top cone base as shown in Figure 3.4. The greater the saturation, the farther the color is from white/gray/black (depending on the intensity). Adjusting the hue will vary the color from red at 0 o, through green at 120 o, blue at 240 o, and back to red at 360 o. When I = 0, the color is black and therefore H is undefined. When S = 0, the color is grayscale. H is also undefined in this case. By adjusting I, a color can be made darker or lighter. By maintaining S = 1 and adjusting I, shades of that color are created. Figure 3.4 Double cone model of HIS color space

6 FACE DETECTION A feature-based three-stage scheme is proposed for face detection. The first stage is used to obtain skin regions. Face candidates are detected in the second stage using the statistical SLBM features. Finally, faces are detected from the face candidates in the third stage using SVM classifier Skin-region Detection In order to detect faces under varying illumination, White is used as reference for lighting compensation. In this method, the top 5% of luminance values in the image is regarded as the reference white if the number of these pixels is sufficiently large (>100). The RGB components of the original image are then adjusted so that the average gray value of the reference-white pixels is linearly scaled to 255. Let i [lu; 255] be the top 5% gray levels and fi be the number of pixels of gray level i in the image. Therefore, the modified RGB components can be estimated Skin color based face detection in RGB color space Crowley et al (2000 ) perceived human color varies as a function of the relative direction to the illumination. The pixels for skin region can be detected using a normalized color histogram, and can be further normalized for changes in intensity on dividing by luminance. And thus converted a [R, G, B] vector is converted into an [r, g] vector of normalized color which provides a fast means of skin detection. This gives the skin color region which localizes face. The output is a face detected image which is from the skin region. This algorithm fails when there exists some more skin region like legs, arms, etc.

7 Skin color based face detection in YCbCr color space The skin color classification algorithm with color statistics gathered from YCbCr color space is detailed. The pixels belonging to skin region exhibit similar Cb and Cr values. Furthermore, it has been shown that skin color model based on the Cb and Cr values can provide good coverage of different human races. The thresholds be chosen as [Cr1, Cr2] and [Cb1, Cb2], a pixel is classified to have skin tone if the values [Cr, Cb] fall within the thresholds. The skin color distribution gives the face portion in the color image. This algorithm is also having the constraint that the image should be having only face as the skin region. Thus, RGB components are to be converted into YCbCr components using Equation (3.1) for removing the effect of luminosity during the image processing process. Y = 0.299R G B Cb = R G B (3.1) Cr = 0.500R G B In the YCbCr components, the luminance (brightness) information is contained in a Y component, and the chrominance information is contained in the Cb (blue) and Cr (red). The Cb and Cr components are independent of luminosity and give a good indication on whether a pixel is part of skin or not.background and faces can be distinguished by applying maximum and minimum threshold values for both Cb and Cr components. Thresholding is done with the condition in Equation 3.2 that if the Cb and Cr values fall in the range shown below the pixels are considered to represent the skin region.

8 <= Cb <= <= Cr <= 165 (3.2) In other words, pixels whose intensities fall in the above range are marked as 1 s, and the other pixels are marked as 0 s. A binary image with all skin color represented by a 1 is created. However, other parts of the body, such as exposed arms, legs, and other skin color background objects can also be captured. A post-processing step (e.g., morphological enhancement) is employed to extract all the skin objects separately for future classification Skin Color based face detection in HIS color space Skin color classification in HSI color space is the same as YCbCr color space but here the responsible values are hue (H) and saturation (S). Similar to YCbCr model, the threshold is chosen as [H1, S1] and [H2, S2], and a pixel is classified to have skin tone if the values [H,S] fall within the threshold and this distribution gives the localized face image. Similar to above two algorithm this algorithm is also having the same constraint Face Detection using Skin Color The skin region detected using the RGB model, YCbCr model and HIS model includes the noise and the removal of noise is important to detect the exact face region. So, Morphological operations are used to improve the face detection technique. A series of morphological operations are used as a post-processing step to extract all the skin objects separately for future classification. Figure 3.5 summarizes all the components in the process.

9 67 Figure 3.5 Morphological operations The various morphological operations are explained below. Flood Fill Operation A flood-fill operation is performed on the binary image to fill the holes created inside the face region. It is because of the eyes and lips as shown in Figure 3.6 or the holes created at the boundary after applying the dilation operation. These holes tend to separate all the objects from each other and therefore should be removed. Dilation Operation Dilation operation is applied on the binary images with holes, using a disk structuring element of radius of 1. This operation fine-tunes the boundary of the regions by connecting small breaks and enlarges the shapes a bit for further processing. Remove Small Regions The region whose area is less than 200 pixels is removed, since they are too small to be a face region.

10 68 Opening Operation Opening and closing operations are performed to remove noise. These are the morphological operations, opening operation means erosion followed by dilation to remove noise. An opening operation is applied using a disk structuring element of size 3 to remove connections between closely connected regions if there are any. Connected Component Labeling The connected component labeling algorithm is applied to obtain a separate component for each object. Ideally, each connected component should contain at most one face. But, it is possible that the connected component may contain two or more faces in them. Now the eyes, ears and mouth are extracted from the binary image by considering the threshold for areas which are darker in the mouth than a given threshold. Figure 3.7and 3.8 shows the detected skin region after the removal of noise, the mouth map. Figure 3.6 Skin detection

11 69 Figure 3.7.Skin with noise removal Figure 3.8. Mouth Map Feature Extraction Using Simplified Local Binary Mean (SLBM) The simplified local binary mean (Priya & Banu 2012) overcomes the disadvantage of the LBP algorithm. The LBP considers the center pixel for LBP operation and allocates the pixel value by comparing with the center pixel value. Using this 256 LBP bins are produced, which takes more space and slows down the speed. The intensity values of the neighbor pixels in the block are thresholded by that of the center pixel. In the 3 by 3 pixel block, there are 8 neighbors around the center pixel, so that a total number of 2 8 = 256 different labels can be calculated by the relation between the value of the center pixel and that of its neighbors. SLBM on the other hand, uses the mean of the 9 pixels for thresholding. This method involves three steps which include subdividing, thresholding and weighing. First, a 3 3 sub image is

12 70 cropped. The pixel values are represented as Ip. Thresholding is done using the mean of the 9 elements of the 3 3 sub image (Im). Thresholding is done based on the condition given in Equation (3.3). = 1, 0, (3.3) The SLBM operator thus generated assigns values for all the 9 pixels present in the 3 by 3 matrix under consideration. The weights are then assigned to all the pixel values and are summed as specified in Equation (3.4) SLMB feature thus obtained are considered for classification purpose: 7 p SLMB f (Ip I m)2 (3.4) P 0 The SLMB features are thus calculated. Many images of different types can have similar histograms. Histograms provide only a coarse characterization of an image. This is the main disadvantage of using histograms. So, the statistical features such as mean and standard deviation of the SLMB features of the total image are calculated. These statistical features are used for face detection using SVM. Figure 3.9 shows the detected occluded face region. Figure 3.9 Detected occluded face

13 FACE DETECTION USING SUPPORT VECTOR MACHINE Support vector machines are a set of related supervised statistical learning methods that analyze data and recognize patterns. They are typically used for classification (machine learning) and regression analysis. Vapnik (1995) invented the original SVM algorithm, and the current standard incarnation (soft margin) was proposed by Cortes & Vapnik et al ( 1995).The basic SVM is a non-probabilistic binary linear classifier. It uses a set of training samples marked as belonging to one of two categories to build a model that predicts whether a new sample falls into one category or the other. Intuitively, an SVM model finds a clear gap that is as wide as possible to separate training samples into two categories. New samples are then predicted to belong to a category based on which side of the gap they fall on. More formally, an SVM constructs a hyperplane or set of hyperplanes in a high-dimensional space for classification, regression, or other tasks. A good separation is achieved by the hyperplane that has the largest distance to the nearest training data points of any class (so-called functional margin), since in general the larger the margin the lower the generalization error of the classifier. Figure 3.10 shows the support vectors for the linearly separable samples. These support vectors are decided by three samples on the dashed line. The solid line represents the hyperplane for the decision function. Figure 3.10 SVM for the linearly separable samples.

14 PREPROCESSING A face recognition system, based on computing the distance between unprocessed gray-level images, fails to recognize all the faces in the database and will confuse the faces. The approaches to this problem can be classified in three main categories: Illumination Normalization: In this approach, face images are preprocessed to normalize the illumination. For example, gamma correction, logarithmic transforms histogram equalization few of the methods are used here. Invariant features extraction: This approach attempts to extract facial features invariant to illumination variations. For example, edge maps, derivatives of the gray-level, Gabor-like filters and Fisher-face etc are few of the methods applicable here. Face modeling: illumination variations are mainly due to the 3D shape of human faces under lighting in different directions. There are researchers trying to construct a generative 3D face model that can be used to render face image with different pose and under varying lighting conditions. Figure 3.11 Varied illumination conditions

15 73 This research has been focused only on the first category, the process, called illumination normalization, attempts to transform an image with an arbitrary lighting condition to an image with standard lighting condition. Subsequently the Gamma Intensity Correction method, the Logarithm transform method, Discrete Cosine transform method and the Histogram Equalization method (global and local) are analyzed to process face images normalizing lighting variations Use of Gamma Intensity Correction (GIC) Gamma Intensity Correction is a technique commonly used in computer graphics. This method can control the overall brightness of an image changing the gamma parameter and it can be used to correct the lighting variations in the face image. The gamma correction is a pixel transform in which the output image is the exponential of the input image. (, ) = (, ) / (3.5) Depending on the output values on the transformed image whether it is darker or brighter, and considering a canonically illuminated face image Ic(x, y), the difference between the transformed image and the predefined normal face image is minimized. = arg [ (, ) (, )], (3.6) The intuitive effect of this transform is that the overall brightness of all processed images is adjust to the same level of the normal face image Ic(x, y).

16 74 In Figure 3.12(a), 3.12(d) and 3.12(g) there are the original images, the images 3.12(b), 3.12(e) and 3.12(h) are transformed with Gamma Correction = 0.5, the images 3.12(c), 3.12(f) and 3.12(i) are transformed with =0.7. a d g b e h c f i Figure 3.12 Gamma intensity correction

17 Logarithm Transform and its application It can be assumed that the image gray level f(x, y) is proportional to the product of the reflectance r(x, y) and the illumination e(x, y) f(x, y) = r(x, y) e(x, y) (3.7) Taking logarithm transform log f(x, y) = log r(x, y) + log e(x, y) (3.8) If the incident illumination e(x, y) and the desired uniform illumination e' are known, there is a new gray-scale image under desired uniform illumination: log f (x, y) = log r(x, y) + log e = log r(x, y) + log e(x, y) - (x, y) = log f(x, y) - (x, y) (3.9) where (x, y) = log e(x, y) log e So the normalized face image can be obtained from the original image and an additive compensation term (x, y). Figure 3.13 Pixel intensity mapping with logarithmic transform intensity

18 76 Figure 3.13 shows the mapping of pixel intensities ranging from 0 to 255. The result of the application of Logarithm Transform to a face image badly illuminated is an intensity enhancement of the shadowed regions. In Figure 3.14 there are some original images and the corresponding log transformed. Figure 3.14 Image Results of Logarithmic transformation

19 Discrete Cosine Transform (DCT) and its application The Discrete Cosine Transform is a novel illumination normalization approach for face recognition under varying lighting conditions that keeps facial features intact. The key idea of this method is that illumination variation can be significantly reduced truncating low frequency DCT coefficients in the logarithm DCT domain. The 2D M N discrete cosine transform (DCT) is defined as: (, ) = ( ) ( ) (, ) cos And the inverse transform is ( ) cos[ ( ) ] (3.10) (, ) = ( ) ( ) (, )cos[ ( ) ]cos[ ( ) ]] (3.11) Where (u)= u=0, u=1,2,, M-1 ( ) = v=0, v=1,2,, M-1 Illumination variation usually lies in the low-frequency band, so it can be reduced by removing low-frequency components. This can be activated by making them (Low frequency component) to zero. The function operation appears like an ideal high pass filter. from 3.10: For example, if the (p, q)th DCT coefficient is set to zero (, ) = (, ) + (, ) (3.12)

20 78 = (, ) (, ) (, ) (, ) where (, ) = ( ) ( ) (, )cos[ ( ) ]cos[ ( ) ] The desired normalized face image in the logarithm domain F 0 (x, y) is basically the difference between the original image gray level F(x, y) and the illumination compensation E(p, q) in logarithm domain. The first DCT coefficient determines the overall illumination of a face image, so it can be set to the same value to obtain the desired uniform illumination, for example: (0,0) = log. The value is normally chosen near the middle level of the original image. This approach, by discarding the DCT coefficients of the original image, only adjusts the brightness, whereas, by discarding the DCT coefficients of the logarithm image, adjusts the illumination and recovers the reflectance characteristic of the face. The logarithm images could be used directly for recognition, skipping the inverse logarithm transform step after inverse DCT. Some shadows could lie in the same frequency band of facial features and not in low-frequency band, as a consequence, illumination variation in small areas could not be correctly compensated. So if the logarithm image is restored to the original one incorrect adjustments can make it worse. This illumination compensation method takes a badly illuminated face image, transforms it in logarithm domain, computes the 2D discrete cosine transform and sets a DCT low-frequency coefficient to zero. The first

21 79 DCT coefficient is set to the mean illumination level of the image to determine the overall illumination. Then the inverse 2D discrete cosine transform and normalization of the image are computed. An half-lighted face image is highly correlated with the (0,1)th basis image, therefore, the illumination compensation has been obtained by discarding the (0,1)th DCT coefficient. Figure 3.15 shows four illuminated human face images and the corresponding DCT transformed images. Figure 3.15 Image Results of Discrete cosine transform

22 Application of Histogram Equalization (HE) An image histogram is a graphical representation of the tonal distribution in a digital image. It plots the number of pixels for each tonal value. The horizontal axis of the graph represents the tonal variations, while the vertical axis represents the number of pixels in that particular tone. Particularly, an image histogram of a gray-scale image has in the horizontal axes the 256 brightness levels, and in the vertical axes the number of times this level appears in the image. In Histogram Equalization the global contrast enhancement is obtained using the cumulative density function of the image as a transfer function. Figure 3.16 Image histogram before equalization The result is a histogram approximately constant for all gray values. While Local Histogram Equalization can enhance details of the image, global Histogram Equalization enhances the contrast of whole image. The problem of the first approach is that the output is not always realistic, but in this case the representation of the face image must be invariant to lighting variations and not a realistic image.

23 81 In Figure 3.16 there is a frontal face image bad illuminated and the corresponding histogram. As it s possible to see in Figure, pixels are not balanced, there are more dark pixels than others. To equalize the image histogram the cumulative distribution function (cdf) has been computed. The cdf of each gray level is the sum of its recurrence and the recurrence of the previous gray level in image. Then the histogram equalization formula is : ( ) = ( ) ( ) ( 1) (3.13) Figure 3.17 Image histogram after equalization where cdfmin is the minimum value of the cumulative distribution function, M and N are the width and the height of image, and L is the number of gray levels used. The result is an equalized and normalized image. In Figure 3.16 the face image before histogram equalization and Figure 3.17 the face image after histogram equalization are shown.

24 Local Histogram Equalization In this case pixels are balanced all over the 256 gray levels. The Local Histogram Equalization method is the same as that of the global one but the image has been divided in two equal parts in vertical direction, assuming to have only half of the image badly illuminated. In Figure 3.18 there are two local histogram. In the first histogram, corresponding to the left part of image, the right side of face, there are more light pixels, whereas in the second histogram, corresponding to the right part of image, the left side of face, there are more dark pixels. In Figure 3.19 the original image and the corresponding Histogram Equalization transformed, in the first column, and the Local Histogram Equalization transformed in the last column. Figure 3.19 shows the image and the histogram of the image after local histogram equalization. Figure 3.18 Image and histogram before equalization

25 83 Figure 3.19 Image and the histogram after equalization(lhe) In Figure 3.20 illustrates the frontal face image badly illuminated and the images normalized by methods including GIC, Log transform, DCT, Histogram equalization and Local histogram equalization. Original GIC Log transform Figure 3.20 (Continued)

26 84 HE LHE Figure 3.20 Illumination Normalization 3.6 SUMMARY In this chapter, skin color based face detection with five preprocessing methods for illumination normalization in face images has been proposed. Comparison has been made for detecting faces, using skin color detection on RGB, YCbCr and HIS color spaces and it is found that YCbCr and HIS color space are more efficient in comparison to RGB to classify the skin region. But still both are not able to give very good results. Based on the results of the three algorithms, are combined with the SLBM statistical features and binary SVM classifier is used to detect the face region from the image. It is robust and efficient in classifying the skin color region and face region. Accuracy of proposed algorithm is 98.18%. The Gamma Intensity Correction method (GIC), the Logarithm Transform method, the Discrete Cosine Transform method (DCT), the global Histogram Equalization method (HE) and the Local Histogram Equalization method (LHE).The experimental results on GTAV face Database show that a simple illumination normalization method, like GIC, Logarithm Transform, DCT or HE, can generally improve the image, and consequently the facial features detection performance, compared with a non-preprocessed image. A more complex method, like LHE, also gives good results in a feature detection system, although the face image is not realistic.