Illumination invariant face detection

Size: px

Start display at page:

Download "Illumination invariant face detection"

Sharlene Baldwin
5 years ago
Views:

University of Wollongong Research Online University of Wollongong Thesis Collection 1954-2016

Cordiner University of Wollongong Recommended Citation Cordiner, Alister, Illumination invariant

1 University of Wollongong Research Online University of Wollongong Thesis Collection University of Wollongong Thesis Collections 2009 Illumination invariant face detection Alister Cordiner University of Wollongong Recommended Citation Cordiner, Alister, Illumination invariant face detection, MComSc thesis, School of Computer Science and Software Engineering, University of Wollongong, Research Online is the open access institutional repository for the University of Wollongong. For further information contact the UOW Library:

3 Illumination Invariant Face Detection A thesis submitted in fulllment of the requirements for the award of the degree Master of Computer Science from UNIVERSITY OF WOLLONGONG by Alister Cordiner School of Computer Science and Software Engineering July 2009

5 Dedicated to Leonard and Sylvia iii

6 Declaration This is to certify that the work reported in this thesis was done by the author, unless specied otherwise, and that no part of it has been submitted in a thesis to any other university or similar institution. Alister Cordiner 6th July 2009 iv

7 Abstract The purpose of face detection is to process input images in order to determine the locations of any faces in the image. Faces are complex objects and detecting them remains a challenging task for computer vision systems, despite the relative ease with which humans are able to do so. One of the major diculties faced by face detection systems is challenging illumination conditions, such as low level lighting and cast shadows. This thesis reviews the state of the art face detection methods (with particular emphasis on the method of Viola and Jones) and explores methods of overcoming adverse illumination conditions. These methods can be broadly classied as invariant features, normalisation and variation modelling. Four novel approaches to overcoming illumination that fall into these 3 categories are proposed in this thesis, namely: (i) log-ratio Haar-like features; (ii) DC Haar-like features; (iii) local variance normalisation; and (iv) classier fusion. Furthermore, a new type of feature called the generalised integral image feature (GIIF) is proposed as an alterative to Haarlike features. The GIIF method is not specically related to illumination invariant face detection, but instead applies to the more general task of face detection and is therefore presented as a separate chapter. Experimental results on standard face databases are provided for all of the proposed methods to verify that they achieve improved accuracy. v

8 Acknowledgements First and foremost, I would like to thank my supervisors, Professor Philip Ogunbona and Dr Wanqing Li. Without their wisdom, guidance, constructive feedback and encouragement this work would not have been possible. I wish to thank all of my student colleagues, particularly the members of the Advanced Multimedia Research Lab at the University of Wollongong, for the opportunities to exchange ideas and share diverse points of view at the group meetings. I also wish to thank my employers, particularly Dr Tarik Hammadou, for being exible and allowing me to complete my thesis while being employed. And nally, I would like to thank my family, especially my parents, Leonard and Sylvia, for being supportive and encouraging over these past two years. To them I dedicate this thesis. vi

9 Contents Abstract v Acknowledgements vi 1 Introduction Overview Contributions of the thesis Publication list Organisation of the thesis Literature review Overview Face detection methods Knowledge-based methods Feature-based methods Template matching methods Appearance-based methods Seminal papers Face databases Evaluating face detection systems Challenges in face detection Low resolution or poor quality images Occlusion by foreign objects Illumination variations Facial expressions and structural obstructions Pose (rotation) Articial faces Evaluation of Viola and Jones face detection vii

10 2.5 Discussion Illumination invariant face detection Theory of light Symbols and notation Overview Lighting environments Eects of illumination Human perception Approaches to illumination invariance Invariant features Intensity derivative (gradient) lters DCT-based features Gabor wavelet lters Haar-like wavelet lters Experiments with invariant features Discussion Normalisation and relighting Monotonic transfer functions Homomorphic ltering Basis sets Ratio images Experiments with normalisation Discussion Variation modelling Illumination cones D linear subspace Principal Component Analysis (PCA) Independent Component Analysis (ICA) Fisher's Linear Discriminant (FLD) Non-negative Matrix Factorisation (NMF) Discussion Proposed methods Invariant features Ratio features viii

11 4.1.2 DC features Experimental results Discussion Normalisation/relighting approaches Local variance normalisation Experimental results Discussion Variation modelling Clustering and classier fusion Experimental results Discussion Comparison of methods Experimental results Example face detection results Discussion Generalised integral image features Background Integral images Feature selection Genetic algorithms Related work Method Features Training Translation invariant features Computational complexity Practical considerations Experiments and results Varying the mutation and crossover rates Varying the value of α Varying the range of g Comparison to Viola and Jones Discussion Conclusion 127 ix

12 A Viola and Jones face detection 131 A.1 Haar-like features A.2 AdaBoost A.3 Cascade generation A.4 Merging algorithm A.5 Extensions to Viola and Jones A.5.1 AdaBoost algorithm A.5.2 Weak learners A.5.3 Haar-like features Bibliography 142 x

13 List of Tables 2.1 Commonly used face databases Average detection rates at original scale Average number of false positives at original scale Comparison of face detection results Examples of false positives and false negatives for all illumination invariant approaches Comparison of AUC and error rate xi

14 List of Figures 2.1 Contoured sketch of a human face [45] Face detection system topology Combining results from multi-resolution, multi-face detection Pattern classication approach to detection Knowledge-based architecture Feature-based architecture Template matching architecture Example of AAM iterative deforming [37] Appearance-based architecture Example images from the AR face database [113] The importance of context Examples of occlusion by sunglasses and a scarf [113] Examples of dierent facial expressions [108] Examples of a frontal face and out-of-plane rotated faces [9] The four main approaches to rotation-invariant face detection Examples of articial faces. [8] Face size vs. detection rate Average intensity vs. detection rate Examples of images with low and high average face intensity which were not detected Examples of images where the face detector performed poorly Specular vs. diuse reection Dierent types of reection from the skin [44] Scattering eect of diuse surfaces blurs the incident light as it is reected Histogram skewness indicates the type of reection Example global and individual geometric models generated from 3-D face data [27]. 41 xii

15 3.6 An image can be represented as the pixel-wise product of a reectance image and an illuminance image Attached and cast shadows from an incident light ray [23] Cube and spherical environment maps showing some possible incident light directions Examples of varying illumination [59] Edge maps are unstable under strong illumination variations and the computed edges do not necessarily correspond to changes in reectance Gabor wavelet lters of varying frequencies and orientations Frequency response of the Haar-like features proposed by Viola and Jones Examples of invariant feature values, where the top left image is the original input image from the Bio ID database Performance of a Viola and Jones classier using dierent invariant features Homomorphic ltering using the discrete Fourier transform (DFT) Example of homomorphic ltering applied to a face Decomposition of an input image sequence into reectance and illumination images using the method of [174] Normalisation methods Mean and variance normalisation Visual representation of the illumination cone of a face [2] (a) NMF illumination basis images. (b) Relighting results (top row are the input images, bottom row are the relit images) DCT, PCA (eigenfaces) and NMF basis images generated for the experiments Haar-like features H 1 to H 4 were used by Viola and Jones. Feature H 5 is the proposed DC feature Example images where the classier fails ranked in order of condence weighting from top left to bottom right Comparison of the error rate of face detectors trained with a single Haar-like feature. Feature H 5 is the proposed DC feature Comparison of the classication margin g i versus illumination angle on the Yale B database for (a) the DC features and (b) the ratio features Comparison of error rate on the Yale B database False positive (a) before and (b) after variance and mean normalisation is applied Comparison of dierent types of variance normalisation xiii

16 4.8 Comparison of the classication margin g i versus illumination angle on the Yale B database Comparison of error rate on the Yale B database Examples of 3 possible illumination classes High-level block diagram of the proposed face detection system Examples of faces clustered into face illumination classes using k-means clustering Example membership matrices for crisp and fuzzy clustering Calculating the output of monolithic and multiple classiers The rst 10 Haar-like features selected for the (a) monolithic and (b) clustered face detectors The ROC curve shows that all of the multiple classier variants outperform the monolithic classier method Confusion matrices showing the detection rates across the dierent face illumination classes Comparison of ROC curves Input image and its corresponding integral image Region A can be computed as y 1 + y 4 (y 2 + y 3 ) [48] Example genetic algorithm [51] Some examples of Haar-like features used to represent h Example integral image look-ups used to calculate a single Haar-like feature value Example integral image look-ups used to calculate a single generalised integral image feature value Approximation of a Viola and Jones Haar-like feature in the GIIF feature space constrained by g { 1, 0, +1} Calculation of an oset integral image Example conversions from a GIIF g to a translation invariant GIIF g Eect of varying the mutation and crossover rates Eect of varying the α value on the error rate and sparsity Equivalent Haar-like features h generated by varying the value of α from a high value (left) to a low value (right) Eect of varying the range of g Comparison of the ROC curves A.1 Haar-like features originally proposed by [170], where the darker rectangles are the positive regions and lighter rectangles are the negative regions xiv

17 A.2 Example of an integral image A.3 AdaBoost viewed as a neural network A.4 Weak classiers with lower training errors are given higher weightings A.5 Structure of the face detector cascade A.6 Examples of multiple detections before merging is applied [142] xv

Face detection using generalised integral image features

University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2009 Face detection using generalised integral image features Alister